Downloading a web page

[JimH44]

I am finding Ketarin very helpful in populating a cloud storage repository at Wuala.com, called LangTran, for Language and Translation software. It exists so people who work in villages without internet connection can update all the installers that they might want to access in the village, while they are still in town and have a connection to the internet.

 

I use Ketarin to keep the installers stored at Wuala.com up-to-date, and it works well for that.

 

I would also like to include selected web pages about these installers in the repository at Wuala.com, but I have not been able to work out how to do that. When I try to set up a recipe for Ketarin to get a web page, I get an error message like this one:

"The downloaded file is not a binary file type (text/html; charset=UTF-8). Possibly there is an error page. Status code: 200 (OK)"

No doubt this happens so we won't get web pages instead of installers if we make mistakes in setting up the Ketarin recipes.

 

Is there a way to ask Ketarin to download the content of a web page and save it somewhere?

 

Thanks,

Jim

[Tomorrow]

Use HTTrack for this purpose. It is able to mimick complete web site structure including files. You can then keep this offline website up to date easily and browse it offline.

Problem is it has a steep learning curve.

[shawn]

You can also use the methods demonstrated by Ketamon here:

https://ketarin.org/forum/index.php/topic/576-ketarin-to-ketamon/

You can then use the before update script to call wget to download the page, which will get you a physical file format, which you can then pipe it to a script.

 

It's better though, IMO, to capture only the portion of the page you require (changelog) than to capture the entire thing. You can store that in a variable and then use it in the after download script to, for example, push the changelog to your site - you can use multireplace for now, but a urlencode function is supposed to be coming in the next build.