Jump to content
Ketarin forum
  • Announcements

Guest g003k

Accessing download link hidden by javascript requiring POST

Recommended Posts

Guest g003k

Any help on accessing hidden divs, that only show download values or redirect urls (i know how to use CURL now) would be appreciated. I'm having trouble understanding how to properly POST my javascript click/submit action to get to the next part. 

 

Figure that I need this before I figure out how to automate sites that require my to login first as well. 

Share this post


Link to post
Share on other sites

Got it! Example for DiffMerge:

 

First of all you need the name of the site called by the download form. If you watch the source code of https://sourcegear.com/diffmerge/downloads.php you will find the line

<form id="DMdownload" name="DMdownload" method="post" action="downloaded.php">

The action part is what we are looking for. So our site we want to POST data to is called downloaded.php

The second thing we need is the POST data itself. Scroll the source code down to the form section. There you will find your POST data. For this download they use radio buttons. What we need now is the names and the values of our files.

                <input type="radio" name="DMdownloadVersion" value="win_installer_64" checked />...
                <input type="radio" name="DMdownloadVersion" value="win_installer_32" />...
                <input type="radio" name="DMdownloadVersion" value="win_program_64" />...
                <input type="radio" name="DMdownloadVersion" value="win_program_32" />...
                <input type="radio" name="DMdownloadVersion" value="mac_pkg" />...
                <input type="radio" name="DMdownloadVersion" value="mac_dmg" />...
                <input type="radio" name="DMdownloadVersion" value="ubuntu_64" />...
                <input type="radio" name="DMdownloadVersion" value="ubuntu_32" />...
                <input type="radio" name="DMdownloadVersion" value="fedora_64" />...
                <input type="radio" name="DMdownloadVersion" value="fedora_32" />...

Now it is time for our variables:

 

 

 

Version (self-explanatory):

 

Contents from URL:

https://sourcegear.com/diffmerge/downloads.php

Use regular expression:

 (?<=Download DiffMerge )[0-9.]{2,}

versionjkfgu.png

 

 

 

 

Download URL (The download URL is the site we want to "POST" data to.):

 

Contents from URL:

https://sourcegear.com/diffmerge/downloaded.php

Use regular Expression:

 Problems with the download.*?href="(.+?)"

url7ycmm.png

 

 

 

 

POST data (Depending on the file you want to download.):

 

Name:

DMdownloadVersion

Value:

One of the values (files you wish to download) from the radion button section.

postdataqauwv.png

 

 

 

 

File (Use {url} as download link to be more flexible.):

downloadxvupe.png

Share this post


Link to post
Share on other sites

That worked perfectly for me! Thank you so much. this was challenging and i spent a long time trying to figure it out. 

I really think these walkthroughs should be added to the Ketarin WIKI on FAQ's.... this can be challenging!

 

I also wanted to ask you if development and progress on this is still underway, as I've thought integrating curl or some other redirect follower could be really handy. There are definite feature additions that would continue grow this. It's a great resource, and I've preferred it to Chocolately because I have full power and control over it, instead of hoping someone maintains packages. 

 

 

I'd consider a few items: 

1) Ability to update my "shared" package with folks... as I fix or resolve issues. 

2) A better "portal" to seeing what apps people have shared. The current forum based interface is a little hard. Not trying to be critical. Sublime plugin portal is a great example of a simple but effective list of addins, and something like this could help make Ketarin shine. 

3) FAQ for newbs to correctly get these more complicated mappings (so we can help more!)

Share this post


Link to post
Share on other sites

One more question, what if I need to login to access the download url, is there a tutorial on how to do this? There are a few sites I use that provide a download link after I login, and I'd love to automate these registered software downloads as well. 

Share this post


Link to post
Share on other sites

You're welcome! I don't know about the development progress but as far as I know Flo is working on the next version.

curl is a good point. You can download curl from here and put the curl.exe and libcurl.dll into your Ketarin folder and create a global variable called curl.

globalvariablesdodgs.png

You can execute it like {curl} -d "{postdata}" {link} before or after downloading something, where {postdata} = your POST variable/value and {link} the link you want to send your data to. This is the way i use curl most of the time.

 

Regarding shared apps. As far as I remember this version of the online database is not the final one and suggestions are welcome. I will try to post all the helpful forum entries to the wiki. But I need some time :)

Share this post


Link to post
Share on other sites

You can try using curl to send login data:

http://curl.haxx.se/docs/httpscripting.html#User_name_and_password

Or you can use wget e.g. this way:

start "" /wait wget −−user=YourUserName −−password=YourPassword –output-document="{root}{outputFilename}"

There are two possibilities using wget. You can submit −−user=YourPassword and −−password=YourPassword for both FTP and HTTP connections or you can use it for a specific protocol. For HTTP −−http−user=YourPassword and −−http−password=YourPassword and for FTP connections  −−ftp−user=YourPassword and −−ftp−password=YourPassword (see wget docs).

Share this post


Link to post
Share on other sites

I forgot the Foxit problem. Instead of using the link above (http://www.foxitsoftware.com/Secure_PDF_Reader/) use the official download page as found in menu on top of the web page.

http://www.foxitsoftware.com/downloads/

In case you are using Firefox, analyze the web code with Firefox's Web Developer Tools (especially the Debugger). There you will see a lot of files  e.g. downloads.js, downloads/ and so on and some redirect to pages like getPackage.php and getForm.php and a lot of variable if you follow all the linked pages, js files and asp files. What you need is the page getPackage.php and the variables.

 

Use the link below to determine your version...

http://www.foxitsoftware.com/downloads/require/getPackage.php?&product=Foxit-Reader&platform=Windows&language=English

...with the regex below to get the version number for the english download or change platform= and language= to whatever you wish for the other versions.

(?<="version":\[")[0-9.]{2,}(?=\",)

version06c77.png

 

 

Your download link is an easy thing. Chose your download from the form on the website and redirect to the next page via download button. The redirection page is:

http://www.foxitsoftware.com/downloads/latest.php?product=Foxit-Reader&platform=Windows&package_type=exe&language=English

and this is what we need. If you download a file it will redirect you to another page which always downloads the latest Version of Windows or any other platform and language (replace Windows with other operating systems and English with any other language).

 

url4wd7c.png

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×