Jump to content
Ketarin forum

Download from dynamic links (e.g. Real Temp from Techpowerup)

Recommended Posts

Hello everybody,

any pointers on how to make downloads work for multiple servers / dynamic links such as Real Temp from techpowerup?

e.g. https://www.techpowerup.com/download/techpowerup-real-temp/


thank you very much! :)


p.s. another example would be Cheat Engine from Cloudfront:



with this one I get "405: method not allowed" error...


thanks again!

Link to comment
Share on other sites

For TechPowerUp you can use the download page (the one you linked to) then collect the hidden "id" value from it. That's 164 for this app. I name this field "downloadid". With that parsed the URL to download becomes:


The site uses POST in the source code, but I tested way back in 2015 or so when I created a download profile for NVcleaninstall and you can use GET and the download will work fine. If that wasn't possible (if they actually checked for POST vs GET) then the next best option would be to perform a POST operation (using the "POST data" button) to get the raw file URL. 


For Cheat Engine you'll need to use a different method. The download URLs generated by cloudfront are unique and evil and rely on HTML5 javascript behaviors that you can't perform with Ketarin alone without massive effort. For these you need to use a browser puppeteer - a scriptable full browser that you can control externally - there's a flavor built for us here by @Ambimind called Kuppet, get it here on the forum.

One of the methods cloudfront uses to "prove" the visitor is human is to insist on a delay before rendering all of the content. For this you'll need to use the "redirection-delay" option. I'm not in a hurry so my redirection delay for any sites that impose one is set to 45 seconds. It feels like a long time (it's not), but it guarantees that sites that impose this delay behavior will work. I'd rather have it work reliably than stall out because I tried to push the envelope.

Just don't go too large because some sites have an idle timer, too, which either refreshes the page or redirects you to their home page after a longer delay, usually 5-15 minutes of inactivity. These webmasters suck because you don't always finish the article before you're thrown to a different page or place on the page when it triggers. Grrr. /soapbox

Please try it and post your XML if it's still not behaving.

Link to comment
Share on other sites

thanks for your support!

unfortunately I couldn't get either of them to work :D

for Real Temp, I end up with either a 404 or 200 error... here is my XML.

Regarding Cheat Engine, I wanted to make sure that Kuppet was set correctly first but I can't seem to get the Logitech Options xml to work (the one downloaded from the kuppet page you linked. I get this error (impossible to connect to the remote server):


I think I got kuppet correctly setup:



unless I am using a (very) old version of it and the chrome that came with it?


thanks again!

Real Temp.xml Logitech_Options.xml

Link to comment
Share on other sites

So the reason why the "downloadid" was failing is because you set the POST data within it. The downloadid should be used to parse *for* the ID value. Then the download_url will work correctly. If the ID remains consistent between all builds then you could probably hard-code it, but I would rather just parse for it to be sure.

Here's the fixed XML:

<?xml version='1.0' encoding='utf-8'?>
  <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="bd752d5c-099d-4187-83e7-2ddb9cf3d546">
    <Category>System Maintenance &amp; Security</Category>
Real Temp is a temperature monitoring program designed for all Intel single Core, Dual Core, Quad Core and Core i7 processors.

Each core on these processors has a digital thermal sensor (DTS) that reports temperature data relative to TJMax which is the safe maximum operating core temperature for the CPU. As your CPU heats up, your Distance to TJMax will decrease. If it reaches zero, your processor will start to thermal throttle or slow down so maximizing your distance away from TJMax will help your computer run at full speed and more reliably too.

[Official Download Sources]


corrections and improvements are welcome - @DHT (Ketarin forums)</UserNotes>
    <DownloadDate xsi:nil="true" />
    <VariableChangeIndicator />
    <HashVariable />
    <HttpReferer />
      <SetupInstruction xsi:type="StartProcessInstruction">
        <EnvironmentVariables />
        <Parameters />
            <Regex>(?&lt;=temp )[\d.]+</Regex>
            <StartText>&lt;title&gt;Real Temp </StartText>
            <EndText> Download</EndText>
            <TextualContent />
            <Regex />
            <Url />
            <Regex />
            <Regex>name="id" value="(\d+)"</Regex>
            <PostData />
            <TextualContent />
    <ExecuteCommand>"{winrar}\winrar.exe" x -ad -ibck -o+ "{file}" "{file:directory}"</ExecuteCommand>
    <ExecutePreCommand />
    <FileHippoId />
    <Name>_ Real Temp (Portable)</Name>


The Logitech Options app profile I posted a while back doesn't work now. Logitech has changed their site so many times recently that this particular code doesn't work anymore, and I don't know if/when I'll be inclined to pursue making one that works again. 


Kuppet requires you to create a firewall rule in order for the local server to run. Assuming you're using the native Windows Firewall, you could whitelist the application or the port. It looks like you're using port 9000 so you can do it with this:

netsh advfirewall firewall add rule name="Kuppet" dir=in protocol=TCP localport=9000 action=allow enable=yes profile=any localip=any remoteip=any interfacetype=any

To later remove it use:

netsh advfirewall firewall delete rule "Kuppet"

This will ensure that the firewall allows Kuppet to operate as a standalone server on your computer.

I also recommend you change the definition for the "run_kuppet" variable to:

if(!(PS kuppet -ea 0)) {START -WindowsStyle hidden BIN\kuppet 9000}

This will open Kuppet interactively - what you'll see is basically just a powershell window showing the log of actions. Use this to see whether it is loading correctly, and whether the requests are being processed correctly. There's a bit of a learning curve.

One issue with Kuppet is that by default it uses the "HeadlessChrome" user-agent, which many web servers block by default. You'll need to configure a kuppet.config file with the user-agent you wish to use. I show how to automate this process here


Link to comment
Share on other sites

Thanks again!

I imported your xml for Real Temp but still get this error:



As for cheat engine, I will try to play with Kuppet some more :)


Update: no luck with Kuppet.. if you can share a working example .xml that uses kuppet, from which I can start building one for Cheat Engine, I would be grateful!

Link to comment
Share on other sites

Looks like TechPowerUp is now using manual server selection, so clicking the button alone doesn't actually download the file, it just shows a page with the server selection. Parse that for "server_id" then use POST to download the file. Since Ketarin can't post to the download URL you'll have to shell it out in a powershell variable or use the pre-download command using something that can (Wget or Curl). This is usually pretty simply once you understand the syntax:

curl.exe -v -X POST -L -o "file.ext" -d "id=164&server_id=12" -4 "https://www.techpowerup.com/download/techpowerup-real-temp/"

This performs a verbose POST request, following redirects, saves the output to "file.exe", and POSTs the data "id=164&server_id=12" using IPv4 to the URL. 

This doesn't actually download the file. There's an nginx proxy there that's refusing to actually allow the download. It's either blocking the UA or the referrer tree. It doesn't matter - the verbose response means that it's still showing you the "real" download URL which you could capture and drop into the download URL in Ketarin and that will actually work. 

If you wrap this in a powershell variable you can capture the download URL. I don't have time today to demonstrate but I will try to come back in the next couple days to show you if you haven't figured it out.


I've attached the app profile for SetDefaultBrowser which uses Kuppet (though it looks like it's now EOL and being replaced by a more powerful tool). 


Link to comment
Share on other sites

thanks so much for your support and thank you for the detailed explanation!

I think this is getting a little bit out of hand for me :D

If you will have the time to make it work, I can wait no problem otherwise, thanks anyway!

Regarding the SetDefaultBrowser.xml, just to make sure the issue is on my side, if you try to update it, your result is different from this, right?


I also tried it with firewall disabled, same result

Link to comment
Share on other sites

Here's a working Real Temp that uses a PowerShell script to read the Location header from the POST response.

Real Temp.xml


SetDefaultBrowser works fine for me. Make sure you're not interfering with it! Kuppet has a built-in timeout that's higher than Ketarin alone, but this specific app uses an imposed 45 second delay *after the last activity*! This is important for anything that needs to bypass CloudFlare or Cloudfront CAPTCHA delays. This is the output I get in Kuppet with logging enabled:

---: Ketarin is playing with her puppet > Port:8080 Time: 3/13/2024 7:28:22 PM

===: port            : 8080
===: kuppetTimeout   : 360
===: pageLoadTimeout : 90
===: userAgent       : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ Safari/537.36
===: verboseLogging  : true
===: logging         : true
===: chromeVisible   : false
===: -----------------

---: R01 | REQUEST  : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/
===: R01 | Post Data String: redirection-delay=45
===: Chrome instance starting
===: The BROWSER user agent is    : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/100.0.4889.0 Safari/537.36
===: Opening new page
===: Set page-load timeout        : 90 seconds
===: Setting PAGE user agent      : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ Safari/537.36
===: Set extra headers
===: Visiting : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/
===: Rendering the javascript
===: Redirection-Delay for: 45 seconds
===: Rendering the Shadow DOM
===: Returning the rendered HTML - everything between <HTML>...</HTML>
===: 'Beautify' the HTML
===: Closing browser
---: R01 | RESPONSE : Cached:false : Duration:50.22 seconds
---: R02 | REQUEST  : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/
===: R02 | Post Data String: redirection-delay=45
---: R02 | RESPONSE : Cached:true  : Duration:0.004 seconds
---: R03 | REQUEST  : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/
===: R03 | Post Data String: redirection-delay=45
---: R03 | RESPONSE : Cached:true  : Duration:0.003 seconds
---: R04 | REQUEST  : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/
===: R04 | Post Data String: redirection-delay=45
---: R04 | RESPONSE : Cached:true  : Duration:0.003 seconds
---: R05 | REQUEST  : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/
===: R05 | Post Data String: redirection-delay=45
---: R05 | RESPONSE : Cached:true  : Duration:0.005 seconds
---: R06 | REQUEST  : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/
===: R06 | Post Data String: redirection-delay=45
---: R06 | RESPONSE : Cached:true  : Duration:0.003 seconds


Link to comment
Share on other sites

One more thing about Kuppet. It has a heavy-duty cache so it doesn't re-request pages unnecessarily, even if you think it should. If you ran it, cancelled, then ran it again you might have issues. Kill Kuppet and try again to make sure it purges it's own cache.

Link to comment
Share on other sites

Real Temp now works, thank you very much!

regarding Kuppet, by looking at your output, I checked the config file included in the 0.8.7 release available on the forum and realized it needed setting up, so I modified that to make it look something similar to yours (see attached) and now I get a similar output:



although Ketarin still times out before Kuppet can finish its job.

What could be still missing here?

thank you once again for your support!


Link to comment
Share on other sites

I recommend removing the older user-agents from the userAgentArray variable. Firefox/65 and Chrome/100 are both *very* old and many security features on websites now block them by default. 

The next troubleshooting step is to change the chromeVisible value in your config to true. This will allow you to watch as Kuppet operates, which might be enough to see what is failing.

Link to comment
Share on other sites

Is it detecting the version number correctly? Or is it just the download that's failing?

You might also need to change the global user-agent (File, Settings, Connection, Default user agent) or at least assign one within the app profile at Advanced Settings, User-agent. By default the user-agent indicates it's Ketarin and might be getting blocked. 

Link to comment
Share on other sites

Yes, my global UA is the default one: Mozilla/4.0 (compatible; Ketarin; +https://ketarin.org/)

is it a good practice to change it to something else? 

In any case, I copied the same UA, that I put in the kuppet.config file, in the app profile, which is the exact version stated in the browser About page:



I tried to check the versionstub variable, but the problem seems to be that it cannot load the page? although I see it in the Chromium browser (it loads correctly after the 3 second Anti-Crawler IP checker page): https://imgur.com/a/Kz9g0Bv


Link to comment
Share on other sites

if you don't have a lot of requests or apps then changing the UA globally is unlikely to help. if you do, then you might want to change it and keep it relatively current.

the format for the version string in Chrome UA is "Chrome/major.0.0.0" - it never includes the minor or build numbers. This is similar to all other modern browsers - they don't include anything but the major version dot zero, and sometimes an additional couple dot-zeroes. Change it to:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ Safari/537.36

That alone might help.

If you haven't set the Version number column to display in the main interface you might want to do that first. This ensures that you can see when new versions are detected and might help you troubleshoot this one. Since this site is one of the ones that is really aggressive about spiders you might want to test with a different one. I've attached another one (aria2) that uses Kuppet.


Link to comment
Share on other sites

thanks for the pointers!

Aria works! https://imgur.com/a/FmsH6yC

I have the version number set, along with previous (See also link above).

At this point, have no idea why the other one isn't working.. but I have a working example, so thank you for that!

EDIT: I noticed that aria works even without kuppet so I don't think it means that kuppet works..

In any case, even with kuppet, I still get the 405 error with Cheat Engine.

At this point, I guess it would be best to stop bothering you guys, thank you for the support so far :)

Cheat Engine.xml

Link to comment
Share on other sites

  • 3 weeks later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.