Jump to content
Ketarin forum

All Activity

This stream auto-updates

  1. Today
  2. thanks for the pointers! Aria works! https://imgur.com/a/FmsH6yC I have the version number set, along with previous (See also link above). At this point, have no idea why the other one isn't working.. but I have a working example, so thank you for that! EDIT: I noticed that aria works even without kuppet so I don't think it means that kuppet works.. In any case, even with kuppet, I still get the 405 error with Cheat Engine. At this point, I guess it would be best to stop bothering you guys, thank you for the support so far Cheat Engine.xml
  3. Yesterday
  4. if you don't have a lot of requests or apps then changing the UA globally is unlikely to help. if you do, then you might want to change it and keep it relatively current. the format for the version string in Chrome UA is "Chrome/major.0.0.0" - it never includes the minor or build numbers. This is similar to all other modern browsers - they don't include anything but the major version dot zero, and sometimes an additional couple dot-zeroes. Change it to: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36 That alone might help. If you haven't set the Version number column to display in the main interface you might want to do that first. This ensures that you can see when new versions are detected and might help you troubleshoot this one. Since this site is one of the ones that is really aggressive about spiders you might want to test with a different one. I've attached another one (aria2) that uses Kuppet. aria2.xml
  5. Yes, my global UA is the default one: Mozilla/4.0 (compatible; Ketarin; +https://ketarin.org/) is it a good practice to change it to something else? In any case, I copied the same UA, that I put in the kuppet.config file, in the app profile, which is the exact version stated in the browser About page: I tried to check the versionstub variable, but the problem seems to be that it cannot load the page? although I see it in the Chromium browser (it loads correctly after the 3 second Anti-Crawler IP checker page): https://imgur.com/a/Kz9g0Bv
  6. Last week
  7. Is it detecting the version number correctly? Or is it just the download that's failing? You might also need to change the global user-agent (File, Settings, Connection, Default user agent) or at least assign one within the app profile at Advanced Settings, User-agent. By default the user-agent indicates it's Ketarin and might be getting blocked.
  8. hmm, I removed the old UAs and manually updated chromium and the string to match the latest v124. With Browser visible, this is what I see: but it still times out.. thank you all for the support kuppet.config
  9. I recommend removing the older user-agents from the userAgentArray variable. Firefox/65 and Chrome/100 are both *very* old and many security features on websites now block them by default. The next troubleshooting step is to change the chromeVisible value in your config to true. This will allow you to watch as Kuppet operates, which might be enough to see what is failing.
  10. Real Temp now works, thank you very much! regarding Kuppet, by looking at your output, I checked the config file included in the 0.8.7 release available on the forum and realized it needed setting up, so I modified that to make it look something similar to yours (see attached) and now I get a similar output: although Ketarin still times out before Kuppet can finish its job. What could be still missing here? thank you once again for your support! kuppet.config
  11. One more thing about Kuppet. It has a heavy-duty cache so it doesn't re-request pages unnecessarily, even if you think it should. If you ran it, cancelled, then ran it again you might have issues. Kill Kuppet and try again to make sure it purges it's own cache.
  12. Here's a working Real Temp that uses a PowerShell script to read the Location header from the POST response. Real Temp.xml SetDefaultBrowser works fine for me. Make sure you're not interfering with it! Kuppet has a built-in timeout that's higher than Ketarin alone, but this specific app uses an imposed 45 second delay *after the last activity*! This is important for anything that needs to bypass CloudFlare or Cloudfront CAPTCHA delays. This is the output I get in Kuppet with logging enabled: ---: Ketarin is playing with her puppet > Port:8080 Time: 3/13/2024 7:28:22 PM ===: KUPPET PARAMETERS ===: port : 8080 ===: kuppetTimeout : 360 ===: pageLoadTimeout : 90 ===: userAgent : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 ===: verboseLogging : true ===: logging : true ===: chromeVisible : false ===: ----------------- ---: R01 | REQUEST : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/ ===: R01 | Post Data String: redirection-delay=45 ===: Chrome instance starting ===: The BROWSER user agent is : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/100.0.4889.0 Safari/537.36 ===: Opening new page ===: Set page-load timeout : 90 seconds ===: Setting PAGE user agent : Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 ===: Set extra headers ===: Visiting : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/ ===: Rendering the javascript ===: Redirection-Delay for: 45 seconds ===: Rendering the Shadow DOM ===: Returning the rendered HTML - everything between <HTML>...</HTML> ===: 'Beautify' the HTML ===: Closing browser ---: R01 | RESPONSE : Cached:false : Duration:50.22 seconds ---: R02 | REQUEST : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/ ===: R02 | Post Data String: redirection-delay=45 ---: R02 | RESPONSE : Cached:true : Duration:0.004 seconds ---: R03 | REQUEST : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/ ===: R03 | Post Data String: redirection-delay=45 ---: R03 | RESPONSE : Cached:true : Duration:0.003 seconds ---: R04 | REQUEST : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/ ===: R04 | Post Data String: redirection-delay=45 ---: R04 | RESPONSE : Cached:true : Duration:0.003 seconds ---: R05 | REQUEST : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/ ===: R05 | Post Data String: redirection-delay=45 ---: R05 | RESPONSE : Cached:true : Duration:0.005 seconds ---: R06 | REQUEST : https://kolbi.cz/blog/2017/11/10/setdefaultbrowser-set-the-default-browser-per-user-on-windows-10-and-server-2016-build-1607/ ===: R06 | Post Data String: redirection-delay=45 ---: R06 | RESPONSE : Cached:true : Duration:0.003 seconds
  13. thanks so much for your support and thank you for the detailed explanation! I think this is getting a little bit out of hand for me If you will have the time to make it work, I can wait no problem otherwise, thanks anyway! Regarding the SetDefaultBrowser.xml, just to make sure the issue is on my side, if you try to update it, your result is different from this, right? I also tried it with firewall disabled, same result
  14. Looks like TechPowerUp is now using manual server selection, so clicking the button alone doesn't actually download the file, it just shows a page with the server selection. Parse that for "server_id" then use POST to download the file. Since Ketarin can't post to the download URL you'll have to shell it out in a powershell variable or use the pre-download command using something that can (Wget or Curl). This is usually pretty simply once you understand the syntax: curl.exe -v -X POST -L -o "file.ext" -d "id=164&server_id=12" -4 "https://www.techpowerup.com/download/techpowerup-real-temp/" This performs a verbose POST request, following redirects, saves the output to "file.exe", and POSTs the data "id=164&server_id=12" using IPv4 to the URL. This doesn't actually download the file. There's an nginx proxy there that's refusing to actually allow the download. It's either blocking the UA or the referrer tree. It doesn't matter - the verbose response means that it's still showing you the "real" download URL which you could capture and drop into the download URL in Ketarin and that will actually work. If you wrap this in a powershell variable you can capture the download URL. I don't have time today to demonstrate but I will try to come back in the next couple days to show you if you haven't figured it out. I've attached the app profile for SetDefaultBrowser which uses Kuppet (though it looks like it's now EOL and being replaced by a more powerful tool). SetDefaultBrowser.xml
  15. Thanks again! I imported your xml for Real Temp but still get this error: As for cheat engine, I will try to play with Kuppet some more Update: no luck with Kuppet.. if you can share a working example .xml that uses kuppet, from which I can start building one for Cheat Engine, I would be grateful!
  16. Welcome aboard @Mundee. This is demonstrated pretty well in this thread. While Ambimind uses variables for the parts of the calling function, if you already know the name, type and form of the variable, then you could use the abbreviated syntax. For Start/End and RegEx you use: $x = $app.variables.MyVersionVariable.CachedContent For Textual Content use: $x = $app.variables.MyVersionVariable.TextualContent In both samples "MyVersionVariable" is the name of the variable you'll be using instead of {MyVersionVariable}.
  17. Hi, I am fairly new to Ketarin and still trying to figure things out. I would like to run a powershell command after downloading that uses the "global" {appname} and the {version} variable I created for the package. However the content of the variable(s) do not seem to be getting forwarded/piped to powershell. Am I doing something wrong? Thank you
  18. Earlier
  19. So the reason why the "downloadid" was failing is because you set the POST data within it. The downloadid should be used to parse *for* the ID value. Then the download_url will work correctly. If the ID remains consistent between all builds then you could probably hard-code it, but I would rather just parse for it to be sure. Here's the fixed XML: <?xml version='1.0' encoding='utf-8'?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="bd752d5c-099d-4187-83e7-2ddb9cf3d546"> <Category>System Maintenance &amp; Security</Category> <WebsiteUrl>http://www.techpowerup.com/realtemp/</WebsiteUrl> <UserAgent>Mozilla/5.0</UserAgent> <UserNotes>[Description] Real Temp is a temperature monitoring program designed for all Intel single Core, Dual Core, Quad Core and Core i7 processors. Each core on these processors has a digital thermal sensor (DTS) that reports temperature data relative to TJMax which is the safe maximum operating core temperature for the CPU. As your CPU heats up, your Distance to TJMax will decrease. If it reaches zero, your processor will start to thermal throttle or slow down so maximizing your distance away from TJMax will help your computer run at full speed and more reliably too. [Official Download Sources] http://www.techpowerup.com/downloads/SysInfo/Real_Temp/ [Changelog] http://www.techpowerup.com/realtemp/ ---------- corrections and improvements are welcome - @DHT (Ketarin forums)</UserNotes> <LastFileSize>330853</LastFileSize> <LastFileDate>2024-03-01T09:34:05.8630984</LastFileDate> <IgnoreFileInformation>false</IgnoreFileInformation> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator /> <HashVariable /> <HashType>None</HashType> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer /> <SetupInstructions> <SetupInstruction xsi:type="StartProcessInstruction"> <EnvironmentVariables /> <FileName>"{file}"</FileName> <Parameters /> <WaitForExit>true</WaitForExit> </SetupInstruction> </SetupInstructions> <Variables> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>StartEnd</VariableType> <Regex>(?&lt;=temp )[\d.]+</Regex> <Url>https://www.techpowerup.com/download/techpowerup-real-temp/</Url> <StartText>&lt;title&gt;Real Temp </StartText> <EndText> Download</EndText> <TextualContent /> <Name>version</Name> </UrlVariable> </value> </item> <item> <key> <string>download_url</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <Url /> <TextualContent>https://www.techpowerup.com/download/techpowerup-real-temp/?id={downloadid}</TextualContent> <Name>download_url</Name> </UrlVariable> </value> </item> <item> <key> <string>previous</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <PostData>3.70</PostData> <TextualContent>3.70</TextualContent> <Name>previous</Name> </UrlVariable> </value> </item> <item> <key> <string>downloadid</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>name="id" value="(\d+)"</Regex> <PostData /> <Url>https://www.techpowerup.com/download/techpowerup-real-temp/</Url> <TextualContent /> <Name>downloadid</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand>"{winrar}\winrar.exe" x -ad -ibck -o+ "{file}" "{file:directory}"</ExecuteCommand> <ExecutePreCommand /> <ExecuteCommandType>Batch</ExecuteCommandType> <ExecutePreCommandType>Batch</ExecutePreCommandType> <SourceType>FixedUrl</SourceType> <DeletePreviousFile>true</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated>2024-03-01T09:34:05.8630984</LastUpdated> <TargetPath>{root}{apps}\{category}\{appname}\</TargetPath> <FixedDownloadUrl>{download_url}</FixedDownloadUrl> <Name>_ Real Temp (Portable)</Name> </ApplicationJob> </Jobs> The Logitech Options app profile I posted a while back doesn't work now. Logitech has changed their site so many times recently that this particular code doesn't work anymore, and I don't know if/when I'll be inclined to pursue making one that works again. Kuppet requires you to create a firewall rule in order for the local server to run. Assuming you're using the native Windows Firewall, you could whitelist the application or the port. It looks like you're using port 9000 so you can do it with this: netsh advfirewall firewall add rule name="Kuppet" dir=in protocol=TCP localport=9000 action=allow enable=yes profile=any localip=any remoteip=any interfacetype=any To later remove it use: netsh advfirewall firewall delete rule "Kuppet" This will ensure that the firewall allows Kuppet to operate as a standalone server on your computer. I also recommend you change the definition for the "run_kuppet" variable to: if(!(PS kuppet -ea 0)) {START -WindowsStyle hidden BIN\kuppet 9000} This will open Kuppet interactively - what you'll see is basically just a powershell window showing the log of actions. Use this to see whether it is loading correctly, and whether the requests are being processed correctly. There's a bit of a learning curve. One issue with Kuppet is that by default it uses the "HeadlessChrome" user-agent, which many web servers block by default. You'll need to configure a kuppet.config file with the user-agent you wish to use. I show how to automate this process here.
  20. thanks for your support! unfortunately I couldn't get either of them to work for Real Temp, I end up with either a 404 or 200 error... here is my XML. Regarding Cheat Engine, I wanted to make sure that Kuppet was set correctly first but I can't seem to get the Logitech Options xml to work (the one downloaded from the kuppet page you linked. I get this error (impossible to connect to the remote server): I think I got kuppet correctly setup: unless I am using a (very) old version of it and the chrome that came with it? thanks again! Real Temp.xml Logitech_Options.xml
  21. For TechPowerUp you can use the download page (the one you linked to) then collect the hidden "id" value from it. That's 164 for this app. I name this field "downloadid". With that parsed the URL to download becomes: https://www.techpowerup.com/download/techpowerup-real-temp/?id={sdownloadid} The site uses POST in the source code, but I tested way back in 2015 or so when I created a download profile for NVcleaninstall and you can use GET and the download will work fine. If that wasn't possible (if they actually checked for POST vs GET) then the next best option would be to perform a POST operation (using the "POST data" button) to get the raw file URL. For Cheat Engine you'll need to use a different method. The download URLs generated by cloudfront are unique and evil and rely on HTML5 javascript behaviors that you can't perform with Ketarin alone without massive effort. For these you need to use a browser puppeteer - a scriptable full browser that you can control externally - there's a flavor built for us here by @Ambimind called Kuppet, get it here on the forum. One of the methods cloudfront uses to "prove" the visitor is human is to insist on a delay before rendering all of the content. For this you'll need to use the "redirection-delay" option. I'm not in a hurry so my redirection delay for any sites that impose one is set to 45 seconds. It feels like a long time (it's not), but it guarantees that sites that impose this delay behavior will work. I'd rather have it work reliably than stall out because I tried to push the envelope. Just don't go too large because some sites have an idle timer, too, which either refreshes the page or redirects you to their home page after a longer delay, usually 5-15 minutes of inactivity. These webmasters suck because you don't always finish the article before you're thrown to a different page or place on the page when it triggers. Grrr. /soapbox Please try it and post your XML if it's still not behaving.
  22. Hello everybody, any pointers on how to make downloads work for multiple servers / dynamic links such as Real Temp from techpowerup? e.g. https://www.techpowerup.com/download/techpowerup-real-temp/ thank you very much! p.s. another example would be Cheat Engine from Cloudfront: https://www.cheatengine.org/downloads.php with this one I get "405: method not allowed" error... thanks again!
  23. Oh! I've posted a link to this image - using the "Other Media" > "Insert Image From URL" forum feature. I've also just tested in FF, Chrome & Edge - all show the image. Not sure why you are not seeing it? in case you're pointing out the lack of text : I intended the image to be self explanatory.
  24. Since the download link is to a different domain, you might be better off using regex to parse for the download URL instead of constructing it from logical elements like version and extension. I use something like this for pretty much all of the downloads where the download is on a different domain: "([^'"]+{version}[^'"]+x64\.msi)" When parsed against {url2} this should capture the complete download URL for the first file that includes the version number and ends with "x64.msi". The URL must be wrapped in double-quotes on-page. If there's a chance that there are multiple matches and you want to control it further then you can fiddle with the beginning exclusionary block as so: "(https://[^'"]+{version}[^'"]+x64\.msi)" or "(https://[^'"]+uvnc\.eu/[^'"]+{version}[^'"]+x64\.msi)" This will let you ensure that the download is from an HTTPS website for the first one and the second adds a specific domain or subdomain to the requirements. If there are language specific builds you can add that as an additional qualifier and just duplicate the "[^'"]+" on either side as so: "(https://[^'"]+en_US[^'"]+{version}[^'"]+x64\.msi)" In the case where they might be using a single quote to wrap attribute values instead of doublequotes (or they use them differently throughout the site) you can use this: ['"](https://[^'"]+{version}[^'"]+x64\.msi)['"] Where this type of thing will fail you is if there is a block of code on the site that is actually commented out but still matches the other syntax requirements or the real URL isn't actually wrapped in quotes. I've seen this several times over the years but unfortunately there's simply no way to completely prepare for when people add comments or leave old code in their page.
  25. UltraVNC links for download are a mess! I needed to create multiple variables to get link after link but now works but i dont khow how long <?xml version='1.0' encoding='utf-8'?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="33d27831-23b8-4560-8cc9-6c7918d52784"> <WebsiteUrl /> <UserAgent /> <UserNotes /> <LastFileSize>8537600</LastFileSize> <LastFileDate>2023-10-21T21:03:47+01:00</LastFileDate> <IgnoreFileInformation>false</IgnoreFileInformation> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>version</VariableChangeIndicator> <HashVariable /> <HashType>None</HashType> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer>https://uvnc.com/downloads/ultravnc.html</HttpReferer> <SetupInstructions> <SetupInstruction xsi:type="StartProcessInstruction"> <EnvironmentVariables /> <FileName>{file}</FileName> <Parameters /> <WaitForExit>true</WaitForExit> </SetupInstruction> </SetupInstructions> <Variables> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>/downloads/ultravnc/\d{3}-ultravnc-\d-\d-\d-\d.html</Regex> <Url>https://uvnc.com/downloads/ultravnc.html</Url> <Name>version</Name> </UrlVariable> </value> </item> <item> <key> <string>download</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <Url>https://www.uvnc.eu/download/1436/UltraVNC_1436_X64.msi</Url> <TextualContent>https://www.uvnc.eu/download/{version2}/UltraVNC_{version2}_X64.msi</TextualContent> <Name>download</Name> </UrlVariable> </value> </item> <item> <key> <string>url1</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>https://uvnc.com{version}</TextualContent> <Name>url1</Name> </UrlVariable> </value> </item> <item> <key> <string>url2</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>/component/jdownloads/summary/\d{3}-ultravnc-\d{4}-msi-x64.html</Regex> <Url>{url1}</Url> <Name>url2</Name> </UrlVariable> </value> </item> <item> <key> <string>version2</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>ultravnc-(\d+)-x64</Regex> <Url>{url1}</Url> <Name>version2</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand /> <ExecuteCommandType>Batch</ExecuteCommandType> <ExecutePreCommandType>Batch</ExecutePreCommandType> <Category /> <SourceType>FixedUrl</SourceType> <PreviousLocation>D:\Backups\Tools\UltraVNC\UltraVNC_1436_X64.msi</PreviousLocation> <DeletePreviousFile>true</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated>2024-02-28T14:56:04.1365006+00:00</LastUpdated> <TargetPath>D:\Backups\Tools\UltraVNC\</TargetPath> <FixedDownloadUrl>{download}</FixedDownloadUrl> <Name>UltraVnc</Name> </ApplicationJob> </Jobs>
  1. Load more activity
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.