Kuppet : Javascript rendered pages for Ketarin

shawn · April 21, 2022

@Ambimindthat seems like a perfect solution. I can roll it out on only those sites where it's necessary without imposing unnecessary delays on other sites. It works perfectly. This is the first time in probably a year that I've been able to check for updates for both of those apps in the same run. Yay!

Seeing the power of using the POST data feature to tune Kuppet for individual sites. Can the other options be assigned there, too? It would be nice to specify a given UA for a site that doesn't like other user-agents, for example. If you plan to implement this as a feature there's no hurry from me. I can see myself using it someday, but currently don't have a need.

When I have some free time I'll be writing a script to update the kuppet.config file with the current Chrome and Firefox versions in a post-update script. I'll share that here.

Ambimind · April 22, 2022

Thanks for your thoughtful feedback @shawn

shawn · April 29, 2022

Alrighty... Here's a post-download command you can use to keep the version information up to date in the kuppet.config file. Change the $kuppetCfg path, and make sure you use a unique $browserIndex for each different browser you want to include. Also, make sure the post-download command is set to use PowerShell.

This one for Firefox:

# For each browser you want to include:
# Use a different Index and different UA pattern
# 0 = Firefox
$browserIndex = 0;
$browserVersion = $app.variables.version.CachedContent.Split(".")[0];
$browserUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:$browserVersion.0) Gecko/20100101 Firefox/$browserVersion.0";
$kuppetCfg = "K:\Ketarin\kuppet\kuppet.config";

# do the work
$kuppetJson = Get-Content $kuppetCfg -Raw | ConvertFrom-Json;
$kuppetJson.userAgentArray[$browserIndex] = $browserUA;
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($kuppetCfg, ($kuppetJson | ConvertTo-Json), $Utf8NoBomEncoding)

This is for Chrome:

# For each browser you want to include:
# Use a different Index and different UA pattern
# 1 = Chrome
$browserIndex = 1;
$browserVersion = $app.variables.version.CachedContent.Split(".")[0];
$browserUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/$browserVersion.0.0.0 Safari/537.36";
$kuppetCfg = "K:\Ketarin\kuppet\kuppet.config";

# do the work
$kuppetJson = Get-Content $kuppetCfg -Raw | ConvertFrom-Json;
$kuppetJson.userAgentArray[$browserIndex] = $browserUA;
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($kuppetCfg, ($kuppetJson | ConvertTo-Json), $Utf8NoBomEncoding)

If you're going to include additional user-agents just make sure to use a new $browserIndex value for each one, otherwise you'll overwrite the wrong one.

2022-05-02: fixed to strip the BOM (byte-order marker) from the file when it's written

2022-08-03: updates user-agent strings to match current patterns used by Chrome and Firefox.

shawn · June 3, 2022

Hi, @Ambimind!

Bug report: redirection-delay works perfectly for my needs, but I've noticed an issue with caching related to this feature.

When I'm working on a new app and I am not yet aware of the need to use the redirection-delay feature, then the content is cached by Kuppet without it and adding the value does not force Kuppet to re-cache the value until the timeout expires or Kuppet is forcefully closed. I've been forcefully closing it, but sometimes I forget and freak out since the page isn't working. Grrr.

I suggest adding a hash of the post-data collection to the key for the cache. This should allow it to break the cache when this data is changed without manually killing Kuppet.

Thanks again! I've been using Kuppet for a ton of new profiles, mostly at Logitech, HP, Netgear, and so on. Loving the improved versatility.

shawn · June 3, 2022

Created an app to monitor this page for updates.

It requires MEGA-Get from MEGA-CMD.

<?xml version='1.0' encoding='utf-8'?>
<Jobs>
  <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="2ca7df65-be33-47ec-860d-71425204d9f4">
    <Category>Tools</Category>
    <WebsiteUrl />
    <UserAgent />
    <UserNotes />
    <LastFileSize>0</LastFileSize>
    <IgnoreFileInformation>false</IgnoreFileInformation>
    <DownloadBeta>Default</DownloadBeta>
    <DownloadDate xsi:nil="true" />
    <CheckForUpdatesOnly>false</CheckForUpdatesOnly>
    <VariableChangeIndicator>version</VariableChangeIndicator>
    <HashVariable />
    <HashType>None</HashType>
    <CanBeShared>true</CanBeShared>
    <ShareApplication>false</ShareApplication>
    <ExclusiveDownload>false</ExclusiveDownload>
    <HttpReferer />
    <SetupInstructions />
    <Variables>
      <item>
        <key>
          <string>schangelog</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</TextualContent>
            <Name>schangelog</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>swebsite</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</TextualContent>
            <Name>swebsite</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>version</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>RegularExpression</VariableType>
            <Regex>.+&gt;(\d+\s\S+\s\d+)\s*Update&lt;</Regex>
            <Url>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</Url>
            <TextualContent />
            <Name>version</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sdownload</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>RegularExpression</VariableType>
            <Regex>href="([^'"]+)"[^&gt;]*&gt;Download link</Regex>
            <Url>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</Url>
            <Name>sdownload</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sdownloadfake</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://www.iana.org/_img/bookmark_icon.ico</TextualContent>
            <Name>sdownloadfake</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sdownloadpath</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>{startuppath:directory}</TextualContent>
            <Name>sdownloadpath</Name>
          </UrlVariable>
        </value>
      </item>
    </Variables>
    <ExecuteCommand />
    <ExecutePreCommand>CALL "%localappdata%\MEGAcmd\mega-get.bat" "{sdownload}" "{sdownloadpath}"
EXIT 3</ExecutePreCommand>
    <ExecuteCommandType>Batch</ExecuteCommandType>
    <ExecutePreCommandType>Batch</ExecutePreCommandType>
    <SourceType>FixedUrl</SourceType>
    <PreviousLocation />
    <DeletePreviousFile>true</DeletePreviousFile>
    <Enabled>true</Enabled>
    <FileHippoId />
    <FixedDownloadUrl>{sdownloadfake}</FixedDownloadUrl>
    <Name>Kuppet</Name>
  </ApplicationJob>
</Jobs>

shawn · August 4, 2022

I updated the Chrome & Firefox post-update commands so they use the newer user-agent string formats.

CybTekSol · November 3, 2023

Greetings to ALL ! I just discovered this thread regarding Kuppet. Interesting... I am going to try to find the time to give it a whirl. I pray you are all doing well ! I MISS working with you guys!

mcdy · February 27, 2025

Hey,

I have a page won't load in Ketarin, so I found this.
But it looks like I missed something, because it still won't work.

I set this:

Name: run_kuppet
Value: if(!(PS kuppet -ea 0)) {START -WindowsStyle hidden BIN\kuppet

Name: >
Value: {run_kuppet:ps} http://localhost:8080/

And I create a folder BIN in my Ketarin folder.
In this I drop the kuppet files.

Hope someone could help.
Thanks!

shawn · March 2, 2025

You're almost there. There should be no space in the value for ">". Then the final step is to use the new ">" variable in the source URL of a your parsed URL as so, replacing example with the website you're parsing from:

{>}https://example.com/path/

If you need additional help please post the URL.

Hulk91 · April 24, 2025

Hello,
I have some problem to downlaod with Kuppet.
Thats the settings i have done :

In Ketarin
And for the application :
but i have this result :

Perhaps someone could help me,

thanks a lot

shawn · April 24, 2025

Use Kuppet to determine versions, change log or download URL. Use Ketarin to actually download the file normally.

Remove "{>}" from the Download URL.

Hulk91 · April 25, 2025

Thanks Shawn,
I remove the "{>}" from download URL but i have this error :

image.png.3d4664f93581e31e5dd08186d35b083d.png

How can I avoid that error?
Perhaps is there a link to kuppet documentation somewhere with examples?

Thanks for your help,

shawn · April 26, 2025

The Kuppet documentation is this thread. The issue you're having isn't a Kuppet issue but a Ketarin issue. The Ketarin docs are mostly here.

Most likely the issue you're having is that you're using either the default or an invalid user-agent. Some sites just block older user-agents or user-agents that they don't want to trust. I change my default UA every year or so (File, Settings, Connection, Default user agent) and change the specific UA for certain apps when I design them if the new "default" isn't functional. It's currently:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36

I recommend switching it to something from the last few months. I've attached my own AnyDesk profile here so you can see how I parse for it.

<?xml version='1.0' encoding='utf-8'?>
<Jobs>
  <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="fd393d9a-4a15-4523-a565-1b33665b236c">
    <Category>Internet</Category>
    <WebsiteUrl>https://anydesk.com/en/</WebsiteUrl>
    <UserAgent />
    <UserNotes />
    <LastFileSize>7682880</LastFileSize>
    <LastFileDate>2025-04-11T10:25:03.8451727</LastFileDate>
    <IgnoreFileInformation>false</IgnoreFileInformation>
    <DownloadBeta>Default</DownloadBeta>
    <DownloadDate xsi:nil="true" />
    <CheckForUpdatesOnly>false</CheckForUpdatesOnly>
    <VariableChangeIndicator>version</VariableChangeIndicator>
    <HashVariable />
    <HashType>None</HashType>
    <CanBeShared>true</CanBeShared>
    <ShareApplication>false</ShareApplication>
    <ExclusiveDownload>false</ExclusiveDownload>
    <HttpReferer />
    <SetupInstructions />
    <Variables>
      <item>
        <key>
          <string>schangelog</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://anydesk.com/en/changelog/windows</TextualContent>
            <Name>schangelog</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>swebsite</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://anydesk.com/en/downloads</TextualContent>
            <Name>swebsite</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>snotes</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent />
            <Name>snotes</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>spc</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>anydesk</TextualContent>
            <Name>spc</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>splatform</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>Windows</TextualContent>
            <Name>splatform</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sbits</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent />
            <Name>sbits</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>svendor</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>AnyDesk Software GmbH</TextualContent>
            <Name>svendor</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>version</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>RegularExpression</VariableType>
            <Regex> -\s([\d\.]+)\s\(Windows</Regex>
            <Url>https://download.anydesk.com/changelog.txt</Url>
            <TextualContent />
            <Name>version</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sdownload</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://download.anydesk.com/AnyDesk.exe</TextualContent>
            <Name>sdownload</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>LastUpdate</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>20250411T1025</TextualContent>
            <Name>LastUpdate</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>LastVersion</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>9.5.1</TextualContent>
            <Name>LastVersion</Name>
          </UrlVariable>
        </value>
      </item>
    </Variables>
    <ExecuteCommand />
    <ExecutePreCommand />
    <ExecuteCommandType>Batch</ExecuteCommandType>
    <ExecutePreCommandType>Batch</ExecutePreCommandType>
    <SourceType>FixedUrl</SourceType>
    <PreviousLocation>K:\_Ketarin\Ketarin\..\Internet\AnyDesk-9.5.1.exe</PreviousLocation>
    <DeletePreviousFile>true</DeletePreviousFile>
    <Enabled>true</Enabled>
    <FileHippoId />
    <LastUpdated>2025-04-11T10:25:03.8451727</LastUpdated>
    <TargetPath>..\{category}\{appname:regexreplace:([\s\t\r\n\-\&amp;]+):_}-{version}.{url:ext}</TargetPath>
    <FixedDownloadUrl>{sdownload}</FixedDownloadUrl>
    <Name>AnyDesk</Name>
  </ApplicationJob>
</Jobs>

Hulk91 · April 26, 2025

Thanks shawn,
I'm looking that monday et try to make it working.
Thank you very much for your help.
Have a nice weekend,

Sorry for my english but i'm french

el_confidente · July 1, 2025

I'm getting this error:

"The contents of the URL can not be loaded: Unable to connect to the remove server"

with this URL:

{>}https://www.qbittorrent.org/download.php

Until recently this URL worked fine without the need for Kuppet, but they appear to be using Cloudflare now.

Any idea what the problem is?

shawn · July 1, 2025

Sadly, @el_confidente, this is the problem with many sites hosted on CloudFlare.

You might have luck by implementing the redirection-delay post-variable detailed at that link, with a delay of at least 45 seconds. This works most of the time for me.

el_confidente · July 1, 2025

The redirection-delay variable didn't have any effect when I tried it, as the error message always popped up within 4-5 seconds.

I finally figured out that all I had to do was delete the ".php" off the end of the URL and it worked as normal. Thanks for your help, though!

shawn · July 2, 2025

When you set it did you set it as a separate standard variable or as a post data variable?

image.png.f9a05fcc8e5ecb366f3378bf219f3bc0.png

el_confidente · July 2, 2025

Post data variable. I think the problem is that they changed the URL and the old one re-directed to the new one through Cloudflare.

shawn · July 2, 2025

@el_confidente, good eyes. Glad you got it sorted.

Sign In

Kuppet : Javascript rendered pages for Ketarin

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Create an account or sign in to comment

Create an account

Sign in

Important Information