Jump to content
Ketarin forum

Kuppet : Javascript rendered pages for Ketarin


Ambimind
 Share

Recommended Posts

@shawn 
For now I've implemented the following solution:
Using post-variables(see image below), each Ketarin-job-variable specifies a "redirection-delay", measured in seconds.
Kuppet will wait for this duration after the initial page load,
during which it is supposed that any landing-page-timers have completed and the actual page has begun to load,
after which kuppet will wait till the page has finished loading(absence of network requests for 500ms).

TWdBxHo.png

Link to comment
Share on other sites

@Ambimindthat seems like a perfect solution. I can roll it out on only those sites where it's necessary without imposing unnecessary delays on other sites. It works perfectly. This is the first time in probably a year that I've been able to check for updates for both of those apps in the same run. Yay!

Seeing the power of using the POST data feature to tune Kuppet for individual sites. Can the other options be assigned there, too? It would be nice to specify a given UA for a site that doesn't like other user-agents, for example. If you plan to implement this as a feature there's no hurry from me. I can see myself using it someday, but currently don't have a need. 

When I have some free time I'll be writing a script to update the kuppet.config file with the current Chrome and Firefox versions in a post-update script. I'll share that here.

Link to comment
Share on other sites

Alrighty... Here's a post-download command you can use to keep the version information up to date in the kuppet.config file. Change the $kuppetCfg path, and make sure you use a unique $browserIndex for each different browser you want to include. Also, make sure the post-download command is set to use PowerShell.

This one for Firefox:

# For each browser you want to include:
# Use a different Index and different UA pattern
# 0 = Firefox
$browserIndex = 0;
$browserVersion = $app.variables.version.CachedContent.Split(".")[0];
$browserUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:$browserVersion.0) Gecko/20100101 Firefox/$browserVersion.0";
$kuppetCfg = "K:\Ketarin\kuppet\kuppet.config";

# do the work
$kuppetJson = Get-Content $kuppetCfg -Raw | ConvertFrom-Json;
$kuppetJson.userAgentArray[$browserIndex] = $browserUA;
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($kuppetCfg, ($kuppetJson | ConvertTo-Json), $Utf8NoBomEncoding)

This is for Chrome:

# For each browser you want to include:
# Use a different Index and different UA pattern
# 1 = Chrome
$browserIndex = 1;
$browserVersion = $app.variables.version.CachedContent.Split(".")[0];
$browserUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/$browserVersion.0.0.0 Safari/537.36";
$kuppetCfg = "K:\Ketarin\kuppet\kuppet.config";

# do the work
$kuppetJson = Get-Content $kuppetCfg -Raw | ConvertFrom-Json;
$kuppetJson.userAgentArray[$browserIndex] = $browserUA;
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding $False
[System.IO.File]::WriteAllLines($kuppetCfg, ($kuppetJson | ConvertTo-Json), $Utf8NoBomEncoding)

If you're going to include additional user-agents just make sure to use a new $browserIndex value for each one, otherwise you'll overwrite the wrong one. 

2022-05-02: fixed to strip the BOM (byte-order marker) from the file when it's written

2022-08-03: updates user-agent strings to match current patterns used by Chrome and Firefox.

Link to comment
Share on other sites

  • 1 month later...

Hi, @Ambimind!

Bug report: redirection-delay works perfectly for my needs, but I've noticed an issue with caching related to this feature.

When I'm working on a new app and I am not yet aware of the need to use the redirection-delay feature, then the content is cached by Kuppet without it and adding the value does not force Kuppet to re-cache the value until the timeout expires or Kuppet is forcefully closed. I've been forcefully closing it, but sometimes I forget and freak out since the page isn't working. Grrr.

I suggest adding a hash of the post-data collection to the key for the cache. This should allow it to break the cache when this data is changed without manually killing Kuppet.

Thanks again! I've been using Kuppet for a ton of new profiles, mostly at Logitech, HP, Netgear, and so on. Loving the improved versatility.

Link to comment
Share on other sites

Created an app to monitor this page for updates. :)

It requires MEGA-Get from MEGA-CMD.

<?xml version='1.0' encoding='utf-8'?>
<Jobs>
  <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="2ca7df65-be33-47ec-860d-71425204d9f4">
    <Category>Tools</Category>
    <WebsiteUrl />
    <UserAgent />
    <UserNotes />
    <LastFileSize>0</LastFileSize>
    <IgnoreFileInformation>false</IgnoreFileInformation>
    <DownloadBeta>Default</DownloadBeta>
    <DownloadDate xsi:nil="true" />
    <CheckForUpdatesOnly>false</CheckForUpdatesOnly>
    <VariableChangeIndicator>version</VariableChangeIndicator>
    <HashVariable />
    <HashType>None</HashType>
    <CanBeShared>true</CanBeShared>
    <ShareApplication>false</ShareApplication>
    <ExclusiveDownload>false</ExclusiveDownload>
    <HttpReferer />
    <SetupInstructions />
    <Variables>
      <item>
        <key>
          <string>schangelog</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</TextualContent>
            <Name>schangelog</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>swebsite</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</TextualContent>
            <Name>swebsite</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>version</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>RegularExpression</VariableType>
            <Regex>.+&gt;(\d+\s\S+\s\d+)\s*Update&lt;</Regex>
            <Url>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</Url>
            <TextualContent />
            <Name>version</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sdownload</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>RegularExpression</VariableType>
            <Regex>href="([^'"]+)"[^&gt;]*&gt;Download link</Regex>
            <Url>https://ketarin.org/forum/topic/4381-kuppet-javascript-rendered-pages-for-ketarin/</Url>
            <Name>sdownload</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sdownloadfake</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>https://www.iana.org/_img/bookmark_icon.ico</TextualContent>
            <Name>sdownloadfake</Name>
          </UrlVariable>
        </value>
      </item>
      <item>
        <key>
          <string>sdownloadpath</string>
        </key>
        <value>
          <UrlVariable>
            <RegexRightToLeft>false</RegexRightToLeft>
            <VariableType>Textual</VariableType>
            <Regex />
            <TextualContent>{startuppath:directory}</TextualContent>
            <Name>sdownloadpath</Name>
          </UrlVariable>
        </value>
      </item>
    </Variables>
    <ExecuteCommand />
    <ExecutePreCommand>CALL "%localappdata%\MEGAcmd\mega-get.bat" "{sdownload}" "{sdownloadpath}"
EXIT 3</ExecutePreCommand>
    <ExecuteCommandType>Batch</ExecuteCommandType>
    <ExecutePreCommandType>Batch</ExecutePreCommandType>
    <SourceType>FixedUrl</SourceType>
    <PreviousLocation />
    <DeletePreviousFile>true</DeletePreviousFile>
    <Enabled>true</Enabled>
    <FileHippoId />
    <FixedDownloadUrl>{sdownloadfake}</FixedDownloadUrl>
    <Name>Kuppet</Name>
  </ApplicationJob>
</Jobs>

 

Link to comment
Share on other sites

  • 2 months later...
  • 1 year later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.