Search the Community

@Ambimind - this is fantastic. Thank you! Can I request a feature? It looks like (according to the log) that it is re-requesting the same URLs repeatedly with each instance of a variable that makes a request. Is it possible to make Kuppet cache the results temporarily until it has moved on to the next Application? Since I gather a lot of content from some pages this could save me a LOT of repeat requests. Are there other switches than the port that it runs on?

hey dunry and Nooban, You can try this: ..\{category}\{appname:regexreplace: (Portable)| (x86)| (x64)| \(.*\):}\{appname} V{version}.{url:ext} it will even if you move the ketarin and downloads to another path in the future (ex: E:\new2019\Soft\Ketarin) Explanation: ".." - will change the current path to one upper in the directory hirarchy tree "{category}" - will change the path to the one sotred in the Category variable directory and create one if not existing "{appname:regexreplace:\sPortable|\sx86|\sx64|\s\(.*\):}" - will change the path to the Application Name variable, while cutting the following: " Portable" or " portable" or x64 or X64 or x86 or X86 and everything in parenthesis including the parenthesis with space before " (example)" "{appname}" - first part of filename taken from the Application Name variable. "{version}" - second part of filename taken from the version variable "{url.ext}" - the extension of the filename, taken from the extension of the url to download from. Example: Application name: LibreOffice (Stable Version) LibreOffice_5.4.5_Win_x64.msi will be saved as: USBStick:\Soft\OfficeSoftware\LibreOffice\LibreOffice (Stable Version) V5.4.5.msi this wiki link may also be useful for you: https://wiki.ketarin.org/index.php/Portability hope you'll have good use in those examples:)

Follow up to this post. Narrowed it down to the use of "Use the following variable as indicator for changes". Fails when using it, succeeds without it. Reproduction: Use a fresh install of Ketarin, keep default settings. Add 2 applications: app name1 = 7-Zip, variables1 = var, contents from url1 = http://7-zip.org, use the following variable as indicator for changes1 = var; app name2 = CCleaner, variables2 = var, contents from url2 = https://piriform.com, use the following variable as indicator for changes2 = var Press "Update all". Result on first try is CCleaner fails but succeeds for every other try. Unless you restart ketarin, then it fails again on 1st attempt. Changing https://piriform.com to https://ccleaner.com makes CCleaner work however then another app fails which worked before. Not using "Use the following variable as indicator for changes" will make all connections work fine. However obviously I need to use it in the first place. What do you mean or need, exactly? Fixing one app makes another completely unrelated one fail which worked before. How should I possibly know what you would need to do to fix that?

Well that was easy. Thanks! Now I just need to figure out how to exclude the root folder that has the version name, or rename the folder once it's extracted.

Sorry for bothering you again, but I have three questions. The first is; how to extract the files then I want to move the files to a specific location then I want to delete the folder that the zip/7z created. Step 1: 7z x -y -o"C:\Program Files\Launchbox\Emulators\Dolphin\Updates" "{file}" Step 2: (Move files) or (Rename folder that is extracted and overwrite files in location)? Step 3: rd /s /q "C:\Program Files\Launchbox\Emulators\Dolphin\Updates\Dolphin-x64" "{file}" My next question is; I assume when you extract the files to a location with the existing files it will automatically overwrite those files, right? My last question is; do you only need Uniextract if 7-Zip can’t extract a certain file extension? It seems to work fine for 7z/zip.

Thank you, @floele! You inspired me to dig through the app again and I discovered there was an unused variable that was opening a defective URL and *that* is what was triggering the error. Once I removed that variable it's working fine. Sigh.

What have you tried? How are you creating your random variable now? Does your random variable need to be unique a la GUID, or is something like the current timestamp {time} sufficient, or does it need to meet another specific format?

Beta 3 : Impressions after a week of use : The new features are extremely useful, but they should be used sparingly. Specifically the use of url-variables-as-scripts, in custom columns : This means each job has a variable which holds a script; this variable is referenced and run to produce the value of a custom column. The script being run, in my case, compares the creation date of each file and calculates the days since update : (New-TimeSpan (GI $app.CurrentLocation).LastWriteTime (Date)).Days Issues were myriad and some intermittent : 5-10s startup times(~100 jobs) crash on start crash during update cycle unresponsive/flickering interface during update cycle app update checks timing out("connection timeout": 30) much slower over-all update cycle("parallel downloads":6) I also had a global (pre-download) script running, which added this 'script-variable' to jobs automatically(another new feature): #### ADD DATE VARIABLE IF NOT EXISTING # Parameters $varnm = "SinceUpdate" $vartp = "Textual" #StartEnd #Textual #RegularExpression # # Derived access property $varcnt = if($vartp -eq "Textual"){"TextualContent"}Else{"CachedContent"} # If it doesn't exist, then create a new one if(-Not $app.variables.$varnm){ ECHO "========== $varnm NOT FOUND ==========" $newvar = New-Object -TypeName Ketarin.UrlVariable -ArgumentList $varnm,$app.variables $newvar.VariableType = $vartp $app.variables.Add($varnm, $newvar) $app.Save() } #Set newly created variable $app.variables.$varnm.$varcnt = '(New-TimeSpan (GI $app.CurrentLocation).LastWriteTime (Date)).Days' #### I've had no issues with multi-segment downloads(using 4 segments). I'm not classing these behaviors as 'bugs', just something to keep in mind when using these features.

I've now tried using this and it does seem to almost always pull the data from the cache instead of re-requesting URLs. I had some problems with the date and filename information so ended up prepopulating those. Here's what I'm currently using: $sfilename = $($App.CurrentLocation); $sfiletime = [datetime]$($App.DownloadDate); $sversion = $app.variables.ReplaceAllInString("{version}", $sfiletime, $sfilename, $true); I would assume that $App.DownloadDate would have the file timestamp from the current download (ideally the date the file was originally published, but the current timestamp if that's not an option). If that's not the case, then is there another variable to pull this information directly? I switched it out with this and it seems to be working ok, any reason I should use something else? $sfilename = $($App.CurrentLocation); $sfiletime = [datetime]$( (Get-Item $sfilename).LastWriteTime ); Now... It looks like it might work, but I'm still having one problem. One of my constructed variables re-requests the URL every time regardless. Is there a reason for it? The variable is schangelog and is defined as https://www.drupal.org{schangelogstub}. schangelogstub is another variable that uses a regex capture from the site to get the actual current version number to identify the specific URL for the changelog. The "new" ReplaceAllInString is still re-requesting the URL to build schangelogstub, and returning: Drupal The remote server returned an error: (403) Forbidden. (https://www.drupal.org/8/download) The file downloads. The rest of the variables are populated. but it is still reported as an error by Ketarin since the PS chokes on re-requesting this content. Darn Drupal.

@jokerfool Check the log window for details! Renaming is not possible if the file already exists. You need to use some delete+rename or copy (with overwrite)+delete.

Some background info: Two commonly used methods for request/response communication between clients and servers are GET and POST. GET requests data and POST submits data. (For a more detailed explanation: W3Schools - HTTP Methods) Regarding Onetastic: In this case, you can reach your goal in two different ways. Method 1: Add &agree=1 to your download link and that's it! For the 32-bit version: https://getonetastic.com/downloadFile&fv=0&file=OnetasticInstaller.x86.zip&agree=1 For the 64-bit version: https://getonetastic.com/downloadFile&fv=0&file=OnetasticInstaller.x64.zip&agree=1 Method 2: Currently, downloading via ketarin only works via method 1, because there are probably also on this page problems with the TLS 1.2 protocol. I hope, floele can fix this issue. But it would work like this: First of all, we need to check which POST variables must be passed in order for the download to start. There are two ways to this. You can take a look at the source code of the web site or you can use web developer tools (like the Firefox Developer Tools, as described here). The first way: We need the web page to which the variable is passed and, of course, the variables. For this we look in the source code for the form with the variable to be passed. In this case, our required data is in line 37 and 38. In line 37 we have the page to which the variables are sent and in line 38 the variable itself (in this case by using checkboxes, which means checked=1 unchecked=0). They called the variable agree. So the value for the checked checkbox called 'agree' will be 1. In the next step we use this data in Ketarin. You can also use the main downloadpage https://getonetastic.com/downloadFile as download url to pass all the other variables instead. Which looks like this: But in this case there is no need to do this. This is just another variation for this website. The second way: The source code (and much more) can also be analyzed with the Firefox developer tools. We need the referer, and all the POST variables submitted. Here is a short video of how to do this for Onetastic.

Can we just allow admins to rename apps after upload? Likewise the system should be set not to rename them on the client during an application profile update if the only changes the user has made are the name and destination information (and of course, leave those alone). On my end, many of my apps are named specifically based on their purpose, not necessarily the application name. For example, the only good reason to download the RK Tools for XP today is for robocopy, thus "XP RK Tools (robocopy)". This wouldn't behave well for me if I had to use actual product names in a normalized convention. For 32 & 64 bit releases, I usually append either "(x86)" or "(x64)" to the end of the app name, such as "SmartSniff (x64)", and only in those cases where bit-specific releases are available. If no bit-specific versions are available, it doesn't make sense to tag the name.

Sure! Define a variable (I scrape from webpage, but I believe almost any variable will work.) You put the variable on the 'advanced' tab in the setup for app. Here's how it works: This variable alone controls Ketarin's downloading. If the contents of this variable change, you want Ketarin to download the file. If the contents have not changed, Ketarin does not download the file, ignoring any other indications that it should download the file. This feature was originally intended to stop the periodic repeat downloading of the same exact file, because some other indicator such as the file dates on the server, had been changed. I do have a couple of apps like that. For whatever reason the file dates on the server change every few days, but the app has not been updated. Every time the file dates changed, Ketarin downloaded the file again, until I put the special variable in. I use version scraped from a page as my indicator -- if the scraped version number changes, Ketarin should get the file. Otherwise, no. (The only reason I didn't like the re-downloading is because it tricked me into thinking there was a new version available, but there wasn't.) So. We can use this special variable to protect against the accidental overwriting of a previous version too, when that variable is part of the disk file name like mine was. My version scrape was still 2.8.3 in this case. Had I been using this for the special variable, Ketarin would not have downloaded the new file (which is really 2.8.4). But I was not using the special variable, so Ketarin rightly detected that a new file was available, and faithfully wrote it to the disk name I specified, which still had 2.8.3 in the name. By not using the special variable to stop Ketarin from downloading, I lost my 2.8.3 version file. Ketarin overwrote it with the newer 2.8.4 version of the file. Then, the second part of this is what I started this thread for. Once I realized my version variable was not correct, I did change the scrape so my version variable was now coming in as 2.8.4. Ketarin had already downloaded that file, so there was nothing new to download, and it did not check further to see if the filename existed (it did not). Hope that helps. Let me know? --appyface

You can use "vacuum" on the jobs.db to shrink it. This will purge unneeded archival data that has been marked for deletion but not actually removed. There are other suggestions at the link, too. However, the safest way of purging your database if you're concerned about privacy matters, is to create a new one. Vacuum only purges data it knows to delete - there's always a chance it won't know, so recreating the jobs.db ensures that it only has access to the new information. From the main Ketarin window: Select all, CTRL+C, open notepad, paste. Also go to the global settings (custom columns, global variables, global actions) and copy those out and label them so you know where they go when you create a new one and remove the ones you don't want/need anymore. Rename the jobs.db to something like jobs-20171213.db, then reopen Ketarin. You'll have a blank new jobs.db, so go back and recreate the global settings first, then copy the original "select all" backup from notepad and paste it right back into Ketarin. This will put all your application profiles back.

This is a referer/redirection problem. You need to set a referer for this download, which is the download link itself. variable: download Content from url: http://www.freefilesync.org/download.php Use regular expression: ([^"]*Setup.exe) Set {referer} as referer for your download: variable: referer Textual content: http://www.freefilesync.org/download.php In case you need the version number: variable: version Content from url: http://www.freefilesync.org/download.php Use regular expression: (?<=Changelog.*?FreeFileSync )[0-9.]{2,} And the final download link: variable: url Textual content: http://www.freefilesync.org/{download}

Hi Flo, Since I'm (still) not a Firefox user (sorry, CybTekSol!) I'd like to dredge this wish back to the top, for reconsideration? Now that Ketarin's logging is so much more comprehensive than it used to be, and we have such a great variable facility now, I think this could be done easily by using variables and a bit of logging...? Here's what I have in mind: 1. I define one or more variables (same way as for any variable) in an app. These might contain entire web page or a section of a webpage, or textual content... whatever content I'm interested in. 2. I have in the app definition a way to list or otherwise designate these variables as my special 'independent watch' variables. A practical limit (maybe a max of 3 such variables?) per app would be fine 3. For each 'independent watch' variable, during update run Ketarin compares the new scraped value with previous stored value (similar to Change Indicator Variable) 4. If the value of any 'watch' variable has changed, make a log entry. Something like this: 04/07/2009 11:42:44: Update started with 1 application(s) 04/07/2009 11:42:45: Download Test: Checking if update is required... 04/07/2009 11:42:45: Download Test: Update not required 04/07/2009 11:42:45: Download Test: Watch variable {changelog} updated 04/07/2009 11:42:45: Download Test: Watch variable {versnews} updated 04/07/2009 11:42:45: Update finished Example: Say I have three 'independent watch' variables defined for this app, the above two are the ones that changed value I would go a step further and ask if the app was NOT downloaded (for any reason, including CIV), but at least one of the watch variables was updated, that an additional entry be made in the log, perhaps something like this. I'd like a different icon on the Ketarin GUI if it's open, perhaps a red exclamation point? 04/07/2009 11:42:45: Download Test: Checking if update is required... 04/07/2009 11:42:45: Download Test: Update not required 04/07/2009 11:42:45: Download Test: Watch variable {changelog} updated 04/07/2009 11:42:45: Download Test: Watch variable {versnews} updated 04/07/2009 11:42:45: Download Test: Alert: No download but watch variable(s) updated Thoughts on this idea? TIA as always, --appyface Edit: Actually, just thinking out loud here -- if a "watch" variable can be a concatenation of other variables, then just one special "watch" variable will do nicely...

I'm getting pretty much from every application the changelog with a variable. Also I run a C# code after updating an application: string downloadFolder = app.Variables.ReplaceAllInString("{download_folder}"); string appName = app.Variables.ReplaceAllInString("{appname}"); string version = app.Variables.ReplaceAllInString("{version}"); string changelog = app.Variables.ReplaceAllInString("{changelog}"); if (changelog == "{changelog}") { changelog = ""; } using (System.IO.StreamWriter w = System.IO.File.AppendText(downloadFolder + @"\changelogs.html")) { w.WriteLine("<hr size=10 noshade>"); w.WriteLine("<h1>{0} - {1}</h1>", appName, version); w.WriteLine(changelog); w.WriteLine("<br><br><br>"); } return; After updating my applications I'm getting a changelogs.html that look like: There are now two ways to continue with this file: 1. Rename the file with the actual date and time as a command "after updating all applications"; move "{download_folder}\changelogs.html" "{download_folder}\changelog-{yyyy}-{MM}-{dd}T{HH}-{mm}.html" 2. sending the content from changelogs.html via e-mail to me therefore I made a powershell script: $smtpServer = "email.server.com" $file = "C:\YOUR\DOWNLOAD\FOLDER\changelogs.html" $body = "$(cat $file)" $from ="Ketarin <ketrain@server.com>" $to = "me@server.com" $subject = "Ketarin Updates" send-MailMessage -SmtpServer $smtpServer -To $to -From $from -Subject $subject -Body $body -BodyAsHtml -Encoding ([System.Text.Encoding]::UTF8) Forgot to mention, that mean you can run Ketarin as a task and you getting after an application update an email.

I know this is years late, but I was looking for something else and saw that one of the issues in this wasn't addressed. The problem with your troubleshooting line is most likely that you didn't quote the URL. This means that your script will crash before it actually echoes anything to the file if there's an ampersand (&) or certain other characters in the {preupdate-url} variable. Try this instead: echo TEST "{preupdate-url}" >> T:\xx.txt

What I did was follow the link on the originating site to download the exe version, then CTRL+J to open the downloads pane, then copied the download URL. It's clear from that URL that the file path is going to be consistent between builds. I pasted that URL into the {url} variable then on the "Advanced settings" tab of the app profile I set the variable change indicator/"use the following variable as indicator for changes" to the {version} variable. It will now properly update itself by running Ketarin updates. Unless you need it exploring the download folder, you could remove that post-update command, too.

Yes. Anything you want to execute can be performed in the pre-update command. You can use this to do something like email yourself, or pipe to a text file, or with the /log parameter. If done correctly, you can use just about any information you like (anything you've stored in a variable) in order to send it to yourself. There is a great example of creating your own changelog here using the "after updating an application" global command. I leave Ketarin open 24/7 and pipe the results of every download to a log file in a global after-update command, and within the pre-update command for a few that use pre-update to perform the downloads or are only used to watch sites. I then use Notepad++ with the logfile open 24/7 so as updates are performed I can review the log and perform any actions based on what's new, such as reading changelogs for web packages like Joomla and installing any that are security updates, or triggering Chrome updates on all client devices and so on.

It seems creating a variable using https://docs.tenable.com/Nessus.htm with regex <?Latest Release Notes.*?blank\">(.*?)</a><! would give you the latest version, which would allow you to do this.

If you're just looking for an Nvidia app profile, here's mine. Be aware that they no longer provide 32-bit drivers, so this is for the latest US-English 64-bit WHQL GeForce driver. It would only take a couple variable changes to modify the language/location, and other details. They're all stuffed into URL parameters so it's pretty easy to fix. <?xml version='1.0' encoding='utf-8'?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="db7445b0-3205-40e7-9c43-3502db1382f1"> <Category>Drivers</Category> <WebsiteUrl>https://www.nvidia.com/Download/index.aspx?lang=en-us</WebsiteUrl> <UserAgent /> <UserNotes>The Nvidia driver package is designed for Nvidia video cards.</UserNotes> <LastFileSize>600971672</LastFileSize> <LastFileDate>2021-04-14T21:38:24.0492945</LastFileDate> <IgnoreFileInformation>false</IgnoreFileInformation> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>version</VariableChangeIndicator> <HashVariable /> <HashType>None</HashType> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer /> <SetupInstructions /> <Variables> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>gridItem"&gt;([\d\.]+)&lt;/td&gt;</Regex> <Url>https://www.nvidia.com/Download/processFind.aspx?osid=19&amp;whql=1&amp;lang=en-us&amp;ptid=1&amp;psid=101</Url> <Name>version</Name> </UrlVariable> </value> </item> <item> <key> <string>dl1</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>['"]([^'']*//[^'']*/en-us)['"]&gt;.*?[^&gt;]*&gt;{version}</Regex> <Url>https://www.nvidia.com/Download/processFind.aspx?osid=19&amp;whql=1&amp;lang=en-us&amp;ptid=1&amp;psid=101</Url> <Name>dl1</Name> </UrlVariable> </value> </item> <item> <key> <string>dl2</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>url=([^&amp;"']+)</Regex> <Url>{dl1}</Url> <Name>dl2</Name> </UrlVariable> </value> </item> <item> <key> <string>snotes</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent /> <Name>snotes</Name> </UrlVariable> </value> </item> <item> <key> <string>swebsite</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>https://www.nvidia.com/Download/index.aspx?lang=en-us</TextualContent> <Name>swebsite</Name> </UrlVariable> </value> </item> <item> <key> <string>schangelog</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex>"([^"]+\.pdf)"&gt;Release Notes</Regex> <Url>{dl1}</Url> <TextualContent>{dl1}</TextualContent> <Name>schangelog</Name> </UrlVariable> </value> </item> <item> <key> <string>spc</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>nvidia</TextualContent> <Name>spc</Name> </UrlVariable> </value> </item> <item> <key> <string>sdownload</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>https://us.download.nvidia.com{dl2}</TextualContent> <Name>sdownload</Name> </UrlVariable> </value> </item> <item> <key> <string>splatform</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>windows</TextualContent> <Name>splatform</Name> </UrlVariable> </value> </item> <item> <key> <string>sbits</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>64</TextualContent> <Name>sbits</Name> </UrlVariable> </value> </item> <item> <key> <string>LastUpdate</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>20210414T2138</TextualContent> <Name>LastUpdate</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand /> <ExecuteCommandType>Batch</ExecuteCommandType> <ExecutePreCommandType>Batch</ExecutePreCommandType> <SourceType>FixedUrl</SourceType> <PreviousLocation>K:\_Ketarin\Ketarin\..\Drivers\nVidia_(x64)-466.11.exe</PreviousLocation> <DeletePreviousFile>true</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated>2021-04-14T21:38:24.0492945</LastUpdated> <TargetPath>..\{category}\{appname:regexreplace:([\s\t\r\n\-\&amp;]+):_}-{version}.{url:ext}</TargetPath> <FixedDownloadUrl>{sdownload}</FixedDownloadUrl> <Name>Nvidia</Name> </ApplicationJob> </Jobs>

You're confusing two languages: batch and Ketarin. Ketarin variables are the native variables as well as those you create yourself on the variables page. The Ketarin variables are, of course, processed before batch variables since the batch script is generated *from* Ketarin. This means that you will never use a "{%whatever%}" syntax, but might very well use a "%{whatever}%" syntax, depending on what you're putting in your variables. You could, for example, use a global Ketarin variable to specify what path you would like to extract to and use that in place of the {whatever} and have it processed by the batch script. "{file}" is the proper way to wrap the filename if it has spaces or other special characters (like ampersands). There are MANY EXAMPLES on this forum of using 7-zip to extract files to specific directories. As for the directory thing - you'll want to look at the functions page on the wiki for stuff like {file:directory}: 7z.exe e -o"{file:directory}" "{file}" *.exe

@Flo, I could definitely use the 'rename variable' feature now as one site I am using is slowly phasing in changes to their redirect method and forces me to use my {redir} variable instead of my {download_url} as the actual download link. To keep things consistent, I currently has to delete both variables and rework them from the clipboard. It would be much easier to delete my {download_url} variable and rename my {redir} variable to {download_url}. Just my take on it for your consideration.

Just wondering: Is there an easy way to rename categories, because it's really annoying having to go to each app and rename the category individually. An idea I was thinking about was to have "Rename category" or "Change category" in the right-click menu. I love this app! Thank you, Kris