Search the Community

How to create such thing or is it possible at all? Its pretty annoying, if I want to change file or path-format for my 130+ apps. It would be much easier to change it in one place and all apps would have that variable in their "save to file" subsection so all it changes for all applications. Help would be: much appreciated...

Keratin is a great piece of goodies. Here is my suggestion. Support array variable Some websites may have a page listing all versions, and I plan to download and keep the last three most updated version. Is there anyway to define an array variable to download first 3 matches? Match from the end of html files when creating variables Sometimes, the links in html are sorted from the old to latest, but regexp can match the first entry only. Can Keratin search in reverse direction? Define max concurrent download from a particular server

Just because of some recent web page changes I was curious if some sort of 'threshold' warning could be implemented. i.e. if the value the variable returns goes from v.123 to </b>This is the best way to eat pudding</b> It would then pop up a warning with the returned value going "some drastic changes may have been made, would you like to go to the variable pages to check?" if the change was 90% different or something along those lines. It could just be an extra tab like "global variables" where you put in the variables you want to keep an eye on and the desired threshold check. Could also make the value allowable to be empty so it returns a true empty for download links.

Again, not sure if it is a bug or not however seemed the most likely place. When pre-defining values with variables i.e. name it seems that Ketarin doesn't process variables with regex that contains < or > (That's < and > to in XML if anyone is curious) For example <title(.*?)/title> won't work, it will say "Cannot determine", however it will retrieve it fine if you check the variable in the settings. title(.*?)/title will work, but not retrieve what I actually want. I assume this is because it doesn't translate the < before trying to define the variable.

Hi, It's possible to add this feature? I explain me, for example, I have winrar and winrar beta, but it isn't beta version ketarin return an error: because doesn't search the beta version. It's possible to add an option to check if variable exist, and if doesn't exist there is no download and no error message. Thx you PS: I see a bug in latest beta version. When edit app, the category, isn't read, it's the first category to take

{url:basefile} tried adding an extra command: 7z x {file} -o"{root}Computer-Repair\{category}\{appname}\" -y ren {url:basefile} {appname} to rename the folder but the {url:basefile} variable isnt working the log only shows this after extracting the file: C:\>ren {url:basefile} MozBackup

Hey floele, and the next one ;-) In the new Beta you´ve implemented the Variable="VariableName". Really great! This is the way to define dropdown menus: <TextualContent><placeholder name="Release?" value="New" options="maverick|lucid|jaunty|hardy" /></TextualContent> Could you implement the variable method for Options too? TextualContent><placeholder name="Release?" value="New" options="{match1}|{match2}|{match3}|{match4}" /></TextualContent> I would grab a whole site, get the whole files on the site and let the user select the desired file per drop down menu: Like: AppName.Portable AppName.X64 Appame.gz AppName.win32 How many functions can I ask for, till you kick me? I have many more *just kidding ;-)

I did some XML cleaning during the 1.6 b3, which worked right up until I upgraded. With b4 the CDATA stuff re-appeared and caused me grief, so I exported all my apps, re-created the .db and cleaned up all my xml and imported that back in. Now the variable version check is stuck again If I re-export however there is no trace of CDATA issues (or at all) and EVERY app is the same except for the regex/url so i'm not sure what the bloody problem is this time. I'll try going back to b3 and see if that solves it and report back on that. <?xml version='1.0' encoding='utf-8'?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="e5f6959c-b5fa-4982-afb6-4957f00562a5"> <WebsiteUrl>http://xkcd.com/</WebsiteUrl> <UserAgent /> <UserNotes /> <LastFileSize>0</LastFileSize> <LastFileDate xsi:nil="true" /> <IgnoreFileInformation>true</IgnoreFileInformation> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>monitor</VariableChangeIndicator> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer /> <SetupInstructions /> <Variables> <item> <key> <string>monitor</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>([^"']+comics[^"']+(\....))</Regex> <Url>http://xkcd.com/</Url> <Name>monitor</Name> </UrlVariable> </value> </item> <item> <key> <string>name</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex><title>.*?(\S.*?)\s*</title></Regex> <Url>http://xkcd.com/</Url> <Name>name</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand>start "" "{property:WebsiteUrl}" & exit 2</ExecutePreCommand> <ExecuteCommandType>Batch</ExecuteCommandType> <ExecutePreCommandType>Batch</ExecutePreCommandType> <Category>Web Comics</Category> <SourceType>FixedUrl</SourceType> <PreviousLocation /> <DeletePreviousFile>true</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated xsi:nil="true" /> <TargetPath>Web Page Monitoring\{appname:regexreplace:([^a-zA-Z0-9\_]+):-}</TargetPath> <FixedDownloadUrl>https://www.google.com/images/logos/mail_logo.png</FixedDownloadUrl> <Name>xkcd</Name> </ApplicationJob> </Jobs> I've deleted/re-created the .db from scrap and re-imported several times.

Hi, Can we have a new global option that allow to set and replace the default "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1)" User Agent ? And maybe use "Ketarin/{ketarin-version}" as default User Agent (which works with the largely used SF.net and should with with most website) ? Regards, Alexandre Edit: and {ketarin-version} as new variable ?

About every other time I run 'Update all' in Ketarin (1.8.11) anymore it crashes after about a minute. It wasn't until recently that I realized that it appears to happen while processing the below app. The error message is attached below, too. Any help anyone can provide is welcome. The error message references powershell, and this may have started around the time I updated my post-script collection routine, which makes sense. This script publishes all the update information to my website so I can automate updates on client devices. Here's the new version of that: #### Support function PKV { Param( $svName ) #### identify variable type and collect stored content $svType = $app.variables.$svName.VariableType; if( $svType -eq "Textual" ){ #### stored text content $svContent = $app.variables.$svName.TextualContent; #### process all child variables if( $svContent -like '*{*' ){ $svContent2 = $app.variables.ReplaceAllInString("{$svName}", $sfiletime, $sfilename, $true); if ( $svContent2 -ne '' ){ $svContent = $svContent2; } } }else{ #### other types of variables must be parsed $svContent = $app.variables.ReplaceAllInString("{$svName}", $sfiletime, $sfilename, $true); } #### parse version for psversion if it exists if( $svName -eq "version" ){ if( $app.variables."psversion".TextualContent -ne "" ){ $svContent = $app.variables.ReplaceAllInString("{$svName}", $sfiletime, $sfilename, $true); } } if( $svContent -eq "{$svName}" ){ $svContent = ''; } Write-Information $svName':'$svContent; $svContent } #### #### Parse cached variable values $sname = $($App.Name); $sfilename = $($App.CurrentLocation); write-host 'Filename:'$sfilename; $sappguid = $($App.Guid); write-host 'Guid:'$sappguid; $scategory = $($App.Category); if($scategory -eq "{category}"){$scategory = '';} write-host 'scategory:'$scategory; $sfiletime = [datetime]$( (Get-Item $sfilename).LastWriteTime ); write-host 'Filetime:'$sfiletime; $sversion = PKV ( 'version' ); $svendor = PKV ( 'svendor' ); $sslug = PKV ( 'spc' ); $stag = PKV ( 'stag' ); $smethod = PKV ( 'smethod' ); $spath = PKV ( 'spath' ); $sparams = PKV ( 'sparams' ); $sformat = PKV ( 'sformat' ); $swebsite = PKV ( 'swebsite' ); $schangelog = PKV ( 'schangelog' ); $sdownload = PKV ( 'sdownload' ); $splatform = PKV ( 'splatform' ); $sbits = PKV ( 'sbits' ); $snotes = PKV ( 'snotes' ); #### #### Append current log details Try{ ( "`n$($scategory)`t$($sslug)`t$($sname)`t$($sversion)`t$($snotes)`t$($schangelog)`t$($swebsite)" | Out-File Updates.txt -Append -Encoding "UTF8" ) } Catch { echo 'Can not write to Updates.txt'; echo "`n$($scategory)`t$($sslug)`t$($sname)`t$($sversion)`t$($snotes)`t$($schangelog)`t$($swebsite)"; } #### #### Update ... is.xml Live # POST data echo 'Posting update information to ...'; $postParams = @{ 'key'='...'; 'vendor'=$($svendor); 'name'=$($sname); 'version'=$($sversion); 'slug'=$($sslug); 'tag'=$($stag); 'method'=$($smethod); 'path'=$($spath); 'params'=$($sparams); 'format'=$($sformat); 'category'=$($scategory); 'website'=$($swebsite); 'changelog'=$($schangelog); 'download'=$($sdownload); 'platform'=$($splatform); 'bits'=$($sbits); 'notes'=$($snotes); 'appguid'=$($sappguid); } echo 'Post body:'; $postParams.GetEnumerator() | % { "$($_.Name)=$($_.Value)" }; $postResult = Invoke-WebRequest -Uri "https://example.com/post.php" -Method POST -Body $postParams echo 'Post results'; echo $postResult.Content; #### #### ADD DATE VARIABLE IF NOT EXISTING # Parameters $varnm = "LastUpdate" $vartp = "Textual" #StartEnd or #Textual or #RegularExpression # # Derived access property, ie. app.variables.$varnm.$varcnt $varcnt = if($vartp -eq "Textual"){"TextualContent"}Else{"CachedContent"} # If it doesn't exist, then create a new one if(-Not $app.variables.$varnm){ # ECHO "========== $varnm NOT FOUND, ADDING IT ==========" # # Add var to job/app $newvar = New-Object -TypeName Ketarin.UrlVariable -ArgumentList $varnm,$app.variables $newvar.VariableType = $vartp $app.variables.Add($varnm, $newvar) $app.Save() } #Set it to the current date, short format(should be sortable) $app.variables.$varnm.$varcnt = Get-Date -UFormat %Y%m%dT%H%M # #### <?xml version='1.0' encoding='utf-8'?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="5e10ff93-d903-42b4-9364-5660e1ab35e5"> <Category>Development</Category> <WebsiteUrl /> <UserAgent /> <UserNotes /> <LastFileSize>9542280</LastFileSize> <LastFileDate>2020-08-02T11:53:48.0018496</LastFileDate> <IgnoreFileInformation>false</IgnoreFileInformation> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>version</VariableChangeIndicator> <HashVariable /> <HashType>None</HashType> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer /> <SetupInstructions /> <Variables> <item> <key> <string>schangelog</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>https://developer.android.com/studio/releases/platform-tools</TextualContent> <Name>schangelog</Name> </UrlVariable> </value> </item> <item> <key> <string>swebsite</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>https://developer.android.com/studio/releases/platform-tools</TextualContent> <Name>swebsite</Name> </UrlVariable> </value> </item> <item> <key> <string>snotes</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent /> <Name>snotes</Name> </UrlVariable> </value> </item> <item> <key> <string>spc</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>adblinux</TextualContent> <Name>spc</Name> </UrlVariable> </value> </item> <item> <key> <string>splatform</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>linux</TextualContent> <Name>splatform</Name> </UrlVariable> </value> </item> <item> <key> <string>sbits</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <Name>sbits</Name> </UrlVariable> </value> </item> <item> <key> <string>svendor</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>Google</TextualContent> <Name>svendor</Name> </UrlVariable> </value> </item> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>([\d\.]+)\s\(</Regex> <Url>https://developer.android.com/studio/releases/platform-tools</Url> <TextualContent /> <Name>version</Name> </UrlVariable> </value> </item> <item> <key> <string>sdownload</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>https://dl.google.com/android/repository/platform-tools-latest-linux.zip</TextualContent> <Name>sdownload</Name> </UrlVariable> </value> </item> <item> <key> <string>LastUpdate</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>20200802T1153</TextualContent> <Name>LastUpdate</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand /> <ExecuteCommandType>Batch</ExecuteCommandType> <ExecutePreCommandType>Batch</ExecutePreCommandType> <SourceType>FixedUrl</SourceType> <PreviousLocation>K:\_Ketarin\Ketarin\..\Development\ADB_(Linux)-30.0.4.zip</PreviousLocation> <DeletePreviousFile>true</DeletePreviousFile> <Enabled>false</Enabled> <FileHippoId /> <LastUpdated>2020-08-02T11:53:48.0018496</LastUpdated> <TargetPath>..\{category}\{appname:regexreplace:([\s\t\r\n\-\&amp;]+):_}-{version}.{url:ext}</TargetPath> <FixedDownloadUrl>{sdownload}</FixedDownloadUrl> <Name>ADB (Linux)</Name> </ApplicationJob> </Jobs>

I propose to standardize some common variable names used in templates. I think this would greatly help database maintainability and template interoperability. Additionally, I think it will raise the quality of entries in online app database. Basically, I think we have all decided to use {version} instead of {vers}, {v} etc. Why not to do the same for other variables ? For example: {download_url} - contains full URL of application to download. Used only when full URL is dynamically scraped. {referrer} - contains a referrer URL {download_filename} - contains file name part of application to download. Used when whole file name is dynamically scraped.

When adding a variable in the EditVariablesDialog window, once you add a variable, it's name immediately appears in the context menu. Attempting to use the variable in it's own definition results in errors. For example, using the variable in it's own URL definition results in an unhandled System.Net.WebException in "UrlVariable:ReplaceInString(string value, bool onlyCached)" because Ketarin attempts an HTTP request for "%Working_Dir%\{varname}", %Working_Dir% being wherever Ketarin is being run from. It wouldn't make sense to use a variable in it's own definition to begin with, so could you maybe exclude the variable being defined from the context menu?

it sounds like what you really want then isn't an execution order change, but the ability to use a script as part of a variable assignment. that's a fantastic idea and doesn't change the execution order for those of use that depend on variables being fully populated in the pre-download script. i see the best way to implement this as an alternative to 'content from URL (start/end)', 'content from URL (regex)', 'textual content' in a variable assignment. like the regex selections, you'd then select the specifics - language of choice, variable return method (either return/exit code or value of x at end of script), and it would probably require some changes or at least exposure to the variable assignment order. currently i *believe* that the variables are calculated in order in the variable list unless they depend on another variable and that variable is then processed prior -- but there's no exposure to that process as far as i can see. this new variable scripting option would require that mechanism to be exposed and would also probably require the ability to reorder variables so you can ensure you have proper scripting context to dependent variable values. the execution order should be automatically imposed based on dependency and maybe a visual indicator for logic loops and similar problems, with the ability to reorder those variables that have similar staging order for dependence but ultimately depend on something else that might alter their proper execution order. if that's what you're really after, we should get it up as a feature request and poll Florian for input.

I'm getting to grips with Ketarin and its mostly excellent. I think i'm the problem. I cant seem to get this package to work when i set the url up. Original http://sourceforge.net/projects/sabnzbdplus/files/sabnzbdplus/sabnzbd-0.5.6/SABnzbd-0.5.6-win32-bin.zip/download I've modified to use the version: http://sourceforge.net/projects/sabnzbdplus/files/sabnzbdplus/sabnzbd-{version}/SABnzbd-{version}-win32-bin.zip/download Also when i create a rule to unzip a package to a certain location, what command do i need to use to copy the extracted files to another location. e.g. i download 7zip and it saves it to "d:\my software" and extracts it to "c:\program files\{app name}" but i would also like to exract it to "d:\my portable apps" Hope that makes and thanks in advance for any help.

Hi Flo, I have the variable for Custom Column set to vers in menu choice File -> Settings. I have enabled the custom value column. I have defined vers as a variable in (almost) all my downloads and populated. As per a previous post, you mentioned if a variable is not utilized, Ketarin is efficient and doesn't populate it, which is nice and efficient. And that appears to be the reason for what I'm seeing: The entries that are displaying a value for vers in the Custom Column, are only the ones that use the variable in URL or download path/file etc. I agree with and like the efficiency. But in this case I feel Ketarin should consider the variable "used" since it is the definition for the custom column, and populate it in every entry that has it? Thanks and regards, --appyface

I'm rather curious as to how we can replace something with the : symbol, I don't see a way of making it treat : as text and not as an operator. Only two of my links are given as %3A which is the : symbol, I want to replace %3A with : obviously. Also has %2F which is the / {download:multireplace:,:%2F,%3A:/,:} That works for the / but the %3A is just replaced with nothing so i'm not sure how to get it. Any pointers?

I found an app where I need to scrape the referer URL. Ketarin interpreted my variable, which I set into the 'referer' box in the app-setup, as a literal. Is it possible to add variable support to the referer box? Thanks in advance as always, --appyface

You can rename a variable with F2. It doesn't rename all instances where the variable is used, but that, IMHO, is a good thing.

While sounding stupid at first I think it will be very useful for templates. You specify a file URL and every field, including the name of the application is populated. Don't like how developers call the app ? Rename it. Simple as that.

Very useful updates, thank you kindly for your efforts floele. Beta 3 : Observations and Examples of the new features : Segmented downloading (enable in Settings -> Connection) Tested with 10MB, ~300MB and ~700MB files; segments set to 3. Integrity of files is maintained after download. Have confirmed 3 open sockets, consuming all available download speed. Made UrlVariable constructor public Works as advertised, thx. It's a little confusing that a variable name is required, both when creating the variable-object and adding it to the job. An example use of this feature, along with context and usage notes Function runpowershell / ps for executing variable content as power shell script and returning its result After testing as shown in examples below, I found no fault in the implementation. This feature brings several new possibilities to Ketarin : a. Derivation of variable value from a Powershell script, in the moment of use.(global,app or column variable) b. Execution of Powershell scripts before 'pre-download' scripts, during checks for updates c. Execution of Powershell scripts to determine the download url d. Execution of Powershell scripts before evaluation of custom-column-variables. NOTE : - Global and app/url-varaibles containing Powershell scripts will not execute unless used. - Where one uses a variable, determines at what stage of the update cycle it is executed. - Custom-column-values will be calculated every update - Custom-column-values will be calculated every time Ketarin starts. Delays in startup time will occur if your are running a powershell script for many jobs. Example 1 : Shows the use of a url-variable as script, used within the download URL. Note that in this specific case, the script returns nothing. Its purpose is just to execute a script before all others, and to do this on every check for updates. Var : {a_script} Use : {a_script:ps} Example 2 : This example demonstrates a ketarin-global-variable as a script, used within the download path. In this case it references a system-global-variable. Var : {COMPUTERNAME} Use : {COMPUTERNAME:ps} Another example, which enables the introduction of arbitrary time delays between updates [21_August_2016] : Added usage note : "Custom-column-values will be calculated every time Ketarin starts. Delays in startup time will occur if your are running a powershell script for many jobs."

Hi all, I can't find any mention to the advanced setting "Use the following variable as indicator for changes": what does it do? Thankyou!

Please rename the topic to "[SOLVED] ...", as the OP is a guest account I'm not able to edit. Thanks Peter

I'd like to define the global variable {root} as something like D:\Downloads and override the default. Then I could define my download path as {root}\{category}\{appname}\ if {appname} existed for each database entry. Would it be possible to create a pre-defined variable like {appname} that corresponds to the first field in the Edit box? It would be nice if you could define some buttons so I could easily paste these pre-defined variables into the Download Location with a few clicks. Thanks for a great tool.

Bug: Using latest 1.8.11 beta posted June 15 and seeing strange behavior. It looks like new variables may not be parsed the same way as variables that already exist for a job. I created a new variable on a job, "svendor". I populated it with a text value (Nirsoft). I use the following PS code in after update to populate a field in a database: $svendor = $app.variables.ReplaceAllInString("{svendor}", $sfiletime, $sfilename, $true); if($svendor -eq "{svendor}"){$svendor = '';} echo 'svendor:'$svendor; Unfortunately, svendor always returns empty. 2019-09-21 11:59:38: PowerShell: svendor: 2019-09-21 11:59:38: PowerShell: 2019-09-21 11:59:38: PowerShell: swebsite: 2019-09-21 11:59:38: PowerShell: https://www.nirsoft.net/utils/browsing_history_view.html I changed svendor in the first param of ReplaceAllInString to swebsite (another variable I use in all my jobs) and it parsed it correctly. 2019-09-21 11:59:03: PowerShell: svendor: 2019-09-21 11:59:03: PowerShell: https://www.nirsoft.net/utils/browsing_history_view.html 2019-09-21 11:59:03: PowerShell: swebsite: 2019-09-21 11:59:03: PowerShell: https://www.nirsoft.net/utils/browsing_history_view.html I've closed and reopened Ketarin and deleted and recreated the variable. I've tried a different variable name. Nothing is getting this value to populate into the powershell script. I've tried this on three jobs now and each is failing. Two are based on a template and I thought that might be the problem, but I tried with one that isn't based on a template and it didn't work either. Any ideas?

Hi Flo, Great to see that Ketarin now has it's own forum Ketarin it is definitely becoming more and more useful with every new release. I was wondering about the "Variable for custom column" feature as I can not seem to get it to work. So far I have only tried with one of the applications I update from FileHippo.com and I suspect that creating custom variables is not supported for FileHippo.com downloads, because of the use of the "FileHippo ID:" field, rather then the "URL:" field? EDIT: I noticed that indeed the "Variable for custom column" feature only works for none FileHippo.com links Thanks and regards, Johnny