CybTekSol

Flo, as my app entries are now above 1100, I have a couple of feature requests to make management of them a little easier. 1) A total entries & selected entries count in the lower right portion of the GUI possibly similar to Windows Explorer? 2) A search and select feature... for example, when I want to update only PortableApps.com entries, I can search PortableApps.com which appears in the application name for the 50+ matching entries and each occurrence will be selected/highlighted and then updated by right-click without scrolling through 1100+ entries to find them for manual selection. Your thoughts? Feasible?

Yes, and for me, the only way to get them to update is to manually reload the pages for the regexs one by one, then update. How freaky is this?

@FranciscoR, I'm having so much trouble with Totalcmd.pl now I've disabled their app entries so they do not update when I perform a 'global' update of apps. I don't have a clue what has changed with their site... which is frustrating. Can't win 'em all.

@FranciscoR, Apparently this is beginning to happen on this side of the Atlantic also. I've experienced difficulties since last night with PortableApps.com downloads. SourceForge appears to be experiencing intermittent issues with other apps for me also.

It must be related to your location as I get the same 5 second routine without all the frills... at some point it may become an issue for me. Can you post your refined template for PortableApps.com in the 'Templates section' at your convenience for those who may have a similar issue now? --------------------------------------------------------------------- 3/30/2009 4:14:41 PM: 7-Zip Portable (PortableApps.com): Replacing {download_url} in '{download_url}' with 'http://downloads.sourceforge.net/portableapps/7-Zip_Portable_4.65.paf.exe' 3/30/2009 4:14:41 PM: 7-Zip Portable (PortableApps.com): Using referer: (none) 3/30/2009 4:14:44 PM: 7-Zip Portable (PortableApps.com): Server source file: /portableapps/7-Zip_Portable_4.65.paf.exe 3/30/2009 4:14:45 PM: 7-Zip Portable (PortableApps.com): Replacing {version} in '{root_folder}\Utilities\Files & Folders\Compression-Decompression\7-Zip_Portable_v{version:replace: :_}.paf.exe' with '4.65' 3/30/2009 4:14:45 PM: 7-Zip Portable (PortableApps.com): Determined target file name: C:\DownLoad\Latest DownLoads\Utilities\Files & Folders\Compression-Decompression\7-Zip_Portable_v4.65.paf.exe 3/30/2009 4:14:45 PM: 7-Zip Portable (PortableApps.com): Checking if update is required... 3/30/2009 4:14:45 PM: 7-Zip Portable (PortableApps.com): Replacing {version} in '{version}' with '4.65' 3/30/2009 4:14:45 PM: 7-Zip Portable (PortableApps.com): Update not required, {version} has not changed 3/30/2009 4:14:46 PM: 7-Zip Portable (PortableApps.com): Replacing {version} in '{version}' with '4.65'

TRY REMOVING THE REFERER and check again... I just updated 50+ PortableApps.com entrys with NO failures (ALL without an HTTP referer AND without the ?download at the end of the FixedDownloadURL).

@FrancisoR, First, I agree that variables in the HTTP referrer would be a welcome addition to Ketarin. In this particular case, with PortableApps.com, I am not currently experiencing the problems you are describing and I do not use a referrer in my template for their apps. The only difference that I see with your app entry for 7-Zip and mine is that I do not add the ?download to the end of the FixedDownloadURL. Try removing this from your entries for PortableApps.com and see if it helps in the interim... before Flo adds this feature.

Try this: <?xml version="1.0" encoding="utf-16"?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>{version}</VariableChangeIndicator> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer>http://www.techpowerup.com/downloads/1363/mirrors.php</HttpReferer> <Variables> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>(?:TechPowerUp\sGPU-Z.*?)(Beta.+?|Build.+?|\d.*?)(?:<|&)</Regex> <Url>http://www.techpowerup.com/downloads/1363/mirrors.php</Url> <Name>version</Name> </UrlVariable> </value> </item> <item> <key> <string>truncated_url</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>(?:Please\sclick\sone\sof\sthe\slinks\sbelow.*?href=")(/downloads/.*?\.exe)(?:")</Regex> <Url>http://www.techpowerup.com/downloads/1363/mirrors.php</Url> <Name>truncated_url</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand /> <Category>System Info</Category> <SourceType>FixedUrl</SourceType> <PreviousLocation /> <DeletePreviousFile>false</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated xsi:nil="true" /> <TargetPath>{target}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath> <FixedDownloadUrl>http://www.techpowerup.com/{truncated_url}</FixedDownloadUrl> <Name>TechPowerUp GPU-Z</Name> </ApplicationJob> </Jobs> It 'scrapes' the first mirror only... if this mirror fails simply adjust the regex for the {truncated_url} variable (or try selecting RTL to 'scrape' the last mirror)... hope this helps.

Man... I'm dizzy from reading all of this. I've never experienced anything similar to this as I have always run Ketarin from the same folder as the Jobs.db file. For example, for my 'all categories' Ketarin, the folder is simply Ketarin, for my 'testing' Ketarin, the folder is Ketarin (Testing), for my 'portable apps' Ketarin, the folder is Ketarin (Portable-Apps) and so on. In other words, a Ketarin.exe, System.Data.SQLite.DLL and Jobs.db in each folder which makes everything portable between all my PCs on my network. Although, it wastes a little HD space, it works well for me so far. The problems being discussed here are indeed puzzling!

@FranciscoR, My template still works for CNET itself, it's that they randomly change the URL for the app with a subsequent version, so any template based on a static app URL will eventually break. I've developed a prototype template based on CNET's own search engine that compensates for this and will be posting it this week as so far it is very reliable. Who knows though, their methods are so cryptic, this one will probably break eventually. I'm beginning to get frustrated trying to keep up with all the site changes that I have seen this week.

I'm glad I could help as I well know it can be frustrating adjusting to site changes. May I ask you a favor in return though? Please remove the profanity in your post above so we can keep things rated "G"... suitable for ALL audiences. Might I suggest darn...

Try this: <?xml version="1.0" encoding="utf-16"?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>{version}</VariableChangeIndicator> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer>http://downloadmirror.intel.com/17348/eng/LF94510J.86A.{version}.EB.EXE&agr=&ProductID=2926&DwnldId=17348&strOSs=&OSFullName=〈=eng</HttpReferer> <Variables> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>(?:alt="Software Search Results".*?\>)([\d]{4})(?:</font)</Regex> <Url>http://downloadcenter.intel.com/Filter_Results.aspx?strTypes=all&ProductID=2926&OSFullname=OS+Independent〈=eng&strOSs=38#BIO</Url> <Name>version</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand /> <Category>BIOS</Category> <SourceType>FixedUrl</SourceType> <PreviousLocation /> <DeletePreviousFile>false</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated xsi:nil="true" /> <TargetPath>{DEST_DIR}\{category}\{appname}_{version}.{url:ext}</TargetPath> <FixedDownloadUrl>http://downloadmirror.intel.com/17348/eng/LF94510J.86A.{version}.EB.EXE</FixedDownloadUrl> <Name>Desktop Board D945GCLF2 BIOS Update</Name> </ApplicationJob> </Jobs>

I imported and tested your Samsung CLP-300 printer drivers XML export and it appears that the version capture configuration is no longer correct... the URL for version appears to be invalid from here. Try to correct that first and then see what happens. BTW, your post for the uTorrent export is incorrect as the utf=16 line is duplicated.

No... CNET has changed their cryptic naming scheme to their app pages which changes randomly with a subsequent app version... VERY frustrating. This scheme renders the Google scrape unreliable as it may return the older version's page. That is exactly what I have experienced even after making the same changes to the Totalcmd.pl template that you have and I cannot determine what has changed with their site either. Puzzling and disturbing day with Ketarin yesterday for me.

Are you able to download the app with your browser without any error?

There was a time when I could envision an application such as Ketarin eventually being able to inventory your locally installed apps and based on a variety of built-in templates covering all the major download sites, create app entries automatically for you. However, after today in which I have noticed how vulnerable app entries (especially those based on templates) are to download site changes, which may not be correctable, I have been jolted back to reality. This goal may never be achieved as a result of the volatility (and inconsistencies) of download site design. The only way that I can conceive of this happening is if the application author is also the owner of a MAJOR download site with unlimited bandwidth and has control of the site's design to provide reliability. This is unrealistic as far as I can see for anytime in the near future. I can only dream. In the interim, Ketarin is still light-years ahead of the old way I accomplished this... manually!

I had discovered the same problems earlier today and made the very same adjustments to your template as it appears that the 'variable scrape order' does make a difference with their site and simultaneous access to their site causes failures now. I was going to post a 'heads up' regarding this but obviously you had already caught it. Props for being 'on the ball' once again FranciscoR! They must have changed something as it was not an issue before in my experience. Do you have any idea what they changed to cause this? I've not had a good day due to site changes that have forced me to modify 30+ app entries and virtually rendered my CNET template useless. I hope this does NOT become a trend.

Due to changes at CNET Download.com, this template has become UNRELIABLE and I do not see any way to fix it at present with the incorporation of Google's search engine. It may appear that your app entries for CNET using this template are working properly, however, some percentage of them (30% in my case) are probably NOT getting the latest version of the application. I have discovered a way to use CNET's search engine with a much higher reliability, so after it has been vetted properly, I will post it in this forum section.

@FranciscoR, That's exactly what I am encountering no matter what POST data I use also FranciscoR... I'm giving up on it for now. Hopefully Flo can figure out what they have changed.

I just played with the POST data changes for nearly an hour and couldn't get it going for JDK 6u13... I think Flo will have to look at this one if time allows as that was all the time I could devote to it for now. Sorry appyface, I tried.

If you mean HTTP referrer, yes it works FranciscoR, I have been doing it for quite some time now... see my CNET template for example. I have tested it with sites that have to have a referrer and it works...

Never scared of changes as long as they are improvements, but you must admit, Ketarin has come a LONG way! Sure, it can be better as almost everything can, although it will adequately get the job done for me now.

Configuring, re-configuring, tweaking, importing, exporting, adding 100 or so apps (now 1100+), tweaking, refining templates, testing new features, updating and smiling the entire time! FranciscoR... you're are beginning to sound like a 'glass is half empty' kinda person. LOL...

Flo, I have put Ketarin 1.0.5 through 10+ hours of torture/testing and I must say I AM VERY IMPRESSED! I can accomplish most anything necessary to 'finalize' my app arsenal now! I want to THANK YOU again for being so responsive to our feature requests and input! My most sincere CONGRATS on a TRULY OUTSTANDING APPLICATION!!

My AnalogX Template: After a four year hiatus, AnalogX is back! I used to use their applications on a regular basis and now that they are updating all their apps and promising new development I decided to create a template to use with their site. I have tested the template with 50+ AnalogX apps and it has a 100% reliability as of today, March 24th, 2009. Try some of their apps and see if you like them. 1) Ketarin (1.0.3) released February 23, 2009 or later is required as this template uses the 'placeholder' feature. 2) The template follows: <?xml version="1.0" encoding="utf-16"?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <DownloadBeta>Default</DownloadBeta> <DownloadDate xsi:nil="true" /> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>{version}</VariableChangeIndicator> <CanBeShared>true</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer><placeholder name="App Info and Download Page URL from AnalogX?" value="http://www.analogx.com/" /></HttpReferer> <Variables> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>(?:version\s)(Beta.+?|Build.+?|\d.*?)(?:<|&)</Regex> <Url><placeholder name="App Info and Download Page URL from AnalogX?" value="http://www.analogx.com/" /></Url> <Name>version</Name> </UrlVariable> </value> </item> <item> <key> <string>truncated_url</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>(?:a href=")(/files/.*?)(?:")</Regex> <Url><placeholder name="App Info and Download Page URL from AnalogX?" value="http://www.analogx.com/" /></Url> <Name>truncated_url</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand /> <Category><placeholder name="Category?" value="AnalogX" /></Category> <SourceType>FixedUrl</SourceType> <PreviousLocation /> <DeletePreviousFile>true</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated xsi:nil="true" /> <TargetPath><placeholder name="TargetPath?" value="{target}\{category}\{appname}_v{version}.{url:ext}" /></TargetPath> <FixedDownloadUrl>http://www.analogx.com{truncated_url}</FixedDownloadUrl> <Name><placeholder name="Application Name?" value="AnalogX App" /></Name> </ApplicationJob> </Jobs> It is a simple 'no frills' template as that was all that was needed. If you are not familiar with templates see this post. Modify the template for YOUR needs. I hope someone finds this useful. Updated April 8th, 2009: Small regex tweak due to site changes... CybTekSol