Jump to content
Ketarin forum

CybTekSol's LATEST Universal Softpedia.com Template (TEST)

CybTekSol

By CybTekSol,
in Templates

 Share

Recommended Posts

CybTekSol Newbie

CybTekSol

Posted
Posted

My LATEST Universal Softpedia.com Template (TEST):

 

****PLEASE TEST it and give me some feedback on this one if you refine it or have

significant problems citing specific examples.

 

****NOTE: This template uses the application's INFO page URL.

 

1) There are two user defined global variables used, download_url={download_url}

(used as an alternative to {url}) and root_folder=Insert YOUR download destination

ROOT folder here (for example C:\Latest-Apps).

 

2) The latest version of Ketarin (1.0.3) released February 24, 2009 or later is

required to overcome their site changes for 'user-agent'.

 

3) The template follows:

<?xml version="1.0" encoding="utf-16"?>
<Jobs>
 <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <DownloadBeta>Avoid</DownloadBeta>
   <DownloadDate xsi:nil="true" />
   <VariableChangeIndicator />
   <CanBeShared>true</CanBeShared>
   <ShareApplication>false</ShareApplication>
   <HttpReferer>{redir1}</HttpReferer>
   <Variables>
     <item>
       <key>
         <string>version</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex><title>Download.*?(\d.*?|\d.*?[ ]Build[ ]\d.*?|\d.*?[ ]Beta[ ]\d.*?)(?:\s\/\s|\s-\s)</Regex>
           <Url><placeholder name="Application Info Page URL from Softpedia" /></Url>
           <Name>version</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>redir1</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=download_bkg.gif.*?<a href=").*?(?=")</Regex>
           <Url><placeholder name="Application Info Page URL from Softpedia" /></Url>
           <Name>redir1</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>download_url</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=DOWNLOAD LOCATIONS.*?<table width=".*?<a href=").*?(?=")</Regex>
           <Url>{redir1}</Url>
           <Name>download_url</Name>
         </UrlVariable>
       </value>
     </item>
   </Variables>
   <ExecuteCommand />
   <Category>[iNSERT CATEGORY HERE]</Category>
   <SourceType>FixedUrl</SourceType>
   <DeletePreviousFile>false</DeletePreviousFile>
   <Enabled>true</Enabled>
   <FileHippoId />
   <LastUpdated xsi:nil="true" />
   <TargetPath>{root_folder}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath>
   <FixedDownloadUrl>{download_url}</FixedDownloadUrl>
   <Name><placeholder name="Application Name" /></Name>
 </ApplicationJob>
</Jobs>

 

To use it:

 

1) Define the user defined global variables under 'Settings' in Ketarin to YOUR

preferences

2) Edit the template externally to reflect these modifications (if you choose not to use

mine) and change the <TargetPath> line to YOUR preferences

3) Then:

a) Import it (Obviously)

B)Enter the information needed as requested as a result of the 'placeholder'

prompts... it will ask for the application's DOWNLOAD page URL (NOT the information

page URL) and the application's name

c) Modify the Category to your preferences

d) Change the 'Advanced Settings' to your preferences

e) VERIFY...VERIFY...VERIFY that ALL regex matches WORK and

MAKE ADJUSTMENTS to them as necessary!

 

I have had a 80-90% success rate with this template; in instances where it failed, I simply tweak the regular expression. ;)

 

CybTekSol

Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted (edited)

Nice indeed,

 

20-02-2009 12:32:34: Everest: Replacing {redir1} in '{redir1}' with 'http://www.softpedia.com/progDownload/EVEREST-Ultimate-Edition-Download-30373.html'

20-02-2009 12:34:04: Update started with 1 application(s)

20-02-2009 12:34:08: Everest: Replacing {redir1} in '{redir1}' with 'http://www.softpedia.com/progDownload/EVEREST-Ultimate-Edition-Download-30373.html'

20-02-2009 12:34:10: Everest: Replacing {download_url} in '{download_url}' with 'http://download.softpedia.com/dl/69b0c84876b9d5f6361db8e5f5e3095e/499ea33d/100030373/software/system/info/everestultimate500.zip'

20-02-2009 12:34:11: Everest: Checking if update is required...

20-02-2009 12:34:11: Everest: Update required, 'D:\Programas\Test\everestultimate500.zip' does not yet exist

20-02-2009 12:34:13: Everest: Replacing {version} in '{version}' with '5.00.1664 Beta / 5.00.1650 Free Trial'

20-02-2009 12:34:13: Update finished

 

Will I actually DL Beta or Trial ?

Edited by FranciscoR
Link to comment
Share on other sites
CybTekSol Newbie

CybTekSol

Posted
  • Author
Posted

@FranciscoR,

This is an example of one of the Softpedia inconsistencies we discussed in the past. This particular example would require some major regex tweaking... OBVIOUSLY! The 80-90% success rate is still better than starting from scratch with every app there though... know what I mean? ;) I don't ever see an infallible template for their site. :(

Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted

Nope, now I have beta version with a trial DL

 

20-02-2009 13:28:38: Update started with 1 application(s)

20-02-2009 13:28:40: Everest: Replacing {redir1} in '{redir1}' with 'http://www.softpedia.com/progDownload/EVEREST-Ultimate-Edition-Download-30373.html'

20-02-2009 13:28:42: Everest: Replacing {download_url} in '{download_url}' with 'http://download.softpedia.com/dl/9850f0f1396405d88f2e6437aadbc0b7/499eb006/100030373/software/system/info/everestultimate500.zip'

20-02-2009 13:28:54: Everest: Checking if update is required...

20-02-2009 13:28:54: Everest: Update required, 'D:\Programas\Test\everestultimate500.zip' does not yet exist

20-02-2009 13:28:57: Everest: Replacing {version} in '{version}' with '5.00.1664 Beta'

20-02-2009 13:28:57: Update finished

 

 

3dsMax {version} is not 100% either

 

20-02-2009 13:30:43: Update started with 1 application(s)

20-02-2009 13:30:46: 3dsMax: Replacing {redir1} in '{redir1}' with 'http://www.softpedia.com/progDownload/3D-Studio-Max-Download-15976.html'

20-02-2009 13:30:48: 3dsMax: Replacing {download_url} in '{download_url}' with 'http://download.softpedia.ro/dl/aeb999d1855cbaa2dc62e88aa7863a2f/499eb084/100015976/software/MULTIMEDIA/GRAPHIC/Autodesk3dsMax2009_ENU_TrialDownload.exe'

20-02-2009 13:30:50: 3dsMax: Checking if update is required...

20-02-2009 13:30:50: 3dsMax: Update required, 'D:\Programas\Test\Autodesk3dsMax2009_ENU_TrialDownload.exe' does not yet exist

20-02-2009 13:30:51: 3dsMax: Replacing {version} in '{version}' with '3ds Max 2009 11.0 Free Trial'

20-02-2009 13:30:51: Update finished

 

 

It needs more time... no pressure from me. I'll be back after some testing on {version}. =)

Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted

As is, it's a very, very, very good template and I must say it already is one of my personal favorites. {version} is good enought to track updates, and I am amazed on how easily Softpedia allowed you to manage the page redirection... I don't have such help on my VersionTracker template, but {version} is so much easy... there's no perfect web design(er) ! ;)

Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted

Me2, running out of time right now. But I do understand the SIZE of your problem... I think that for the moment I will be using 'Last Updated:' row, e.g. 'November 3rd, 2008, 14:48 GMT', properly customized, to track versions. Easy to-do and less unpredictable.

 

P.S. You're using HTTP referer for any specific reason ? Seems to work fine without it.

Link to comment
Share on other sites
CybTekSol Newbie

CybTekSol

Posted
  • Author
Posted
P.S. You're using HTTP referer for any specific reason ? Seems to work fine without it.
If my memory serves me correctly, it was an issue for a couple of the apps I get from there for some freaky reason, so I left it in the template... can't remember the specifics.
Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted

This is just my personal preference, but until they include a somewhat standardized version format, I will be using ([a-z]+?\s\d{1,2}[a-z]{2}\,\s\d{4}\,\s\d{2}\:\d{2})(?:\sGMT)? . It works for me with every Adobe and Autodesk app and plugin I tested with, and allows me to solve problems like Adobe Photoshop Lightroom 2.3 RC Build 534370 / 2.2 Build 523352.

 

20-02-2009 16:49:31: Update started with 1 application(s)

20-02-2009 16:49:33: Everest: Replacing {redir1} in '{redir1}' with 'http://www.softpedia.com/progDownload/EVEREST-Ultimate-Edition-Download-30373.html'

20-02-2009 16:49:35: Everest: Replacing {download_url} in '{download_url}' with 'http://download.softpedia.com/dl/fb8af7afa01229399789c3ff567275fc/499edf1b/100030373/software/system/info/everestultimate500.zip'

20-02-2009 16:49:36: Everest: Checking if update is required...

20-02-2009 16:49:36: Everest: Update required, 'D:\Programas\Test\everestultimate500.zip' does not yet exist

20-02-2009 16:49:37: Everest: Replacing {version} in '{version}' with 'February 20th, 2009, 08:12'

20-02-2009 16:49:37: Update finished

 

 

I'm quite happy with template, thanks a lot CybTekSol. =)

Link to comment
Share on other sites
CybTekSol Newbie

CybTekSol

Posted
  • Author
Posted
This is just my personal preference, but until they include a somewhat standardized version format, I will be using ([a-z]+?\s\d{1,2}[a-z]{2}\,\s\d{4}\,\s\d{2}\:\d{2})(?:\sGMT)? . It works for me with every Adobe and Autodesk app and plugin I tested with, and allows me to solve problems like Adobe Photoshop Lightroom 2.3 RC Build 534370 / 2.2 Build 523352.

 

I'm quite happy with template, thanks a lot CybTekSol. =)

I'm glad you could use it. ;) Would you please take my template, make changes that you have implemented to suit your needs and post it to this same thread to give other users better coverage of Softpedia apps? Thanks in advance...
Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted

Like I said, it's *only* my personal preference. If a user wants to use last updated date instead of file version, all he/she has to do is to copy that regex into the {version} variable. But your template is working great, let users experience it and decide, if a request is forward I will do that. Still, a community input would be most useful to improve it... Let it rest now, no worries, we cannot be too demanding with DL sites such as this. Well done again. ;)

Link to comment
Share on other sites
CybTekSol Newbie

CybTekSol

Posted
  • Author
Posted
If a user wants to use last updated date instead of file version, all he/she has to do is to copy that regex into the {version} variable.
Agreed... I think the average user will not have any problems doing that. Users can create two different versions of the template on their own with a simple copy/paste replacement of the {version} regex with this: 
([a-z]+?\s\d{1,2}[a-z]{2}\,\s\d{4}\,\s\d{2}\:\d{2})(?:\sGMT)?

either within an external editor, since there are no converted characters (new users see more on this below) or within Ketarin and exporting it as template. When confronted with a situation like this at Softpedia (where multiple versions and download locations are combined), they can import the alternative template which captures the 'Last updated' date and then tweak the {download_url} by inserting an additional anchor from the app's download page for the download location displayed ABOVE the location they actually want to use, such as 

Softpedia Secure Download \(RO\).*?

inserted which would change the {download_url} scrape from:

(?<=DOWNLOAD LOCATIONS.*?<table width=".*?<a href=").*?(?=")

to: 

(?<=DOWNLOAD LOCATIONS.*?<table width=".*?Softpedia Secure Download \(RO\).*?<a href=").*?(?=")

and in the case of "Everest Ultimate Edition" would capture the third download url (the Beta version).

 

I must caution new users regarding pasting text directly into a template with an external editor as certain characters are converted on export from Ketarin. For example: If you paste the regular expression above into Ketarin it would be exported to a template as:

(?<=DOWNLOAD LOCATIONS.*?<table width=".*?Softpedia Secure Download \(RO\).*?<a href=").*?(?=")

Notice the < character is converted to < (its hypertext equivalent). This is the case for several other characters as well and can result in template import errors if you are not careful. Just 'food for thought' to give new users a head start! ;)If you are unsure, the safest method is to paste the expression into Ketarin first, then export, and work with the template in an external editor afterwards.

 

CybTekSol

Link to comment
Share on other sites
  • 2 months later...
FranciscoR Newbie

FranciscoR

Posted
Posted (edited)

Now using

 

<Regex>(?:(?<=\<title\>Download.*?<placeholder name="Stable release ?" value="\/.*?" />\s)(\d+[\.\d+]+?.+?)(?=\s[\-\/\[]<placeholder name="Version format is 1.x.x ?" value="?" />)<placeholder name="Capture date ?" value="|(?:[a-z]+?\s\d{1,2}[a-z]{2}\,\s\d{4}\,\s\d{2}\:\d{2})(?:\sGMT)?" />)</Regex>

 

to capture version, based on my SF template. Works with all my SP jobs, probably will do the same with the {download_url} variable. Or not. =)

 

-----------------------

Edited by FranciscoR
Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted (edited)

I have a restricted universe here, that's no miracle for sure. 'Stable release' should be used only when you are presented multiple versions in a format like version #1/version #2 (#2 is usually stable, whereas #1 is beta). If #3 and #4 exist, adjust regex. 'Version format' excludes 'free trials' stuff from capture, but remenber using it will also exclude 'builds', 'betas', etc. from capture (i'm using it for stable releases). As for date, I could just remove it but maybe tomorow i need it, so i prefer to delete it upon pop-up. =)

 

-------------------------

Edited by FranciscoR
Link to comment
Share on other sites
FranciscoR Newbie

FranciscoR

Posted
Posted (edited)

Try this one instead. You need to escape ( and ) with the "Paste link text" placeholder: 'Softpedia Secure Download \(US\)' or 'Softpedia Secure Download \(RO\)'.

 

<?xml version="1.0" encoding="utf-16"?>
<Jobs>
 <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <DownloadBeta>Default</DownloadBeta>
   <DownloadDate xsi:nil="true" />
   <VariableChangeIndicator />
   <CanBeShared>true</CanBeShared>
   <ShareApplication>false</ShareApplication>
   <ExclusiveDownload>false</ExclusiveDownload>    
   <HttpReferer><placeholder name="Download Page URL" /></HttpReferer>
   <Variables>
     <item>
       <key>
         <string>version</string>
       </key>
       <value>
         <UrlVariable>
           <RegexRightToLeft>false</RegexRightToLeft>
           <VariableType>RegularExpression</VariableType>
           <Regex>(??<=\<title\>Download.*?<placeholder name="Stable release ?" value="\/.*?"  />\s)(\d+[\.\d+]+?.*?)(?=\s[\-\/\[]<placeholder name="Version format is 1.x.x ?" value="?"  />)<placeholder name="Capture date ?" value="|(?:[a-z]+?\s\d{1,2}[a-z]{2}\,\s\d{4}\,\s\d{2}\:\d{2})(?:\sGMT)?" />)</Regex>
           <Url><placeholder name="Info Page URL" /></Url>
           <Name>version</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>download_url</string>
       </key>
       <value>
         <UrlVariable>
           <RegexRightToLeft>true</RegexRightToLeft>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=href\=")(\w+\:\/\/[\w][\w\-?\w.:]+\/?[\w\.?=%&=\-/,\)\(]*)(?=".*?<placeholder name="Paste link text"  />)</Regex>
           <Url><placeholder name="Download Page URL" /></Url>
           <Name>download_url</Name>
         </UrlVariable>
       </value>
     </item>
   </Variables>
   <ExecuteCommand>{rem} copy /y {file} {shared}</ExecuteCommand>
   <ExecutePreCommand>{rem} del /q {shared}\{appname}*.*</ExecutePreCommand>
   <Category><placeholder name="Category" value="Backup-Benchmark-Clean-Compression-Grep-Internet-Multimedia-Misc-Office-PDF-Recovery-Registry-Scrapbook-Security-Test-Virtualization" /></Category>
   <SourceType>FixedUrl</SourceType>
   <DeletePreviousFile>false</DeletePreviousFile>
   <Enabled>true</Enabled>
   <FileHippoId />
   <LastUpdated xsi:nil="true" />
   <TargetPath><placeholder name="Save to" value="{target}\{category}\" /><placeholder name="Class" value="Diagramming\DL_Manager\Extensions\Graphics\Network\Notepad\Plugins\ScreenCapture\Test\Uninstall\USB\"/><placeholder name="Rename" value="{appname}_{version}.{url:ext}" /></TargetPath>
   <FixedDownloadUrl>{download_url}</FixedDownloadUrl>
   <Name><placeholder name="AppName" /></Name>
 </ApplicationJob>
</Jobs>

Edited by FranciscoR
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.

  I accept