CybTekSol's CNET Download.com (using Google Search) Template: (TEST)

[CybTekSol]

My CNET Download.com (using Google Search) Template: (TEST)

 

****IMPORTANT: I believe this template WILL WORK for the MAJORITY (75-80%)

of applications listed at CNET Download.com; located at:

http://download.cnet.com/.

 

IT IS A WORK IN PROGRESS and I will post improvements

as they develop.

 

 

****NOTE: This template uses the application's NAME embedded in the

application INFO page URL as designated by CNET Download.com.

In other words, go to the application's INFO page at CNET Download.com

and copy the NAME that appears within the URL. For example, for

Avast Home Edition, at CNET Download.com:

hxxp://www.download.com/Avast-Home-Edition/....html

the NAME you would paste at the FIRST prompt from the template

would be Avast-Home-Edition and the ACTUAL NAME of the

application Avast Home Edition at the SECOND prompt

from the template. This template uses Google's SEARCH ENGINE

which should make the apps fairly impervious to CNET Download.com's

cryptic URL changes!

 

1) There are two user defined global variables used, download_url={download_url}

(used as an alternative to {url}) and root_folder=Insert YOUR download destination

ROOT folder here (for example C:\Latest-Apps) and two local variables, {find} and

{redir1}.

 

2) Ketarin (1.04) released March 8, 2009 or later is required as this revision

uses the new 'variables in regular expressions' feature implemented by Florian.

 

3) The template follows:

 

<?xml version="1.0" encoding="utf-16"?>
<Jobs>
 <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <DownloadBeta>AlwaysDownload</DownloadBeta>
   <DownloadDate xsi:nil="true" />
   <VariableChangeIndicator />
   <CanBeShared>true</CanBeShared>
   <ShareApplication>false</ShareApplication>
   <HttpReferer>{redir1}</HttpReferer>
   <Variables>
     <item>
       <key>
         <string>CNetDownloadName</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>Textual</VariableType>
           <Regex />
           <TextualContent><placeholder name="CNet Download.com App Name from URL [Example: Avast-Home-Edition]" /></TextualContent>
           <Name>CNetDownloadName</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>find</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=Search[ ]Results.*?a href=")(http://www.download.com/{CNetDownloadName}.+?\.html|http://www.download.com/\d.+?\.html)(?="|\?)</Regex>
           <Url>http://www.google.com/search?hl=en&q={CNetDownloadName}+site%3Awww.download.com</Url>
           <Name>find</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>version</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=id="contentMain".*?<[a-z][\d]>.*?\s)[\d].+?(?=<)</Regex>
           <Url>{find}</Url>
           <Name>version</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>redir1</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=class="dlNowIcon".*?class="[a-z]+">.*?a[ ]href=")http://dw.com.com.+?(?=")</Regex>
           <Url>{find}</Url>
           <Name>redir1</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>download_url</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=META[ ]HTTP-EQUIV="Refresh"[ ]CONTENT="\d;[ ]URL=)[a-z]+://.+?(?="/>)</Regex>
           <Url>{redir1}</Url>
           <Name>download_url</Name>
         </UrlVariable>
       </value>
     </item>
   </Variables>
   <ExecuteCommand />
   <Category>[iNSERT CATEGORY HERE]</Category>
   <SourceType>FixedUrl</SourceType>
   <PreviousLocation />
   <DeletePreviousFile>true</DeletePreviousFile>
   <Enabled>true</Enabled>
   <FileHippoId />
   <LastUpdated xsi:nil="true" />
   <TargetPath>{root_folder}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath>
   <FixedDownloadUrl>{download_url}</FixedDownloadUrl>
   <Name><placeholder name="Application Name" /></Name>
 </ApplicationJob>
</Jobs>

 

TO USE IT:

 

1) Define the user defined global variables under 'Settings' in Ketarin to YOUR preferences

2) Edit the template externally to reflect these modifications (if you choose

not to use mine) and change the <TargetPath> line to YOUR preferences

3) Then:

a) Import it (Obviously)

Enter the information needed as requested as a result of the 'placeholder'

prompts... it will ask for the application URL's NAME as described above and

the actual application's name

c) Modify the Category to your preferences

d) Change the 'Advanced Settings' to your preferences

e) VERIFY...VERIFY...VERIFY that ALL regex matches WORK and

MAKE ADJUSTMENTS to them as necessary!

 

I hope somebody can use it... feel free to modify it in any way you wish for your use or make suggestions.

 

Update: Added variable within the regex for {find} to make the template more reliable on March 19th, 2009... Since this feature was just added a couple of Beta Releases ago, make sure you use Ketarin version 1.04 or higher.

 

March 27th, 2009: Tweaked the regex for {find} AGAIN, due to rolling URL changes at CNET, however, it is still flaky. See my post below and I would refrain from using this template extensively in the interim.

 

CybTekSol

Edited March 27, 2009 by CybTekSol

[EY]

Even *I* was able to use it just now to download Advanced SystemCare Free from CNET. Now THAT'S got to make you feel like you're doing something right, huh?

 

Thanks very much for this, I hadn't counted on getting things from CNET too! Off to make a list...

[FranciscoR]

Wow !!! Congrats CybTekSol, MAJOR template ! =D

[JaseF]

Hiya,

I'm very very new to Ketarin (and already LOVE IT THANKS SO MUCH), so I might be missing out some basics, but I'm trying to use this to download Smart Defrag (see here: http://download.cnet.com/Smart-Defrag/3000-2094_4-10759533.html?part=dl-6271865&subj=dl&tag=button&cdlPid=11006601).

 

All I've down is cut & pasted the script provided above into an xml file (Cnet_Download_Com.xml), then using Ketarin selected new from file, answered Smart-Defrag, then Smart Defrag, then updated the local path to Install\OS Tools\!Default\Ketarin_SmartDefrag.{url:ext}

 

When attempting to update, I get the following error:

"The URL '{find}' of variable 'redir1' is not valid.

 

Thanks for your help!

[CybTekSol]

When attempting to update, I get the following error:

"The URL '{find}' of variable 'redir1' is not valid.

1)Be sure you have followed this post step-by-step to create the template.

 

2)Make sure you are using Ketarin version 1.04 or higher (1.0.5 currently) available here.

 

3)Input Smart-Defrag at the first prompt and Smart Defrag at the second as you have apparently already done.

 

4)I personally tested this app with the 'CNET template' and it works.

 

Hope this helps.

 

CybTekSol

[JaseF]

Thanks for such a quick reply

 

Aaaahhh... It IS the version! Seeing as I'd only downloaded Ketarin yesterday, I was sure it must be up-to-date, but I guess this must be a beta (or at least not active on the the main link yet)

 

Apologies for the oversight, and thanks again!

[CybTekSol]

Thanks for such a quick reply

You're VERY welcome... I'm glad I could help. Regarding the version, Ketarin has developed dramatically in the past few betas, which is why I updated the CNET template. The Google search portion of the template became much more reliable with the ability to add a 'variable within a regular expression' feature that Florian implemented with version 1.0.4.

[CybTekSol]

Due to changes at CNET Download.com, this template has become UNRELIABLE and I do not see any way to fix it at present with the incorporation of Google's search engine. It may appear that your app entries for CNET using this template are working properly, however, some percentage of them (30% in my case) are probably NOT getting the latest version of the application. I have discovered a way to use CNET's search engine with a much higher reliability, so after it has been vetted properly, I will post it in this forum section.

[zimboi]

Hey, does anyone know how to sort the version variable out so it only shows the version.

 

Using as default:

{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}

 

ends up as this for me:

Compression Utilities\7-Zip_v7-Zip_4.57_User_Reviews_.{url:ext}

 

And the best i can get it to is:

{category}\{version}.{ext}

 

which ends up as this:

Compression Utilities\7-Zip 4.57 User Reviews.{ext}

 

If i could even just get rid of the User Reviews part i wouldn't have to rename many downloads at all.

 

And thanks CybTekSol, been having to download manually for a while and couldnt get the SP template to work, so this is really helpful.

 

Cheers.

[CybTekSol]

Which template are you using for this zimboi? It appears the 'version scrape' needs to be tweaked?