Jump to content
Ketarin forum

CybTekSol's BetaNews FileForum Templates (Uses THEIR Search)


CybTekSol
 Share

Recommended Posts

My Universal BetaNews FileForum Templates: (TEST)

 

****IMPORTANT: I believe these templates WILL WORK for the VAST MAJORITY of

applications listed at BetaNews FileForum; located at:

http://fileforum.betanews.com/. This is the FIRST version

of my templates and they have been adjusted for their new site

design. IT IS A WORK IN PROGRESS and I will post improvements

as they develop. ;)

 

There are TWO VERSIONS of the template; the FIRST is for all

applications that are NOT HOSTED by SourceForce and the SECOND

is for applications that ARE HOSTED by SourceForge. It is YOUR

RESPONSIBILITY to determine which to use by clicking on the apps

download link to determine the host location.

 

****NOTE: These templates use the application's NAME embedded in the

application INFO page URL as designated by BetaNews FileForum.

In other words, go to the application's INFO page at BetaNews FileForum

and copy the NAME that appears within the URL. For example, for

Spybot - Search & Destroy, at BetaNews FileForum:

hxxp://fileforum.../detail/Spybot-Search-Destroy/1043809773/1

the NAME you would paste at the FIRST prompt from the template

would be Spybot-Search-Destroy and the ACTUAL NAME of the

application Spybot - Search & Destroy at the SECOND prompt

from the template. These templates use BetaNews FileForum's SEARCH

ENGINE which should make the apps fairly impervious to their cryptic

URL changes!

 

1) There are two user defined global variables used, download_url={download_url}

(used as an alternative to {url}) and root_folder=YOUR Download Destination

ROOT Folder (for example C:\Latest-Apps) and two local variables, {find}

and {redir1}.

 

2) Ketarin (.0.9.9.22) released January 18, 2009 or later is required as this revision

uses the new 'placeholder' feature implemented by Florian.

 

3) The template for applications NOT HOSTED by SourceForge follows:

 

<?xml version="1.0" encoding="utf-16"?>
<Jobs>
 <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <DownloadBeta>AlwaysDownload</DownloadBeta>
   <DownloadDate xsi:nil="true" />
   <VariableChangeIndicator />
   <CanBeShared>true</CanBeShared>
   <ShareApplication>false</ShareApplication>
   <HttpReferer>http://fileforum.betanews.com/detail/{find}</HttpReferer>
   <Variables>
     <item>
       <key>
         <string>BNFileForumName</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>Textual</VariableType>
           <Regex />
           <TextualContent><placeholder name="BetaNews FileForum Name ['Their Name' for App within App Page URL (i.e. TreeSize)]" /></TextualContent>
           <Name>BNFileForumName</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>find</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=results[ ]found[ ]for.*?A HREF="http://fileforum.betanews.com/detail/).+?(?=")</Regex>
           <Url>http://fileforum.betanews.com/search?s={BNFileForumName:replace:-: :}</Url>
           <Name>find</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>version</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<="main_file_details".*?<[a-z][\d]>.*?)\d[.].+?[ ]??(?=[ ]+[\r\n]|[ ][\r\n]|[\r\n])</Regex>
           <Url>http://fileforum.betanews.com/detail/{find}</Url>
           <Name>version</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>download_url</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=meta[ ]http-equiv="refresh".*?URL=)http://.+?(?=")</Regex>
           <Url>http://fileforum.betanews.com/download/{find}</Url>
           <Name>download_url</Name>
         </UrlVariable>
       </value>
     </item>
   </Variables>
   <ExecuteCommand />
   <Category>[iNSERT CATEGORY HERE]</Category>
   <SourceType>FixedUrl</SourceType>
   <PreviousLocation />
   <DeletePreviousFile>false</DeletePreviousFile>
   <Enabled>true</Enabled>
   <FileHippoId />
   <LastUpdated xsi:nil="true" />
   <TargetPath>{root_folder}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath>
   <FixedDownloadUrl>{download_url}</FixedDownloadUrl>
   <Name><placeholder name="Application Name" /></Name>
 </ApplicationJob>
</Jobs>

 

4) The template for applications HOSTED by SourceForge follows:

 

<?xml version="1.0" encoding="utf-16"?>
<Jobs>
 <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <DownloadBeta>AlwaysDownload</DownloadBeta>
   <DownloadDate xsi:nil="true" />
   <VariableChangeIndicator />
   <CanBeShared>true</CanBeShared>
   <ShareApplication>false</ShareApplication>
   <HttpReferer />
   <Variables>
     <item>
       <key>
         <string>BNFileForumName</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>Textual</VariableType>
           <Regex />
           <TextualContent><placeholder name="BetaNews FileForum Name ['Their Name' for App within App Page URL (i.e. TreeSize)]" /></TextualContent>
           <Name>BNFileForumName</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>find</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=results[ ]found[ ]for.*?A HREF="http://fileforum.betanews.com/detail/).+?(?=")</Regex>
           <Url>http://fileforum.betanews.com/search?s={BNFileForumName:replace:-: :}</Url>
           <Name>find</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>version</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<="main_file_details".*?<[a-z][\d]>.*?)\d[.].+?[ ]??(?=[ ]+[\r\n]|[ ][\r\n]|[\r\n])</Regex>
           <Url>http://fileforum.betanews.com/detail/{find}</Url>
           <Name>version</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>redir1</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=If[ ]your[ ]download.*?)http://.+?(?=")</Regex>
           <Url>http://fileforum.betanews.com/download/{find}</Url>
           <Name>redir1</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>download_url</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=You[ ]are[ ]downloading.*?)http://.+?(?=")</Regex>
           <Url>{redir1}</Url>
           <Name>download_url</Name>
         </UrlVariable>
       </value>
     </item>
   </Variables>
   <ExecuteCommand />
   <Category>[iNSERT CATEGORY HERE]</Category>
   <SourceType>FixedUrl</SourceType>
   <PreviousLocation />
   <DeletePreviousFile>false</DeletePreviousFile>
   <Enabled>true</Enabled>
   <FileHippoId />
   <LastUpdated xsi:nil="true" />
   <TargetPath>{root_folder}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath>
   <FixedDownloadUrl>{download_url}</FixedDownloadUrl>
   <Name><placeholder name="Application Name" /></Name>
 </ApplicationJob>
</Jobs>

 

TO USE THEM:

 

1) Define the user defined global variables under 'Settings' in Ketarin to YOUR preferences

2) Edit the template externally to reflect these modifications (if you chose

not to use mine) and change the <TargetPath> line to YOUR preferences

3) Then:

a) Import it (Obviously)

B) Enter the information needed as requested as a result of the 'placeholder'

prompts... it will ask for the application URL's NAME as described above and

the actual application's name

c) Modify the Category to your preferences

d) Change the 'Advanced Settings' to your preferences

e) VERIFY...VERIFY...VERIFY that ALL regex matches WORK and

MAKE ADJUSTMENTS to them as necessary!

 

****NOTE:

 

IF YOU WISH TO CREATE A TEMPLATE from one of YOUR XML exports:

 

1)It is important that the Guid="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" string be

removed from the XML export to use it as a template, otherwise you'll just keep

overwriting the previously saved application entry that was created using the

imported template

2) The <DownloadDate> line must be changed to <DownloadDate xsi:nil="true" />

3) The <LastUpdated> line must be changed to <LastUpdated xsi:nil="true" />

4) Remove the entire <PreviousLocation> line or replace it with <PreviousLocation />

5) Create 'placeholder' entries as needed (see THIS thread)

 

I hope somebody can use it... feel free to modify it in any way you wish for your use or make suggestions.

 

CybTekSol

Link to comment
Share on other sites

  • 2 weeks later...

CybTekSol - this is wonderful, I have just implemented it with both types and it worked flawlessly (apart from Eventum as this requires a MySQL dev account despite being listed at SF, there will no doubt be others...)

 

A few observations though, (may help others)

 

you state

"root_folder={Insert YOUR download destination ROOT folder here} (for example C:\Latest Downloads)"

 

I found if I included my download path in the {} it was prefixed with the Ketarin install folder BEFORE the path and caused an error. The root_folder=C:\Latest Downloads ie WITHOUT the {} works without problems

 

you also state

2) The <DownloadDate> line must be changed to <DownloadDate xsi:nil="true" />

 

again on my system even after a successful download and export this value is the same/not updated (all others are valid as expected) could this be a bug in Ketarin, can you reproduce/test?

 

and finally the

 

<TargetPath>{root_folder}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath>

 

is an excellent little tip..

 

I will be trying a {root_folder}\{category}\{sub_cat} approach for real geek value lol (TODO: maybe some of these could be integrated into a drop down menu within Ketarin)

 

I suspect it's answered elsewhere but where are the 'share this application on-line' stored/sent do you know?

 

This is really the start of being able to add support for BetaNews File forum to Ketarin if you ask me (Softpedia anyone?)

 

Ready Rolled Example

 

PSPad_editor - NOT SF HOSTED - 1.94MB * Cat is set to 'Developer Tools' you need to create this or change to suit

 

<?xml version="1.0" encoding="utf-16"?>
<Jobs>
 <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
   <DownloadBeta>AlwaysDownload</DownloadBeta>
   <DownloadDate xsi:nil="true" />
   <VariableChangeIndicator />
   <CanBeShared>true</CanBeShared>
   <ShareApplication>false</ShareApplication>
   <HttpReferer>http://fileforum.betanews.com/detail/{find}</HttpReferer>
   <Variables>
     <item>
       <key>
         <string>BNFileForumName</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>Textual</VariableType>
           <Regex />
           <TextualContent>PSPad_editor</TextualContent>
           <Name>BNFileForumName</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>find</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=results[ ]found[ ]for.*?A HREF="http://fileforum.betanews.com/detail/).+?(?=")</Regex>
           <Url>http://fileforum.betanews.com/search?s={BNFileForumName:replace:_: :}</Url>
           <Name>find</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>version</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<="main_file_details".*?<[a-z][\d]>.*?)\d[.].+?[ ]??(?=[ ]+[\r\n]|[ ][\r\n]|[\r\n])</Regex>
           <Url>http://fileforum.betanews.com/detail/{find}</Url>
           <Name>version</Name>
         </UrlVariable>
       </value>
     </item>
     <item>
       <key>
         <string>download_url</string>
       </key>
       <value>
         <UrlVariable>
           <VariableType>RegularExpression</VariableType>
           <Regex>(?<=meta[ ]http-equiv="refresh".*?URL=)http://.+?(?=")</Regex>
           <Url>http://fileforum.betanews.com/download/{find}</Url>
           <Name>download_url</Name>
         </UrlVariable>
       </value>
     </item>
   </Variables>
   <ExecuteCommand />
   <Category>Developer Tools</Category>
   <SourceType>FixedUrl</SourceType>
   </ PreviousLocation>
   <DeletePreviousFile>false</DeletePreviousFile>
   <Enabled>true</Enabled>
   <FileHippoId />
   <LastUpdated xsi:nil="true" />
   <TargetPath>{root_folder}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath>
   <FixedDownloadUrl>{download_url}</FixedDownloadUrl>
   <Name>PSPad Editor</Name>
 </ApplicationJob>
</Jobs>

Link to comment
Share on other sites

A few observations though, (may help others)

 

you state

"root_folder={Insert YOUR download destination ROOT folder here} (for example C:\Latest Downloads)"

 

I found if I included my download path in the {} it was prefixed with the Ketarin install folder BEFORE the path and caused an error. The root_folder=C:\Latest Downloads ie WITHOUT the {} works without problems

1) I did not intend for the 'curly brackets' to be included in the definition of the variable... just that its contents was what defined the variable. Sorry for the confusion. I will probably remove them from my template posts as I can see where they could be misleading users. ;)

 

Again on my system even after a successful download and export this value is the same/not updated (all others are valid as expected) could this be a bug in Ketarin, can you reproduce/test?
2) Great observation... I had not picked up on this. Out of my 687 apps, the <DownloadDate> data is only included in 7 of them in my 'export all' xml! I don't know if it's a bug or not... sure seems like it. Florian, do you have any ideas on this issue? Could it be related to inserting <DownloadDate xsi:nil="true" /> in the templates? It appears to me that Ketarin is no longer storing this data as the last entry I have is 12/31/08...

 

I suspect it's answered elsewhere but where are the 'share this application on-line' stored/sent do you know?
I'll let Florian answer this one... I'm not sure, but I suspect to a database on one of his servers.

 

(Softpedia anyone?)
3) I have developed a template for Softpedia but chose not to post it as their site has so many inconsistencies it was almost impossible to refine a 'universal' regex scrape. At best, I got to a 50-70% reliabilty... not quite ready for the masses. However, we would welcome that addition if someone can donate the time and effort to achieve a more reliable version. I had to lighten up as my wife voiced her opinion regarding my long hours. :(
Link to comment
Share on other sites

1. The root_folder=C:\Latest Downloads ie WITHOUT the {} works without problems

- Yeah, that's the way to go. If you have problems with spaces, remember you can use "" arround path, 8.3 shortfilenames or DOS environment variables.

 

2.Could it be related to inserting <DownloadDate xsi:nil="true" /> in the templates?

- I've noticed this for a while, but if you use Filehippo "template" <DownloadDate> is not updated either, so I would say this is not a template problem. What is updated is the <LastUpdated> tag, e.g., <LastUpdated> 2009-02-03T18:26:28.7522851</LastUpdated>, and this is the data Ketarin displays in main window. It doesn't seem to be causing any problems.

 

3. <TargetPath>{root_folder}\{category}\{appname:replace: :_}_v{version:replace: :_}.{url:ext}</TargetPath> is an excellent little tip...

- This is highly flexible, you can do anything with it. Some of my target paths:

{target}\{category}\

{target}\{category}\{appname}\

{target}\{category}\{appname}_{version}.{url:ext}

{target}\{category}\{url:basefile}.{url:ext}

{target}\{category}\{url:basefile}_{version:replace: :.}.{url:ext}

{target}\{category}\wlx_{appname}_{version}.{url:ext}

{target}\{category}\wcx_{appname}_{version:replace: :_}.{url:ext}

{target}\{category}\{appname}_{version:replace:Vwr_Port:.}.{url:ext}

{target}\{category}\{appname}_{version:replace: (Build :.}.{url:ext}

{target}\{category}\{appname}_{version:replace:PortablePreRel:.}.{url:ext}

 

4. I got to a 50-70% reliabilty...

- Workaround is to import template and do some minor tweaks to regex prior to update. This works for me even in sites where we can't get 100% reliability. I am yet to create a template I'm not using. =)

Link to comment
Share on other sites

Workaround is to import template and do some minor tweaks to regex prior to update. This works for me even in sites where we can't get 100% reliability. I am yet to create a template I'm not using. =)
AGREED! I ALWAYS add a new app using one of my templates, even for generic 'one app from a site' additions using my 'generic' template. It reduces redundant entry of data and gives me a starting point from which a minimal amount of tweaking is required! ;)
Link to comment
Share on other sites

  • 2 weeks later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.