SF.net, wget, .. ¿

[u2111]

Hi,

 

I tried today to use (great) Ketarin to download from SourceForge using auto mirror, and after having tried lots of configuration (url, referer, user agent), I'm unable to make this work in Ketarin.

 

If I try using wget, everything is fine :

Pyxis% wget 'http://downloads.sf.net/mpc-hc/MPC-HomeCinema.1.4.2499.0.x86.exe?download'
--2011-01-15 18:52:14--  http://downloads.sf.net/mpc-hc/MPC-HomeCinema.1.4.2499.0.x86.exe?download
Resolving downloads.sf.net... 216.34.181.129
Connecting to downloads.sf.net|216.34.181.129|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://downloads.sourceforge.net/project/mpc-hc/MPC%20HomeCinema%20-%20Win32/MPC-HC%20v1.4.2499.0_32%20bits/MPC-HomeCinema.1.4.2499.0.x86.exe?download= [following]
--2011-01-15 18:52:15--  http://downloads.sourceforge.net/project/mpc-hc/MPC%20HomeCinema%20-%20Win32/MPC-HC%20v1.4.2499.0_32%20bits/MPC-HomeCinema.1.4.2499.0.x86.exe?download=
Resolving downloads.sourceforge.net... 216.34.181.59
Connecting to downloads.sourceforge.net|216.34.181.59|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://freefr.dl.sourceforge.net/project/mpc-hc/MPC%20HomeCinema%20-%20Win32/MPC-HC%20v1.4.2499.0_32%20bits/MPC-HomeCinema.1.4.2499.0.x86.exe [following]
--2011-01-15 18:52:15--  http://freefr.dl.sourceforge.net/project/mpc-hc/MPC%20HomeCinema%20-%20Win32/MPC-HC%20v1.4.2499.0_32%20bits/MPC-HomeCinema.1.4.2499.0.x86.exe
Resolving freefr.dl.sourceforge.net... 88.191.250.132
Connecting to freefr.dl.sourceforge.net|88.191.250.132|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5165571 (4.9M) [application/x-msdownload]
Saving to: `MPC-HomeCinema.1.4.2499.0.x86.exe.1'

100%[=========================================================================>] 5,165,571    873K/s   in 6.4s

2011-01-15 18:52:22 (782 KB/s) - `MPC-HomeCinema.1.4.2499.0.x86.exe.1' saved [5165571/5165571]

 

HTTP Headers gives all redirections :

http://downloads.sf.net/mpc-hc/MPC-HomeCinema.1.4.2499.0.x86.exe'>http://www.webrankinfo.com/outils/header.php?url=http://downloads.sf.net/mpc-hc/MPC-HomeCinema.1.4.2499.0.x86.exe

http://www.webrankinfo.com/outils/header.php?url=http://downloads.sourceforge.net/project/mpc-hc/MPC%2520HomeCinema%2520-%2520Win32/MPC-HC%2520v1.4.2499.0_32%2520bits/MPC-HomeCinema.1.4.2499.0.x86.exe

 

And even in Ketarin, the file seems downloaded when I put "http://downloads.sf.net/mpc-hc/MPC-HomeCinema.1.4.2499.0.x86.exe" in a "Content URL" and the referer is set to "http://sourceforge.net" : the content that appear after a few second is "MZP" (executable header I suppose ?).

But when updating, Ketarin says

MPC The downloaded file is not a binary file type (text/html; charset=utf-8). Possibly there is an error page. Status code: 200 (OK) (http://downloads.sf.net/mpc-hc/MPC-HomeCinema.1.4.2499.0.x86.exe)

 

So : what am I missing ?

I saw that there's a new way to download from SF using credentials (though it's not documented yet), but this seems useless in this case.

 

Any help or explanation about why it's not working ?

 

Thanks in advance,

Regards,

Alexandre

[shawn]

Hi, Alex,

 

What you're seeing is how SF gives certain clients a pass on how URLs operate. The easiest solution, assuming your app profile works otherwise, is to set the "user agent" option to wget on the "advanced settings" tab of the app profile. Using "wget" alone may work, but I always stuff it with a full version, too: "wget/1.9"

 

Good luck!

[Tomorrow]

Alex i just made a very simple xml.Try and see if it works for you.I used OVH as it's a France based SF mirror so should be the best speed for you

 

<?xml version='1.0' encoding='utf-8'?>
<Jobs>
 <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="836246b7-6844-4bf3-b472-b1a2b32d8447">
   <SourceTemplate><![CDATA[]]></SourceTemplate>
   <WebsiteUrl>http://sourceforge.net/projects/mpc-hc/files/MPC%20HomeCinema%20-%20Win32/</WebsiteUrl>
   <UserAgent />
   <UserNotes />
   <LastFileSize>0</LastFileSize>
   <LastFileDate xsi:nil="true" />
   <IgnoreFileInformation>true</IgnoreFileInformation>
   <DownloadBeta>Avoid</DownloadBeta>
   <DownloadDate xsi:nil="true" />
   <CheckForUpdatesOnly>false</CheckForUpdatesOnly>
   <VariableChangeIndicator />
   <CanBeShared>true</CanBeShared>
   <ShareApplication>false</ShareApplication>
   <ExclusiveDownload>false</ExclusiveDownload>
   <HttpReferer />
   <SetupInstructions>
     <SetupInstruction xsi:type="StartProcessInstruction">
       <EnvironmentVariables />
       <FileName>{file}</FileName>
       <Parameters>/verysilent</Parameters>
       <WaitForExit>true</WaitForExit>
     </SetupInstruction>
   </SetupInstructions>
   <Variables>
     <item>
       <key>
         <string>version</string>
       </key>
       <value>
         <UrlVariable>
           <RegexRightToLeft>false</RegexRightToLeft>
           <VariableType>StartEnd</VariableType>
           <Regex />
           <Url>http://ovh.dl.sourceforge.net/project/mpc-hc/MPC%20HomeCinema%20-%20Win32/?C=N;O=D</Url>
           <StartText>Parent Directory</a></td><td> </td><td align="right">  - </td></tr>
<tr><td valign="top"><img src="/icons/folder.gif" alt="[DIR]"></td><td><a href="MPC-HC%20v</StartText>
           <EndText>_32%</EndText>
           <Name>version</Name>
         </UrlVariable>
       </value>
     </item>
   </Variables>
   <ExecuteCommand>"{file}" /verysilent</ExecuteCommand>
   <ExecutePreCommand />
   <ExecuteCommandType>Batch</ExecuteCommandType>
   <ExecutePreCommandType>Batch</ExecutePreCommandType>
   <Category>Video Player</Category>
   <SourceType>FixedUrl</SourceType>
   <PreviousLocation />
   <DeletePreviousFile>true</DeletePreviousFile>
   <Enabled>true</Enabled>
   <FileHippoId />
   <LastUpdated xsi:nil="true" />
   <TargetPath>{Apps x86}</TargetPath>
   <FixedDownloadUrl>http://ovh.dl.sourceforge.net/project/mpc-hc/MPC%20HomeCinema%20-%20Win32/MPC-HC%20v{version}_32%20bits/MPC-HomeCinema.{version}.x86.exe</FixedDownloadUrl>
   <Name>Media Player Classic Home Cinema x86</Name>
 </ApplicationJob>
</Jobs>

[u2111]

Thanks !

I only tried on this app, but using "Ketarin/{ketarin-version}" with ketarin-version as 1.6.0.400 works too.

Is there any way to set a global user agent (not using templates) ? Shouldn't Ketarin prefer this solution (if not already) ?

There were a lot of complicated and (most of them) not working solutions, and I missed the simple user agent solution .

 

after having tried lots of configuration (url, referer, user agent)

"Real" UserAgent like "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.13 (KHTML, like Gecko) Chrome/9.0.597.47 Safari/534.13" isn't working though (that's what I tried).

 

Edit: Tomorrow thanks too.

Edited January 15, 2011 by Alex131089

[Omniferum]

Don't quite get why people make sourceforge so confusing in general. Just adding anything in the user agent field still works for me, mine is

 

(!)

 

However i've written

 

poo

 

in there and it still works fine. I have a fully functional template running around somewhere.

[u2111]

Yep, actually when you know it, it is really simple. This case is even in the troubleshooting page of the wiki, but maybe it should explicitly point out SF.net.

[Tomorrow]

I think i'll post a guide how to make SF downloads the way i do.It's really simple and requires no referers or user agents.Max 2-3 variables and generally very fast as all data is taken from mirror's plain text page.

[u2111]

Does your solution rely on a single & defined mirror ?

[Tomorrow]

Does your solution rely on a single & defined mirror ?

Yes.I tend to use the biggest mirrors.Smaller ones could have more downtime.

For my downloads i use either Mesh(Germany) or Sunet(Sweden).

[u2111]

I'm not sure about this, but I already noticed (and someone else on the forum) that all files are not on all mirors, that's why I left this solution (and for the reliability).

[Tomorrow]

That's why i use the biggest mirrors.So far they have always had all files i needed.Besides during new app creation i can already see if the mirror has the files or not.