Requests for 1.0

[Dante]

I consider the following to be requirements for a fully usable version 1.0:

• Mirror support: more URLs that represent the same resource
  
• Group support: an application is usually part of a group, which represents the same product in a different language, packing solution (MSI,EXE,ZIP),type (installation,addons,sourcecode,JAD descriptor,file updates)
  
• A fully functional regexp engine for HTML parsing
  
• SHA-1 signatures, maybe even signed with a trusted certificate (I fear the recently revealed MD5 vulnerabilities)
  

Edited January 3, 2009 by Dante

[floele]

I doubt that all your requests will make it into 1.0. In particular, I don't see a need for this grop thing.

 

Could you explaon the last two a bit more? So far I'd say that there already is a fully functional Regex engine. What would I do with SHA1/MD5 signatures?

[Stalker]

What would I do with SHA1/MD5 signatures?

I'm guessing that there could be a variable which retrieves the signature which is verified by Ketarin upon download.

[Dante]

Group support for v2

I want to keep an installer for the Opera browser. This would include the Opera Mini version for MIDP phones (2 files), the latest desktop English versions for Win32 and Ubuntu64, and the latest Opera Mobile version for my ARM PocketPC. Right now, I have to setup an entry for each of these. Ideally from the online database I would select "Opera Browser", then check/select the needed items. (Right now there are lots of conflicting names for similar applications, but that's a separate issue)

 

RegExp engine

I would need the number between the brackets of this RegExp: <option value="([^<]*)" selected="selected">

I have no idea how to get it inside a variable, is there a tutorial for the RegExp expressions?

 

SHA1/MD5 signatures

I need to make sure the downloaded files are genuine. FileHippo naturally offers "no warranties whatsoever". Most application sites offer a MD5 signature (whose URL could be part of the application group , sometimes PGP signed. I would want Ketarin to double-check this after download: that what is transferred is what is expected. The (signed) hashes could be part of the database.

Edited January 3, 2009 by Dante

[floele]

@Groups: You might want all Opera packages, others might not want them. In particular, it would never work for the online database, since most users submit "unpolished" data (as you have probably noticed, duplicate application names and the like).

 

@RegExp: You can already do that I'd say. If you use () within a regular expression, Ketarin will use the value matched within these. If you do not use any (), it will use the full matched value.

 

@SHA1/MD5: They should not be part of the online database (if that is what you mean), because they could be faked on the client side when submitting the applications. My suggestion would be to implement special variables. Say, if you define a custom variable called "md5", Ketarin will compare the value of the variable and the actual value of the file. The same goes for "sha1".

[Dante]

Thanks, I'll look into RegExp/MD5 var.

[floele]

Well, the MD5 stuff is not implemented yet. It's just an idea

[floele]

...in other words, I'd appreciate some comments on my suggestion.

[Stalker]

For MD5/SHA1 that was what I was talking about. For FileHippo it could be automatic because they already provide MD5 on "Technical" tab.

While we're at it please note that not all sites provide the has as a single string. Some, like KeePass prefer to get creative with spaces.

Edited January 3, 2009 by Stalker

[Dante]

The problem remains to trust a HTTP stream that contains the hash. As I understand it, Ketarin ignores certificate errors even if the source of the MD5 message could be trusted as untampered, i.e. coming from a HTTPS URL. Like it is now, the MD5 FileHippo reports still has to be compared with the application author's hash (I'm ignoring MITM/DNS poisoning attacks while getting the two hashes)

 

Regarding the suggestion, I don't see MD5 as being a global/local variable; the checksum mechanism should be active for each download: Ketarin should verify the integrity of the download each time; for this, each application should include a trusted URL to the hash (e.g. PGP-signed MD5)

Edited January 3, 2009 by Dante

[Stalker]

for this, each application should include a trusted URL to the hash (e.g. PGP-signed MD5)

And where would you find such a source ? Of aprox. 70 applications in my Ketarin db only those on FileHippo and then a one or two have some sort of hash information available.

Basically all these checksum verifications, even if we consider that we could get correct checksums in 100% of cases, would not guarantee you that you have downloaded the same package the author of the application intended for you to download. It makes sure that you have downloaded the same package, more or less, that the server served to you.

Edited January 3, 2009 by Stalker

[floele]

Indeed, I'd not so much like MD5 as security measure but rather to prevent corrupted downloads. If you need to retrieve an MD5 hash securely, you'll need an HTTPS connection. And currently, I don't know any application which offers such a secure hash retrieval.

 

And yes, Ketarin ignores bad certificates. Since some sites have bad certificates but you still might want to download from such sites, this is necessary. All in all I don't think that Ketarin is suited for secure downloads, nor do I think that Ketarin should be responsible for that.

[Dante]

Right. I guess separately downloading and checking the ASC/MD5/SHA will have to do for now.