shawn

Hi, Flo! The recent thread with all the minor feature requests appears to have been lost in the recent db corruption, so here's a thread intended to recreate it. # variables support in user agent (ex: assign "{ie32}", which is defined as a global variable) # contextual variables support within global variables (ex: {version} could be parsed within a global variable) # iterative/heirarchial variables support in global variables (ex: {sfprefix} could contain {sfuser} and {sfpass} as "sf://{sfuser}:{sfpass}@") # variable structures within the DOWNLOAD url should all be parsed and if they do not exist, be replaced with an empty string. This one requires a more detailed example, I think. Imagine the following download URL: {sfprefix}{downloadurl} On my computer this would be parsed to something like this: sf://user:pass@http://download.sourceforge.net/example/files/example/1.2.3/example-123.exe But should be parsed to something like this on OTHER computers that don't have "{sfprefix}" defined: http://download.sourceforge.net/example/files/example/1.2.3/example-123.exe # {property:HttpReferer} is no longer working in custom columns. (actually, it looks like all {property:*} values are now broken references) # cosmetic/language issue - replace "updating" and "update" with "downloading" and "download" to indicate the difference in meaning now that the installation functionality is included. # add a link to the Forum in the Help menu. # add ability to purge the log at will (especially useful when developing a new app profile) # add ability to right-click on a cell and "copy contents" # add more custom columns (I could readily use five custom columns) # add ability to enable inline editing within a custom column (if the custom column is mapped to a user-assigned variable, this would enable inline editing of that variable within the [variables] value for the specific app profile) # add ability to style custom column value/cell based on a regex pattern (ex: "bg:red:(^$)" and "fg:orange:([^\d\.\-]+)") That's all I remember. Anyone else have others?

Info here: http://ketarin.canneverbe.com/forum/viewtopic.php?pid=4535#p4535

Don't take this the wrong way, but addressing the issues with assigning header values this way is very funky, and I'm concerned about proper encoding. For example, cookies have multiple values separated by ampersands (potentially multiple on the same line) and carriage returns within the HTTP protocol. How do we accomplish this with the new httpx protocol? Will the ampersands be URL-encoded automatically, or do we need to do that ourselves? Do we need to include multiple "cookie:name=value" parameters? I really feel that this implementation is going to be a burden to maintain, and that your best shot is to use a datagrid, as you've done with global variable assignments and post data. Preferably with the ability to sort values in order of assignment, so the headers can be composed to comply with certain user-agent profiles (a common anti-spidering/anti-caching method is to enforce header sequence and conformity for popular UA's, and forbid other UA's). A datagrid that enabled us to assign both the sequence, form and structure of these headers would allow us to correctly obtain content from these strict sources.

Based on the syntax, I assume it's written such that what you're doing is invoking a method of the "app" object. That is, the string itself may not be exposed, but this method simply executes it ensuring that the proper executor (batch vs c#) is called.

Feature request: link to the forum from the Help menu.

Tomorrow, here's a working uTorrent profile: <?xml version='1.0' encoding='utf-8'?> <Jobs> <ApplicationJob xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Guid="98e52edb-3990-43de-b09d-06c5515c9d84"> <SourceTemplate><![CDATA[]]></SourceTemplate> <WebsiteUrl>http://www.utorrent.com/</WebsiteUrl> <UserAgent>{ie32}</UserAgent> <UserNotes /> <LastFileSize>328568</LastFileSize> <LastFileDate>2010-10-16T03:55:24.9037236</LastFileDate> <IgnoreFileInformation>false</IgnoreFileInformation> <DownloadBeta>Default</DownloadBeta> <DownloadDate>2009-05-20T22:20:53</DownloadDate> <CheckForUpdatesOnly>false</CheckForUpdatesOnly> <VariableChangeIndicator>updated</VariableChangeIndicator> <CanBeShared>false</CanBeShared> <ShareApplication>false</ShareApplication> <ExclusiveDownload>false</ExclusiveDownload> <HttpReferer /> <SetupInstructions /> <Variables> <item> <key> <string>swebsite</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>http://www.utorrent.com/downloads</TextualContent> <Name>swebsite</Name> </UrlVariable> </value> </item> <item> <key> <string>schangelog</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>"([^"]+)">Changelog</Regex> <Url>http://www.utorrent.com/downloads/?</Url> <EndText>">Changelog</EndText> <Name>schangelog</Name> </UrlVariable> </value> </item> <item> <key> <string>snotes</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent /> <Name>snotes</Name> </UrlVariable> </value> </item> <item> <key> <string>updated</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>\(US\)\s*-\s*(\S+\s*\d+,\s*\d+)<</Regex> <Url>http://www.utorrent.com/downloads/?</Url> <Name>updated</Name> </UrlVariable> </value> </item> <item> <key> <string>aversion</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>/([\d\.]+)/[^"]+exe</Regex> <Url>http://www.utorrent.com/downloads/complete?os=win</Url> <Name>aversion</Name> </UrlVariable> </value> </item> <item> <key> <string>bversion</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>Version ([\d\.]+\s\(build\s*[\d]+)\)</Regex> <Url>{schangelog}&</Url> <Name>bversion</Name> </UrlVariable> </value> </item> <item> <key> <string>url</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>RegularExpression</VariableType> <Regex>http://download.utorrent.com/[\d\.]{3,}/utorrent.exe</Regex> <Url>http://utorrent.com/downloads/complete?os=win</Url> <Name>url</Name> </UrlVariable> </value> </item> <item> <key> <string>version</string> </key> <value> <UrlVariable> <RegexRightToLeft>false</RegexRightToLeft> <VariableType>Textual</VariableType> <Regex /> <TextualContent>{bversion:regexreplace:[^\d\.]+:.}</TextualContent> <Name>version</Name> </UrlVariable> </value> </item> </Variables> <ExecuteCommand /> <ExecutePreCommand /> <ExecuteCommandType>Batch</ExecuteCommandType> <ExecutePreCommandType>Batch</ExecutePreCommandType> <Category>Internet</Category> <SourceType>FixedUrl</SourceType> <DeletePreviousFile>true</DeletePreviousFile> <Enabled>true</Enabled> <FileHippoId /> <LastUpdated>2010-10-16T03:55:24.9037236</LastUpdated> <TargetPath>.\{category}\{appname:regexreplace:([\s\t\r\n\-\\&\/]+):_}-{version:replace: build :.}.exe</TargetPath> <FixedDownloadUrl>http://download.utorrent.com/{aversion}/utorrent.exe</FixedDownloadUrl> <Name>uTorrent</Name> </ApplicationJob> </Jobs>

Yay!

I think this would be great, especially now that Google Reader has abandoned webpage monitoring. I think Omni is more after forking Ketarin to provide this functionality as an alternative tool, not doing this (webpage monitoring) in the core Ketarin. I know I would keep them separate in every way, even with the portability - they just don't belong in the same app. I've not used WebMon, but I'll check it out.

In which situation would you do that? It's not necessarily a situation, but a user behavior. Many people *expect* to see it, so they get nervous when it isn't there. Personally, I could care less if an "apply" button exists, but it would be nice if there were a "dirty" indicator in the title bar (just an asterisk or something to indicate the changes haven't yet been saved).

Yep. This, along with other meta-data, such as a "dependencies" field which would include such things as global variable assignments, sourceforge account, 7z or ahk. each of these should be listed as a dependency of the app profile. Dependencies could be detected automatically from the XML. Also, I think you really need to settle on naming conventions. I get all hot and bothered every time I try to describe a single app profile. What's the correct terminology? The profile itself isn't actually the "app" so calling it an "app" really bothers me. And what do you call the XML exports for one or many profiles? Me, too! It would seriously minimize the amount of crap I have to prefix within many profile fields. Maybe global vars (or even, maybe, ALL variables) should be recursively processed in order to process all variables, even those that are built dynamically during the variable assignment process.

the problem is when you need to assign a useragent to a bunch of records. currently, the only way is to manually assign it to each one as a static string. when i need to go change that (which I am sure I inevitably will), I have to find a way to first locate, then update all those records. it would be far easier to use global variables to assign the UA string you want, then just insert something like {curl}, {ie64} or {chrome} in the UA box. if I want to update the UA I have been using for chrome, I just have to edit the value in one place - global variables (where this kind of global option belongs).

yep. that would be cool. i'd LOVE to be able to put links on my sites to enable users to monitor my apps (or other apps) with Ketarin. nifty. bug report (IMHO): the "useragent" field doesn't parse variables (contrast with the "referer" which does appear to parse vars).

this is possible in the current beta, available here: http://ketarin.canneverbe.com/forum/viewtopic.php?id=557 in a specific app profile, click the "advanced settings" tab. Here's a few UA strings to start you out: ie32 Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C) ie64 Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Win64; x64; Trident/4.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Media Center PC 5.0; SLCC1; Tablet PC 2.0; .NET4.0C) firefox Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.10) Gecko/20100914 Firefox/3.6.10 opera Opera/9.80 (Windows NT 6.1; U; en) Presto/2.6.30 Version/10.62 chrome Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3 wget wget/1.9+cvs-stable+(red+hat+modified) curl pycurl/7.18.2

No, i don't think an installer is a good idea UNLESS you're going to map a filetype/extension to the program. As it is, I *love* the simplicity and portability of the app, which would only be reduced by the inclusion of an installer. I do understand that it still wouldn't REQUIRE an installer, and that those of us that really prefer the portability of Ketarin could continue to use the simple "unzip" format -- but including an installer would likely conduse some people about the applications function and portability. However, I do like the idea of mapping a certain filetype to Ketarin. For example, when I open a URL with a content-type of "text/ketarin" (which is simply an XML formatted file like an app dump file) which opens directly as an import in Ketarin, possibly with a potential new element to provide the ability to immediately download+install+save the app profile, download only + save the app profile, or either of those two but operating as a temp, without storing the app profile permanently within Ketarin. On that, it'd be kinda nifty to be able to build the contents of the xml file in such a way as to refer to external files to import, locally or by URL (similar to the format of sitemaps)

winscp does: http://winscp.net/eng/index.php

make sure you're quoting the URL, in case it has an ampersand in it. ampersand in a command line is treated as a new command (kinda like a carriage return). most url's have ampersands, so anytime you're writing it to a command line, you have to quote it to avoid that behavior. start "" /wait curl.exe -O "{preupdate-url}" & exit 2

What you're actually seeing is a special behavior applied to SF for the curl and wget engines. Because SF is used as a distribution network for Linux libraries and other packages, allowing curl & wget a direct "pass" to the real files, effectively bypassing the whole framed and redirection distribution network. I really should have realized that before when I was creating the template, 'cause it could save a lot of time just by adding a wget or curl UA header to the SF apps. Sigh.

Begin/End only works on certain HTML, too, though - so that's not really a good justification to exclude XSL. Even Regex is only useful if you can find a *unique* pattern that works for the specific data you're trying to capture. XSL is really a useful tool when it *can* work, and if it becomes an option within Ketarin, it could be very useful to those geeks like me that will find ways to make it work. Also, if it's implemented as a function, instead of only a capture type, we would be able to utilize it with well-formed data once we've mostly sanitized it using other methods. That said, I would really like the simplicity of using the IE "slice" method of simply tagging the returned region with an anchor reference in the URL.

Maybe I was a little too excited. All SF apps need updated again, and again, and again... 10/12/2010 2:30:43 AM: 7-Zip cmd: Update required, file modified dates do not match ... 10/12/2010 2:30:45 AM: 7-Zip Portable: Checking if update is required... 10/12/2010 2:30:45 AM: 7-Zip Portable: Update required, file modified dates do not match The SF behavior is broken in this sense. The anticipated behavior is that it *should* be updating the "file modified date" on local files to ensure that the timestamps are accurate with the remote (and to better enable sorting and stuff). Meanwhile, I can use a manually-assigned "version" variable check for changes, but this would have to be updated on all my SF apps, which would be a PITA to add and remove.

Yay! I added a global variable "sfprefix" with my SF login credentials in this format: sf://username:password@ ...then added that variable to prefix the "download" field for all my SF downloads, as so. {sfprefix}{dl} {sfprefix}http://downloads.sourceforge.net/project/xbmc/XBMC%20for%20Windows/{tag}%20-%20{version}/xbmc-{version}.exe {sfprefix}http://downloads.sourceforge.net/project/dqsd/dqsd/{filename} They ALL work! Some modification to my apps were required for a couple apps that returned the "Source file on server is empty (ContentLength = 0)" error. This indicates that the URL is *wrong* for these files, so correcting the regex capture to eliminate extraneous characters resolved it in every case.

Anything that uses recursive subfolders, like portableapps or winmerge.

Thanks, Gozi, but I'm actually more interested in obtaining a random source link from a regex-matching pattern from an HTML result. Such as the download source lists on MySQL, IrfanView, and similar sites where the list uses something much different than an appended number to identify the different sources.

The other option is to use something like this: {myvar:regex:\-([^\-]+)$} You can also regex pretty much any other variable to extract contents from it using that function...however, I would **really** love to have the ability to perform an XSL transform or use XSL selectors to obtain more specific info from HTML (like "slices" and such).

Unfortunately, that method won't work for me on many of my SF sources.

ours are comparable , but if it's not an issue, it's not an issue.