NetNewsWire script: Subscribe to full-text feed of current subscription

Update Dec 2010: it looks like FullTextRSS is down. I’ll leave this up, but be forewarned: it probably won’t work. FiveFilters and WizardRSS provide similar services, so you may want to look there.

Here’s a script that attempts to subscribe to a full-text feed of the current subscription in NetNewsWire. It does this using EchoDittoLabs’ excellent FullTextRSS service.

To use, select a headline or subscription title in NetNewsWire and run the script. The full-text feed will appear in your top-level items.

Thanks to harvey.nu for the URL encoding routine. The script worked without the routine but it seemed safer to include it.

[Updated 2/22 with change suggested by Pascal.]

(*
Fulltextrss.scpt v 0.1b

Attempts to subscribe to the current subscription via
EchoDittoLabs’ FullTextRSS service (echodittolabs.org/fulltextrss)

Contains no error checking; use at your own risk

Dan Byler dbyler@gmail.com
*)


property fulltextpre : "http://labs.echoditto.com/projects/fulltextrss/?url="

tell application "NetNewsWire"
        if exists selectedHeadline then
                set this_headline to selectedHeadline
                set stdfeed to RSS URL of subscription of selectedHeadline
        else if exists selectedSubscription then
                set stdfeed to RSS URL of selectedSubscription
        end if
        try
                set theTextEnc to my urlencode(stdfeed)
                set fulltextfeed to fulltextpre & theTextEnc
                set theresult to subscribe to fulltextfeed
        on error
                display dialog "Oops—something went wrong."
                return
        end try
end tell

— urlencode routine taken from http://harvey.nu/applescript_url_encode_routine.html
on urlencode(stdfeed)
        set theTextEnc to ""
        repeat with eachChar in characters of stdfeed
                set useChar to eachChar
                set eachCharNum to ASCII number of eachChar
                if eachCharNum = 32 then
                        set useChar to "+"
                else if (eachCharNum ≠ 42) and (eachCharNum ≠ 95) and (eachCharNum < 45 or eachCharNum > 46) and (eachCharNum < 48 or eachCharNum > 57) and (eachCharNum < 65 or eachCharNum > 90) and (eachCharNum < 97 or eachCharNum > 122) then
                        set firstDig to round (eachCharNum / 16) rounding down
                        set secondDig to eachCharNum mod 16
                        if firstDig > 9 then
                                set aNum to firstDig + 55
                                set firstDig to ASCII character aNum
                        end if
                        if secondDig > 9 then
                                set aNum to secondDig + 55
                                set secondDig to ASCII character aNum
                        end if
                        set numHex to ("%" & (firstDig as string) & (secondDig as string)) as string
                        set useChar to numHex
                end if
                set theTextEnc to theTextEnc & useChar as string
        end repeat
        return theTextEnc
end urlencode

Download a copy here.

On Evernote

Let’s talk about the elephant in the room.

I am, of course, referring to Evernote, a tool that’s designed to remember everything you throw at it – then provide access to your information from virtually anywhere.

If you’re unfamiliar with the tool, this video will give you a quick picture of what it does:

Despite the obvious awesomeness of its sync/access capabilities, earlier iterations of Evernote failed my acquired-information management criteria on numerous counts:

  • It couldn’t accept basic documents like PDFs (flexibility fail)
  • It wasn’t scriptable (extensibility fail)
  • You couldn’t import/export data en mass (openness fail)

But the Evernote team have been hard at work, and with the recent addition of scriptability and an API, it’s worth a serious second look.

Accessibility: 9/10

Accessibility is undoubtedly where Evernote shines: you can access your data on the web client, your Mac or PC, or your iPhone or Windows Mobile device – and it all stays synchronized. Ergo, you still have your data when the network goes down. (The mobile Evernote clients act more as search/input portals into your Evernote data, though the latest iPhone version now stores your “favorites” locally so you can access critical notes offline. Also, if you prefer to keep some data private, you can selectively opt out of synchronization.)

Besides being Evernote’s killer feature, accessibility is also the cornerstone of the product’s business model: Evernote itself is free, but if you need more than 40MB/month you’ll need to upgrade to the premium version ($5/month or $45/year). The most exciting Evernote use cases will probably be mashups that use its API, which means increased bandwidth needs – and, therefore, subscriptions. (N.B.: Evernote’s API documentation describes the bandwidth elements as the “accounting structure”.)

Flexiblity: 4/10

Evernote accepts text/RTF files, images, web clippings, PDFs, and audio notes. To get files in, you can drag-and drop files onto the desktop client, email them to a special Evernote email address, or use the bookmarklet to clip content directly from any web browser. Handily, if you have part of a web page selected, the bookmarklet just saves the selection. Read: Evernote is most in its element when used for web clippings.

In addition, images can be snapped from your webcam or iPhone camera. More on this later.

Unfortunately, Evernote can’t handle many common document types, including Word documents (though RTF documents work passably). Most other filetypes (mind maps, outliner documents, etc) are out of the picture as well.

In the Mac client, the built-in content editor is little more than a glorified text editor. Text formatting is limited to font/size, bold/italic/underline, and alignment, though you can also attach images. (Not to mention the insulting font selection, which includes Arial but not Helvetica – shame!) The Windows client also provides some drawing tools (useful with a tablet PC), a broader font selection, and outlining functions. Update 11/10/08: Version 1.1.6 for Mac introduces orderd/unordered lists and tables.

Scalability: 6/10 (est.)

I haven’t thrown a tremendous amount of data at Evernote yet, so it’s unclear how performance is affected by a large data set. (I was hoping to put it to the test via the Delicious bookmark import, but Evernote just imported the bookmarks as links, rather than scraping the bookmarks’ targets.)

From a user interface perspective, scalability may be a problem. Items are accessed by browsing (by tags and metadata) as well as search. Aside from the ability to use multiple “notebooks”, there is no standard hierarchical organization. Users with a large number of tags or documents might become frustrated by this.

Searchability: 8/10

Basic search functions are solid, though generally unremarkable (no regular expressions, no advanced operators). You can combine search terms with tag filters.

In addition, Evernote has a couple search tricks up its sleeve:

  • Images pass through Evernote’s OCR engine when synchronized, turning image text into searchable data. This even works, to a large extent, on handwriting – slick!

  • Evernote metadata includes standard text tags as well as optional location data, so you should be able to search by location as well as content

Extensibility 7/10

The Mac client now includes a basic AppleScript dictionary, which allows for integration with other apps. No content-level scripting, but the most important feature – note creation – is available.

One example of extensibility in action: Justin adapted my NetNewsWire » DEVONthink batch archiving script to work with Evernote, including tagging and notebook selection. Check it out here.

In addition, the Evernote API provides full access to Evernote data. I’m not aware of any Evernote-based applications yet, though Pelotonics is planning some level of integration. If useful integration emerges, this is will be a big win for users.

Openness 4/10

Disappointingly, despite claiming export options as a feature, Evernote maintains a tight grip on your data. Need to send a file to your colleague? Forget drag-and-drop: you’ll need to go through Evernote’s export or email functions.

On the Mac side, the only export option produces a proprietary Evernote-formatted XML file with document contents embedded. The Windows client can also export in HTML, web archive, and text formats.

When emailing files, Evernote wraps most notes in an ad-encrusted PDF document before sending. (Yes, it’s as bad as it sounds. Look for the “Plain text note” here. That PDF is what you get when you try to email a text file.) Mercifully, you can email PDFs in their original form – though whether this is by design or oversight is unclear. What is clear is that Evernote doesn’t make it easy for you to use your data as you wish.*

A third option is to share your documents in a public notebook (like this one). This of course not the same as export, but it does provide a refreshing level of social openness uncommon in tools of this nature.

Final thoughts

Evernote deftly handles web clippings and snapshots, and ubiquitous access makes it a viable tool for web research and data management.

Perhaps paradoxically, Evernote’s impressive accessibility also limits how I use it. Synchronizing data through Evernote’s server means I won’t use it for sensitive data. So although it’s hard to imagine a situation in which I won’t have access to my iPhone, Mac, or a web browser, it’s also hard to imagine a situation in which said access is truly critical. So I typically use Evernote for less important (but nonetheless useful) tasks:

  • Clipping captioned images and business cards for OCR
  • Jotting beer-tasting notes
  • Snapping photos of the same
  • Misc. data capture when I’m away from my computer

Bottom line: despite its limitations, Evernote is a great tool. At the free price point, you’re unlikely to find a more robust tool… so give the elephant a whirl.


*Comparisons could be drawn to DRM-laden music purchased from iTunes: it’s quite likely that you’ll never want to use it outside the iTunes/iPod ecosystem. But if (or when) that day comes, you won’t want to deal with their restrictions on your data. Same principle.

Archive newsfeeds in DEVONthink Pro via NetNewsWire

Newsfeeds provide an invaluable service: direct access to content of specific interest. NetNewsWire has long been my preferred newsreader, and the recent addition of synchronized online access makes it, for me, the clear best-in-class news client. Yet although NetNewsWire does a fine job aggregating news feeds, it’s a poor long-term information management solution.

Enter DEVONthink Pro, a highly reviewed research tool/information manager.

The NetNewsWire > DEVONthink bridge is a logical one, so it’s no surprise that DEVONthink Pro comes with preloaded scripts to archive information directly from NetNewsWire. (Sorry, DEVONthink Personal doesn’t include scripting support.) But the feature I really needed – the ability to archive entire RSS feeds into DEVONthink – is not included in these scripts.

To fill this gap, I wrote a script to archive entire newsfeeds (or subsets thereof) to DEVONthink Pro.

This script saves items from your selected feed (or folder of feeds) to DEVONthink Pro as web archives, with the following options:

  • From the selected feed, folder, or smart folder, save:

    • All items

    • Flagged items

    • Unread items

    • Read items

    • Date range (archive all items within a specified date range)
  • Archive options:

    • Feed saves just the content of the news item (typically the best option if the site provides full-text feeds)

    • Site saves the content of the item’s target website (this should be the whole article – particularly useful if the newsfeed is truncated)

    • Site/Print goes to the item’s target website and looks for a link that includes the string “Print”. If one exists, it saves the print-ready page to DEVONthink. If none exists, reverts to “Site” behavior.

  • Perform post-archive actions: mark read, unflag, or do nothing. This action occurs after DEVONthink has archived the article, so you can see archive progress in action and ensure you don’t double-archive an article

  • Optional: introduce a random time delay between archiving articles if using “Site” or “Site/Print”. (Useful if the site you’re reading from has countermeasures to prevent content scraping. Read: if they’re trying to keep you from saving their content for later use.)

My personal workflows for this script:

  • Flag items of interest from all newsfeeds. Then run this script periodically to import all flagged items into DEVONthink. Archive target: Flagged items (feed); Post-archive action: Unflag.

  • Save full-text content of a magazine I subscribe to but don’t want to keep hard copies of. Run this script every week, when the magazine’s RSS feed is updated. Archive target: unread items (Site/Print); Post-archive action: Mark as read.

Get the script here.

Update 2008-08-12: Added an additional Growl-free script to the download for users who don’t have Growl installed. The only difference is that the lines referencing GrowlHelperApp are commented out. Same download link applies.

Update 2010-05-26: Fixed a problem encountered when news items don’t have a description. Added an option to archive items from a specific date range. The download link has been updated.

Get Newsfeeds via Safari and NetNewsWire – script updated

NetNewsWire often does a good job finding the newsfeed of a site — but it does sometimes choke.

The updated script will still attempt to get the newsfeed straight from NNW. If NNW has instant success, the script will display the feed and offer to copy it to your clipboard:

Success

After one second, if NNW was unsuccessful, the script will direct Safari search for the feed. The searches initiated (in new tabs, of course) include Google (“site:x rss OR subscribe”) and RSS Micro.

Quick note about RSS Micro: unlike most blog search engines (Google blog search included), it provides direct links to the relevant feeds. Just click on the RSS icon:

RSS Micro

Finally: the download includes a script that initiates these searches from Safari without invoking NetNewsWire. Useful if you don’t use NNW or don’t want to open it.

Download here.

Find URL feed via Safari and NetNewsWire

Guy Kawasaki asked for quick ways to find the RSS feed of a web site. NetNewsWire does a fine job subscribing to sites so I thought it might be up to the task. It is.

Run this to pull the current website from Safari, subscribe in NetNewsWire, and return the RSS feed URL, optionally copying it to the clipboard.

tell application "Safari"
        set thisPage to do JavaScript "document.URL" in document 1
end tell
tell application "NetNewsWire"
        activate
        set subcount to (count of subscriptions)
        set theresult to subscribe to thisPage
        delay 2  --gives NNW time to retrieve the RSS feed
        set thefeed to RSS URL of subscription (subcount + 1)
        if thefeed is equal to thisPage then
                display dialog "No feed found"
                return
        end if
        display dialog thefeed buttons {"Clipboard", "OK"} default button 1
        if the button returned of the result is "Clipboard" then
                set the clipboard to thefeed
        end if
end tell

Click here to open the script and give it a whirl.

Update: Renamed “NetNewsWire 3” to “NetNewsWire” since that’s what the app is called when you download it.

Also, NetNewsWire takes a split second to work its magic. If the script isn’t working for a site, try increasing the “delay 2” line to “delay 5” or something.