David R. MacIver's Blog

reddilicous: Automatically import your links from other sites into delicious

I appear to have done something highly out of character and created a tool which is simply useful, without any real theoretical interest to it.

I followed up on the useful scripts I posted a while ago and decided to turn it into something slightly more complete and robust. The result is reddilicious, a tool for automatically importing from various sites into delicious. (Note: It’s written in Ruby, mostly due to the rather excellent HTTParty and Mechanize libraries. If you’re coming from planet scala expecting to see my awesome scala code, sorry, instead you get some rather grim Ruby code).

It currently handles:

• Reddit: Pulls in any pages you’ve voted up. Tags them with the subreddit and via:reddit
• Stumbleupon: Similarly pulls in any pages you’ve thumbed up. Pulls in your blog entry on it if there is one as an extended comment. Pulls in any tags. Tags it via:stumbleupon
• Twitter: Pulls in any links mentioned in your friends timeline. Tags them via:twitter, from:user, to:users (anyone mentioned @ in the tweet) and with the hashtags mentioned in the tweet

It correctly handles historical data for all of them (twitter it only goes back to the limit of your friends timeline, not everything ever mentioned by a friend of yours), with timestamps set appropriately (on reddit the timestamp is the post date rather than the date you thumbed it up).

It’s all very rough around the edges at the moment, but it does work rather well if you’re prepared to put up with its quirks. The basic mechanism is as follows: There’s a script “reddilicious” in the distribution (note: It currently gets unhappy if you refer to it with a symlink due to path handling issues. Fixing that is on my todo list) which handles all its operations. A reddilicious instance corresponds to a directory where it stores all its data (including passwords in plaintext. Sorry). You create an instance with:

reddilicious create somedir

add accounts to it with e.g.

reddilicious twitter somedir

and update it with

reddilicious update somedir

update will pull in new items and post them to delicious. It should be pretty well behaved about not stomping existing bookmarks when there are duplicates (but will add in additional tags to them, add an extended description if there isn’t one already there, etc).

Its logging output is very chatty (an artifact of my spending way too much time debugging it), and currently logs to stdout. I currently have it running in cron redirected to a file.