rssmailer.py – Convert RSS feeds to a digest email

I finally scratched a long-pending itch and wrote a python application that:
  • Reads a list of feeds (that update more frequently than I prefer) once a day,
  • Extracts the last day’s posts,
  • and mails it to me.
I used mailgun’s neat APIs to send out the mails, and feedparser to do the rss/atom parsing.
The application hinges on twisted to asynchronously parse multiple feeds at a time.
Here’s the package in PyPi: https://pypi.python.org/pypi/rssmailer/0.2
And here’s the product page: https://tools.indeliblestamp.com/
The mail below is what a sample mail looks like when I get it for one of the sites.
—– Original message —–
From: RSS Digest Mailer
To:
Subject: Digest mail for 3quarksdaily
Date: Thu, 05 May 2016 13:02:46 +0000

Dangerous Fictions: A Pakistani Novelist Tests the Limits

Dexter Filkins in The New Yorker: Hanif got the idea of writing about a nurse in a decrepit hospital. Alice Bhatti (named for his old editor) is a ferociously strong young woman: smart, independent, and rebellious to the point of…

Our brain uses statistics to calculate confidence, make decisions

From PhysOrg: The directions, which came via cell phone, were a little garbled, but as you understood them: “Turn left at the 3rd light and go straight; the restaurant will be on your right side.” Ten minutes ago you made…

Trump-Sanders Phenomenon Signals an Oligarchy on the Brink of a Civilization-Threatening Collapse

Sally Goerner in Evonomics: The media has made a cottage industry out of analyzing the relationship between America’s crumbling infrastructure, outsourced jobs, stagnant wages, and evaporating middle class and the rise of anti-establishment presidential candidates Donald Trump and Bernie Sanders….

Stop telling kids you’re bad at math

Petra Bonfert-Taylor in the Washington Post: Why do smart people enjoy saying that they are bad at math? Few people would consider proudly announcing that they are bad at writing or reading. Our country’s communal math hatred may seem rather…

Blockchain technology will revolutionise far more than money: it will change your life

Dominic Frisby in Aeon: The impact of record-keeping on the course of history cannot be overstated. For example, the act of preserving Judaism and Christianity in written form enabled both to outlive the plethora of other contemporary religions, which were…

Warsan Shire: the Somali-British poet quoted by Beyoncé in Lemonade

Rafia Zakaria in The Guardian: She writes of places where many Beyoncé fans rarely go, the portions of London where the faces are black and brown, where men huddle outside shop-front mosques and veiled women are trailed by long chains…

The Essence of Mathematics, in One Beatles Song

Ben Orlin in Math With Bad Drawings: Okay, here’s a life regret: No one has ever stopped me on the street, grabbed me by the collar, and demanded that I explain to them the essence of mathematics. Me: So, you…

How should we live in a diverse society?

Kenan Malik in Pandaemonium: ‘Can Europe be the same with different people in it?’ So asked the American writer Christopher Caldwell in his book, Reflections on the Revolution in Europe, published a few years ago. It is a question that…

Sarah Palin, Jimmy Kimmel and Scientists on Climate Change

Why did the death of a single lion cause a sustained uproar?

Jason Goldman for Conservation Magazine: When the story of Cecil the lion’s death at the hands of an American hunter hit the media, the global response was “the largest reaction in the history of wildlife conservation,” according to a new…

Email is still King

In 2012 I wrote this post, describing my rss2email setup. Since then, I’ve moved on to a host of other rss readers. Notable ones among them are:
  • Newsblur
  • Digg Reader
  • Feedly
Having tried so many of them, I had a clear idea of what was most important for my needs. Newsblur was feature rich, but moving from one post to the next took just that little bit longer than I liked. Digg Reader was lightning fast in that aspect, and is what I’d recommend if someone wants a simple, fast, hosted solution.

But along the way I also ended up owning a couple of non-android phones. The sparse collection in Microsoft’s app store made me realize how important it was not to depend on apps that were tied to a single product.

So I’ve now come full circle, back to rss2email. Fastmail is my email host of choice. I still have more than 70 feeds, some of which are pretty high traffic. So I’ve setup three folders: ‘alerts’, ‘hightraffic’ and ‘toread’.

Once I had some rules set up to redirect most mails to ‘alerts’, and the noisy ones to ‘hightraffic’, my workflow was ready:

During my daily commute, I use the incredible K-9 Mail app on my android phone to skim through the feeds. Any feed that deserves a deeper look gets moved to the ‘toread’ folder. The hightraffic ones are read only when I have time to spare. When I have access to my laptop or desktop, I use the fastmail web interface, which supports vim-like keybindings for quick navigation.

I’ve also moved my self-hosted wordpress blog (again!) back to blogger, which has an option to send a post by email. So I’m typing this as en email that will be sent to blogger shortly.

To recap, I have a solution that is fast, self-hosted, and flexible enough to allow convenient workflows tuned to my free time and connectivity.

However hard a service tries to ‘kill’ email, it’s a testament to how conveniently flexible it is in allowing setups like this.

My Email Client is my RSS Reader

rss2email is a kick-ass RSS solution that I’ve been using for quite awhile now. It works exactly the way it sounds: You provide it a list of feeds, and it mails you hourly when one or more of them get updated. I now have 100+ feeds added to it. Every morning I log in and skim through the 30-40 emails that come straight to my ‘alerts’ folder. Here is what a random day’s email looks like for me:
I hit 10000 mails in this folder last week. Here is what I’ve learnt:

r2e-thumb-500x298-12
  • Don’t use this to subscribe for web sites that update frequently. I have direct bookmarks for sites like the Guardian and Metafilter.
  • Pictures/Videos don’t work too great. Obviously, because any decent email client blocks external images.
  • Some good sites have feeds for specific tags in case you don’t want the rest of the content. Not Exactly Rocket Science’s great link round up is a fine example.

I’m always on the lookout for newer feed readers, but this is the one I’ve settled with quite comfortably.