Code exploration in C with GNU GLOBAL

For a long time I’ve been getting by with using cscope at work. I’m not a great C programmer by any stretch, and I need to have multiple windows open and a note book by my side to get any sort of understanding of the code.

globals
GLOBAL’s HTML output

So I was happy to discover GNU GLOBAL recently. Getting it setup is a matter of running gtags on the root of the repository. Running htags also produces nice HTML files that make it much easier to browse through.

It has quite a few integrations as well. The snippet I added to my emacs init.el is as follows:

;; gtags/gnu globals
(setq load-path (cons "/usr/share/emacs/site-lisp/" load-path))
(autoload 'gtags-mode "gtags" "" t)
(setq c-mode-hook
 '(lambda ()
 (gtags-mode 1)
 ))
(setq gtags-suggested-key-mapping t)
(setq gtags-auto-update t)

Here is a table that compares similar tools.

web2pdf – A tool to export bookmarks to PDF

Here’s a follow up to my earlier post on archiving. I spent a couple of days coming up with a quick Python app to fill my needs.

Here it is: web2pdf.

Once installed, the configuration simply expects a bookmarks.html file on the filesystem. It reads it, stores the contents to an sqlite DB and starts  saving PDF versions of each link there-in.

You can kill the script and re-run it at a later point in time and it will continue where it stopped. The output looks like this:

(pdf) bash-4.3 ~/code/web2pdf/web2pdf$ ./web2pdf.py 
Found 2599 links in the bookmark file
Found 2599 rows in the bookmark db
..of which 81 links are already saved
..and 2506 are pending
Hit enter to start downloading pending PDFs
Downloading https://www.quantamagazine.org/20170207-bell-test-quantum-loophole/ | experiment-reaffirms-quantum-weirdness

There are more details in the github link. I’m not happy with how fast slow it is but I seem to be limited by the library I’m using and the use-case itself: fetching a page is trivial but it has to render it before exporting it.

As always there is more to do but it works pretty well already. It tags failed bookmarks separately in the DB in case it needs retrying later. I’ve tried to speed it up using Python3’s native async/await, but the performance improvements are not noticeable so far. I’ll try with multiprocess instead and commit whichever one works better.

rssmailer.py – Convert RSS feeds to a digest email

I finally scratched a long-pending itch and wrote a python application that:
  • Reads a list of feeds (that update more frequently than I prefer) once a day,
  • Extracts the last day’s posts,
  • and mails it to me.
I used mailgun’s neat APIs to send out the mails, and feedparser to do the rss/atom parsing.
The application hinges on twisted to asynchronously parse multiple feeds at a time.
Here’s the package in PyPi: https://pypi.python.org/pypi/rssmailer/0.2
And here’s the product page: https://tools.indeliblestamp.com/
The mail below is what a sample mail looks like when I get it for one of the sites.
—– Original message —–
From: RSS Digest Mailer
To:
Subject: Digest mail for 3quarksdaily
Date: Thu, 05 May 2016 13:02:46 +0000

Dangerous Fictions: A Pakistani Novelist Tests the Limits

Dexter Filkins in The New Yorker: Hanif got the idea of writing about a nurse in a decrepit hospital. Alice Bhatti (named for his old editor) is a ferociously strong young woman: smart, independent, and rebellious to the point of…

Our brain uses statistics to calculate confidence, make decisions

From PhysOrg: The directions, which came via cell phone, were a little garbled, but as you understood them: “Turn left at the 3rd light and go straight; the restaurant will be on your right side.” Ten minutes ago you made…

Trump-Sanders Phenomenon Signals an Oligarchy on the Brink of a Civilization-Threatening Collapse

Sally Goerner in Evonomics: The media has made a cottage industry out of analyzing the relationship between America’s crumbling infrastructure, outsourced jobs, stagnant wages, and evaporating middle class and the rise of anti-establishment presidential candidates Donald Trump and Bernie Sanders….

Stop telling kids you’re bad at math

Petra Bonfert-Taylor in the Washington Post: Why do smart people enjoy saying that they are bad at math? Few people would consider proudly announcing that they are bad at writing or reading. Our country’s communal math hatred may seem rather…

Blockchain technology will revolutionise far more than money: it will change your life

Dominic Frisby in Aeon: The impact of record-keeping on the course of history cannot be overstated. For example, the act of preserving Judaism and Christianity in written form enabled both to outlive the plethora of other contemporary religions, which were…

Warsan Shire: the Somali-British poet quoted by Beyoncé in Lemonade

Rafia Zakaria in The Guardian: She writes of places where many Beyoncé fans rarely go, the portions of London where the faces are black and brown, where men huddle outside shop-front mosques and veiled women are trailed by long chains…

The Essence of Mathematics, in One Beatles Song

Ben Orlin in Math With Bad Drawings: Okay, here’s a life regret: No one has ever stopped me on the street, grabbed me by the collar, and demanded that I explain to them the essence of mathematics. Me: So, you…

How should we live in a diverse society?

Kenan Malik in Pandaemonium: ‘Can Europe be the same with different people in it?’ So asked the American writer Christopher Caldwell in his book, Reflections on the Revolution in Europe, published a few years ago. It is a question that…

Sarah Palin, Jimmy Kimmel and Scientists on Climate Change

Why did the death of a single lion cause a sustained uproar?

Jason Goldman for Conservation Magazine: When the story of Cecil the lion’s death at the hands of an American hunter hit the media, the global response was “the largest reaction in the history of wildlife conservation,” according to a new…

Java factorial program with recursion and BigIntegers

Another initiative at work expects me to have a certain amount of Java knowledge. Oh well. Here are some good resources I am currently going through:

Learneroo’s first chapter has a factorial method recursively. No biggie. I wrote a quick program, ran and tested it for small values. When I tried it for ‘100’, though:

Enter a number for which I will find the factorial:
100
0

Oops. The same program worked fine with Python, what was I doing wrong? A quick check showed that my program returned the result as an int. The highest Int you can get is 2^(31)-1. So, enter the BigInteger class. It is not a native data type and I had to tweak the program a bit to handle it. This is what it shows as the output now:

Enter a number for which I will find the factorial:
100
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000

And here is the full program:

import java.math.BigInteger;
import java.util.Scanner;

public class Factorial {

    public static void main(String[] args) {
        Scanner read = new Scanner(System.in);
        System.out
                .println(“Enter a number for which I will find the factorial:”);
        int num = read.nextInt();
        Factorial test = new Factorial();
        BigInteger result = test.findfact(num);
        System.out.println(result);
        read.close();
    }

    public BigInteger findfact(int n) {
        if (n == 1) {
            return BigInteger.ONE;
        } else {
            return BigInteger.valueOf(n).multiply(findfact(n – 1));
        }
    }
}

Turtle Power

Thanks to this great website, I’ve been having some more fun with Python.  I’m now in the chapter on Recursion, which introduces Python’s Turtle module. Turtle is an implementation of Logo, a language I remember playing with happily in my younger years. What you can do with it in Python is quite amazing sometimes.  There are some demo scripts that come pre-installed, and can be accessed this way:

python -m turtledemo

I’ve still not wrapped my head around the more complex ones, but the Sierpinski example in the tutorial is a nice start. I learned from their example and wrote my own (fairly similar) version. The result looks pretty nice:

sierpinski

A couple of weeks back I wouldn’t have been able to come up with something like this at all, so I’m pretty happy with what I’ve learned so far.
Below is the script. Just pasting it in a python3 prompt should suffice.

import turtle
def drawTriangle(points, color, t):
    t.fillcolor(color)
    t.penup()
    t.goto(points[0][0],points[0][1])
    t.pendown()
    t.begin_fill()
    t.goto(points[1][0],points[1][1])
    t.goto(points[2][0],points[2][1])
    t.goto(points[0][0],points[0][1])
    t.end_fill()
def findMid(p1,p2):
    return( (p1[0]+p2[0])/2, (p1[1]+p2[1])/2 )
def sierpinski(points,depth,t):
    colormap=['purple2','tomato2','khaki3','gold','orchid','spring green','turquoise']
    drawTriangle(points,colormap[depth-1],t)
    if depth>1:
        sierpinski([points[0],findMid(points[0],points[1]),
                    findMid(points[0],points[2])],depth-1,t)
        sierpinski([points[1],findMid(points[1],points[2]),
                    findMid(points[0],points[1])],depth-1,t)
        sierpinski([points[2],findMid(points[2],points[0]),
                    findMid(points[2],points[1])],depth-1,t)
def main():
    t = turtle.Turtle()
    myWin = turtle.Screen()
    t.speed(0)
    points=[[-300,-150],[0,150],[300,-150]]
    sierpinski(points,6,t)
    t.hideturtle()
    myWin.exitonclick()
main()

Fetch unique words in a web page (in 7 lines of Python)

I got a chance to enroll for some Python training at work and had a nice time with it. I’ve tried getting into other languages myself before, and this has been my freshest experience so far. The three days of training were enough to show me how insanely easy it is to get productive in Python.

A quick example we saw was to:

  • Get the contents of a web page, and
  • Find all the unique words in it.
This simple example introduces the usage of lists, sets, modules and loops. The code looks like this:

import urllib2
words=[]
for line in urllib2.urlopen(‘http://slashdot.org’):
    words.extend(line.split(‘ ‘))
print ‘number of words is’, len(words)
uniq=set(words)
print ‘number of unique words is’, len(uniq)
That’s it! The whole job takes only eight lines. Running it gives an output like this:
C:Python27mypy>19.py
number of words is 13297
number of unique words is 3672

The first line imports the urllib2 module. 
The next line initializes an empty list.
The third line is where most of the magic happens. A for loop iterates over a list returned by urllib2 when we ask it to open Slashdot’s home page. For each line, we split the content into individual words and append it to our list.
That gives us all the words in the page, which we print.
Python supports a set data structure that we copy this list to, which ends up removing recurring elements automatically (since sets support only unique elements).
And that’s it, we’re done. Of course there is always more to do beyond this (such as in not counting html tags), but this was probably my favourite example from the course that captured how easy it was to get seemingly complicated tasks done in Python.