Notes on notes

Shortly after the 24:00 mark in the latest episode of Gabe Weatherhead’s Generational podcast, Gabe’s guest, Walton Jones, starts talking about his system for annotating and summarizing academic papers. If you can listen to that 8- to 10-minute stretch without being inspired to improve your own methods for managing the flood of information in your job, then you’re dead to me.

Walton’s system

To be sure, Walton’s system is highly tuned to the specifics of his profession. As a scientist, a good portion of his time is spent analyzing and synthesizing the research of others. That research comes to him in the form of PDFs of journal papers. He adds color-coded annotations to the PDFs as he reads them: red for summaries, green for references, yellow for results, and so on. This may sound like nothing more than a digital version of Post-it notes, but Walton has an amazing trick up his sleeve. When he’s done reading a paper, he runs an AppleScript that goes through the PDF and creates a Markdown document with all the paper’s annotations listed by page number and organized according to category (summary, reference, result, etc.). The Markdown is then turned into a new page in a VoodooPad document.

So he has this VoodooPad document with his notes on the papers he’s read, which is nice, but that’s not the end. Each individual note in VoodooPad is linked to the page of the PDF to which it refers. The power of this system is that he can search through his VoodooPad document, which has his notes and therefore uses terminology that come naturally to him while searching, and when he finds what he’s looking for, he can click a link and be taken immediately to the right spot in the right paper. This is so much better than simply searching through abstracts or lists of keywords, all of which are words chosen by others.

But don’t just go by my description, read Walton’s own explanation of his system.

My system

While I don’t pore over research papers anymore, I do deal with a menagerie of documents—drawings, photographs, videos, test reports, deposition testimony, presentation slides, email trails—that are increasingly in some sort of electronic format. I try to organize this mess by turning everything except the photos and videos into PDFs. Like Walton, I make notes on these documents as I go through them, but I don’t do it the way he does.

My system is based on talking. Long ago, I talked into a voice recorder. Later, I started talking into my iPhone, using Griffin’s iTalk app. With both of these systems, I’d replay the recording to myself and type up the notes, usually cleaning up the sentence structure as I went along. For the past two months, though, I’ve had a much better system: Siri.

Say all the mean things you want about Siri; for me, she’s a great dictation transcriber. The individual notes I make as I read through a document are typically one or two sentences long, which is just about the perfect length for Siri. In Notesy, I tap the microphone button on the keyboard, say my one or two sentences, and tap Done. A few seconds later, the note appears. Unless I’ve hemmed and hawed or there’s a peculiar word, the transcription needs no editing and I move on.

I have a particular format I prefer, with the page number on a line of its own, then the note itself, then a blank line. A typical session would be me saying something like

Fifty-two. New line. A solid or liquid to a change in direction will be as great as a ton per square inch. Period. There are many transformations of motion. Period. New paragraph.

which comes out in Notesy as

Siri dictation in Notesy

Notesy syncs to Dropbox, so the file will be on my Mac when I’m done making notes. The format is not exactly Markdown, but it’s easy to run a global search-and-replace to add a pair of space characters after each page number to provide the line breaks I want in the output. Marked then turns the text file into a PDF.

This is a pretty good system, but what’s missing—and what Walton inspired me to add—are links from my notes to the page numbers in the original documents. Since I keep my summaries in the same directory as the original documents, the links could be added this way in Markdown:

A solid or liquid to a change in direction will be as
great as a ton per square inch. Period. There are many
transformations of motion.

I’m currently working on a script that’ll do this. It works, but it isn’t especially robust and there’s too much “by hand” work in turning the Markdown into a PDF. I’ll do a complete post when I get those problems solved.

I should mention here that neither Preview (under Lion) nor PDFpenPro handles page number links correctly. Preview opens the original document (sometimes—other times it refuses and says I don’t have permission to open it, which is probably some kind of sandboxing stupidity) but won’t go the linked page number. PDFpenPro doesn’t even get that far; it opens a blank document that it claims in the title bar is original document.

Skim, on the other hand, handles page number links like a boss. This was a little surprising to me, because Walton says in another post that it doesn’t and that he had to write a script to word around that limitation. All I can say is that Skim has worked fine in all my tests so far. I just need to get that script working so I can start using summaries with links.

9 Responses to “Notes on notes”

  1. Stuart Dootson says:

    Automating Markdown to PDF? The way I do it is to install Pandoc and MacTex and use pandoc -o output.pdf. I’ll admit that the 2+GB download of TeX always seems rather excessive, but as I’m converging on a LaTeX based workflow, that doesn’t bother me too much…

  2. Walton says:

    Thanks for the interest and the kind words Dr. Drang. I am finally satisfied with my solution, but I am still curious about how you are getting the Skim links to work. My problem was having a Skim-specific url scheme that obeys the page command. I was able to use file://example-report.pdf#page=52 to load the local file in a browser and hit the right page, but then the Skim annotation file is not loaded and I have to look around the page for the thing I had noted. If the file is launched in Skim, the annotations are loaded simultaneously and I can see right away what I am looking for. Is Skim moving to the right page for you?

  3. Dr. Drang says:

    I guess the reason page-specific links in Skim are working for me is that I start with my summary document (which is itself a PDF) in Skim, so file:// type links are staying in the same application. I realize now that this is an important difference between your setup and mine: you need a link that opens a file to a particular page in a particular application, whereas I just need a link that opens a file to a particular page.

    I’ll use MultiMarkdown for the conversion to PDF. It’s not a big deal; I just need to get myself up to speed on the current version. I’ve been using a customized version for years that’s tuned to the format I use for reports—not appropriate for these summary documents.

    As for Pandoc, I’ve had difficulty installing GHC in the past and am not interested in trying again.

  4. Rohit Sharma says:

    Thank you for posting this, Dr. Drang. As a Master’s student who reads 100s of pages a week for school, it’s nice to find new ways to get a handle on my notes.

    Have you found a method for reading PDFs that works well for you, too?

    I like using iBooks on my iPad because of the full-screen view, offline access, and retina display, but copying papers via USB stopped being fun a long time ago.

  5. Bill says:

    Rohit, Try ReaddleDocs for viewing/annotating PDFs on iOS. Like GoodReader, it is highly versatile, reads many formats, and has Dropbox syncing. Unlike GoodReader, it has a pleasing UI. Annotations persist across platforms, so notes and highlights made on iPad can be seen on Mac.

    There are a number of good PDF-specific iPad readers, but none of them have turned me away from ReaddleDocs. PDFPen is notable in that it enables easy PDF creation; iAnnotate in that it allows you to open one PDF in more than one tab.

  6. Bill says:

    And let me just add that I keep GoodReader on my iOS devices for one good reason: it will pretty much open anything you can throw at it!

  7. Dr. Drang says:

    I tend use Preview or Skim (no iPad) or, horror of horrors, I print the PDFs out and read that quaint combination of ink and paper. Drawings are almost always better viewed on paper because you can spread them out across a table and follow the cross-references more easily.

  8. John says:

    Dr. Drang: Pandoc has been available for some time as a Mac installer package with no dependancies, and its extensive support, with user templates, for other input and output formats has led to it replacing MultiMarkdown for me.

    Stuart Dootson: For converting Markdown to PDF with Pandoc there is no need for the full MacTex download. It is possible to use the 64 MB download of BasicTeX which includes all the packages required to use the default Pandoc templates.

  9. Rohit Sharma says:

    Bill, ReaddleDocs was a great recommendation. The Dropbox sync works well, and it seems to even render PDF pages quicker than iBooks. Thank you!

    Dr. Drang, As nice as the iPad is, you’re right that dead trees are still the best experience for reading papers - probably even moreso compared to the Mini. Thank you for recommending Skim, though - I haven’t come across it in grad school yet, and it seems like a nice alternative to Preview.