Archive for the ‘software’ Category
Repeating dates in OmniFocus and iCal
June 30th, 2008
In trying to create a repeating OmniFocus task, I learned that its handling of repeating dates is pretty rudimentary. iCal is much better, but is still lacking some niceties that can come in handy.
My company uses an outside bookkeeper who comes to our office on the last Friday of every month to do the accounting that pays our bills, taxes, and salaries. Some paperwork needs to be done by one of the partners before her visit, and if I’m in town, I’m usually the one who does it. Last week I tried to set up an OmniFocus task to remind me of the necessary paperwork, but I was stopped short by some of OmniFocus’s limitations.
I started with what I thought was the obvious approach. I created an Accounting project and added a repeating task to get the needed paperwork ready. Two problems:
- When I’m showing Next Actions—which is my usual working Perspective—all the Next Actions appear. But the paperwork for the bookkeeper shouldn’t be done until a few days before salary checks are cut, so this task will just clutter up my view for most of the month. I may be able to get around this problem by fiddling with start dates or by scheduling an AppleScript to toggle the task status from On Hold to Active near the end of the month.
- OmniFocus’s Repeat Every setting is just too limited. It allows you repeat a task every so many days, weeks, months, or years, but it can’t handle something like “the last Friday of the month.”
This second problem is the killer, and it’s a pretty embarrassing lack of functionality. The algorithms for this kind of repetition are well established and available. Papers and books on the topic by Reingold and Dershowitz can be found here, and you can always look at the source code for the Calendar/Diary module in Emacs, or for the old BSD calendar program, which comes with OS X.
iCal has a much better repeating date implementation. I’ve used it to create an event for the bookkeeper’s visit on the morning of the last Friday of every month.
This is great for keeping track of the visits, but isn’t ideal for keeping track of my paperwork task. David Allen saith that the calendar is only for things that must be done on that day. The paperwork can be done any time in the week before the last Friday of the month and is best done a day or two before the bookkeeper’s visit.1 Since OmniFocus can’t do what I need on its own, the iCal entry will have to suffice until the OmniGroup improves its date repetition code, or until I can figure out a way to force OmniFocus to do my bidding.
I should mention here that iCal’s repeated date handling isn’t perfect. It can’t, for example, handle US election days or the US federal holiday for Christmas. Election day is not the first Tuesday in November, it’s the day after the first Monday in November. And the federal Christmas holiday—the day you get off work—is December 25 only if December 25 falls on a weekday. If it falls on a weekend, the Christmas holiday is the weekday nearest to December 25. Try doing either of these as a repeating event in iCal.
Although I have no solution now, I’m not giving up. Maybe I can use iCal’s ability to schedule AppleScripts to run a few days before an event to create an OmniFocus task telling me to get the paperwork ready. I need to dig into OmniFocus’s AppleScript library.
OmniFocus and OmniOutliner Pro
June 27th, 2008
My free test period for OmniFocus ran out today and I couldn’t open my GTD lists. This bothered me, which I took to be a sign that it’s a program I really use and should buy.
OmniFocus normally sells for $80, but if you’re a licensed owner of OmniOutliner Pro, you can buy OmniFocus for $60. I don’t own OO Pro, but I do have the regular version of OO that was bundled with my iMac. Since that can be upgraded to OO Pro for $30, doing the upgrade and then buying OmniFocus was just $10 more than getting OmniFocus alone. OK, OmniGroup, you win—I’ll get both.
And yes, I’ll get OmniFocus for the iPhone when it comes out. I’m turning into the OmniBitch.
iPhone ringtones flowchart
June 20th, 2008
I don’t convert songs to iPhone ringtones very often, so I made this flowchart to help me remember the process. And by posting it here, I’ll always know where it is. Oddly, I find that Googling “create iphone ringtone” or something like that often brings up out-of-date information. This flowchart works with iTunes 7.6.2 and iPhone 1.1.4 system software, I’ll try to keep it current as the versions change.
Here it is:
You can see a larger version by clicking on it.
This is a Mac-centric workflow, and it takes advantage of the software I own. I perform the actions in the yellow boxes using Rogue Amoeba’s Fission audio editor, but you could substitute any sound file editor, like Freeverse’s SoundStudio or the free Audacity or whatever’s popular on Windows.
Converting to AAC is done by choosing the Convert Selection to AAC command from iTunes’ Advanced menu. If you don’t see a Convert Selection to AAC item in that menu, it’s because your iTunes Importing Preference is set to use a different encoder. Change the encoder to AAC and the menu item will change, too.
A clip about 10-20 seconds long seems to work well. Once the clip is in the iTunes Ringtone Library, it will transfer to your iPhone the next time you sync.
Automated charts with Gnuplot
June 19th, 2008
Every month since the fall of 2006, I’ve written a post like this one, in which I include a chart of US and coalition fatalities. This is a reduced version of the latest:
I get the data from icasualties.org and generate the chart using Gnuplot, the venerable Unix plotting program. I figured it was about time I showed how I do it.
Gnuplot began its life as a terminal program, and it’s still driven by typed commands rather than by clicks and drags. This makes it a bit difficult to learn, but because it allows sequences of commands to be stored in files, it makes the creation of several similarly-formatted graphs a snap. I started using it when I switched to Linux in the mid ’90s because there were no good Excel-like programs on that platform. I’ve kept using it after switching back to Macintosh because it’s so good at producing the kinds of graphs common in science and engineering. I’m not big on pie charts.
The data for the graphs are stored in a file called “icasualties.txt” that I update every month. The first several lines of that file look like this:
Month US UK CO
2003-03 65 27 0
2003-04 74 6 0
2003-05 37 4 1
2003-06 30 6 0
2003-07 48 1 0
2003-08 35 6 2
2003-09 31 1 1
2003-10 44 1 2
2003-11 82 1 27
2003-12 40 0 8
The columns are separated by spaces; it doesn’t matter that the number of spaces differs from column to column or row to row. I think all columns but the last are self-explanatory. The “CO” column contains the count of military deaths from coalition countries other than the United States and United Kingdom.
The file of Gnuplot commands that generates the graph is called “icasualties.gp.”
1: # input format for dates
2: set timefmt "%Y-%m"
3:
4: # horizontal (time) axis layout
5: set xdata time
6: set format x "%b\n'%y"
7: set xtics "2003-01", 60*60*24*365.2425/4, "2008-12" # quarterly
8: set mxtics 3 # monthly
9:
10: # left vertical axis layout
11: set ylabel "Military Deaths" 2,0
12: set yrange [0:150]
13: set ytics 25
14: set mytics 5
15:
16: # overall layout
17: set title "Iraq War Timeline" 0,-.5
18: set grid
19: set key at "2007-10",20 right width -3 samplen 1.5 box
20:
21: # Make labels for the totals
22: ustot = `perl -e '$s=0;while(<>){($m,$a,$b,$c)=split;$s+=$a}print$s;' icasualties.txt`
23: tot = `perl -e '$s=0;while(<>){($m,$a,$b,$c)=split;$s+=$a+$b+$c}print$s;' icasualties.txt`
24:
25: # Make label for data source
26: set label 3 "Data source: http://icasualties.org" at '2003-05',10 left
27:
28: # choose output and plot it
29: set terminal aqua 0 title "Timeline" size 800 600\
30: fname "Helvetica" fsize 14
31: plot "icasualties.txt" using 1:2 title sprintf("US only (%d)",ustot)\
32: with linespoints pointtype 2 linetype 3 linewidth 3,\
33: "icasualties.txt" using 1:($2+$3+$4) title sprintf("Coalition (%d)",tot)\
34: with points pointtype 3 linetype 1
Like Perl, Python, and the shell, Gnuplot comments start with a hash mark (#). As you can see, most of the commands “set” a Gnuplot parameter that controls either the input or output formatting. Two of the other commands create variables for later use, an the final command creates the plot itself. Here’s the explanation:
Line 2 tells Gnuplot that some of the input data will be time values and that they will be formatted with a 4-digit year (%Y) followed by a hyphen and a 2-digit month (%m). The codes follow the conventions of the well-known strftime C library.
Lines 5-8 cover the formatting of the horizontal (x) axis. Line 5 says that the x-axis will consist of time values. Line 6 sets the format of the axis labels to a 3-letter abbreviation of the month name (%b), new line (\n), an apostrophe, and a 2-digit year (%y). Line 7 sets the major tic marks and the labels to the start of every quarter, which is kind of tricky. The three arguments to set xtics are
- Where we should start counting: 2003-01. This is the January before the war began, and I chose this date to insure that the major tics marks fall on the usual quarterly start dates: January 1, April 1, July 1, and October 1.
- The spacing between the major tic marks, in seconds. Using 356.2425 as the number of days in a year is overly precise, but I’m a big fan of the Gregorian calendar reform.
- Where we stop counting. I’ll have to change this next year.
Line 8 tells Gnuplot to split the space between major tic marks into three parts and put minor tics at the splits.
Lines 11-14 cover the formatting of the vertical (y) axis. The numbers after the axis label in Line 11 nudge the label a little to the right, because I thought the default location was too far from the axis. Since Gnuplot will choose the range and tic locations if they’re not specified, Lines 12, 13, and 14 set the range and spacing to get a consistent vertical axis every time I generate a graph.
Line 17 sets the title at the top of the graph and nudges it down a bit from its default location. Line 18 tells Gnuplot to put faint gridlines that run the full width or height of the graph at every major tic mark.
Line 19 puts the key (or legend) near the bottom of the graph and puts a box around it. The at "2007-10",20 right part positions top right corner of the key at those coordinates. The width -3 part makes the box a bit smaller than its default width. The samplen 1.5 part makes the blue and red point/line examples a bit wider than the default.
Lines 22 and 23 are tricky. I wanted to put the US and coalition casualty totals on the graph, but as far as I can tell, Gnuplot doesn’t have a builtin way to get those figures. But it does have a way of calling another program. So these two lines contain short Perl scripts that scan through the “icasualties.txt” file and sum up the US and full coalition figures. The sums are stored for later use in the Gnuplot variables ustot and tot.
Line 26 puts the acknowledgement text near the bottom left of the graph. The commands for positioning are similar to those in Line 19.
Lines 29-30 is one long Gnuplot command split over two lines. It tells Gnuplot to display the graphs in an 800x600 AquaTerm window with 14-point Helvetica as the base font. The set terminal command is difficult to learn, but makes Gnuplot very flexible in its output. Although I want the graphs in the form of a PNG file, I chose aqua terminal over the png terminal because the precompiled Gnuplot I got with Octave (see this post) doesn’t have support for Macintosh fonts built into its png terminal. And I don’t feel like gathering all the libraries necessary to compile my own version.
Lines 31-34 are what we’ve been leading up to. This is one long Gnuplot command that actually created the plot according to the specifications given in the previous lines. It makes one graph for the US casualties and one for the total coalition casualties. The key labels these data series include the total casualty counts calculated back in Lines 22 and 23. The presentation styles (line type, line width, color, and point type) have numbers rather than names, so there’s usually a bit of trial and error before you hit on a combination you like.
I create the graph by typing gnuplot icasualties.gp into Terminal. AquaTerm launches and shows the graph. I then do a screenshot of the graph and upload it to my server as a PNG file. The whole thing takes less than a minute.
As you can see, Gnuplot is very flexible but very complex. I find that when I’m using it a lot—for example, while writing a report that reduces and presents a lot of data—I get into a rhythm and the command come naturally to my fingertips. But when I’ve been away from Gnuplot for a few months, there’s always some frustration when I start back again.
Good documentation would go a long way toward relieving that frustration. Unfortunately, Gnuplot’s documentation, while quite complete, is terribly difficult to use because it’s organized alphabetically by command. So it’s great if you know the command, but if you knew the command you probably wouldn’t be looking in the manual.
It looks like help is on the way. Gnuplot in Action is a book by Philipp Janert that is scheduled to be published by Manning later this year. Janert and Manning have graciously given me a free review copy of the Early Access Edition of the book—basically a PDF of most of the book in a pre-publication state—and I’m optimistic. The best part of the book can be seen from its table of contents. Janert presents Gnuplot concept by concept rather than command by command. There’s a chapter on axes, a chapter on styles, a chapter on scripting, etc.
He has not, however, simply taken the official Gnuplot manual and rearranged it (although that would be valuable in itself). The chapters are filled with examples showing both the Gnuplot command and the graphical result. The official manual has many examples, but because it’s still text-based, it can only tell you what the commands will do—it can’t show you. I do wish Manning would put the commands and output side by side as is done in The LaTeX Companion, but the sequential layout still gets the job done.
One aspect of the book I don’t expect to like is hinted at in its subtitle, Understanding Data with Graphs. In addition to Gnuplot itself, Janert apparently wants to teach me how to use graphs generally to analyze data. Since that section of the book hasn’t made its way into the Early Access Edition yet, I don’t know what he’s going to say, but I’ve been analyzing data with graphs for 30 years, and I doubt he’s come up with anything new. But that section may be helpful to others.
This isn’t a review. The book isn’t close enough to publication quality for a review to be fair. It is, however, already better at teaching Gnuplot than the official manual is. In some ways, its better than the manual even as a reference because it’s better organized. I’m keeping an eye on it and will write up a review when it’s near its final form.
TypeIt4Me problem with AppleScript snippets
June 13th, 2008
Two weeks ago, I posted the following bug report to the the TypeIt4Me technical support forum:
I am having a problem with TypeIt4Me’s AppleScript clipping feature on my iBook G4. I have a clipping with the following AppleScript: tell application “Safari” to get URL of front document When I run this clipping from within the text editor TextMate, it comes out looking like thisThe “spaces” before each character are not really spaces, they’re unprintable characters. I’ve saved this in a file called tm-g4.txt and run the xxd utility on it to see what the characters are. Here are the results:h t t p : / / m y . y a h o o . c o m /The unprintables are nulls. At first I thought this was some weird interaction between TypeIt4Me and TextMate, but… When I run the same clipping from within TextEdit, the spaces don’t appear, but saving it to a file called te-g4.txt and running xxd on it gives me the same thing:% xxd tm-g4.txt 0000000: 0068 0074 0074 0070 003a 002f 002f 006d .h.t.t.p.:././.m 0000010: 0079 002e 0079 0061 0068 006f 006f 002e .y...y.a.h.o.o.. 0000020: 0063 006f 006d 002f .c.o.m./So the nulls are there, but TextEdit just isn’t showing them. I don’t get this behavior with the same clipping on my Intel iMac. On this machine, there are no blanks before the characters when the clipping is inserted in a TextMate document. Saving the document to a file called tm-intel.txt and running xxd on it yields% xxd te-g4.txt 0000000: 0068 0074 0074 0070 003a 002f 002f 006d .h.t.t.p.:././.m 0000010: 0079 002e 0079 0061 0068 006f 006f 002e .y...y.a.h.o.o.. 0000020: 0063 006f 006d 002f .c.o.m./No nulls, which is how it should be. I got the same result from using the clipping in TextEdit on the Intel iMac:% xxd tm-intel.txt 0000000: 6874 7470 3a2f 2f6d 792e 7961 686f 6f2e http://my.yahoo. 0000010: 636f 6d2f com/Running the wc utility on these files also shows that the G4 versions have two bytes for every byte in the Intel versions:% xxd te-intel.txt 0000000: 6874 7470 3a2f 2f6d 792e 7961 686f 6f2e http://my.yahoo. 0000010: 636f 6d2f com/Is this some sort of big-endian/little-endian thing? Can it be fixed?% wc tm-g4.txt 0 1 40 tm-g4.txt % wc tm-intel.txt 0 1 20 tm-intel.txt % wc te-g4.txt 0 1 40 te-g4.txt % wc tm-intel.txt 0 1 20 tm-intel.txt
The problem arose when I tried to use the snippets I discussed in this post on my iBook G4. I really like the idea of having programmable snippets available system-wide, not just in TextMate. But I can’t use the feature if it keeps spitting out these spurious nulls.
So far, there’s been no response from Ettore Software, which is a bit annoying. I’m pretty sure the forum is where Ettore wants bugs reported.
While waiting for a response, I downloaded the trial version of TextExpander to see if it has the same problem. It doesn’t; AppleScript snippets in TextExpander expand perfectly on both my Intel and G4 machines.
Were it not for the AppleScript snippet bug, I’d prefer to use TypeIt4Me, because it has the AutoCue feature that allows me to enter variable text at placeholder locations within the snippet—similar to the tab stops in TextMate’s snippets. With the bug, however, the scale tips to TextExpander. If I don’t get some encouraging response from Ettore, I’ll have to go ahead and buy it when the trial period is over.
New calculator for iPhone 2.0
June 9th, 2008
For me the hidden gem of the upcoming iPhone 2.0 software is the new scientific calculator application. I’ve been using Belfry’s SciCalc webapp, which is nice enough, but it doesn’t seem as responsive to key taps as the builtin calculator, probably because it’s not a native application.
Here’s what the new calculator looks like according to Apple’s calculator page.
It has the functions I use often, and may be hiding some more goodness behind that 2nd key in the upper left corner. Of course, I also have some complaints, even though I haven’t used it yet.
First, I think the calculator’s switch from basic to scientific when you turn the iPhone from portrait to landscape is a cute idea that will quickly wear out its welcome. Even though my favorite real-world calculator, the spectacular HP 15C, was landscape oriented, I don’t want my calculator switching back to basic mode just because I’ve turned it a bit. And I suspect people who just want the basic mode don’t want it shifting on them, either.
Next, Apple’s choice of some functions seems weird. The x3 key is a waste of space; in thirty years of calculator use, I’ve found cubing to be no more common than raising to any other power—the xy is sufficient. And while there’s a ex key to complement the ln, there’s no 10x key to complement the log. It wouldn’t bother me if there was no log button—calculators make base 10 logs much less useful than their continued presence in math handbooks would suggest—but if you’re going to have log, you really should have 10x. And percentage keys really have no business being on a scientific calculator; if you don’t know that percents are just decimals, you should stick to a basic calculator.
Finally, it looks like there’s just one memory location. One memory location!? On a computer with 8 or 16 gigabytes of memory? A computer with more power than what got Apollo 11 to the moon? Jesus!
Now, there are some good things about the calculator. The big = key is a nice touch. The pi key is something no scientific calculator should be without, but many keep it hidden behind a function key. And I like the idea of having a Rand key, even though I don’t know why you’d use a random number generator on a non-programmable calculator.
Speaking of which, why isn’t this a programmable calculator? Oh well, we’ll probably see a few of those in the App Store.
Address Book URLs, revisited
June 5th, 2008
In an earlier post I discussed the Mac’s addressbook URI scheme and how you can open a particular contact in your Address Book with a command like
open addressbook://A1A2AA41-FA30-40AE-9925-FD6DB270B0A5:ABPerson
from the Terminal. Everything after the double slash is the ID of the contact, accessible via the id property in AppleScript. In a similar way, you can create links in HTML documents which, when clicked, will open Address Book to the contact with that ID. (On my work computer, the above ID is for Apple, Inc.; the ID for Apple on your computer will be different.) I’ve been using such links a lot recently—for a serverless wiki-like system I’ll be writing about soon—and needed a utility for quickly extracting the ID of an Address Book contact.
The Address Book opener described in that earlier post used a pair of scripts—one AppleScript and one bash script. I never liked that setup, because the AppleScript was somewhat convoluted and because it used two script to do basically one thing. So even though the ID extraction script would be almost identical to the Address Book opener, I didn’t want to reuse my old code.
The script works like this: entering
abid john smith
at the Terminal prints the ID of the first contact with the names “john” and “smith.” Any number of arguments can be given to the abid command; the name of the contact must have each of the arguments as a substring. The search through the Address Book is case-insensitive. If no match is found, abid prints “No match!” instead of the ID.
Here’s the code. It’s written in Python using the appscript module.
1: #!/usr/bin/python
2:
3: import sys
4: from appscript import *
5:
6: # Find the contacts that have all the names in the given list.
7: def searchName(nameList):
8: names = [ x.lower() for x in nameList ]
9: # Get everyone whose last name matches.
10: matches = [ x for x in app('Address Book').people.get() if names[-1] in x.name.get().lower() ]
11: # Look for matches with the other names if there are any.
12: while len(names) > 1:
13: del names[-1]
14: matches = [ x for x in matches if names[-1] in x.name.get().lower() ]
15: # Return the list of matches.
16: return matches
17:
18: # Print the ID of the top match. Or print an error message.
19: try:
20: print searchName(sys.argv[1:])[0].id.get().replace(':', '%3A')
21: except IndexError:
22: print "No matches!"
The searchName function does almost the same thing as the identically-named function in my old AppleScript, but is much shorter and the logic is cleaner. Which is why I wanted to get away from AppleScript. I suppose the really cool, Lispy way to write searchName would be to make it recursive instead of iterative, but the while loop works fine.
The name of a contact is the full name—prefix, first, middle, last, suffix—as a string. If the contact is a company, the name is the company name. The list comprehensions in Lines 10 and 14 are basically substring filters.
You can see in Line 20 that I’ve URL-encoded the colon near the end of the ID, replacing it with %3A. The encoding isn’t necessary to make the URL legal, but eliminating that colon from the string makes the ID easier to use in the serverless wiki I mentioned at the top of this post.
The program works through the argument list from back to front instead of front to back. I expected to give it the names in “first last” order, and I thought that filtering by last name first would be faster because identical last names are rarer than identical first names and the list comprehension in the while loop would therefore be working with smaller lists. This notion could be completely off base; I haven’t done any benchmarking.
There’s no need to pass abid more than one name if one of the contact’s names is unique. I have only one Adolf in my Address Book, so
abid adolf
is all I need to get his ID. Nicknames will work if the nickname is a subset of the name given in Address Book. For example, if someone is listed as Robert Johnson, then
abid rob johnson
will find him, but
abid bob johnson
will not.
This script would likely be unnecessary if Address Book would show us the ID and let us copy it into the clipboard. But I haven’t found any way to get Address Book to show us the ID; if you know of a way, I’d like to hear about it.
Best weather webapp for the iPhone
May 5th, 2008
A few days ago, I wrote a post describing a little webapp I’d written that scraped a National Weather Service page to deliver a detailed text forecast to the iPhone. The next morning, I saw that @dsandler had Twittered that night about a Weather Underground service that did the same thing. And more.
The iPhone-optimized page is i.wund.com, and it provides
- Current conditions: temperature, wind, humidity, visibility
- Sun and moon rise and set times
- A radar image of your area
- Forecasts for the next several days
- Severe weather warnings
- A search field to get the weather for another city
Some of this is information that could be scraped from the NWS, and I had been thinking about improving my webapp by adding the current conditions and a search field. But there’s no reason for me to go any further; the Weather Underground page gives me everything I want in a compact, fast-loading package. I’ve bookmarked the page for Naperville and added an icon for it to my home screen. Had I known of i.wund.com earlier, I never would have bothered to develop my webapp. At least I learned a bit about Beautiful Soup.
You might wonder why I didn’t learn about the Weather Underground webapp before starting to develop my own. It is, after all, on Apple’s page for popular weather applications. Well, the sad fact is that I’ve been disappointed by so many of the webapps listed on Apple’s iPhone pages that I’ve gotten out of the habit of looking there first. I guess I should get back into the habit.
Trip card with OmniOutliner
May 4th, 2008
Two or three years ago, I began printing itinerary information on index cards so I had one place to look for flight, hotel, and rental car information. I’ve used this idea for business trips and vacations and it’s very convenient. In this post, I described a script that took its input from a plain text file and formatted and printed a nice looking card. This has worked well, but its one drawback is that its formatting is pretty rigid and I have to remember the best places to put the formatting codes to get good-looking output. If a month or two go by without a trip, I forget and have to look up an old card to remember how to do it.
I’ve been using OmniOutliner (the regular version that was bundled with my Mac) for several writing projects recently, and I’ve come to enjoy it. My trip cards are basically an outline, so it seemed a good fit. My only concern was whether I could coerce it to make a page that fit on an index card. That concern was answered when I looked at its Page Setup… options:
This sheet allowed me to set print margins that would place the outline at the top center, perfect for the way I manually feed index cards into my printer.
I played with the formatting until I got what I wanted, then saved the result as an OmniOutliner template file, which you can download here. Unzip the file, and you’ll get a template that looks like this when it’s opened:
Change or add the information for your trip, print it out on an index card, and you’re good to go.
Update
I’ve improved the formatting for the flight times, and the improvements are reflected in both the screen shot above and in the zip file. The template was made with OmniOutliner 3.6.5, which is the latest version as of this writing. I’m not sure if it will open and print properly on earlier versions.
In tweaking the formatting and print settings of this template, I’ve learned how touchy (that is, buggy) OO is with respect to margin and ruler settings. I’d upgrade to the Pro version if I were confident it didn’t have these bugs. I suppose I should download the Pro version and test it out.
Further update
I’ve resolved the margin problems by getting rid of the column title, putting that information in the top level of the outline, and shifting all the other items one level to the right. The template in the zip file reflects those changes, the screen shot above does not.
A DRM adventure
April 15th, 2008
Today I bought an engineering standard from a standards body that I won’t name (it’s an institute that promulgates national standards in America). I’ve bought many “electronic” standards from this group; they’ve always been plain old PDF files with marginal notes explaining that I’m the only one licensed to use the file, that I’m not allowed to make copies for others, and that I probably shouldn’t even let anyone else read the standard. But there’s never been any DRM to enforce these restrictions.
After downloading the standard I bought today, I soon realized that something was different. When I double-clicked on it, Preview launched, but all the pages were blank and the table of contents in the sidebar was filled with gibberish. OK, I said, let’s try Adobe Reader. I really hate Reader because it takes forever to load and acts like a Windows program by filling the entire screen, but I’ll use it if I have to. Reader launches and soon tells me that it needs a plugin to open the file. It sends me to a website where I can download and install the appropriately-named FileOpen plugin. That done, the PDF opened in Reader and I could start looking through the standard.
Reader soon alerted me that there was a newer version—I had version 7.x, and version 8.x was now available—would I like to download and install the newer version? Being all for progress, I said yes, and soon had Reader 8 in my Applications folder.
When I tried to open the standard in Reader 8, it went through the FileOpen plugin installation and then informed me that it couldn’t open the file because it had already been authorized to “another computer.” That computer was, of course, my computer, the very same computer I was now using, but apparently FileOpen’s authorization is actually applied to a particular application, not to the computer on which the application runs. Since I had authorized the PDF with Reader 7, it could only be opened with Reader 7 (and, presumably, only that particular copy of Reader 7; I’ll bet if I ever had to reinstall Reader 7, the replacement wouldn’t be able to open the file).
So I got on the phone to the standards body and complained, first about the authorization problem and second about forcing me to use Reader instead of Preview. I didn’t, and still don’t, expect anything to come of the second complaint—Windows users seem to think Reader is a fine program. And besides, it’s free, so what’s there to complain about? I did, however, manage to get them to let me to download another copy of the standard, one that I could authorize with Reader 8. The authorization of the new PDF worked—OK, Reader crashed the first time it tried to open the file, but it worked the second time—and the standard was sitting happily on my Desktop.
The first thing I did was print out a copy for safe keeping. In the old days, paper copies were all we had, and I didn’t want to put all my trust in a computer file and program that had already acted flaky. When I hit Command-P, the most amazing dialog box appeared. I didn’t take a screenshot of it at the time, and I’m writing this on a different computer, so I can’t take one now, so a description will have to suffice. (Update: Back at my work computer—here’s a screenshot.)
The dialog box didn’t look like any Print dialog I’d ever seen. There was no opportunity to choose a printer, no option to choose a paper tray, no option for collating or duplex printing, no way to print a range of pages. Here’s what you could choose:
- whether to print the entire document or just the current page (update: or the selection);
- how many copies to print.
The file would print out, on one side of the paper, on the default printer using the default paper tray. Don’t like that? Talk to the hand.
Apparently, Adobe was scared shitless of Apple’s standard Print dialog, with its dangerous Save as PDF… menu command.
This was a terrible state of affairs. I had just paid for a file that won’t open in Preview, that almost certainly won’t open with the next version of Reader, and that I can’t print the way I want. As I wrote on the Twitter, I was starting to turn into Cory Doctorow. I had to free that PDF!
And with a bit of work, I succeeded. I now have an unlocked, free to copy version of the PDF. I can open it in Preview, I can copy it to my laptop or a thumbdrive, and I can print it on whatever printer I want using whatever options I want. I won’t sell or distribute copies. I’m not interested in piracy; I just want a copy of the standard that’s usable and won’t turn into a pumpkin in a year or two.
I won’t tell you how I unlocked the file (because I am scared shitless of the standards body’s lawyers), but I will remind you of a few things you probably already know:
- Printer drivers create temporary files, called spool files, when you issue a print command.
- The spool file for a PostScript printer is a PostScript file.
- When a printer is paused, the spool file will sit in a particular directory until the printer is unpaused or the print job is deleted.
- PostScript files can be turned into PDFs using utilities like Ghostscript and ps2pdf.
- Google is your friend.
Godspeed, John Glenn.