# The Siri data logger

I’ve been taking measurements professionally—forces, lengths, times, temperatures, speeds, voltages, etc.—for over half my life. Sometimes I’ve been lucky and the measurements have been collected and recorded automatically, but usually it’s been me with a pen and notebook, writing the numbers down by hand and then typing them into a computer later. Since getting an iPhone 5, though, I’ve been handwriting and transcribing a lot less. Many people complain about Siri, but she’s great with numbers.

Today, for example, I had to weigh several dozen small parts. I created a file with all the part identification numbers—basically a copy/paste—and saved it to Dropbox. I then opened the file in Notesy, put the cursor after the first part number (which already had a separating space after it) and started a process of weighing and dictating for each part.

I’d tap to move the cursor to the next part while the scale was settling down, then lift the phone to my ear, dictate the weight, and set the phone back down on the bench. Siri would automatically go into and out of dictation mode as I repositioned the phone, and I could check the number she typed as I reached in to take the piece off the platform. It went very quickly, and Siri never made a transcription error, probably because numbers are pretty easy to transcribe. When I walked back from my lab to my office, there was a file with 144 part weights waiting for me. Heaven.

I’ve been doing similar things for over a year, and have developed a few techniques for making the transcription go smoother.

1. I make it a habit to say “zero” instead of “oh.” Hence, the weight in the photo was “three point seven zero five eight.” In the early days, I’d often slip and say “oh.” While that never led to a transcription error, it seemed like asking for trouble.
2. Siri’s style is to spell out the integers 1–9 as “One” through “Nine”1 instead of typing numerals. The way around this to use the old programmer’s trick of promoting the integer to a float. I now say “two point zero” when I want a 2. If the “.0” part needs to be deleted, I do it with a search/replace in post-processing.
3. When I need to put more than one number on a line, I separate them with commas. “Five point five comma six point seven two” turns into

5.5, 6.72


If the comma shouldn’t be in the data file, I delete it later with a search/replace. You might think that saying “space” between the two numbers would work, but I’ve found that Siri is too literal for that—she types out the word “space” between the numbers.

Update 2/25/14
Unsurprisingly, there are direct ways to handle both the small integer and space issues:

@drdrang Try saying “numeral two spacebar numeral five point one” to Siri. Produces “2 5.1”, with no need for “point zero”s or commas.
— Jason Robinson (@VafeR) Tue Feb 25 2014 5:35 AM

When I saw this last night, my initial reaction was that I could see myself using “spacebar,” but that I’d never get into the habit of prefixing numbers with “numeral.” After sleeping on it, I’m not so sure—I’ll have to give both a try to see what sticks. Thanks to Jason for the suggestions.

There are still times when I can’t use Siri when taking measurements. The background noise may be too loud or my phone’s connection may be too spotty. In these cases, I write down the numbers on paper as in the old days, but I dictate them into a file using Siri when I’m back in better conditions. And now with Mavericks, I don’t even have to use my phone. I can sit at my desk, notepad in hand, and talk the numbers right into a file—no special headset or microphone needed. While this isn’t as good as direct dictation, at least it eliminates the typing.

To get more immediate feedback as you dictate into your computer, turn on Enhanced Dictation in the Dictation & Speech panel of System Preferences.

It requires the download of a 785 MB file, but it’s worth it, especially if you want to use Siri dictation for things other than numbers. If you want more ideas on using dictation to make your life easier, David and KatieFloyd did an excellent Mac Power Users episode on dictation, with both Siri and Dragon Dictate, back in December.

1. Actually, she usually renders 2 as “to,” because when given a single word, she guesses wrong.

# Weather Underground in Pythonista

I woke up yesterday morning, checked the weather on my phone, and got a vague sense of dissatisfaction that’s probably familiar to you. This weather app doesn’t have Feature A; that app does, but it doesn’t have Feature B; this other app has both, but they’re inconveniently hidden behind a series of taps and swipes. What I want is a custom-built app that presents just the information I use in a format that’s convenient to me. Last night I took the first step toward such an app by building a simple, locally hosted webapp in Pythonista.1

Longtime readers may remember my first attempt at something like this, a CGI script built on the pywapi library that I access through Mobile Safari and a home screen button. It produces a page that looks like this:

Three years later, I still use it occasionally, but it has the distinct disadvantage of being hard-coded to the western suburbs of Chicago. This is fine most of the time, but is useless when I’m traveling.

The difference between now and three years ago is that now I have Pythonista and its location module. There are two ways to take advantage of this:

1. Rewrite the CGI script to accept latitude and longitude as parameters and have it produce a page for that location. Use Pythonista to get the current location and open the location-encoded URL.
2. Write the whole thing in Pythonista.

I chose the second because I thought a self-contained system would be more interesting to write.

The new script would need a radar map, which ruled out using pywapi and Dark Sky (now Forecast) as the underlying API. That led me to the Weather Underground API, which offers both a simple calling convention and a rich set of data, including maps.

I will say, though, that although the Weather Underground API returns an abundant data set, it’s very poorly organized. Numerical data sometimes comes in the form of a floating point number and other times as a Unicode string. The documentation is thin, so you need to run experiments to learn, for example, that actual temperature is a float but the “feels like” temperature is a string. Madness. Sunrise and sunset times are even worse; they’re returned as two strings, one for the hour and another for the minute.

Anyway, here’s my first pass at WeatherUnderground.py. It doesn’t have any forecast information yet, and it’s woefully short on error checking (by which I mean it has no error checking). Those will come later, as will a rewriting of the cruftier bits.

python:
1:  #!/usr/bin/python
2:
3:  import json
4:  import requests
5:  import time
6:  from datetime import datetime
7:  import location
8:  import webbrowser
9:  import BaseHTTPServer
10:
11:  ########################## Functions ############################
12:
13:  def wunder(lat, lon, wukey):
14:    "Return a dictionary of weather data for the given location."
15:
16:    # URLs
17:    baseURL = 'http://api.wunderground.com/api/%s/' % wukey
18:    conditionURL = baseURL + 'conditions/q/%f,%f.json' % (lat, lon)
19:    astroURL = baseURL + 'astronomy/q/%f,%f.json' % (lat, lon)
21:                       + '?centerlat=%f&centerlon=%f' % (lat, lon - 1) \
23:                       + '&timelabel.x=10&timelabel.y=350' \
24:                       + '&newmaps=1&noclutter=1'
25:
26:    # Collect data.
27:    c = requests.get(conditionURL)
28:    current = c.json()['current_observation']
29:    a = requests.get(astroURL)
30:    astro = a.json()['moon_phase']
31:
32:    # Turn sun rise and set times into datetimes.
33:    rise = '%s:%s' % (astro['sunrise']['hour'], astro['sunrise']['minute'])
34:    set = '%s:%s' % (astro['sunset']['hour'], astro['sunset']['minute'])
35:    sunrise = datetime.strptime(rise, '%H:%M')
36:    sunset = datetime.strptime(set, '%H:%M')
37:
38:    # Mapping of pressure trend symbols to words.
39:    pstr = {'+': 'rising', '-': 'falling', '0': 'steady'}
40:
41:    # Construct the dictionary and return it.
42:    wudata = {'pressure': float(current['pressure_in']),
43:              'ptrend': pstr[current['pressure_trend']],
44:              'temp': current['temp_f'],
45:              'desc': current['weather'],
46:              'wind_dir': current['wind_dir'],
47:              'wind': current['wind_mph'],
48:              'feel': float(current['feelslike_f']),
49:              'sunrise': sunrise,
50:              'sunset': sunset,
51:              'moon_pct': float(astro['percentIlluminated']),
52:              'moon_age': int(astro['ageOfMoon']),
54:    return wudata
55:
56:
57:  def wuHTML(lat, lon, wukey):
58:    "Return HTML with WU data for given location."
59:
60:    d = wunder(lat, lon, wukey)
61:
62:    # Get data ready for presentation
63:    sunrise = d['sunrise'].strftime('%-I:%M %p').lower()
64:    sunset = d['sunset'].strftime('%-I:%M %p').lower()
65:    temp = '%.0f&deg;' % d['temp']
66:    pressure = 'Pressure: %.2f and %s' % (d['pressure'], d['ptrend'])
67:    wind = 'Wind: %s at %.0f mph' % (d['wind_dir'], d['wind'])
68:    feel = 'Feels like: %.0f&deg;' % d['feel']
69:    sun = 'Sunlight: %s to %s' % (sunrise, sunset)
70:    moon = 'Moon: %s%% at %s days' % (d['moon_pct'], d['moon_age'])
71:
72:    # Assemble the HTML.
73:    html = '''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
74:    <html>
76:    <meta name="viewport" content = "width = device-width" />
77:    <title>Weather</title>
78:    <style type="text/css">
79:      body { font-family: Helvetica; }
80:      h1 { font-size: 175%%;
81:        text-align: center;
82:        margin-bottom: 0; }
83:      h2 { font-size: 125%%;
84:        margin-top: 0;
85:        margin-bottom: 0; }
86:      #now { margin-left: 0; }
87:      #gust { padding-left: 2.75em; }
88:      div p { margin-top: .25em;
89:        margin-left: .25em; }
90:    </style>
92:    <body onload="setTimeout(function() { window.top.scrollTo(0, 1) }, 100);">
93:    <h1>%s &bull; %s </h1>
94:
95:    <p><img width="100%%" src="%s" /></p>
96:
97:    <p id="now">%s<br />
98:    %s<br />
99:    %s<br />
100:    %s<br />
101:    %s<br /></p>
102:
103:    </body>
104:    </html>''' % (temp, d['desc'], d['radar'], wind, feel, pressure, sun, moon)
105:
106:    return html
107:
108:
109:  ######################### Main program ##########################
110:
111:  # My Weather Underground key.
112:  wukey = 'xxxxxxxxxxxxx'
113:
114:  # Get the GPS info.
116:  time.sleep(2)
117:  loc = location.get_location()
118:
119:  # Generate the HTML.
120:  html = wuHTML(loc['latitude'], loc['longitude'], wukey)
121:
122:  # Create the request handler.
123:  class MyHandler(BaseHTTPServer.BaseHTTPRequestHandler):
124:    def do_GET(s):
125:      """Respond to a GET request."""
126:      s.send_response(200)
129:      s.wfile.write(html)
130:
131:  # Start the server and show the page.
132:  server = BaseHTTPServer.HTTPServer(('', 8888), MyHandler)
133:  webbrowser.open('http://localhost:8888')
134:  server.handle_request()


The wunder function makes two calls to Weather Underground and returns a dictionary with the data I want to present. Whatever conversions need to be made between strings, floats, and times are done here. I’ll need to add one or two new calls to get forecast information, and I may need to restructure the dictionary.

The wuHTML function returns the HTML for the page being served. This is pretty crude code, written quickly to see if I could get something working. As I add more information, and maybe some interactivity, I’ll have to move to a real templating system.

The main program starts with my Weather Underground API key. If you want to do something like this, you’ll have to get your own. The key is free, but there are restrictions on how many API calls you can make. They’re generous for personal use—10 calls per minute, up to 500 calls per day—but couldn’t be used for a commercial product. That requires a paid account.

The script then gets the phone’s location and feeds it to wuHTML to generate the page. At this point, I originally thought I could display the HTML using the webbrowser module, but that doesn’t seem to be the case. In fact, I couldn’t figure out any method to simply shoot the HTML to a browser. Either I’m using the wrong Google search terms or it really isn’t allowed.

That led me to the BaseHTTPServer code. Lines 123–129 set up a handler class that serves the HTML in response to a GET request, and Lines 132 and 134 run the server. I have no previous experience with BaseHTTPServer, and I may be doing this part wrong. I have occasionally gotten “Address already in use” errors on Line 132, so I clearly don’t have everything right yet.

Line 133 launches Pythonista’s built-in browser to display the page, which looks like this:

As you can see, I stole most of the HTML from my old CGI script. I think I’ll be dumping the moon information, and I’ll definitely be adding wind gusts once I learn whether the WU API always includes a gust item in the returned JSON.

The radar map, by the way, isn’t centered on my location. I’ve done that deliberately because I’m more interested in what’s west of me than what’s east of me. The positioning of the map is done in Line 21, where the center of the map is set one degree west of the current location. That’s the kind of customization you can achieve when you write the code yourself.

I’m not sure how I’ll handle the forecasts, but I’m leaning toward a detailed view of the remainder of today; morning, afternoon, and evening summaries for tomorrow; and maybe just high and low temperatures for the following day or two. There will be a lot of messing around as I learn what works and what doesn’t, but I don’t want to clutter ANIAT with every detail. At the moment, the code is in a gist; as it grows, I think I’ll move it to a proper GitHub repository.

1. I’m not a real programmer, just a dabbler, so Objective C and Cocoa are out of the question.

# Leveraging little lab labels

I’m starting a long series of tests at work involving many different parts exposed to a variety of chemicals. Devising the test setup was interesting, and I expect analyzing the results to be interesting, too. But this middle section—actually doing the tests—is going to be pretty boring. Lots of bookkeeping, not much thinking. This is the stage that’s easy to screw up because it seems too easy to screw up. Organization is key.

Altogether, I have almost 150 part/chemical combinations to keep track of simultaneously. Some of the parts are easy to distinguish visually, but many aren’t; none of the chemicals are visually distinct. The parts themselves can’t be labeled because that might interfere with the chemical treatment. Because the treatment will take place in small glass jars, the best way to keep track of the tests is, pretty obviously, to label each jar with the part and chemical.

How to do the labeling? Labs that perform the same kinds of tests in large batches over and over have standard procedures for running standard tests on standard samples, and have standard lab management software that produces standard labels. I don’t have that kind of lab or that kind of software, but I do have a label printing script that’s flexible enough to handle most of what I run into. I just need to create the proper input file.

Let me stop here for a bit and say that writing the information by hand on the lid of each jar is certainly an option, and that’s what I’ve done for smaller scale tests. But writing on 150 jars is not only horrifically dull, it’s error-prone. The same is true of using a template; time can be saved by copying and pasting, but it’s easy to paste in the wrong spot. The time-saving aspects of automation are what people usually talk about, but its accuracy is just as important.

I had a list of all the parts, one part per line. What I needed was to convert that list into the input format needed for my ptlabels (“print tiny labels”) program. That meant transforming something like this:

0001
0002
0003
0004


into something like this:

#0001|Temperature X
Chemical A

Chemical B

Chemical C

Chemical D

#0002|Temperature X
Chemical A

Chemical B

Chemical C

Chemical D

#0003|Temperature Y
Chemical A

Chemical B

Chemical C

Chemical D

#0004|Temperature X
Chemical A

Chemical B

Chemical C

Chemical D


Did I mention that the tests were going to be carried out at different temperatures? That’s another bit if information that has to be on the label.

There are many ways to do this sort of transformation, but because I’m using BBEdit, its command came to mind. The first transformation was this,

which added the temperature information and the hash and vertical bar formatting symbols. I then went through the list and changed Temperature X to Temperature Y where appropriate—about a half-dozen places. As there was no relationship between part number and testing temperature, brute force was the fastest approach.

The next step was to add a suffix only.

You can’t see everything in the little text field, but you can probably guess that the suffix is

&Chemical A&&Chemical B&&Chemical C&&Chemical D&


Why didn’t I have this as part of the suffix in the previous step? Because that would have made changing those half-dozen Xs into Ys more difficult because they’d be buried in the middle of their lines.

The last step is a regular expression (grep) find/replace that turns all the ampersands into newlines.

There are probably better ways to transform the text, but these are the steps I thought of first. Trying to think of a better solution would’ve just wasted time.

I saved the transformed file and passed it to ptlist, which printed out all the labels on a couple of sheets of Avery 5167 paper. Avery 5167 is meant to be used for return addresses but I’ve used it label several types of lab sample.

The jars come from the supplier in boxes of 24. An unplanned benefit of using stickers instead of hand-printing the information was that I could label all the jars without removing them from the boxes.

This kept my limited bench space tidier. It also kept the jars in order, which will speed the adding of parts and chemicals in the next stage.

This post, there’s nothing bright about it, and there’s probably nothing in it that anyone else can use directly. But by leveraging previous work and using the built-in features of my text editor,1 I was able to save myself at least an hour of work, and I ended up with something that was neater and better organized than I would have had otherwise.

1. Those of you with access to multiple cursor editing are probably thinking of a faster way to transform the parts list.

This is a followup to my Mavericks Mail post of a couple of days ago. I think I have a better understanding of one of the problems I was having and a workaround for the other.

I can hear many of you saying “Just drop Mail already. It isn’t worth the effort.” You’re probably right, but I’m not in a position right now to switch. Because of the nature of my work, I have a hybrid system in which some of my mail is saved in local mailboxes and some is saved in IMAP mailboxes. Before I move to another mail client, I have to be sure I can export/import all of that structure without losing anything. Maybe that’ll be easy, maybe it won’t, but to be on the safe side, I don’t want to start the migration until my work schedule is a little more open than it is right now.

There’s also the issue of which client to switch to. I mentioned MailMate in the earlier post, but several friends on Twitter have recommended Airmail. And I should probably look into Postbox, too. Whichever I choose, it isn’t going to be a drop-in replacement for Mail. And wouldn’t it be wonderful if Apple fixed Mail and made it so I don’t have to decide?

As for the problems I’ve been having with Mail, let’s start with an update on searching. Many of you suggested that the problem was with Spotlight and that a reindex of its database might be a solution. I think that’s exactly right, and I kind of thought that running the command (which I mentioned in the update at the bottom of the post) would do just that. Apparently not, although it may have started the reindexing process.

After posting the update Monday morning, my disk chitter-chattered for about an hour. Activity Monitor told me one of the md* processes was running all that time, not taking up much CPU time, but definitely exercising the read/write heads. As best I can tell, searching in Mail has worked correctly since then.

I’m willing to accept a fair amount of the blame for this. The clearly incorrect search results I was getting should have prompted me to think of Spotlight. I have only two things to say in my defense:

1. I almost never use Spotlight proper, by which I mean the thing in the menubar. If I use it more than ten times a year, I’d be greatly surprised. I’m more comfortable using grep and ack to search for words in text files, and my world is almost entirely text files. So Spotlight just isn’t something that comes to mind, and I haven’t had enough experience with it to immediately recognize the signs when it’s gone flaky.
2. Apple seemingly provides no automatic Spotlight diagnosis. If you don’t know your search results are incomplete, what would inspire you to reindex your drive? Why don’t Apple’s set of system launchd processes keep Spotlight in tune or a least warn you when it’s out of whack?

Speaking of launchd, I have a workaround for Mail’s no send/no receive problem. It starts with this AppleScript, which runs the two menu commands Apple recommends:

applescript:
1:  tell application "System Events"
2:    tell process "Mail"
4:        tell menu bar item "Mailbox"
6:            click menu item "Take All Accounts Offline"
7:            delay 2
8:            click menu item "Get New Mail"
9:            --say "reset"
10:          end tell
11:        end tell
12:      end tell
13:    end tell
14:  end tell


The two-second delay in Line 7 came after some experimentation—with no delay, the “Get New Mail” command wouldn’t work.

The AppleScript is named Reset Mail.scpt, and it’s stored in ~/Library/Scripts/Applications/Mail which makes it available to FastScripts if I want to run it manually.

Mostly, though, I want it to be run periodically by launchd. So I created this launch agent plist file,

xml:
1:  <?xml version="1.0" encoding="UTF-8"?>
2:  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3:  <plist version="1.0">
4:  <dict>
5:    <key>Disabled</key>
6:    <false/>
7:    <key>Label</key>
8:    <string>com.leancrew.ResetMail</string>
9:    <key>ProgramArguments</key>
10:    <array>
11:      <string>/usr/bin/osascript</string>
12:      <string>/Users/drang/Library/Scripts/Applications/Mail/Reset Mail.scpt</string>
13:    </array>
14:    <key>StartInterval</key>
15:    <integer>600</integer>
16:  </dict>
17:  </plist>


named it com.leancrew.ResetMail.plist, and saved it in ~/Library/LaunchAgents. Items in that folder are supposed to get loaded into launchd when you log in. You can also force an agent to load and start with these commands from the Terminal:

launchctl load ~/Library/LaunchAgents/com.leancrew.ResetMail.plist
launchctl start com.leancrew.ResetMail


The best quick introduction to launchctl—all killer, no filler—is this post by Nathan Grigg. I return to it every time I need a refresher, which is basically every time I need to use launchctl.

A few words on these files:

• The commented out Line 9 in the AppleScript is a leftover from the debugging process. I used that line in the early stages of development to make sure launchd was running the script when it was supposed to. I decided to leave it there, inactive, in case I need to debug again in the future.
• The plist file will need some editing if you want to use on your computer. You could change the label string in Line 8, but you don’t need to. You definitely need to change the path to the AppleScript in Line 12 to wherever you keep Reset Mail.scpt. Line 15 tells launchd to run the script every 10 minutes (600 seconds); change it to whatever feels comfortable to you.

Whenever I get around to switching mail clients, I’ll have to remember to tell launchd to stop running the script. This will stop it immediately:

launchctl stop com.leancrew.ResetMail

I think I can leave the plist file where it is if I change the Disabled setting in Line 6 from <false/> to <true/>, or maybe I’ll have to use the -w option with launchctl unload. Most likely, though, I’ll just move the file out of the LaunchAgents folder so there’s no chance of it loading the next time I log in.