October 3, 2016 at 10:09 PM by Dr. Drang
When PCalc 3.6.1 came out a little over a week ago, I saw something disturbing in the release notes:
Round function now always rounds half upwards to the nearest integer.
Perhaps the release notes were wrong. I opened PCalc and began testing.
- 1.5 rounded to 2, which is consistent with PCalc’s previous behavior and correct.
- 2.5 rounded to 3, which is new and wrong.
I got on the Twitter and shot off a direct message to PCalc’s developer, James Thomson, asking, in effect, WTF? His reply was pretty much what I expected. He’d been getting complaints from people who expected (for positive numbers, anyway) that halves would always be rounded up.
This is an all-too-common belief, instilled in children by parents and elementary school teachers who were taught it by previous generations of misguided souls. You may believe it yourself; I certainly did until I became enlightened sometime in grad school.
The problem with always rounding halves up is that in doing so, you introduce a persistent bias in whatever calculations you do with the rounded number. If you’re adding a list of rounded numbers, for example, the sum will be biased high.2
If you round halves to the nearest even number, though, the bias from upward roundings tends to be negated by an equal number of downward roundings.3 Overall, you get better results.
James expected the rounding change to raise hackles, and he was thinking about how to provide rounding options before I DM’d him. PCalc 3.6.2, released today, is the result. There’s a new Rounding item in the Advanced Settings. Tapping it gives you these six choices:
The top option is what the great unwashed expect. The second is the One True Rounding. The others are specialized options that are often needed when doing integer calculations. The floor and ceiling roundings, for example, get used in things like the Doomsday Algorithm.
PCalc’s “round” function rounds to integers, but rounding also applies when you have the display set to show a limited number of decimal places.
With Nearest or Even rounding and two decimal places chosen, the numbers ⅛, –⅛, ⅞, and –⅞ will display like this:
Note that although this display rounding follows the same rule as the round function, the numbers in the display still retain, behind the scenes, all their digits. Therefore, if we enter ⅛, we’ll see 0.12 in the display, but the number is really 0.125. Multiplying by 2 will give us 0.25, not 0.24. This differs from the round function, which really does change whatever number it’s applied to into an integer.
I was perfectly happy with PCalc’s old behavior, which always rounded halves to even, but, shockingly, James has to consider the needs and wants of customers other than me. Now there’s rounding for everyone.
Yes, IEEE 754 does define other rounding rules, but round to nearest, ties to even is the default. ↩︎
I’m assuming positive numbers in this example. When numbers come from measurements, that’s the most common case. ↩︎
If you’re thinking Benford’s Law screws up the notion that there will be an equal number of upward and downward roundings, think again. Benford’s Law applies to the most significant digit; rounding is done to the least significant digit. Only if you’re rounding to one digit will Benford’s Law lead to significant bias. And even then, rounding halves to even is still less biased than rounding them up. ↩︎
October 2, 2016 at 8:40 PM by Dr. Drang
I mentioned Supertop’s Castro podcast player a couple of days ago and decided I should devote a full post to it. It got a lot of coverage when Version 2.0 was released in mid-August, so it’s very unlikely you haven’t heard about its unique approach to organizing your playback queue via “triage” of your inbox of episodes. But since I’m still using it several weeks later, after it’s no longer a novelty, I thought it worth a second look.
Every review of Castro spent most of its time talking about the inbox, and rightly so. If you’re the kind of person who subscribes to many podcasts, but doesn’t have the time to listen to every episode, the inbox is a great way to swiftly look through your available episodes, queue up those that look the most interesting, and get rid of those that you’ll never get around to.
Here’s my inbox as of a few minutes ago:
Tapping on an episode brings up the dispatch buttons.
You can either
- Play the episode right from the inbox. I never do this.
- Send it to the top of the playback queue. If the episode currently at the top of the queue is in progress, the one in the inbox will go just under it.
- Send it to the bottom of the playback queue.
- Archive it. “Archive” is Supertop’s word, and I think it’s a poor choice. The episode, if downloaded, isn’t stored on your device, it’s deleted and only its metadata is kept. Basically, Archive = Delete.
One of the nice usability touches of Castro is that after you’ve dispatched an episode and it disappears from the inbox, the dispatch buttons appear automatically on the next episode in the list. Castro assumes—correctly in almost every case—that once you start triaging you want to continue, and it doesn’t force you to tap on the next episode to make the dispatch buttons reappear. This makes processing your inbox go very quickly.
Don’t be concerned, by the way, that the dispatch buttons limit you to the top and bottom of the playback queue. When you switch to the queue, you can rearrange the episodes by grabbing the dotted handles at the right and dragging them up or down.
One thing that bothered me about the initial reviews of Castro 2 was that they left the impression that you have to go through the triage procedure, even for those podcasts where you listen to every episode. That’s not the case. Although starting in the inbox is the default, you can change the settings for certain podcasts to go straight to the top (or bottom) of the queue.
In my set of subscriptions, new episodes of In Our Time, 99% Invisible, and Slate’s Political Gabfest go straight to the top of the playback queue. I never skip an episode. The Talk Show, on the other hand, too often has guests that offer no insight,1 so it gets the triage treatment. So does The Incomparable, although I’m beginning to think that’s a mistake. I set it up so new episodes would go to the inbox because I thought I wouldn’t want to listen to episodes about books and movies I haven’t read or seen. But I’ve found that I so rarely skip an episode, it isn’t worth the triage step.
The downsides to Castro that most reviewers mentioned back in August still hold true:
- Speeded-up playback in Castro is nowhere near the quality of speeded-up playback in Overcast or Pocket Casts. And it has nothing like Overcast’s Voice Boost feature. These deficiencies don’t affect me because I always listen at normal speed and the shows I subscribe to tend to be recorded with fairly uniform levels among the guests.
- Podcast discovery in Castro pales in comparison to discovery in other apps. Again, this doesn’t bother me, because I learn about new shows from Twitter and blogs. And I’m really subscribed to too many shows as it is.
I’ve been pretty promiscuous in my use of podcast players. Only Downcast and Overcast have stuck for more than a year (and I used Downcast back when there were few decent alternatives). Supertop could make a UI change to Castro tomorrow that sours me on it. But right now, Castro fits my way of listening better than the others.
These are almost always the tech reporters for “real” news outlets. ↩︎
September 30, 2016 at 9:41 PM by Dr. Drang
Yesterday morning, a new podcast episode popped up in my Castro inbox: a Relay B-Side called The Mac Draft.1 In it, Stephen Hackett, Christina Warren, Brian Sutorius and Ed Cormany did a four-round draft of significant or favorite Macintosh models. It’s a fun show, and you should give it a listen. (If you’re used to the draft shows on The Incomparable, fear not. This one’s only 48 minutes long.)
If you’ve been a Mac user for any length of time, you’ll probably think of which machines you would have drafted had you been on the show. I sure did, and that’s what this post is about. One restriction I’m going to impose on myself that the youngsters didn’t is that I will only draft Macs that I owned or was the primary user of at work. I like the personal touch.
Round One: SE/30
Yes, I know Ed drafted the SE/30 with his first round pick. I’m just going to assume that I went before him and sniped this pick. It’s my blog, and I’ll do what I want.
The SE/30 is the favorite of a lot of Mac users of a certain age. During the Mac’s 25th anniversary year, it was hailed as the greatest Mac ever by no less than Adam Engst, John Gruber, and John Siracusa. Introduced to the world at the beginning of 1989 and retired almost three years later, it was the apotheosis of the original Mac form factor.
I was teaching in 1989, and that fall one of my departmental colleagues got an SE/30. I was immediately filled with jealousy. When I changed jobs at the beginning of 1990, I persuaded my new employer to provide me with an SE/30, and the sour feeling of inadequacy washed away as I set it up at the corner of my desk.
That was the great thing about the SE/30. It was a serious workstation—the 68030 processor was a beast for its time—in a package that could fit almost anywhere. And when I was 30 years old, my eyes could handle the 9″, 512×342 pixel, black-and-white screen.
Round Two: 2010 13″ MacBook Air
No need for a stock image of this one; it’s sitting on my lap right now. This is not the much-maligned original Air. It’s the second-generation model, the one they got right. Real ports, no flip-down door, and Flash storage instead of a 4200 rpm, iPod-class hard disk.
This machine was a revelation when I first got it. It booted up so fast. In fact, everything seemed extra quick, an indication that most of what I was doing was constrained more by I/O than by processing. And of course, it was so thin and light. Even after six years, I don’t really feel it’s a museum piece—it still gets the job done.
Although I drafted the SE/30 first, that was a tactical decision. This is my favorite Mac.
Round Three: 512K
This was my first Mac. I bought it in grad school and wrote my Ph.D. thesis on it, using MacWrite, MacDraw, an ImageWriter, and Andy Hertzfeld’s amazing Switcher.
Although the formal name was 512K, based on its RAM, everyone called it the Fat Mac. Mine was configured with an external single-sided diskette drive to compliment the internal drive. That gave me a whopping 800 kB of storage for the system, the applications, and my files.
But the best thing about the Fat Mac was its RAM, something that didn’t become apparent until Switcher was released. Because applications had to be written to fit in the 128 kB of the original Mac, Switcher allowed you to fit the Finder plus two or three other applications into RAM and quickly flip back and forth between them. It wasn’t multitasking, but it was sure better than the usual launch/use/quit, launch/use/quit cycle. Given that I typeset by hand the equations for my thesis in MacDraw and then pasted them into MacWrite, Switcher was a godsend.
Round Four: Cheating
I have three Macs that I’d like to put into the fourth slot, and I don’t know which one to choose. So I’m just going to cheat by squeezing them all into one round. Again, it’s my blog.
- The Macintosh II. I won this computer in an Apple software contest run shortly after the Mac II was introduced. It wasn’t the grand prize, but it was good enough for me. The 68020 processor was a big step up from the 68000 of the Fat Mac and the Mac Plus, and of course it was the first Mac with color. It was a gigantic thing, though, and took up almost all of the desk I had it on.
- The PowerBook 170. This replaced my SE/30 as I started traveling more for my job in the early 90s. It was the high end model of Apple’s first three real notebook Macs (I don’t count the Macintosh Portable, which weighed about 16 pounds and used—you can look it up—a lead-acid battery) and was quite peppy. I remember it took a while for me to get used to the trackball after 5–6 years of mousing.
- The 12″ iBook G4. Lots of people love the 12″ PowerBook G4 (Brian picked it in Round Two). The iBook was about half the price and had close-enough specs. I got a lot of work done on this guy, but when OS X became Intel-only, its days were numbered. I got it at the end of 2004—my return to the Mac after years of using Linux—and held onto it until I got the Air in the fall of 2010. A sturdy performer for many years.
Surprisingly, the SE/30 is the only overlap between my choices and the ones made by Stephen, Christina, Brian, and Ed. A good part of this is because my choices are biased toward the early days of the Mac—a lot of beige, as Stephen would say. But it’s also because 30+ years of design includes something for everyone.
September 22, 2016 at 7:15 PM by Dr. Drang
Although Jake Arrieta still has good looking statistics for the season, my sense is that he’s been a mediocre pitcher since the All Star break. Am I right about this? Because it’s easy to get game-by-game statistics for almost any aspect of baseball, I figured it wouldn’t be hard to answer this question. More important, I could use this as an opportunity to practice my Python, Matplotlib, and Pandas skills.
Modern, stats-loving baseball fans can probably come up with a dozen ways to measure the quality of a pitcher, but I wasn’t interested in cutting-edge sabremetric research, so I stuck with pitching quality statistic of my youth, the earned run average. Remember, this is mostly an excuse to keep my Python skills sharp. Don’t write to tell me about your favorite pitching stat—I don’t care.
I started with a simple text file that looked like this:
Date,IP,ER 4/4,7.0,0 4/10,7.0,3 4/16,8.0,0 4/21,9.0,0 4/28,5.0,1 etc.
Each line represents one game. The date is in month/day format, the earned runs are integers, and the innings pitched are in a hybrid decimal/ternary format common to baseball, with the full innings before the period and the partial innings after it. The full innings are regular decimal numbers, but the partials represent thirds of an inning. The number after the period can take on only the values 0, 1, or 2.
Because I knew Pandas would look at the IP values and interpret them as regular decimal numbers, I decided to massage the input file first. Also, I figured it wouldn’t hurt to add the year to the game dates. After a couple of find-and-replaces in BBEdit, the file looked like this:
Date,FIP,PIP,ER 4/4/2016,7,0,0 4/10/2016,7,0,3 4/16/2016,8,0,0 4/21/2016,9,0,0 4/28/2016,5,0,1 etc.
Now there are separate fields for the full innnings and partial innings pitched. Time to fire up Jupyter and commence to analyzin’.
python: import pandas as pd from functools import partial import matplotlib.pyplot as plt %matplotlib inline plt.rcParams['figure.figsize'] = (12,9)
This is mostly standard preamble stuff. The odd ones are the
rcParams call, which makes the inline graphs bigger than the tiny Jupyter default, and the
functools import, which will help us create ERAs over small portions of the season.
Next, we read in the data, tell Pandas how to interpret the dates, and create an innings pitched field in standard decimal form:
python: df = pd.read_csv('arrieta.txt') df['Date'] = pd.to_datetime(df.Date, format='%m/%d/%Y') df['IP'] = df.FIP + df.PIP/3
Now we can calclulate Jake’s ERA for each game.
python: df['GERA'] = df.ER/df.IP*9
To get his season ERA as the season develops, we need the cumulative numbers of innings pitched and earned runs given up. That turns out to be easy to do with the Panda’s
python: df['CIP'] = df.IP.cumsum() df['CER'] = df.ER.cumsum() df['ERA'] = df.CER/df.CIP*9
Let’s see how his ERA has developed:
python: plt.plot_date(df.Date, df.ERA, '-k', lw=2)
The rise in May, though steep, doesn’t indicate poor pitching. Arrieta just started the season out so well that even very good pitching looks bad in comparison. It’s the jump in late June and early July that looks really bad. And now that I think of it, the Cubs did have a terrible three-week stretch just before the All Star break. Arrieta’s performance was part of that, so his slide started earlier than I was thinking.
But even after the big jump, there’s been a slow climb. So even though the Cubs have played excellent ball since mid-July, Arrieta hasn’t. To get a better handle on how poorly he’s been pitching, we could plot the game-by-game ERAs, but that’s likely to be too jumpy to see any patterns. A way to smooth that out is to calculate moving averages.
But what’s the appropriate number of games to average over? Two is certainly too few, but three might work. Or maybe four. I decided to try both three and four. To do this, I defined a function that operates on a row of the DataFrame to create a running average of ERA over the last n games:
python: def rera(games, row): if row.name+1 < games: ip = df.IP[:row.name+1].sum() er = df.ER[:row.name+1].sum() else: ip = df.IP[row.name+1-games:row.name+1].sum() er = df.ER[row.name+1-games:row.name+1].sum() return er/ip*9
if part handles the early portion of the season when there aren’t yet n names to average over. Looking at the code now, I realize this could have been simplified by eliminating the
if/else and just using
python: max(0, row.name+1-games)
as the lower bound of the slice. Oh, well. I’ll leave my crude code here as a reminder to think more clearly next time.
With this general function defined, we can use the
partial function imported from
functools to quickly define running average functions for three and four games and add those fields to the DataFrame.
python: era4 = partial(rera, 4) era3 = partial(rera,3) df['ERA4'] = df.apply(era4, axis=1) df['ERA3'] = df.apply(era3, axis=1)
Now we can plot everything:
python: plt.plot_date(df.Date, df.ERA3, '-b', lw=2) plt.plot_date(df.Date, df.ERA4, '-r', lw=2) plt.plot_date(df.Date, df.GERA, '.k', ms=10) plt.plot_date(df.Date, df.ERA, '--k', lw=2) plt.show()
As you can see, I didn’t bother to make this look nice. I just wanted to dump it all out and take a look at it. I don’t see much practical difference between the three-game average (blue) and the four-game average (red). Arrieta did have a good stretch there in late July and early August (which I hadn’t noticed), but he’s not been pitching well since then. It’s not unlike his early years with Baltimore. I’ll be curious to see where he’s put in the playoff rotation.
As is usually the case with my baseball posts, I’ve learned more about programming tools than I have about the game. I’ve used
partial many times in the past, and I always feel like a real wizard when I do. But I’ve never used
cumsum before, and I’m really impressed with it. Not only does it perform a useful function, it’s implemented in a way that couldn’t be easier to use for common cases like the ones here. I hope I don’t forget it.
Update Sep 24, 2016 9:33 PM
Of course I take full credit for Arrieta’s performance the day after this was posted. Jake obviously reads this blog, especially for the Python-related posts, and realized he needed to step up his game after seeing the clear evidence of how he’s gone downhill recently.