XSLT, of course
May 17th, 2012 at 12:35 am by Dr. Drang
A couple of days ago, I stole Jason Snell’s idea of using Yahoo! Pipes to filter the 5by5 After Dark podcast feed to limit it to just the After Darks of shows I listen to. In the first comment on the post, Gabe (Macdrifter) asked whether it was a good idea to rely on Yahoo! for this kind of service.
For reasons I can’t fully explain, I have a little more faith in Yahoo! than almost every other sentient being on the planet. Even though I can’t think of a single thing of value the Yahooers get out of Pipes, I don’t think they’ll be shutting it down anytime soon. And even if they did, losing a sightly more convenient form of a podcast feed wouldn’t drive me to the depths of despair.
But the notion of creating and hosting my own filtered feed was appealing. I asked Twitter for examples of existing libraries or frameworks that I could use for that purpose. I got a few answers, but none that I’d feel comfortable using. Using someone’s half-finished implementation in a language that only half understand does not appeal.
Tonight, though, I realized that a standards-compliant solution was staring me in the face: XSLT. RSS is just XML, and the way to filter XML is through XSLT. I’m hardly an XSLT expert, but I’d used it several years ago when I modified Fletcher Penney’s MultiMarkdown, so I knew I could do it again, especially for a simple filter.
As a language, XSLT blows, but once you accept its absurd verbosity you can get on with life and do the job. And there are plenty of examples on the internet to crib from. I found this one to be close enough to what I was doing. I downloaded the After Dark feed and started experimenting. After several dead ends, I wound up with this:
xml:
1: <?xml version="1.0" encoding="utf-8" ?>
2: <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
3:
4: <xsl:output method="xml" encoding="utf-8" indent="yes"/>
5:
6: <!-- First, get everything. -->
7: <xsl:template match="node() | @*">
8: <xsl:copy>
9: <xsl:apply-templates select="node() | @*"/>
10: </xsl:copy>
11: </xsl:template>
12:
13: <!-- Then restrict to just certain items. -->
14: <xsl:template match="/rss/channel/item">
15: <xsl:if test="title[contains(., 'Incomparable') or contains(., 'Talk Show') or contains(., 'Back to Work') or contains(., 'Hypercritical')]">
16: <item>
17: <xsl:apply-templates select="node()" />
18: </item>
19: </xsl:if>
20: </xsl:template>
21:
22: </xsl:stylesheet>
The pattern used here—start out with a stanza that grabs everything and then follow up with one that picks out the nodes you need—seems to be standard; I saw it in several examples. What makes it useful in this situation is that the first stanza gives you all the channel information—which you need to keep—without having to write any code specific to that structure.
Let’s be more concrete. Here’s the skeleton of the After Dark RSS feed:
xml:
<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:admin="http://webns.net/mvcb/" xmlns:atom="http://www.w3.org/2005/Atom/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:media="http://search.yahoo.com/mrss/" version="2.0">
<channel>
<title>After Dark</title>
<link>http://5by5.tv/afterdark</link>
<pubDate>Wed, 16 May 2012 18:00:00 GMT</pubDate>
<description>This is what happens after Dan and his co-hosts hit "STOP" and their official shows are over. Behind the scenes, casual, unedited, and uncensored. Hosted by Dan Benjamin.</description>
<!-- many nodes with info about the podcast in general -->
<item>
<title>After Dark 157: After Amplified #7</title>
<!-- more info about this episode -->
</item>
<item>
<title>After Dark 156: After Back to Work #67</title>
<!-- more info about this episode -->
</item>
<item>
<title>After Dark 155: After Build and Analyze #77</title>
<!-- more info about this episode -->
</item>
<!-- and so on, with more items -->
</channel>
</rss>
My goal is to get everything in this feed except the <item>s for shows I don’t listen to. I don’t care about the structure of the nodes that provide the general podcast information, I just know that I want them in the output. The “get everything” stanza in Lines 6-11 provides that.
Similarly, the stanza in Lines 13-20 get all the <item>s I want based entirely on what’s in their <title>s. I don’t need to know anything else about the internals of an <item>.
With this filter, I can generate my own After Dark feed via
curl -s http://feeds.feedburner.com/5by5-afterdark \
| xsltproc rssfilter.xslt - \
| sed '/^ *$/d' > afterdark.rss
Aren’t pipelines fun? I’ve split it over several lines to make it easier to read. The command
- Downloads the feed.
- Filters it as described above.
- Deletes the blank lines.
- Saves it to a file.
Deleting the blank lines with sed wasn’t really necessary, but it neatened up the output feed.
So what good is this? Well, if you have shell access to a server with curl and xsltproc,1 you could set up a cron process to run this pipeline periodically and then subscribe to that afterdark.rss file. If you don’t have shell access on your server, you can set up a schedule to run the pipeline locally and upload the resulting file.
Either way, you have a filtered feed that doesn’t rely on a free online service that could be yanked at any time and which only uses standard tools.
-
Or equivalent programs. ↩
Enhancing screenshots with PIL
May 15th, 2012 at 9:16 pm by Dr. Drang
When I wrote a post last week about using Flickr as a CDN, there was a paragraph that bothered me:
One thing did change when I started using Flickr to host screenshots: I had to change my screenshot format from PNG to JPEG. Flickr accepts PNGs, but it saves the resized images as JPEGs, and any transparent areas in the original PNG become black in the resized JPEGs. Since many of my screenshots are of windows with the semi-transparent shadow, the resized versions on Flickr look like shit. With JPEG screenshots, the window shadow backgrounds are white, which is just what I want.
While a white background to the window shadow is generally OK when I include a screenshot here, it isn’t always “just what I want.” For example, when I want to put a window screenshot in an update to a post, the white background of the screenshot looks stupid against the light green of the update <div>. And when I need to mix screenshots of multiple windows with those of single windows, there’s a clash of styles because my multiple window screenshots usually have a bit of my Desktop color (Solid Aqua Dark Blue) showing.1
Unfortunately, the screencapture command, which my snapflickr program calls, doesn’t have an option to layer in a background color. So I needed to do the layering myself. Since snapflickr is written in Python, the natural way to add the background is through the Python Imaging Library (PIL).
I was a little apprehensive. I’ve used PIL several times in the past and never had any real trouble installing it, but I hadn’t installed it under Lion, and in particular I hadn’t installed it under Lion after Xcode moved from /Developer to /Applications. Gabe Weatherhead (Macdrifter) has tweeted recently about the problems he had getting PIL installed, and I was not inclined to follow in his footsteps (he eventually got it installed via Macports, a system I never want to see again). The comments on this post suggest that Apple’s movement of Xcode, combined with the fact that it no longer installs the command line tools with Xcode by default, has turned the installation of PIL into an almost superhuman effort.
But I really wanted that background color in my screenshots, and I didn’t want to do it through AppleScripting Acorn if I could help it. So I started by installing the Command Line Tools through Xcode’s Preferences:
Then I crossed my fingers and typed
sudo easy_install pil
It worked on the first try! Oh, there were compilation warnings aplenty, most of which had to do with casting 32-bit values as 64-bit (or something like that, I didn’t save them), but when I was done, I had a PIL that was usable. Apparently, PIL just likes me better than it likes Gabe.
The right library makes all the difference in the world. Adding a background to my screenshots was very easy with PIL. Here are the lines I had to add to snapflickr:
95: # Add a desktop background if it's a shadowed window screenshot.
96: snap = Image.open(fn)
97: if snap.mode == 'RGBA':
98: # Crop it to a more reasonable size,
99: # and paste it over a solid-colored background.
100: cropbox = (margin[0],
101: margin[1],
102: snap.size[0]-margin[2],
103: snap.size[1]-margin[3])
104: snap = snap.crop(cropbox)
105: bg = Image.new('RGB', snap.size, bgcolor)
106: bg.paste(snap, None, snap)
107: bg.save(fn)
We start on Line 96 by opening the screenshot, which has already been saved to the Desktop as a PNG file and has been given the name stored in the fn variable. Since I want to add the background color only to window screenshots, and window screenshots in PNG format have an alpha channel to handle the shadow transparency, the test on Line 97 will return true only for window screenshots. The remaining lines will be skipped for rectangular area screenshots.
I like the shadow Apple adds to windows, but when you take a screenshot with a shadow, there’s a lot of extra margin beyond the shadow itself. Lines 100-104 crop out the extra using a margin tuple defined earlier in the program as
margin = (25, 5, 25, 35)
These are how many pixels get cropped off the left, top, right, and bottom sides. I chose them after a bit of experimentation because I thought they gave a decent look to the result.
Line 105 creates a solid background image of the same size as the newly-cropped screenshot and filled with the color defined near the top of the program as
bgcolor = (61, 101, 156)
which are the decimal RGB values for Solid Aqua Dark Blue.
Line 106 pastes the screenshot on top of the background. The third argument to paste is the mask. According to the PIL documentation, calling paste with three arguments
updates only the regions indicated by the mask. You can use either “1”, “L” or “RGBA” images (in the latter case, the alpha band is used as mask).
So using snap as both the source and the mask is like using two layers in a program like Photoshop or Acorn. The screenshot is set in a layer in front of the background layer, which is visible through the transparent parts. The image is then flattened into a single layer.
The new image with the background color is then saved, overwriting the original screenshot. Because the bg image is in RGB mode (no alpha channel), so is the saved image. When the image is uploaded to Flickr, the original size will be a PNG and all the other sizes will be JPEGs, but there’ll be no funny black background areas in the resized images because Flickr won’t have to deal with transparency in the original.
It’s entirely possible that I’ll fiddle around some more with the margins, trying different crop amounts to see if I like something better. But I think the basic structure of the code is set.
-
Actually, what’s usually showing in a multi-window screenshot is Backdrop. Although I like to keep a tidy Desktop, there are usually at least a few icons on it. Backdrop is an app that covers up the Desktop—and any app windows it’s stacked in front of—with a nice solid color. Great for screenshots and screencasts. ↩
Stealing ideas
May 14th, 2012 at 10:24 pm by Dr. Drang
Tonight I saw these two tweets from Jason Snell and smacked my forehead. Of course!
The “Smart Playlists” feature in Instacast 2 isn’t smart. I’d like only @5by5 After Darks from certain podcasts, but it can’t filter that.
— Jason Snell (@jsnell) Mon May 14 2012
(Solution: I made a Yahoo Pipe that takes in the After Dark feed, filters by show title, and then outputs a new RSS feed.)
— Jason Snell (@jsnell) Mon May 14 2012
The After Dark feed includes After Darks from all the 5by5 shows. This is probably a good way for Dan Benjamin to introduce all his shows to those who currently listen to just a couple (and it’s certainly easier to manage one After Dark feed than a dozen), but I’ve had to declare podcast bankruptcy1 more than once and don’t need any more suggestions.
Hence the value of Jason’s tweets. Yahoo! Pipes, which was big news four or five years ago, is still there and still working, but I never would’ve thought of it, even though it’s absolutely perfect for this sort of situation. It’s hard to imagine how Yahoo! makes any money off of Pipes, but as long as it’s there, we should take advantage.
I’m not privy to what Jason’s After Dark pipe looks like, but here’s mine:
This is about as simple as a Pipe can get. The original feed is
http://feeds.feedburner.com/5by5-afterdark
and I allow only those items whose titles contain the key title words of the podcasts I listen to. I should probably add Mac Power Users to the list, even though it’s never had an After Dark—there’s always a first time.
Sometimes you don’t need chapter and verse; just a word or two is sufficient. My thanks to Jason for providing the words.
-
You know, like email bankruptcy, but with podcasts. ↩
Links
May 13th, 2012 at 11:43 pm by Dr. Drang
Last week I was going to write a post about Leslie speakers. My hook was going to be the use of “Tomorrow Never Knows” in last week’s episode of Mad Men. A famous story about the recording of “Tomorrow Never Knows” is that John’s voice was fed through a Leslie speaker to get that ethereal sound. This was in lieu of
- a chorus of Tibetan monks, or
- suspending John from a rope tied to the ceiling and spinning him around as he sang,
both of which were John’s ideas.
Leslie speakers were one of the many clever audio devices that people came up with the the pre-digital age. Inventors had to be clever back then because you couldn’t just throw massive amounts of computing power at a problem to get a solution. The Leslie achieved its unique sound by spinning its speakers around to get both a tremolo, because volume changed as the speakers went from pointing toward you to pointing away from you, and a vibrato due to the Doppler effect.
(Image taken from Clifford Hendricksen article.)
It’s hard, though, to write an article about the Leslie without also writing about the Hammond tonewheel organ, with which it was usually paired. In fact, there was no microphone input to the Leslie at Abbey Road; it was so intimately tied to the Hammond that the engineers had to open up the speaker cabinet to wire the mic in directly.
The Hammond was perhaps an even greater example of pre-digital cleverness. It was, in effect, an electromechanical synthesizer, generating pure tones in harmonic series that were added together to create particular timbres. The pure tones were made by spinning toothed wheels—gears, basically—in front of electromagnetic pickups. The spinning gear changed the magnetic field in a regular, sinusoidal pattern, inducing a sinusoidal current in the pickup’s coil. If this sounds suspiciously like the way an electric guitar works, it’s because it is, with the spinning gear taking the place of the vibrating string. In fact, it’s so much like an electric guitar, you’d think it would’ve been invented by Les Paul.
(Image taken from Keyboard Exchange International.)
The organist had great control over how the harmonics were added through the manipulation of a series of drawbars. The similarity between the drawbars on a Hammond and the stops on a pipe organ can’t be coincidental. The Hammond was, and still is, marketed largely to churches who want that churchy organ sound but can’t afford a pipe organ.
Soon, the Hammond sound, especially the B3 model, worked its way into blues, R&B, and rock and roll. My article got sidetracked for a day or two as I spent hours on YouTube looking at videos of Keith Emerson, Jon Lord, Jimmy Smith, Gregg Rolie, and Rod Argent. Research, don’t you know.
This morning I woke up to the news that Duck Dunn had died, leaving only half of the great Stax house band still with us. I doubt more than a week ever goes by without me listening to at least one song with Dunn on it.
I don’t think I’ll ever write that Tomorrow Never Knows/Leslie/Hammond post. As with a lot of my ideas, it just didn’t come together. But this time I decided to give you the links so you can make up your own post.
And I’ll add this: the quintessential R&B instrumental with the classic lineup of Al Jackson, Jr. on drums, Donald “Duck” Dunn on bass, Steve Cropper on guitar, and Booker T. Jones on the Hammond B3.
Bolt upright
May 12th, 2012 at 11:20 pm by Dr. Drang
Yesterday, my older son mowed the lawn, and I helped him bag the clippings (we’d gone too long since the last mowing to be able to use the mower’s mulching feature). Late in the process, I noticed half a bolt lying near the edge of the sidewalk. I poked around in the grass nearby, looking for the other half. No luck. I guess the mower blade will find it sooner or later.
The bolt is one of the four shoulder bolts that connect the bottom of the mower’s handle to its deck. The handle was a little floppy with only three bolts, but my son managed to finish the lawn anyway.
As you can see, the bolt fractured in the first thread, right next to the unthreaded part of the shank. This is a pretty common place for bolts to fail, as the threads create a stress concentration, and the first thread is typically where that concentration is the greatest.
Flipping the half-bolt up to look at the fracture surface, we see a classic fatigue failure. Fatigue is the gradual accumulation of damage under repeated loads.
In this photo, the origin of the fracture is at the bottom. It grew upward through at least half the cross-section, leaving behind the concentric arcs known as beach marks. This growth occurred slowly as the load on the bolt was applied and released, applied and released, applied and released. The beach marks represent the edge of the crack front at various points in its growth history.
The upper half of the fracture surface doesn’t appear to have beach marks, but it’s possible that there was fatigue crack growth in that zone, too. I’d have to look at the bolt under a microscope to be sure.
This fracture surface has another interesting feature that’s common in bolt failures: the radial lines that run inward from the outer edge of the bolt. This comes from the struggle between two tendencies of cracks:
- They like to run flat in a plane.
- They like to run through the weakest part.
The weakest part of a bolt is through the root (the bottom of the valley) of the thread. Because the root runs around the outer edge of the bolt in a helix, the crack can’t be planar and run through the root. It compromises by making little jumps—and occasionally big jumps—at the outside edge. The result is a sort of spiral staircase of cracks running around the helix of the thread root. If you were to shrink down1 and start walking counter-clockwise along the edge of the fracture, you’d find yourself climbing up those stairs. When you’d gone all the way around, you would have climbed one thread pitch, and could look down to where you started. You can see the one thread-pitch drop in the top photo.
Because of the particular geometry of the shoulder, I can’t get replacement bolts at the local hardware store. Luckily, there’s a lawn mower specialty store one town over that has most replacement parts in stock. After a quick look through the parts manual, I’ll be giving them a call Monday.
Update 5/14/12
You know, if I were more like a real blogger, I would’ve linked to my earlier posts on fatigue. Two of those posts have photos of other items in my house that failed through fatigue: the plastic flushing arm from a toilet, and the oil-tempered steel torsion spring from the garage door.
