Reshaping output

I often need to take a long list of character strings—one per line—and reformat them into a table so they’ll be easier to read. In the past, I’ve done this by pasting the data into a spreadsheet and moving the cells around to build the table by hand. Today, I decided there had to be a Unix command line tool that would do this for me, and I set out looking for it. I found two.

The data set I was dealing with was a list of devices, each identified by serial number and having a set of characteristics. I wrote a short Python script that filtered the data, printing out only the serial numbers of the devices that fit particular criteria. The easiest way to write the script was to have the serial numbers come out one per line, but that’s not how I wanted to present them in an email to my client. I wanted them laid out in a nice table.

I found a Python module, pycolumnize that will do this, but I wanted to learn a more general tool, one that I could use on any set of textual data, not just a list within a Python program.

The first command I ran across was column, which is pretty easy to use and doesn’t have many options. Here’s an example:

$ jot 36 | column -c 32
2       11      20      29
3       12      21      30
4       13      22      31
5       14      23      32
6       15      24      33
7       16      25      34
8       17      26      35
9       18      27      36

The jot command generates the numbers 1–36, one per line. Then column rearranges them into columns. The number of columns is determined by the value of the -c option; column packs as many columns as it can into an output block that many characters wide.1 While I can understand why someone might like this way of specifying the output, I’d rather specify the number of rows or columns directly.

Also, column puts a tab character between each column.2 Again, I can see why some people might like that, but I prefer spaces between the columns.

My dissatisfaction with column led me to rs, the reshape command. This is a much more flexible command than column. In its most basic form, you give it the number of rows and columns (in that order) you want in the output.

$ jot 36 | rs 4 9
1   2   3   4   5   6   7   8   9
10  11  12  13  14  15  16  17  18
19  20  21  22  23  24  25  26  27
28  29  30  31  32  33  34  35  36

Having to specify both rows and columns shouldn’t be necessary, and it isn’t. You can use 0 as a dummy value for either the row or column count, and rs will figure out what that value ought to be.

$ jot 36 | rs 0 4
1   2   3   4
5   6   7   8
9   10  11  12
13  14  15  16
17  18  19  20
21  22  23  24
25  26  27  28
29  30  31  32
33  34  35  36

As you can see, by default rs enters the values row by row instead of column by column. That can be changed with the -t (transpose) option:

$ jot 36 | rs -t 0 4
1   10  19  28
2   11  20  29
3   12  21  30
4   13  22  31
5   14  23  32
6   15  24  33
7   16  25  34
8   17  26  35
9   18  27  36

Each column of rs output is as wide as the widest entry. Other values are padded with spaces. By default, the entries are left justified within that width, but you can make them right justified by using the -j option:

$ jot 36 | rs -tj 0 6
 1   7  13  19  25  31
 2   8  14  20  26  32
 3   9  15  21  27  33
 4  10  16  22  28  34
 5  11  17  23  29  35
 6  12  18  24  30  36

The “gutter” is the space between the columns, and rs uses a two-space gutter by default. That can be changed with the -g option:

$ jot 36 | rs -tj -g4 0 4
 1    10    19    28
 2    11    20    29
 3    12    21    30
 4    13    22    31
 5    14    23    32
 6    15    24    33
 7    16    25    34
 8    17    26    35
 9    18    27    36

Be careful with -g. The width number has to come right after it. If you put a space between the g and the width number, rs will misinterpret it.

The rs command has, perhaps unfortunately, many more options, but -t, -j, and -g are probably the most useful.

By the way, I don’t want to leave the impression that you need to have a full table. If you ask for 36 entries to be arranged in 5 columns, you’ll get this:

$ jot 36 | rs -tj -g4 0 5
 1     9    17    25    33
 2    10    18    26    34
 3    11    19    27    35
 4    12    20    28    36
 5    13    21    29
 6    14    22    30
 7    15    23    31
 8    16    24    32

Or this if you don’t transpose:

$ jot 36 | rs -j -g4 0 5
 1     2     3     4     5
 6     7     8     9    10
11    12    13    14    15
16    17    18    19    20
21    22    23    24    25
26    27    28    29    30
31    32    33    34    35
36

For the data I was dealing with today, the serial numbers were all four digits long and there were, after filtering, about 200 of them, so you can see why I wanted to reshape them into a table. I chose six columns and a four-space gutter to get a table that was relatively easy to read. I piped the output of my Python script through rs -t -g4 0 6 and got output that looked something like this:

1013    2337    3908    5808    7374    8919
1021    2358    3962    5819    7384    8932
1095    2419    3980    5843    7467    8936
1125    2481    4007    5843    7494    8960
1173    2485    4059    6028    7497    8998
1194    2501    4076    6110    7510    9118
1250    2595    4119    6128    7600    9168
1255    2603    4260    6206    7623    9196
1314    2724    4315    6233    7688    9208
1326    2867    4363    6260    7766    9307
1330    2904    4372    6310    7850    9308
1346    2905    4376    6352    7914    9394
1355    3021    4379    6407    7935    9411
1404    3032    4502    6450    7990    9435
1408    3196    4503    6457    7991    9442
1449    3234    4572    6501    8022    9473
1462    3246    4699    6652    8078    9536
1477    3246    4767    6660    8127    9544
1477    3293    4819    6665    8158    9563
1529    3293    4988    6724    8222    9603
1537    3355    5035    6747    8299    9603
1551    3356    5042    6765    8333    9630
1552    3372    5101    6783    8402    9742
1809    3421    5111    6845    8436    9749
1930    3458    5261    6861    8474    9783
1996    3499    5315    6862    8507    9892
2045    3569    5522    6875    8519    9906
2047    3575    5525    6897    8524    9944
2070    3597    5526    7036    8586    9948
2078    3686    5569    7050    8664    9959
2124    3691    5583    7077    8720    
2146    3722    5592    7109    8751    
2158    3739    5617    7310    8891    
2336    3810    5755    7333    8905    

Yes, it’s long, but it’s probably as easy to read as 200 serial numbers can be.


  1. Sort of. In practice, I’ve found that you usually need to give -c a greater number than is absolutely necessary. For example, in the table above, each line is only 26 characters long, but column -c 28 would result in only three columns of data. I don’t know why. 

  2. HTML doesn’t like tab characters, so to match the look of the output in Terminal, I’ve shown the output of column with spaces between the columns. 


I’m not a historian, but I play one on podcasts

One upside of getting older is that if you can avoid drooling, younger people often think you know stuff about the “olden days.” This is how I ended up explaining what it used to be like working with computers on the most recent episode of Technical Difficulties.

Lightspeed Ad

A downside of getting older is that although you can remember perfectly well an advertisement from the ’80s for Lightspeed Pascal, you can’t remember what you did a few months ago. This episode was recorded back in August, and while I recall enjoying myself and talking too much, I have no idea what I actually said. It’ll be as new to me as it will be to you.


I and iPad

I don’t own an iPad, and I don’t much care for articles about the business side of the computer industry. But I do have a blog, which automatically makes me an expert on both.

iPad Sales slide

When Tim Cook showed this slide during the recent Apple Special Event, it was a way of avoiding showing an updated version of this slide from last year’s October event:

iPad S curve

In an updated graph, the downward curvature would be more prominent.1 For the past three quarters, iPad sales have been down from the year-ago quarter, and the unthinking wing of the tech press has finally picked up on it. Stories have shifted from “iPad will take over all of computing” to “iPad is a failure.” Tim’s slide comparing the iPad to all of Lenovo, HP, etc. puts this “failure” in perspective. A more nuanced view may be in order.

When the iPad was first announced, I gave it a lot of thought. My late-2004 iBook G4 was stuck on Leopard, the last version of OS X that supported the PowerPC, and third-party developers were starting the shift to Intel-only applications that would leave my old G4 even further behind. Even worse, the power supply connection was getting flaky, and it seemed foolish to pay for repairs to a machine that was on the way out.

The question was: could an iPad take the place of my iBook G4? After the iPad was released, the answer was clearly no. Luckily, Apple came out with the redesigned and actually usable version of the MacBook Air later that year, and that solved my replacement computer problem. I’m still using it.

The time I spend thinking about that first iPad wasn’t wasted, though. When the iPad 2, with its A5 chip2 and front-facing camera, came out the next year, I got one for my wife, and it’s been her main computer ever since. It fits her needs very well, but the areas where it doesn’t fit illustrate why we haven’t yet shifted into the post-PC era.

First, there’s the problem of web sites that don’t work with iOS devices. I’m not talking about responsive design here,3 I’m talking about sites with “clever” JavaScript-based interaction features that simply ignore the possibility that users may be using a tablet or phone and reveal their capabilities through cursor-only interactions like hovering. And there are still sites that insist on Flash. This isn’t Apple’s fault, and it’s a problem that’s slowly receding, but it still crops up now and then.

Second, there’s the printing problem. I know lots of young people think printing is dead, but adults still live in a world where forms have to be printed out and taken or physically mailed to offices. I solved some of this problem by getting an AirPrint printer, but pages printed from the iPad often come out poorly formatted. Some of this is certainly not Apple’s fault—the world would be a better place if form makers would just use PDF—but it could help out by giving users a little more control over printing. Even just allowing users to choose which pages to print would be a big step forward.

Finally, there’s the big problem: storage. How does someone on a iPad access all of the photos and music and video and other files that are part of the modern digital life that Apple wants us to lead? None of us can be post-PC until all of our stuff is where we can get at it without a PC. That there’s been no clean, obvious, and reliable solution to this problem is definitely Apple’s fault, and it’s kept the iPad from being a complete PC replacement.

The thing is, Apple is almost uniquely unqualified to provide the kind of online storage the iPad needs. The history of Apple cloud-based services is so laden with bad ideas and worse execution that it’s impossible not to be skeptical when a new service is announced. Still, I hold out hope that iCloud Drive is the solution, because the iPad Air 2, when combined with iOS 8’s new openness, is a compelling machine.

The performance of the A8X and the additional RAM really do make the new iPad look like a viable replacement for a notebook computer. Its Geekbench scores are well beyond those of the machine I’m typing this on. But that’s not the whole story. Apple’s always had a deep understanding that it’s software that makes hardware live up to its potential. Let’s hope it’s finally learned that network services are just as important.


  1. That’s downward curvature, not downward slope. Keep your derivatives straight. 

  2. Still being used in the low-end iPad mini. 

  3. Glass houses. 


An iOS 8 regression

Driving along, listening to a podcast. My iPhone is plugged in to top off its battery. The message alert sounds, interrupting John Siracusa’s complaint about Apple still selling 16 GB iPads and iPhones.

“Hey, Siri.”

“Da-ding?”

“Read me the last text message.”

“There are no messages.”

Yes there are! You just made the new message sound! What the hell is wrong with you? You used to know how to do this!”