Markdown reference links in BBEdit

Markdown has two link styles: inline and reference. Inline links look like this:

This is [an inline link](http://daringfireball.net/projects/markdown/syntax#link).

where the link text in brackets is immediately followed by the URL in parentheses. Reference links look like this:

This is [a reference link][ref]. Note how the URL does not
directly follow the link text but is put elsewhere in the
document, often at the end of the file or at the end of a
section.

[ref]: http://daringfireball.net/projects/markdown/syntax#link

where the link text in brackets is followed by a reference, also in brackets. The URL is kept elsewhere in the document, on a line that starts with the reference and a colon.

I’ve always preferred the reference style to the inline style because it looks cleaner and makes the original Markdown document easier to read directly. I put the references at the end of the document, sort of the way references would be placed in an academic paper. It is harder to make reference links, though, because you have to jump from the link text to the list of references and back.

In 2006, with the help of the nice folks on the TextMate mailing list, I made a couple of macro/command/snippet combinations for quickly inserting reference links. The first assumed the link text was about to be written. The second assumed the link text had already been written and was selected. Both took advantage of my habit of using numbers for the references instead of words or abbreviations and would automatically increment the reference number as I added more links. These commands have served me well for over six years, and I missed them greatly when I stopped using TextMate. I needed replacements that would work in BBEdit while I gave it a try.

(Gabe Weatherhead uses reference links, too, and he blogged about his technique for adding them back in January. His system uses Keyboard Maestro, which I don’t have, so I couldn’t just steal it.)

Getting the next reference number

The first order of business was writing an editor-agnostic script that would scan whatever text was given to it, pick out the reference-style links, and return the appropriate number for the next reference link. This was pretty easy to write in Python:

python:
 1:  #!/usr/bin/python
 2:  
 3:  import sys
 4:  import re
 5:  
 6:  text = sys.stdin.read()
 7:  reflinks = re.findall(r'^\[(\d+)\]: ', text, re.MULTILINE)
 8:  if len(reflinks) == 0:
 9:    print 1
10:  else:
11:    print max(int(x) for x in reflinks) + 1

The text to be scanned is read from standard input on Line 6. The findall command in Line 7 returns a list of all the numbers used in reference-style links. Line 11 picks out the largest of those numbers and increments it. This is the value written to standard output.

Update 8/26/12
Ah, the dreaded edge case! In the original version of this script, I forgot to check to see if there were no numbered reference links. That’s been fixed.

If I decide to dump BBEdit—which is starting to look unlikely—this script, nextreflink, will still work.

Turning a command line argument into standard input

As we’ll see later, the cursor movement and text insertion necessary to insert a reference link will be handled by an AppleScript. To get the next reference link number, I needed the AppleScript to call nextreflink, feeding it the text of the file being edited and collecting its output. AppleScript has the very useful do shell script construct for running scripts like nextreflink, but it doesn’t have a good mechanism for specifying the standard input to the script.

In this technical note, Apple recommends using the echo command to turn a command line argument into standard output, which can then be piped as standard input into another command. Here’s an example:

applescript:
tell application "BBEdit"
  set myText to content of front document
  set myRef to do shell script "echo " & quoted form of myText & " | wc"
end tell

There are two problems with this approach when scripting BBEdit:

  1. echo adds a newline to the end of the text it’s echoing. For some scripts this won’t matter; for some, like wc, it will.
  2. BBEdit, as we’ve seen before, considers the lines of the files it’s editing to be ended by carriage returns, regardless of the files’ line ending settings. This is fine for BBEdit’s internal consistency, but it doesn’t work well when you need to feed those lines to a script that assumes linefeeds to be the only just and proper way to end a line. The example above will return 2 no matter how many lines there are in the front document, because wc treats everything ending in CR to be one long single line. Only the LF added by echo is recognized; it makes wc think there’s one long line before the LF and one zero-length line after it.

The solution is a little script I call bbstdin:

bash:
#!/bin/bash
echo -n $1 | perl -015 -l12 -p -e0

This uses echo’s -n switch to turn off the otherwise automatically supplied trailing linefeed. The output of echo, which just the first argument supplied on the command line, is then piped to what may be my favorite Perl program, a program that makes such clever use of Perl’s switches that it doesn’t even need a body.

The -015 switch sets the input record separator (i.e., the line ending character on input) to the carriage return, whose octal representation is 15. The -l12 switch sets the output record separator (i.e., the line ending character on output) to linefeed, whose octal representation is 12. The -p switch tells Perl to read the input a line at a time an print the output. The -e0 switch sets the body of the program to 0, which tells Perl to do nothing to the lines as they pass through (other than what the -0 and -l switches say).

In a nutshell, perl -015 -l12 -p -e0 turns the old-style, pre-OS X Mac line endings that BBEdit uses internally into Unix line endings. I’m afraid I can’t credit the wonderfully twisted genius who came up with this gem, but I’ve had this aliased to cr2lf in my .bashrc file for ages.

Returning to our example, the right way to feed the text of the front BBEdit document to wc is

applescript:
tell application "BBEdit"
  set myText to content of front document
  set myRef to do shell script "~/bin/bbstdin " & quoted form of myText & " | wc"
end tell

Making a Markdown reference link

Now we come to the cursor movement and text insertion, and this is where BBEdit really shines. I always felt my TextMate system for inserting reference links was a lucky kluge. TextMate’s cursor control macro commands were pretty rudimentary and only an unlikely trick or two got it to work. BBEdit, on the other hand, has a full suite of cursor control commands available. Yes, they have to be accessed through AppleScript, but you can’t have everything.

I was even able to combine the logic for both types of reference link—when the link text is not yet written and when it’s written and selected—into a single command

applescript:
 1:  set myURL to the text returned of (display dialog "URL:" default answer "http://")
 2:  
 3:  tell application "BBEdit"
 4:   set myText to contents of front document
 5:   set myRef to do shell script "~/bin/bbstdin " & quoted form of myText & " | ~/bin/nextreflink"
 6:   
 7:   if length of selection is 0 then
 8:     -- Add link with empty text and set the cursor between the brackets.
 9:     set curPt to characterOffset of selection
10:     select insertion point before character curPt of front document
11:     set selection to "[][" & myRef & "]"
12:     select insertion point after character curPt of front document
13:     
14:   else
15:     -- Turn selected text into link and put cursor after the reference.
16:     add prefix and suffix of selection prefix "[" suffix "]" & "[" & myRef & "]"
17:     select insertion point after last character of selection
18:   end if
19:   
20:   -- Add the reference at the bottom of the document and reset cursor.
21:   set savePt to selection
22:   select insertion point after last character of front document
23:   set selection to "[" & myRef & "]: " & myURL & return
24:   select savePt
25:  end tell

The script starts by asking for the link’s URL. I’ll usually paste this in or use one of my URL-gathering TextExpander snippets.

Reference link dialog

Lines 4-5 get the content of the front document and have nextreflink provide the next number for a reference link. We use the bbstdin trick of the previous section to make sure nextreflink’s input is in the format it needs.

Line 7 checks the length of the selection. If it’s zero, Lines 9-12 insert [][n], where n is the next reference link number, and set the cursor between the first two brackets. If there is selected text, Line 16 puts brackets around the text and adds the reference link afterward: [link text][n]. Line 17 then puts the cursor after the final bracket.

Lines 21-24 save the location of the cursor, move to the end of the document, insert the reference and URL, and return to the saved cursor location. Boom. Instant reference link, regardless of whether you start with a blinking cursor or a text selection.

I call the AppleScript “Reference link” and keep it in my BBEdit Scripts file. It’s keyboard shortcut is ⌃L.

Notes


8 Responses to “Markdown reference links in BBEdit”

  1. Patrick says:

    Imho, those kind of hacks are exactly the reason why I would not want to use editors such as BBEdit and TextMate that don’t have a proper plugin system.

    Implementing such a plugin in python should be pretty trivial in Sublime Text 2 for example as you have total control over everything from a simple script and you don’t have to hack together anything in AppleScript or the like for it to work with the editor.

  2. Nathan Grigg says:

    There are some subtleties to the way BBEdit handles selection and insertion point

    I have also run into these. It helps a little to remember that everything is a text_object, which is how BBEdit refers to pretty much any contigous stream of text. BBEdit refers to most of these as character ranges, counting from the beginning of the document, e.g. characters 5 thru 15 of document 1. The special case is zero-width objects, which is where BBEdit uses insertion point before character 5 of document 1, or equivalently, insertion point after character 4 of document 1.

    Instead of using select some_text_object and then set selection to "foo", you can just say set some_text_object to "foo". Not that this changes what happens in any way, because whatever you insert into the document gets selected, so you still have to use your “save cursor and then put it back” trick. It would be nice if you could say insert "foo" after last character instead of set insertion point after last character to "foo". Then again, AppleScript is all about saying this in the most awkward English possible.

  3. Dr. Drang says:

    Nathan,
    I had this page of yours open in a Safari tab throughout the process of writing the reference link script. The section on getting and setting the cursor location, short as it is, was essential.

  4. Gabe says:

    Gotta say, I really love the way BBEdit handles cursor movement in general. You know I’m a fan of the jump mark but selection is very easy too. If only they would choose a real scripting language. ;)

  5. Nathan Grigg says:

    The markdown scripts are great, but using echo to pass the entire document to an external script has a lot of potential for me. For some reason, I’ve never used this to bridge AppleScript and shell scripts, probably because every time I tried, something went wrong.

    Case in point: The bbstdin thing behaves differently if I put it straight into an AppleScript instead of in a separate file like you did. After much confusion, I remembered that echo is a shell builtin, and AppleScript uses sh instead of bash. For example, the echo that sh uses doesn’t accept the -n option and always expands escape sequences. If you want to use echo directly in an AppleScript, you should use /bin/echo, which, just like the echo that bash uses, accepts the -n option and does not expand escape sequences.

  6. Dr. Drang says:

    Nathan, I’m sorry you suffered the echo rigamarole. I went through exactly the steps you described and was going to write about it but decided it added too many words to an already long post. Also, once I decided to make the reusable bbstdin script, the ambiguity over which echo would be used disappeared. Had I followed my first instinct and written it up, it would have saved you time and frustration.

    It is, I think, an unfortunate choice by Apple to have do shell script run the command through sh instead of bash. Not because bash is more capable—although it is—but because bash has been the default login shell since (I think) Tiger. It’s what Mac users think of when they think “shell.”

  7. Arjan Boerma says:

    Regarding sh vs. bash: If I recall correctly, /bin/sh is symlinked to /bin/bash on OS X. In fact, this tech note (see ‘Gory Details’) says that bash ‘emulates’ sh, but you can do stuff like $[5+2] arithmetic in a do shell script that sh proper can’t (couldn’t) do.

  8. Dr. Drang says:

    Well, Arjan, on my computer ls -l /bin/sh yields

    -r-xr-xr-x  1 root  wheel  1371712 Jul 24  2011 /bin/sh
    

    and ls -l /bin/bash yields

    -r-xr-xr-x  1 root  wheel  1371648 Jul 24  2011 /bin/bash
    

    so sh isn’t just a link to bash, but they are suspiciously close in size. Perhaps the 54-byte difference is the code that tells bash emulate sh? I can’t think of any other reason for plain ol’ sh to be that big.

    Interestingly, ls -l /bin/*csh yields

    -rwxr-xr-x  2 root  wheel  772992 Jul 24  2011 /bin/csh
    -rwxr-xr-x  2 root  wheel  772992 Jul 24  2011 /bin/tcsh
    

    which shows that csh is just a link to tcsh.