# Tags and copies

In yesterday’s post, I talked about how I’ve been using file tags to organize my work photographs according to both date and subject.1 This works pretty well, but sometimes a set of Smart Folders that collects photos according to their tags isn’t the right solution. In those cases, I have a couple of scripts that allow me to replicate my set of Smart Folders into a set of real folders with copies of the photos organized by subject.

The problem with tagging and Smart Folders is that they are too Mac-centric. If I need to share project photos with a colleague or a client—who are almost always Windows users—I can’t just copy a file structure like this onto a USB stick and give it to them:

The JPEGs are fine, as are the dated directories, but the Smart Folders are just gibberish on a Windows machine—a set of XML files with .savedSearch extensions. If I need to have the photos broadly available in folders organized by subject, I need to make real folders and put copies of the photos in them.

Fortunately, this is pretty easy to do if I’ve already done the tagging and created the Smart Folders for each tag. I have a command-line script called tags2dirs which, when run from the parent directory (e.g., the test directory in the example above), creates a set of traditional folders that parallel the Smart Folders. After running tags2dirs, I get this:

The tags2dirs script is this short bit of Python:

python:
1:  #!/usr/bin/python
2:
3:  import os
4:  import glob
5:
6:  tagDirs = glob.glob('*.savedSearch')
7:  newDirs = [ x.split('.')[0] for x in tagDirs ]
8:  map(os.mkdir, newDirs)


Line 6 looks through the current directory and collects the names of all the Smart Folders into the tagDirs list. Line 7 goes through that list, strips the .savedSearch extension off of each name and saves the result to the newDirs list. Line 8 then makes new directories with the names in newDirs. Simple.

Now comes the harder part: copying each photograph in the dated folders into the corresponding folders named for the tags. As you might expect, I have a script for doing this, too. It’s called cp2tags, and when invoked like this from the test directory,

cp2tags */*.jpg


it will copy every JPEG file one directory level down into all of the appropriate directories that were created by tags2dirs. For example, the left track folder will look like this:

This uses a lot of disk space—if a photo has six tags, it will be copied six times—but both of my Macs have 3 TB Fusion Drives, so I can afford to be profligate, at least occasionally.

Here’s the source code for cp2tags:

python:
1:  #!/usr/bin/python
2:
3:  import os.path
4:  import subprocess as sb
5:  import sys
6:  import shutil
7:
8:  tagCmd = '/usr/local/bin/tag'
9:  for f in sys.argv[1:]:
10:    tagString = sb.check_output([tagCmd, '--no-name', '--list', f]).strip()
11:    if tagString:
12:      tags = tagString.split(',')
13:      for t in tags:
14:        shutil.copy(f, t)


It goes through the arguments, which are expected to be file names, one at a time. For each file, it runs James Berry’s tag command, mentioned in yesterday’s post, to determine all the tags applied to each. The output of tag is a string of comma-separated tags, which is split apart on Line 12 to create a Python list of the file’s tags. Line 14 then copies the file to all the directories named after those tags.

For most projects, I don’t need tags2dirs or cp2tags, because I don’t need to send others my photos. But it’s nice to have the scripts ready.

One last thing: Tags created on the Mac can be used to filter files in the iOS Files app but only if the files are saved in iCloud Drive. File tags are synced to Dropbox, but the Files app doesn’t seem to know it. And I haven’t seen anything in the Dropbox app to suggest that it can filter by tags.

I’ve added tags to the sidebar through the clumsy tap-and-hold technique, but I haven’t worked out a quick, automated way to get a tag list into the Files sidebar.

It’s too bad the Files app knows no more about Smart Folders than Windows does.

1. For me, the subject of a photo is usually a piece of machinery, a structural component, or a fragment of a broken part. Many of my photos include more than one subject, which makes them a natural for tagging.

# A little tagging automation

I resisted tagging my files for quite a while. Why bother with tags when we already have a hierarchical file system for our first method of organization and hard links when we need another method? But I’ve come around to using tags, partly because I got used to them on iOS and partly because the advantages of hard links are lost when you use a cloud system like Dropbox to keep two computers in sync.1 So I’ve gradually developed a set of scripts and macros for dealing efficiently with tags.

I use these automation tools almost exclusively with the photographs I take for work, so I should start by saying that I don’t use the Photos app and don’t ever expect to, despite its internal tools for categorizing photos. The photos I take for work are project- and client-specific. Each project has to be kept separate, and I often need to be able to copy my entire project file for either archiving or sending to a client. Hard experience has taught me that folders of JPEGs are better for my situation than any structure within Photos.

My usual baseline method of organizing photos is to put them in folders according to date. The photos themselves, which come out of the camera with names like IMG_1234.JPG or DSCN1234.JPG, are renamed according to date, photographer, and image number like this:

20181004abc-001.jpg


where “abc” are the initials of the photographer, usually me. The photos are put into folders named according to the date using the same yyyymmdd format as the file name prefix. On small projects with only a few dozen photos, this is sufficient, and my photo organization starts and ends with a system like this:

But when I have a more complex set of photos, and particularly when I have photos of many objects that have to be analyzed separately (residential units in a building, machine parts, chunks of an exploded boiler, etc.) its useful to have them also organized by object. The key here is “also”—I don’t want to lose the date organization, I want object organization in addition to date organization.

The natural way to do this on a Mac is to use tags. There’s a nice command-line tool called tag, written by James Berry, that does pretty much whatever you might want with tags: adding, removing, listing, and finding files. If you use Homebrew, you can get it via brew install tag.

I have a Keyboard Maestro macro that uses tag. When I want to organize photos by object, I open a Finder window for one of my dated photo folders in icon view with the icon size set large enough for me to identify the photo subject(s). I then work my way through the folder, selecting photos and calling the macro with the ⌃⇧T keystroke. This brings up a window with a selection box listing the tags I can apply. I select one or more tags, hit the OK button, and move on.

Here’s the Keyboard Maestro macro that does the work,

The first step defines a list of tags, one per line. I change this for every project. The second step is an AppleScript that takes those tags, uses them to create the window with the selection box, and then applies the chosen tags to the selected files. Here’s the AppleScript in full:

applescript:
1:  -- Create a list of tags from the variable defined above
2:  tell application "Keyboard Maestro Engine" to set tagString to getvariable "tagListString"
3:  set oldDelimiters to AppleScript's text item delimiters
4:  set AppleScript's text item delimiters to linefeed
5:  set tagList to every text item of tagString
6:  set AppleScript's text item delimiters to oldDelimiters
7:
8:  -- Add chosen tags from the list to the selected Finder items
9:  tell application "Finder"
10:    set finderItems to selection as alias list
11:
12:    -- Ask user to choose tags from the list
13:    set tags to choose from list tagList with title "Add tags" with prompt "Choose file tags(s)" with multiple selections allowed
14:    if tags is false then
15:      -- do nothing
16:    else
17:      -- Assemble the chosen tags into a quoted, comma-separated string
18:      set oldDelimiters to AppleScript's text item delimiters
19:      set AppleScript's text item delimiters to ","
20:      set tagString to tags as text
21:      set AppleScript's text item delimiters to oldDelimiters
22:      set tagString to quote & tagString & quote
23:
24:      -- Add the tags to each file in turn
25:      -- The tag command is from https://github.com/jdberry/tag
26:      -- It can be installed via brew install tag
27:      repeat with p in finderItems
28:        set cmd to "/usr/local/bin/tag --add " & tagString & " " & quote & (POSIX path of p) & quote
29:        do shell script cmd
30:      end repeat
31:
32:    end if
33:  end tell


Much of the script’s length is due to AppleScript’s clumsy tools for moving between lists and strings. The script works like this:

• Lines 2–6 import the list of tags defined in the first step (which come in as a string) and convert them into an AppleScript list through the ol’ text item delimiters shuffle.
• Line 10 gets the list of files selected in the Finder.
• Line 13 asks the user which tags to apply to the selected files, using the tagList defined in Line 5. Selecting more than one tag is allowed, which is important, as my photos of machinery often include more than one part.
• If the user cancels (Line 14), the script does nothing. Otherwise…
• Lines 18–22 do the reverse text item delimiters dance to turn the list of tags chosen by the user into a string of tags separated by commas. This is the format the tag program wants.
• Lines 27–30 then go through each of the selected files and add the tags. Line 28 sets up a tag command that looks like this:

tag --add "tag A,tag B" "Photo 27.jpg"


to apply the tags. The quotes are needed because both the tag list and the file name can have spaces that need to be protected from shell interpretation. The command is then run by do shell script in Line 29.

After the tags are applied, I can use Smart Folders keyed to a particular tag to collect all the photos of that object in one spot. Unfortunately, making a Smart Folder that searches for tags involves more scrolling and clicking than I can tolerate.

So I wrote a command-line script called mktagdirs to do it for me. Running it from the directory that contains all the dated photo subdirectories results in a new Smart Folder for each tag.

Here’s the Python source code for mktagdirs:

python:
1:  #!/usr/bin/python
2:
3:  import plistlib
4:  import sys
5:  import os
6:  import subprocess as sb
7:
8:  # The tag command can be found at https://github.com/jdberry/tag
9:  # This is where I have it installed (via brew install tag)
10:  tagCmd = '/usr/local/bin/tag'
11:
12:  # Get the working directory and all of the tags in files under it
13:  cwd = os.getcwd()
14:  tagString = sb.check_output([tagCmd, '--no-name', '--recursive']).strip()
15:  tagString = tagString.replace(',', '\n')
16:  tags = set(tagString.split('\n'))
17:
18:  for t in tags:
19:    # Build the dictionary for the smart folder
20:    rawQuery = '(kMDItemUserTags = "{}"cd)'.format(t)
21:    savedSearch = {
22:    'CompatibleVersion': 1,
23:    'RawQuery': rawQuery,
24:    'RawQueryDict': {
25:      'FinderFilesOnly': True,
26:      'RawQuery': rawQuery,
27:      'SearchScopes': [cwd],
28:      'UserFilesOnly': True},
29:    'SearchCriteria': {
30:      'CurrentFolderPath': [cwd],
31:      'FXScopeArrayOfPaths': [cwd]}}
32:
33:    # Make the smart folder
34:    plistlib.writePlist(savedSearch, '{}.savedSearch'.format(t))


The first thing to note is that this is using the Python that comes with macOS, which is Python 2.7. This script will not work on recent versions of Python without a little tweaking, because the plistlib module has been rewritten.

The key to understanding mktagdirs is recognizing that Smart Folders aren’t folders at all; they’re just plist files with a .savedSearch extension. It’s the contents of the plist file that determines which files appear when you open a Smart Folder in the Finder.

We’re going to use tag again, this time to gather all the tags of the photo files. Line 10 defines where I have it saved, and Line 14 runs it via the subprocess library. The invocation would look like this on the command line:

/usr/local/bin/tag --no-name -recursive


This returns all of the tags for all of the files within the current directory, including files nested in subdirectories. Each file’s tags are output as a comma-separated list, one line for each file, like this:

boom,bucket,cab,left track,right track,stick
bucket
boom,stick
boom
cab


Lines 15 and 16 take this output and turn it into a Python set called tags, which we start iterating through on Line 17. By using a set instead of a list, repeated entries are reduced to a single item.

Lines 21-31 define a dictionary of the items needed for a Smart Folder. I learned some of this from Kirk McElhearn’s old Macworld article and some from just playing around with the plist of a Smart Folder I’d made “by hand” and seeing what could be deleted without impairing its function. For our purposes, the most import things are the following:

• The SearchScope, CurrentFolderPath, and FXScopeArrayOfPaths items, which we set to the “current working directory,” which is the directory from which mktagdirs was invoked.
• The RawQuery items, which are defined in Line 20 as a kMDItemUserTags search for the tag name. The cd means the search is insensitive to case and diacritical marks.

Finally, Line 34 converts the dictionary to a plist and writes it to disk with a .savedSearch extension.

Now I have a way to look at my photos by date and by subject. Each Smart Folder is populated with the files that have its tag.

This works well when I’m the only one who needs to see my photos, but not when I have to share them with others. The next post will have a couple of scripts to handle that situation.

1. Dropbox treats the hard links as separate files and uploads them accordingly. So when your other computer downloads files from Dropbox to stay in sync, it downloads multiple copies of each. This means the links are lost and one of your computers is using much more disk space for the files than the other.

# You really like me

If you’re like me, you tend to think of automation as a way to streamline the performance of complex, many-step actions. And while that’s certainly an important use of automation, sometimes its the simple things, things that barely seem worthy of automating, that are the most satisfying. Today I used Shortcuts to replace a bit of lost functionality in Tweetbot that’s been bugging me for quite a while.

Once upon a time in Tweetbot, you could see who liked one of your tweets by swiping to the left to see it by itself and then tapping on the number of likes.

You can still do this with retweets, but not with likes. As a fundamentally insecure person, desperate for the approval of others, I miss being able to quickly see who liked my tweets. That’s what my “You Really Like Me” shortcut addresses.1

Here it is, a Share Sheet shortcut with just two steps:

It takes advantage of Twitter’s simple URL system. If you want to see the likes of the tweet with URL

https://twitter.com/drdrang/status/1047637958130642945


you just stick /likes onto the end of it:

https://twitter.com/drdrang/status/1047637958130642945/likes


That’s what the first step of the shortcut does. The second step shows the web page for that URL.

Boom. Lost functionality replaced. To be sure, what used to be a single tap in Tweetbot is now three (Share→Shortcuts→You Really Like Me), but at least I don’t have to mess around with launching another app and either pasting in a URL or navigating through my tweets.

Update Oct 6, 2018 10:13 AM  I didn’t include a download link for this shortcut, because I figured it was too short to bother. Then I heard from Sebastian Peitsch:

@drdrang Everything’s different in German but I managed to get this done via comparing the icons

Thanks Doc!

— Sebastian Peitsch (@SPeitsch) Sat Oct 6 2018 2:57 PM

If you’re using Shortcuts in a different language, the names of the actions aren’t the same, which makes it harder to copy than it should be. But an imported shortcut will be localized to your language. So here’s a link to download: You Really Like Me.

Thanks to Sebastian for the help.

Update Oct 6, 2018 10:30 AM  If you find that the shortcut isn’t working for you, that it opens the tweet in a browser view but not the list of likes, it’s because you aren’t logged in to Twitter in that browser. I thought I had tested it under those conditions, but I hadn’t. Thanks to Grant Buell for finding the error and Sean for explaining it.

I should also point out that John Cutler has a two-tap system for getting to his likes:

@oscargong1995 @drdrang @OpenerApp Tapping the time/date stamp on a tweet in Tweetbot will also open the tweet in the mobile twitter site. As long as you’re logged in, you’ll be able to view likes there too.
— John Cutler (@JohnCutler) Sat Oct 6 2018 1:46 PM

This takes one less tap, but I find it takes slightly more time than my shortcut, as it loads the entire mobile Twitter site after you tap on the time/date stamp. Still, you might prefer it.

1. With apologies to Sister Bertrille.

# Shortcuts as subroutines

I was too busy at work last week to spend any time with Federico Viticci’s excellent screenshot framing shortcut for iOS, but I dug into it over the weekend and used it as the basis for a shortcut that does the things I usually need done after creating a screenshot:

• Resizing if the screen shot is excessively wide.
• Optimizing to reduce the file size.

I renamed Federico’s shortcut from “XS Frames” to “Frame Screenshot.” My extended shortcut is called “Frame and Upload” and I keep it next to “Frame Screenshot” in my list of shortcuts to make it easy to find.

It would typically be called from the Share Sheet within the Photos app after selecting one or more screenshot images to frame. After selecting “Frame and Upload” from the Shortcuts list, the screenshots are processed and then the user is asked to resize the resulting image.

Because I wrote this to handle images posted here on the blog, and because width is the controlling dimension, the resizing is done by specifying a new width—the height is adjusted automatically to maintain the aspect ratio.

Note that the current width of the framed screenshot is given as the default. If the user just taps the OK button without editing the width field, no resizing is done.

Next, the user is asked for a file name.

Actually, what gets entered here is just a part of the file name. I have a particular naming scheme I use for images here:

yyyymmdd-Description of image.ext


The date the image is uploaded, like 20181002, is included as a prefix. Then comes a description of the image, which can, and usually does, include spaces—this is the part that the user is asked for. I have other scripts that extract this portion of the file name to use as the alt and title attributes in the <img> tag. The extension for these framed screenshots is png.

Assuming everything goes well—as Federico said in his post, the shortcut sometimes fails, probably for lack of memory; I’ve always found that I could run it successfully immediately after a failure—I have a framed screenshot in Photos and a new file in the blog’s images directory on the server.

Here’s the complete shortcut, with the various sections marked:

Now you can see why I’m not interested in specifying the height of an image.

A few things worth expanding on:

• Although it may not be obvious (it wasn’t to me), Federico’s shortcut puts the framed image into the “flow.” This is why we can run it unchanged and put its output into the framedImage variable for later use.
• I create a new images directory on the server each year and call it imagesyyyy. That’s why there’s a thisYear variable.
• Because autocompletion leaves a space after the word it just filled in, it’s easy to have trailing whitespace in the file name. The Replace Text action that’s called right after the file name is entered eliminates both leading and trailing whitespace so the user doesn’t have to be extra careful about it.
• Note that the yyyymmdd prefix is created with a formatting string of yyyyMMdd. The uppercase M’s are not a typo. The date/time formatting system used by Shortcuts doesn’t comply with the old Unix strftime library conventions.
• I find the cat command in the SSH action to be really weird. In a normal command-line environment, that command would basically be a no-op, and I can’t think of any time I’ve used cat like that in over 20 years of Unix/Linux experience. But because Shortcuts is sending the data flow to the server as standard input, that weird usage works perfectly. As I was putting this together, I asked Federico, Jason Snell, and Rosemary Orchard about uploading through Shortcuts, and Rosemary was the first to respond with this short, clever solution. Just what I was looking for.
• The call to optipng in the SSH action at the end will only work if OptiPNG is installed on the server.

Chances are you don’t want to enter all these steps by hand. You can download a template of it and just change the stuff in the SSH action.

I’m still not a big fan of the Workflow/Shortcuts programming environment—I prefer typing to dragging, and really prefer a text environment when I need to edit a program. But there’s no question of its power, especially when you can take entire shortcuts and use them as subroutines. Thanks to the Workflow boys for that. I have a distinct sense that if Shortcuts had been developed within Apple from the start, that feature, essential for building up complex programs, would not have been included.

At present, Federico’s framing shortcut can add only X🅂 and X🅂 Max frames around your screenshots. I assume he didn’t bother with iPad frames because he’s expecting new versions to arrive soon and didn’t want to waste his time on something that would have to be changed almost immediately. Presumably, when Steve Troughton-Smith and Guilherme Rambo uncover the hero images for the new iPads, Federico will be on them like a duck on a June bug. Do they have June bugs in Italy?