Tags and copies

In yesterday’s post, I talked about how I’ve been using file tags to organize my work photographs according to both date and subject.1 This works pretty well, but sometimes a set of Smart Folders that collects photos according to their tags isn’t the right solution. In those cases, I have a couple of scripts that allow me to replicate my set of Smart Folders into a set of real folders with copies of the photos organized by subject.

The problem with tagging and Smart Folders is that they are too Mac-centric. If I need to share project photos with a colleague or a client—who are almost always Windows users—I can’t just copy a file structure like this onto a USB stick and give it to them:

Photo organization with smart folders for tags

The JPEGs are fine, as are the dated directories, but the Smart Folders are just gibberish on a Windows machine—a set of XML files with .savedSearch extensions. If I need to have the photos broadly available in folders organized by subject, I need to make real folders and put copies of the photos in them.

Fortunately, this is pretty easy to do if I’ve already done the tagging and created the Smart Folders for each tag. I have a command-line script called tags2dirs which, when run from the parent directory (e.g., the test directory in the example above), creates a set of traditional folders that parallel the Smart Folders. After running tags2dirs, I get this:

Photo organization with folders for copies

The tags2dirs script is this short bit of Python:

python:
1:  #!/usr/bin/python
2:  
3:  import os
4:  import glob
5:  
6:  tagDirs = glob.glob('*.savedSearch')
7:  newDirs = [ x.split('.')[0] for x in tagDirs ]
8:  map(os.mkdir, newDirs)

Line 6 looks through the current directory and collects the names of all the Smart Folders into the tagDirs list. Line 7 goes through that list, strips the .savedSearch extension off of each name and saves the result to the newDirs list. Line 8 then makes new directories with the names in newDirs. Simple.

Now comes the harder part: copying each photograph in the dated folders into the corresponding folders named for the tags. As you might expect, I have a script for doing this, too. It’s called cp2tags, and when invoked like this from the test directory,

cp2tags */*.jpg

it will copy every JPEG file one directory level down into all of the appropriate directories that were created by tags2dirs. For example, the left track folder will look like this:

Left track copies folder

This uses a lot of disk space—if a photo has six tags, it will be copied six times—but both of my Macs have 3 TB Fusion Drives, so I can afford to be profligate, at least occasionally.

Here’s the source code for cp2tags:

python:
 1:  #!/usr/bin/python
 2:  
 3:  import os.path
 4:  import subprocess as sb
 5:  import sys
 6:  import shutil
 7:  
 8:  tagCmd = '/usr/local/bin/tag'
 9:  for f in sys.argv[1:]:
10:    tagString = sb.check_output([tagCmd, '--no-name', '--list', f]).strip()
11:    if tagString:
12:      tags = tagString.split(',')
13:      for t in tags:
14:        shutil.copy(f, t)

It goes through the arguments, which are expected to be file names, one at a time. For each file, it runs James Berry’s tag command, mentioned in yesterday’s post, to determine all the tags applied to each. The output of tag is a string of comma-separated tags, which is split apart on Line 12 to create a Python list of the file’s tags. Line 14 then copies the file to all the directories named after those tags.

For most projects, I don’t need tags2dirs or cp2tags, because I don’t need to send others my photos. But it’s nice to have the scripts ready.

One last thing: Tags created on the Mac can be used to filter files in the iOS Files app but only if the files are saved in iCloud Drive. File tags are synced to Dropbox, but the Files app doesn’t seem to know it. And I haven’t seen anything in the Dropbox app to suggest that it can filter by tags.

I’ve added tags to the sidebar through the clumsy tap-and-hold technique, but I haven’t worked out a quick, automated way to get a tag list into the Files sidebar.

Tags in Files sidebar

It’s too bad the Files app knows no more about Smart Folders than Windows does.


  1. For me, the subject of a photo is usually a piece of machinery, a structural component, or a fragment of a broken part. Many of my photos include more than one subject, which makes them a natural for tagging. 


A little tagging automation

I resisted tagging my files for quite a while. Why bother with tags when we already have a hierarchical file system for our first method of organization and hard links when we need another method? But I’ve come around to using tags, partly because I got used to them on iOS and partly because the advantages of hard links are lost when you use a cloud system like Dropbox to keep two computers in sync.1 So I’ve gradually developed a set of scripts and macros for dealing efficiently with tags.

I use these automation tools almost exclusively with the photographs I take for work, so I should start by saying that I don’t use the Photos app and don’t ever expect to, despite its internal tools for categorizing photos. The photos I take for work are project- and client-specific. Each project has to be kept separate, and I often need to be able to copy my entire project file for either archiving or sending to a client. Hard experience has taught me that folders of JPEGs are better for my situation than any structure within Photos.

My usual baseline method of organizing photos is to put them in folders according to date. The photos themselves, which come out of the camera with names like IMG_1234.JPG or DSCN1234.JPG, are renamed according to date, photographer, and image number like this:

20181004abc-001.jpg

where “abc” are the initials of the photographer, usually me. The photos are put into folders named according to the date using the same yyyymmdd format as the file name prefix. On small projects with only a few dozen photos, this is sufficient, and my photo organization starts and ends with a system like this:

Photos organized by date

But when I have a more complex set of photos, and particularly when I have photos of many objects that have to be analyzed separately (residential units in a building, machine parts, chunks of an exploded boiler, etc.) its useful to have them also organized by object. The key here is “also”—I don’t want to lose the date organization, I want object organization in addition to date organization.

The natural way to do this on a Mac is to use tags. There’s a nice command-line tool called tag, written by James Berry, that does pretty much whatever you might want with tags: adding, removing, listing, and finding files. If you use Homebrew, you can get it via brew install tag.

I have a Keyboard Maestro macro that uses tag. When I want to organize photos by object, I open a Finder window for one of my dated photo folders in icon view with the icon size set large enough for me to identify the photo subject(s). I then work my way through the folder, selecting photos and calling the macro with the ⌃⇧T keystroke. This brings up a window with a selection box listing the tags I can apply. I select one or more tags, hit the OK button, and move on.

Setting tags

Here’s the Keyboard Maestro macro that does the work,

Set file tags Keyboard Maestro macro

which you can download and edit for your own use.

The first step defines a list of tags, one per line. I change this for every project. The second step is an AppleScript that takes those tags, uses them to create the window with the selection box, and then applies the chosen tags to the selected files. Here’s the AppleScript in full:

applescript:
 1:  -- Create a list of tags from the variable defined above
 2:  tell application "Keyboard Maestro Engine" to set tagString to getvariable "tagListString"
 3:  set oldDelimiters to AppleScript's text item delimiters
 4:  set AppleScript's text item delimiters to linefeed
 5:  set tagList to every text item of tagString
 6:  set AppleScript's text item delimiters to oldDelimiters
 7:  
 8:  -- Add chosen tags from the list to the selected Finder items
 9:  tell application "Finder"
10:    set finderItems to selection as alias list
11:    
12:    -- Ask user to choose tags from the list
13:    set tags to choose from list tagList with title "Add tags" with prompt "Choose file tags(s)" with multiple selections allowed
14:    if tags is false then
15:      -- do nothing
16:    else
17:      -- Assemble the chosen tags into a quoted, comma-separated string
18:      set oldDelimiters to AppleScript's text item delimiters
19:      set AppleScript's text item delimiters to ","
20:      set tagString to tags as text
21:      set AppleScript's text item delimiters to oldDelimiters
22:      set tagString to quote & tagString & quote
23:      
24:      -- Add the tags to each file in turn
25:      -- The `tag` command is from https://github.com/jdberry/tag
26:      -- It can be installed via `brew install tag`
27:      repeat with p in finderItems
28:        set cmd to "/usr/local/bin/tag --add " & tagString & " " & quote & (POSIX path of p) & quote
29:        do shell script cmd
30:      end repeat
31:      
32:    end if
33:  end tell

Much of the script’s length is due to AppleScript’s clumsy tools for moving between lists and strings. The script works like this:

After the tags are applied, I can use Smart Folders keyed to a particular tag to collect all the photos of that object in one spot. Unfortunately, making a Smart Folder that searches for tags involves more scrolling and clicking than I can tolerate.

New Smart folder for tags

So I wrote a command-line script called mktagdirs to do it for me. Running it from the directory that contains all the dated photo subdirectories results in a new Smart Folder for each tag.

Photo organization with smart folders for tags

Here’s the Python source code for mktagdirs:

python:
 1:  #!/usr/bin/python
 2:  
 3:  import plistlib
 4:  import sys
 5:  import os
 6:  import subprocess as sb
 7:  
 8:  # The tag command can be found at https://github.com/jdberry/tag
 9:  # This is where I have it installed (via `brew install tag`)
10:  tagCmd = '/usr/local/bin/tag'
11:  
12:  # Get the working directory and all of the tags in files under it
13:  cwd = os.getcwd()
14:  tagString = sb.check_output([tagCmd, '--no-name', '--recursive']).strip()
15:  tagString = tagString.replace(',', '\n')
16:  tags = set(tagString.split('\n'))
17:  
18:  for t in tags:
19:    # Build the dictionary for the smart folder
20:    rawQuery = '(kMDItemUserTags = "{}"cd)'.format(t)
21:    savedSearch = {
22:    'CompatibleVersion': 1,
23:    'RawQuery': rawQuery,
24:    'RawQueryDict': {
25:      'FinderFilesOnly': True,
26:      'RawQuery': rawQuery,
27:      'SearchScopes': [cwd],
28:      'UserFilesOnly': True},
29:    'SearchCriteria': {
30:      'CurrentFolderPath': [cwd],
31:      'FXScopeArrayOfPaths': [cwd]}}
32:  
33:    # Make the smart folder
34:    plistlib.writePlist(savedSearch, '{}.savedSearch'.format(t))

The first thing to note is that this is using the Python that comes with macOS, which is Python 2.7. This script will not work on recent versions of Python without a little tweaking, because the plistlib module has been rewritten.

The key to understanding mktagdirs is recognizing that Smart Folders aren’t folders at all; they’re just plist files with a .savedSearch extension. It’s the contents of the plist file that determines which files appear when you open a Smart Folder in the Finder.

We’re going to use tag again, this time to gather all the tags of the photo files. Line 10 defines where I have it saved, and Line 14 runs it via the subprocess library. The invocation would look like this on the command line:

/usr/local/bin/tag --no-name -recursive

This returns all of the tags for all of the files within the current directory, including files nested in subdirectories. Each file’s tags are output as a comma-separated list, one line for each file, like this:

boom,bucket,cab,left track,right track,stick
bucket
boom,stick
boom
cab

Lines 15 and 16 take this output and turn it into a Python set called tags, which we start iterating through on Line 17. By using a set instead of a list, repeated entries are reduced to a single item.

Lines 21-31 define a dictionary of the items needed for a Smart Folder. I learned some of this from Kirk McElhearn’s old Macworld article and some from just playing around with the plist of a Smart Folder I’d made “by hand” and seeing what could be deleted without impairing its function. For our purposes, the most import things are the following:

Finally, Line 34 converts the dictionary to a plist and writes it to disk with a .savedSearch extension.

Now I have a way to look at my photos by date and by subject. Each Smart Folder is populated with the files that have its tag.

Left track smart folder

This works well when I’m the only one who needs to see my photos, but not when I have to share them with others. The next post will have a couple of scripts to handle that situation.


  1. Dropbox treats the hard links as separate files and uploads them accordingly. So when your other computer downloads files from Dropbox to stay in sync, it downloads multiple copies of each. This means the links are lost and one of your computers is using much more disk space for the files than the other. 


You really like me

If you’re like me, you tend to think of automation as a way to streamline the performance of complex, many-step actions. And while that’s certainly an important use of automation, sometimes its the simple things, things that barely seem worthy of automating, that are the most satisfying. Today I used Shortcuts to replace a bit of lost functionality in Tweetbot that’s been bugging me for quite a while.

Once upon a time in Tweetbot, you could see who liked one of your tweets by swiping to the left to see it by itself and then tapping on the number of likes.

Tweetbot view of a tweet

You can still do this with retweets, but not with likes. As a fundamentally insecure person, desperate for the approval of others, I miss being able to quickly see who liked my tweets. That’s what my “You Really Like Me” shortcut addresses.1

Here it is, a Share Sheet shortcut with just two steps:

You Really Like Me shortcut steps

It takes advantage of Twitter’s simple URL system. If you want to see the likes of the tweet with URL

https://twitter.com/drdrang/status/1047637958130642945

you just stick /likes onto the end of it:

https://twitter.com/drdrang/status/1047637958130642945/likes

That’s what the first step of the shortcut does. The second step shows the web page for that URL.

List of likes after running shortcut

Boom. Lost functionality replaced. To be sure, what used to be a single tap in Tweetbot is now three (Share→Shortcuts→You Really Like Me), but at least I don’t have to mess around with launching another app and either pasting in a URL or navigating through my tweets.

Update Oct 6, 2018 10:13 AM  I didn’t include a download link for this shortcut, because I figured it was too short to bother. Then I heard from Sebastian Peitsch:

@drdrang Everything’s different in German but I managed to get this done via comparing the icons

Thanks Doc!

  — Sebastian Peitsch (@SPeitsch) Sat Oct 6 2018 2:57 PM

If you’re using Shortcuts in a different language, the names of the actions aren’t the same, which makes it harder to copy than it should be. But an imported shortcut will be localized to your language. So here’s a link to download: You Really Like Me.

Thanks to Sebastian for the help.

Update Oct 6, 2018 10:30 AM  If you find that the shortcut isn’t working for you, that it opens the tweet in a browser view but not the list of likes, it’s because you aren’t logged in to Twitter in that browser. I thought I had tested it under those conditions, but I hadn’t. Thanks to Grant Buell for finding the error and Sean for explaining it.

I should also point out that John Cutler has a two-tap system for getting to his likes:

@oscargong1995 @drdrang @OpenerApp Tapping the time/date stamp on a tweet in Tweetbot will also open the tweet in the mobile twitter site. As long as you’re logged in, you’ll be able to view likes there too.
  — John Cutler (@JohnCutler) Sat Oct 6 2018 1:46 PM

This takes one less tap, but I find it takes slightly more time than my shortcut, as it loads the entire mobile Twitter site after you tap on the time/date stamp. Still, you might prefer it.


  1. With apologies to Sister Bertrille. 


Shortcuts as subroutines

I was too busy at work last week to spend any time with Federico Viticci’s excellent screenshot framing shortcut for iOS, but I dug into it over the weekend and used it as the basis for a shortcut that does the things I usually need done after creating a screenshot:

I renamed Federico’s shortcut from “XS Frames” to “Frame Screenshot.” My extended shortcut is called “Frame and Upload” and I keep it next to “Frame Screenshot” in my list of shortcuts to make it easy to find.

It would typically be called from the Share Sheet within the Photos app after selecting one or more screenshot images to frame. After selecting “Frame and Upload” from the Shortcuts list, the screenshots are processed and then the user is asked to resize the resulting image.

Resize dialog

Because I wrote this to handle images posted here on the blog, and because width is the controlling dimension, the resizing is done by specifying a new width—the height is adjusted automatically to maintain the aspect ratio.

Note that the current width of the framed screenshot is given as the default. If the user just taps the OK button without editing the width field, no resizing is done.

Next, the user is asked for a file name.

File name dialog

Actually, what gets entered here is just a part of the file name. I have a particular naming scheme I use for images here:

yyyymmdd-Description of image.ext

The date the image is uploaded, like 20181002, is included as a prefix. Then comes a description of the image, which can, and usually does, include spaces—this is the part that the user is asked for. I have other scripts that extract this portion of the file name to use as the alt and title attributes in the <img> tag. The extension for these framed screenshots is png.

Assuming everything goes well—as Federico said in his post, the shortcut sometimes fails, probably for lack of memory; I’ve always found that I could run it successfully immediately after a failure—I have a framed screenshot in Photos and a new file in the blog’s images directory on the server.

First and second home screens

Here’s the complete shortcut, with the various sections marked:

Frame and Upload

Now you can see why I’m not interested in specifying the height of an image.

A few things worth expanding on:

Chances are you don’t want to enter all these steps by hand. You can download a template of it and just change the stuff in the SSH action.

I’m still not a big fan of the Workflow/Shortcuts programming environment—I prefer typing to dragging, and really prefer a text environment when I need to edit a program. But there’s no question of its power, especially when you can take entire shortcuts and use them as subroutines. Thanks to the Workflow boys for that. I have a distinct sense that if Shortcuts had been developed within Apple from the start, that feature, essential for building up complex programs, would not have been included.

At present, Federico’s framing shortcut can add only X🅂 and X🅂 Max frames around your screenshots. I assume he didn’t bother with iPad frames because he’s expecting new versions to arrive soon and didn’t want to waste his time on something that would have to be changed almost immediately. Presumably, when Steve Troughton-Smith and Guilherme Rambo uncover the hero images for the new iPads, Federico will be on them like a duck on a June bug. Do they have June bugs in Italy?