When and why I automate

Way back in February, there was a Back to Work episode in which Merlin talked about how easy it is to waste time on “productivity”—spending more time on automating a task than you’ll ever get back on the savings that automation gives you. I was going to write a little post on that topic but work got in the way and then I forgot about it.

At the end of March, Clark Goble wrote a post on the same topic, complete with a funny graph taken from a now-lost recently resurrected post by Jason Verly. Clark made this point:

[G]eeks generally relax writing these sorts of things. I know probably half of my scripts were as much me relaxing and trying to clear my head from whatever I was working on at the time. That they make me much more efficient is just icing on the cake.

This was one of the points I had planned to make: For many of us, automating tasks is a form of recreation. A strict, green-eyeshade accounting of time spent doesn’t capture the value to us of writing a script. There was another point or two I wanted to make on the topic, but I was on spring break with my family at the time and wasn’t blogging. When we returned, I had some unexpected travel, which delayed my return to posting even further.

So now that the topic is cold and dead, I’m finally getting around to writing about it. This is why I’m not a link blogger.

There are five things I weigh when deciding whether to automate a task:

  1. The potential time savings. This is based on estimates of how often I think I’ll have to repeat the task, how long it takes to do manually, and how long it’ll take to write a script (or macro or Automator action or whatever). The stuff Merlin talked about.
  2. The entertainment value of creating the automation. For example, I’m currently beta testing an app that’s adding a sort of scripting language to an upcoming release. Learning the language by writing a few scripts in it was fun. I’ll certainly use the scripts I wrote, but I’ll never get back the time I put into them. That’s OK with me.
  3. The annoyance I feel when doing a task manually that could be automated. This is the flip side to the entertainment and relaxation value Clark talked about. I hate hate hate doing repetitive tasks when I know I could write a script to turn several steps into one. I know this is a personality defect that keeps me from being as efficient as I could be, and I try to keep it in check, but it’d be foolish to deny it exists.
  4. The value of learning a new technique or library and keeping sharp with what I already know. I write scripts to get my professional work done, and the more nonessential, recreational scripts I write, the more efficient I am at writing the scripts I need to get my job done.
  5. The value it might provide to others. I’ve learned so much from people who share their knowledge on the web, I feel compelled to reciprocate. This is, I confess, almost never my initial motivation, but it the reason I sometimes add a little extra polish to a script that’s already working.

Sharing scripts and scripting techniques also changes the accounting of time. A script that takes an hour to write but only saves me 15 minutes may seem like a waste, but if it saves three other people 15 minutes, too, the overall accounts are in balance. I’ll get my “lost” 45 minutes back when others share the scripts they’ve written.

17 Responses to “When and why I automate”

  1. Dexter Ang says:

    Those 5 reasons… are pretty much my personal reasons as well.

    I guess us “geeks” are pretty forever curious and lazy. I find that the “helping others” part also keeps us sharp, in that it hopefully makes us improve our scripts, and maybe learn from what others contribute.

    Thank you very much for all your contributions. Especially how to change the battery on a Toyota’s remote because that was a particular pain in the a**.

  2. Aristotle Pagaltzis says:

    I always considered it to be not about time but tedium saved – a matter of psychology, not accounting. It’s a way to turn an unrewarding task into a different one that is (or at least is more so). Outside of trivial cases it’s hard to make a straight-up economic argument about it. Even its inefficiency is not necessarily easily argued – sure I could nominally have been more efficient, had I eschewed automation in some example case, but what if in spite of its small size I find a task exceedingly painful? Better to spend half a day coming up with a way for the machine to do it than to keep putting off those 2 hours of work for months on end, isn’t it? So then doing it in a less efficient way has made me more effective overall.

    Other aspects that merit consideration: automation is far more reliable at doing things the same way repeatedly – humans make mistakes. Although an automation can be faulty too; or just unprepared for a new situation which the human would have dealt with without thinking twice about it. And on the converse, relying on automation may atrophy the skills and experience at doing the task manually that you need to automate it well in the first place.

    There is a lot going on. Efficiency is almost beside the point, certainly a superficial substitute criterion to latch onto.

  3. Doug Adams says:

    You said it.

  4. Tom S. says:

    I’ll add one more reason that I think you’ll agree with, at least as far as writing scripts for professional work.

    I analyze a lot of data for a living. Its non-trivial often involving a lot of steps in the process with multiple software programs. I find that by writing scripts to analyze the data, it leaves behind an exact record of what I did. Years later I can come back, look at the script and know exactly what I did to come to the conclusion I did. Sometimes I get back the time I put in writing the script. Some times I don’t. But the record is something you can’t put a value on.

  5. RobbieP says:

    I do automation to eliminate mistakes as much as possible on repetitive monthly bills and statements both paper and paperless. I don’t know how to write code but I do know how to find more efficient ways of doing mundane work. Since I don’t know how to write automation, I use Hazel to watch my stuff and stuff it where I won’t need it till next tax season. I realize this is very simple and not geekie, but it helps me not make mistakes. So in my world, I need more correctness and gathering of those repetitive items than initial time savings.

  6. Paul DeCarli says:

    I can’t understand why my coworkers think that I’m “overdoing it” when I develop sophisticated automation workflows. I try to explain to them: if I create an automation that saves me 4 seconds per repetitive action, and trigger that automation 400 times in a day, I’ve saved over 26 minutes. Even if that automation took 3 hours to develop and fully implement, I make that time back in less than a week. I’ve been using an automation like this for almost 3 years now; I calculate that I’ve saved more than a full month of time in front of my computer.

  7. Tony McDaniel says:

    I wanted to add to what Tom mentioned above. My research involves applying the same analysis to multiple test cases and comparing the results. Automating the process not only removes the tedium of repeating all the steps, it also prevents human error which could result in incorrect results.

    For the accounting aspect, I’m able to reuse some pieces of code, and know that by keeping functions modular, I can use what I’m writing now in later projects.

  8. Thomas S says:

    Ooooh - look - I am not alone ;) Great - thanks for your post.


  9. Tom Reid says:

    Thank you. This is great.

  10. Jason Verly says:

    Link to the post Dr Drang mentioned above has been fixed. Full link is: Geek Task Management.

    Not exactly the way I wanted to find a problem. ;-)

  11. Andre Kibbe says:

    There’s also the multiplier of sharing the automation. Publishing a script that gets used by dozens or hundreds of other people can easily save exponentially more time than the script took to develop.

  12. Dr. Drang says:

    Tom and Tony,
    You’re absolutely right about scripting leaving a reproducible trail in data analysis. Like you, I always turn my data analysis steps into a script, but until you mentioned it, I didn’t even think of that as automation. I think of it more as documentation.

    Aristotle and Robbie,
    Accuracy is another good reason to automate—one that I’ve mentioned in other posts but inexplicably omitted here. Thanks for the reminder!

  13. Clark says:

    As I mentioned the one nice thing about scripting regularly as a hobby is that you can usually take script fragments to repurpose quickly when you really do need an important script. With employees who don’t always understand what they are doing the accuracy and speed both count a lot.

    Ultimately while I agree 100% with everything Drang says, I also see it akin to exercise. When I’m running on a treadmill I’m not likely to take people complaining that I’m not getting anywhere seriously. Some skills need practice to be maintained. Scripting for fun does that and it really is a useful skill to have.

  14. Jeffrey Nonken says:

    Several commenters have already talked about automating repetitive tasks to minimize errors, and that’s also one of my reasons. I want to add to that.

    If you screw up something in your process, doing it by hand means re-doing it by hand. That seriously eats into any times saved not creating your automation, and seriously increases chances of mistakes. Whereas if you automate the task, if you make a mistake — all you have to do is fix it in the automation, and then re-run the automation. Fixed!

    If something in the task changes, tweak your automation and run it again. Fixed!

    Next week you find a similar task. Adapt your code to the new task and run. Done!

    Does this happen every time? No. Sometimes you just spend more time automating than you save. But like a formal process, the idea is to reduce expended resources in the long run. Think of it as an investment. Or hedging your bets.

    As well as all the other reasons mentioned here. They all count. Automate your heart out, and laugh at anybody who claims you’re wasting your time. Hah hah! I laugh at your presumption, you small person of limited imagination and love of tedium.

    Well, maybe not quite like that.

  15. Artem Sapegin says:

    Goog post!

    I have one more reason to automate things: how easy to do something wrong or forget essential details. And it’s also true for things you don’t do very often because often making simple script isn’t harder than then saving sequence of shell command in Evernote and manually replacing data when you want to do the same thing next time.

  16. Designer and Geek says:

    I wholeheartedly agree! (Hence my alias!)

  17. Aristotle Pagaltzis says:

    And for the time-savings side of the argument…