222. Automating Processes with Software is HARD
We have decades of experience trying to automate processes. My biggest lesson is that automation is not about the easy and known flow, but about exception handling.
Something about automation that I believe is that it is *way* more difficult to do than most imagine. Most look at problems to automate from the outside—a process they find tiresome, slow, yet repetitious. They often don’t think of their own skills and tasks as easily automated. The reality is much more difficult.
Most people who have built an automation know, or at least come to know, just how fragile it is. It is fragile because steps are not followed. Tools and connections fail in unexpected ways. Or most critically because the inputs are not nearly as precise or complete as they need to be. And they come to know that addressing any of those is wildly complex. Ask any engineer that has designed their own CBCI, build, or deployment process and they will tell you “don’t touch it” unless I am around.
There’s a deep reality about any process—human or automated—that few seem to acknowledge. Most all automations are not defined by the standard case or the typical inputs but by exceptions. It is exceptions that drive all the complexity. Exceptions make for all the insanely difficult to understand rules. It is exceptions that make automations difficult, not the routine. And over time most all systems are about exception handling not routine.
The very first business system I built was an inventory system that made it easy to assemble an order for customers and even printed a shipping label. Then one day the business owner (my father) said “hey this customer is going to come by and pick up the order so don’t print a shipping label.” Yikes had to figure that out. Then one day another customer said “hey can we pay cash on delivery?” C.O.D. used to be a thing. Then I had to add that. Then there were different commission rates for the new sales person. And on and on. Oh and then one day UPS came out with an entirely different way to compute shipping costs. Pretty soon my perfectly automated system was not just a UI mess but all the logic was about handling different circumstances for running the business.
This is what defines all modern business systems and processes. Anyone who has ever sat in a business review meeting knows that all the interesting questions are not in the standard reported. Who has not lived through “this is our template” for review meetings only to be confronted with an entire discussion about what is not on the template? That’s because all the interesting questions are exceptions to the preconceived notion of what we need to know.
The best diagnosis for exception handling I can think of is to wait on line at the post office. If you’ve ever done that, you know the thought of “doesn’t anyone just want to mail a package” comes to mind. As it turns out the entire flow at the post office (or DMV or tax office) is about exception handling. No amount of software is going to get you out of there because it is piecing together a bunch of inputs and outputs that are outside the bounds of a system.
The ability to automate hinges not just on the ability to know the steps to take for predefined inputs, and not even the steps to take if some inputs are erroneous or incomplete, but what to do if you can’t even specify the inputs.
Almost all travel automation is about not having the inputs. Are dates flexible? Seating flexible? Specific airline flexible? I recently saw a great tweet about how “programming will go away when people can just specify the goal of the code and then the code will get created”. My first thought was “Software engineering and product management. You’ve invented software engineering and product management.”
This was the argument about compilers when they were invented. Many said “don’t worry now that we have a compiler we don’t have to worry about the [assembly] code.” Unfortunately for a very long time to use a compiler meant debugging the compiler. And for any system that matters knowing the output of a compiler continued to matter for decades. And of course knowing what code would be generated matters too (eg recursion is great in a textbook but not in practice).
No current automation example is as interesting (to me) as medical diagnosis. There are tons of claims about how we can automate diagnosis (and treatment protocols). The problem is medicine is incredibly uncertain as a baseline. But also patients are incredible uncertain. It isn’t just that people are not always complete in even saying their symptoms. For example, shoulder pain might be an injury from playing tennis the patient does 3 times a week and has been around for a couple of weeks. Or it might be metastatic cancer going undetected. That’s quite a range (real example by the way.) So they might not even mention it to a doc. Or if they did, no doctor is going to run and do a CT or even lab work based on shoulder pain. But sure enough there are case histories and thus training models that immediately leap to a worst case. This same holds even for routine preventative work. It is wildly common for routine lab work to fall out of range (most every CBC or Chem20 shows something out of range, defined as 2 SD). This could easily be a lab aberration, a recent exposure to an unknown allergen, dehydration from fasting, or forgot to fast, or ate ice create 14 hours ago. Sure enough a trained model will say If you think healthcare expenses are high now, imagine what happens when the worst case based on incomplete or missing information is used as a baseline.
In all cases this could easily be said that the opportunity is to automate because the reality is that “most” (however that is defined) people in roles are not very good at handling exceptions and software would indeed be better. There is some truth to that—for every story I could produce about a missed diagnosis someone will have one about an astute doctor that had seen a similar zebra earlier in career. It isn’t that simple though. In most routine automations the way exceptions are handled is by the escalation path of humans. The frustration is real in dealing with this—re-explaining your problem, finding the expert, and so on. But it might also be the best way to handle escalation.
Rightfully one might suggest that automation is the way to handle the front line, at least for now. The problem then becomes not just the error rate in that but that the tools are currently designed to always produce a fine sounding answer rather than say “I don’t know.” This works fine or even better than fine for generating on the fly words or images for basic use cases. But in practice that is because those uses do not have financial, physical, or societal costs. So even the most basic of agents and automation will be seen to require oversight.
I am a proponent of that because that is how AI should be thought of—as a tool for the right people to use. The AI safety world was in a rush to say AI safety requires that AI not be used or should be regulated in whole new ways. Except AI as a tool is regulated just as a PC or calculator or PDR is regulated—the regulation is not the tool but the person using the tool. The liability is with the person that deployed or employed the tool, not the tool itself. There are many edge cases here, such as when a tool was designed with malicious intent or negligence. Those are higher bars as they should be.
We can see this at work with how self-checkout is evolving. You’d think 15 years into the smart phone revolution most people could operate an order kiosk or self-checkout without help. That’s certainly what stores had hoped. But as these are rolling out you can see how these systems are now staffed by people there to handle the exception. Amazon Go will be surly seen ahead of its time, but those are now staffed full time and your order is checked on the way night. And special orders at McDonalds? Head to the counter :-) (I’m picky)
I’m super optimistic about automating things. I’ve spent a career on automation. But part of what that career taught me—from some examples like automating finances in excel, to document templates, to creating good looking charts in PowerPoint, scheduling in Outlook, workflows in SharePoint…all fairly basic tasks—is that the baseline is easy but highly unsatisfactory. The success in those tools was not because they could help people get started but because they could help people get finished.
Automation will come but the breakthrough is going to look a lot like product management showing up and spending a huge amount of energy on what the inputs to a system are and what it means to have and handle exceptions.
In the end, that might look a lot like programming…again.
My favorite example of automation is scheduling meetings. I never met an important big boss with multiple administrative assistants who didn't tell me scheduling meetings is easy and asked why they still need to have admins or why doesn't Outlook solve this. Oy vey.
The least empathetic managers I knew had no idea how hard it was to schedule meetings. There is literally no standard process. Everyone who ever told me it was standard ("I [big boss] always let people use my blocks of time MW 10-12") simply wasn't thinking about how the organization contorted around their rules and how every reschedule was a cascade of scrambling and frustration. Problem solved.
I will go on a limb and say this, much like travel planning and medical diagnosis, scheduling that most valuable asset, time, will be the very last task to be completely automated from end-user to completion in one step. They will be assisted, like the way the web totally changed travel planning but really didn't automate it as much as just augment it. These systems will be automated by an elaborate dialog of collecting and structuring inputs such that the output becomes deterministic and repeatable.
We thought we'd solved scheduling in Outlook when we invented "free/busy" time and even had the first "schedule over the internet" in 2000 I think. Fast forward two decades+ and there are now systems where one side does the up front work to structure a calendar and then just asks the other side to do the work of picking an open time with a web site. This is not unlike what we first did as free/busy. I don't know about you, but I am so far trending 100% on meetings rescheduled where I pick the time off someone else's calendar without their specific input.
Automation is difficult even for seemingly simple things. It is highly likely that what will ultimately be automated by AI falls into two classes:
Totally different tasks than what we view as laborious or tedious today. In other words, not the first things that jump to mind.
Work that is open to be completely redefined to function in some new way.
My favorite example of the latter is how the arrival of IBM computing in the 60s and 70s totally changed the definition of accounting, inventory control, and business operations. Every process that was "computerized" ultimately looked nothing at all like what was going on under those green eyeshades in accounting. Much of the early internet (and still most bank and insurance) look like HTML front ends to mainframe 3270 screens. Those might eventually change, just not quickly. It might be that the "legacy" or "installed base" of many processes is such that the cost to change is too monumental. It might be that AI will be more fundamental changes to whole countries that were basically not automated some old way or maybe the most recently automated countries—the way much of Africa only knew mobile phone computing or that East Asia is primarily cloud-based computing.
There's a huge opportunity for the US and WE to use AI as a chance to reconsider digital infrastructure.
That's a heavy lift.
—Steven Sinofsky
# # # # #