I enjoyed Samuel Arbesman’s first book, The Half-Life of Facts, which was a discussion of the exponential pace of change, as exemplified by Moore’s Law, among other things. When I saw the title of his new book,Overcomplicated, I assumed that it would be a warning that we technologists had gone too far in creating complex systems. It would advocate moving to simpler systems, just as a doctor might advise an overweight person to go on a diet. I was prepared to argue against such a conclusion, but as I discovered upon reading the book, Arbesman does not say that complexity is necessarily bad or that we should seek simplicity. Instead, he maintains that systems are now unknowably complex, that they will become even more so, and we should…just get over it.

Much of the book is spent in discussing the reasons why complexity is inevitably increasing. Arbesman writes that “almost everything we do in the technological realm seems to lead us away from elegance and understandability, and toward impenetrable complexity and unexpectedness.” He cites three main factors driving this increase—“accretion,” “interconnection,” and “edge cases.” Accretion is the result of large systems being built on top of smaller and older systems, often via the incorporation of legacy code, producing what we call kludges. As these subsystems become interconnected, the resulting entanglement can change what was simply intricate to truly complex. Finally, complexity is exacerbated by the inevitable existence of edge cases—that myriad of individually negligible exceptions and rarities that yet constitute the long tail of cases that must all be accounted for in system design. The complexity resulting from these factors has passed a tipping point where no single person can fully understand a complete system.

At this point in reading the book I’m thinking that this is all familiar territory to us engineers. The question is: What do we do about it? Arbesman passes quickly by our usual strategies to manage complexity—the abstraction of subsystems to hide the complexity of lower levels, and “good hygiene” in coding. Good things to do,…[Read more]