Whenever people ask me about my philosophy for building software, top of my list is always to say that I am a proponent of modular design. To explain what I mean by that, it is first necessary to understand a few of the fundamental theories that underpin how I view software engineering as a discipline, and then we can explore how those come together in a philosophy of modularity.
Systems
Software engineering is a discipline for the design and implementation of complex systems of interacting elements. If a program is intended to serve a single purpose by completing just one task, that task can be broken down into smaller tasks, which can themselves be broken down further and further.
The result of conceptually atomising monolithic operations is a network of interdependent operations that comprise the wider system. The boundaries of this system are wide and varied - how data is written and read from hardware devices, communication of information between geographically separated networks, even the knowledge and opinions of humans interacting with websites or smartphone apps. Whatever we are building, our control extends only partway through the network.
Systems thinking is the embracing of the vastness and complexity of all the moving parts that influence the operation of software. The challenges we face therefore come from the limits of our own capacity for comprehension.
Abstraction
To address this complexity, the primary philosophical tool in the software engineer’s belt is abstraction. To abstract something is to hide its complexity by defining only the way that it interacts with the world around it, and to deliberately behave in a way that ignores the inner workings of the thing.
This is the fundamental thought process in software development, first articulated by Dijkstra in his essay “The Humble Programmer”. The central thesis of that essay, written in the 1970s before the invention of the microprocessor, was that as programmers we are developing systems too complex to hold fully within our mind all at once. At the time, this was a unique and forward-thinking insight - today it is obvious.
Abstraction teaches us that software engineering is a philosophical discipline - one in which our goal is to enhance our ability to reason about complex systems by effectively abstracting away complexities that lie outside our immediate sphere of interest. Components that are behind or within the components we directly interact with are ignored - only the immediate effect on us is relevant.
Encapsulation
To make this kind of abstraction possible, it is necessary that components within the system encapsulate their complexity such that others can abstract it away when interacting with them.
This concept is often talked about within the domain of object-oriented design - a discipline that is perhaps less fashionable than it once was, but still underpins much of the fundamental way in which software is best built. The core idea is that a component should encapsulate its functionality and only present what it does to the outside world, not how it does it. It is not necessary to understand what happens to the engine of a car when you depress the accelerator pedal - only to know that it goes faster.
In normal language, we would call the surface of encapsulated functionality an interface, but our industry loves three letter acronyms, and so we universally tend to use the clunky term Application Programming Interface for this, or API for short.
Recursion
Recursion is a simple-to-explain, difficult-to-grasp context from mathematics that is all over software design. The simple explanation is that it is just the concept of a system in which components can be self-referential within their category. To give an example, I have two parents, and each of them had two parents, and so on. The “and so on” in this sentence is the recursion. This one very simple relationship belies a hugely complex tree of connections - my family tree, in this case.
Through this lens, there are no limits to encapsulation, where each layer of abstraction can be nested within arbitrary other layers of abstraction. Components encapsulate components, recursively. A component has many components within it, each of which has many more. And so on.
The ability to zoom into the right layer of abstraction within this tree of complexity is how we accomplish the task of writing software. By well defining interfaces around components, by abstracting the details of how they work and instead just understanding *what *they do, and by navigating through a recursive tree of components, we are able to reason about the behaviour of our system at all levels of abstraction.
Modularity
What then, is modular software design? Well, it is an acceptance of the above philosophy, and an embracing of the implications, specifically in thinking about how systems evolve over time. For me, this fundamentally comes down to designing systems such that you maximise your ability to effect change through adding, moving or replacing components, rather than modifying existing ones. This motivation comes from the hard-earned experience that modifying working software is hard, but replacing or adding software can be easy if your setup is optimised for it.
By thinking of systems as networks of interconnected and nested components, and adopting a strategy of ensuring all of those components have well defined interfaces at each layer of abstraction, we build software that is flexible, resilient and easy to reason about.