The impending death of everything…
…and the continuing need for architectural change
Enterprise Architecture, along with the role of the Architect, is dead…
As is Design apparently… maybe. Of course, it’s well documented that Blockchain is dead(and many times over in the case of Bitcoin). DevOps might be too (although we’re not sure), Artificial Intelligence as well. Quantum Computing may be both dead and alive, but we won’t know till we check! 😉
It seems we have a certain preoccupation with predicting the impending demise of almost everything around us. We do this when we feel something is maybe too popular, no longer trendy or fashionable or perhaps has just been around for a while. Maybe this is just human nature being what it is and that perverse inner voice of ours that wants to take things down a peg or two, or maybe it’s the more optimistic side of us that always looks to the horizon for ways to be better. But that’s a very philosophical question…
Regardless, rather than being a death knell, these sentiments are often a good barometer for identifying the need for change in a given area. The clamour and noise around these sentiments are usually outlining why something no longer works as well as it used to or why it’s going to soon stop being effective. From a software architecture standpoint and the role of the architect, rumours of its impending demise have been circulating for years.
So what has needed to change along the way and what still needs to?
Traditionally, the role of the software architect focused primarily around the initial physical and logical design of the system as a whole, with attention paid to the non-functional requirements such as the ability to scale, to be secure, to be available and other similar factors, based on the needs of the specific system. The functionality of the system was often left to a functional design team to dictate, while the architect just dealt with questions like:
- how to keep it running,
- how many servers it’s going to need,
- and how not to lose any data.
Maybe the functional team also got inputs from the sales team for a product, or direction from key sponsors (and maybe even users once in while!) but the responsibilities were very siloed and scope was defined up front and written in stone (the less said about the graffiti of change requests, the better!).
Over the years, the main changes that occurred in the way that an architect went about their work were based around the architectural styles they used to design systems, e.g. the fat client-server model, n-tier architectures, SOA, Microservice design, containers, IaaS/SaaS etc. These changes were primarily driven by the need for solutions to do things in a way they hadn’t before, as well as the capabilities that evolving technologies (hardware, software & networking) could provide. E.g. when the internet became mainstream, the need to build globally accessible systems became relevant for the first time for a lot of systems. Before broadband became the norm, “always-on” applications were limited to very niche mission critical requirements.
The other major changes in ways that architects worked were brought on by the evolution of work practices for designing and building software. Extreme Programming (XP) and later Agile introduced a different cadence of design & development iteration than was previously possible (or wise) using older Waterfall techniques or its variants (anyone else remember the dreaded V-Model? Looking back it seems more like a pyramid scheme; expensive to get in to and not returning on it’s promise in the end!). This meant radical changes for architects who usually did their work up-front, got them agreed & signed off and then handed them off to the development team to think about and try to remember when they were writing code. This change led to the need to disseminate ownership of the architecture throughout the team, at least to some extent, and have the architects working closely with the team on day-to-day decisions and impacts. This also put paid to the concept — but unfortunately not the practice in all cases — of the Ivory Tower architects that had sprung up as the norm in a lot of organisations. This is where the architects declaimed their solutions from on-high with the risk of not being close enough to the implementing and supporting teams to understand the impacts of their decisions in the real systems or on real users.
As an aside, an interesting thing is that the outputs produced by an architect have remained broadly similar down through the years, being often (digital) paper-based documents containing diagrams and supporting text. There have been many approaches to try to model and automate the production of these outputs (and even to use them to automatically write bad code!) but a lot of these attempts have not been as successful as they could or at least have not progressed enough into broad adoption to become fully universal and so they remain as they are.
So, if all that’s better, what still needs to change today?
The process of building a software system, is still fundamentally similar to what it has been since software became large enough to require teams of people. The elements are still all recognisably present — plan, design, build, test, deploy, run, monitor — but the arrangement of how and when has changed, along with who does what.
But what are the areas that we still have a tendency to do badly when going through the architectural process?
1. Decision making
Implications can range from none at all to the large scale pain of being stuck with a solution that doesn’t meet your needs
The biggest and most crucial one in my opinion is decision making. Not the actual act of choosing a direction, which is often easy, but not recognising the decision being made, leading to implicit rather than explicit decisions. Implicit decisions occur when we choose a direction without actually evaluating alternatives. Often these only come to light when we get asked the dreaded question, “why did you choose X and not Y?” Depending on the implicit choice, this could be an easy answer or an embarrassing “ah, I don’t know. We never thought about it any other way…”. Implications can range from none at all, to needing a quick reshuffle, to the large scale problem of being stuck with a solution that doesn’t meet your needs and is expensive to remediate.
The other aspect of decision making problems that I often see (and do myself if I’m not paying attention to it!) happens even with explicit decisions and comes from evaluating a decision on only a subset of the necessary criteria. E.g. “What language should we use for our new backend microservice? Let’s use Golang because the service API will get quite a lot of concurrent traffic, it’s a technology that’s gaining a lot of traction in the market, it works within a serverless context with most vendors and it has a reputation for being more performant than a lot of the other languages.” Sounds like a reasonable rationale, right? But what are the other dimensions that we should be evaluating before we reach that decision? Who will be the long term owners of solution? What’s their skill set? What are our other existing services written in? Are we happy to commit to a polyglot support structure? Does it matter whether or not the language is more performant or not since the performance is likely to be network or disk/DB limited for most APIs in any case? Is super high level performance a key requirement? These questions are not to imply that Golang is the wrong choice, only that we may not have answered the question before deciding. A decision is only worth the effort that’s been put into evaluating it.
As well as the criteria we need to evaluate against and prioritise, we also need to look at all the relevant alternatives that are available and consider those too. E.g. how would Java, Python or JavaScript score against the same prioritised criteria? Would that change or decision?
The bad news after all that is that you may have been right in the first place! It becomes clear why this behaviour is difficult to change, it’s like dieting or quitting a bad habit: it needs discipline, willpower and intent. Of course, if we do this it means we can also quickly record those results and then refer back to them as the basis for our decision later. This helps us create a core part of our lightweight documentation for the architecture at the time that we make each decision. It also allows us to understand the impacts when we need to revisit a decision.
2. Architecture treated as a job or a position, not a mindset
Our industry is too full of elitism and mystique on the “dark arts” of architecture
The second problem is one of structures. Being an Architect cannot be about a job title or even a career choice. It has to be a role you play and a mindset that you adopt. Whether the person playing that role is a full-time dedicated architect who does nothing else or whether architectural decisions are shared and made by the team as a whole makes no difference. It’s the mindset and approach that’s important.
Architectural thinking is a mindset than can be adopted by anyone who has had enough experience to recognise the potential impacts of decisions and to understand the trade-offs they are making by choosing one alternative or another. Our industry is too full of elitism and mystique on the “dark arts” of architecture when it is simply the ability to take a systems view on a problem and apply experience and a structured decision making processes to understand the trade-offs being made.
The architecture should be able to be contributed to by anyone on the team, but in saying that it should not be expected that everyone must contribute. In the same way that it is of benefit to any developer to understand and have experience across the full stack, having experience of practising architectural thinking is useful — but that in no way means that every developer must be required to be a full stack developer, or indeed an architect. A successful team needs to combine and encourage the growth of both focused specialists and versatile generalists.
Organisations need to correct their career tracks to ensure that architects are not seen as the pinnacle of a career goal for every developer and that architectural experience is not seen as only available to or useful for people on an architectural career track.
3. Dedicated followers of fashion
The problem here is what I like to call “Conference Driven Development”!
Trends, trends, trends. There is an almost insatiable urge to follow them or to at least try them out for most architects, as they are often new, interesting and intriguing. When approached properly, they can lead to new understandings and new options as well as professional growth and fulfilment. Organisationally, they can also provide options to boost retention of talented team members as the work involved is often interesting. Win, win? Surely. However, problems can easily arise!
The problem here is what I like to call “Conference Driven Development”! This is the pattern seen when someone, be they a developer, an architect or even the CTO/CEO, returns from a technical conference having seen what’s possible with the newest technology/technique/platform/pattern and for the next 3 months finds it necessary to insert said trend into the organisation in some shape or form. This can lead to the exploration of new areas, which is good. However, often the pain of trying to shoe-horn a new technology or approach into a solution that doesn’t need it can be very large, both in the immediate term in project delivery, but also in the long term, with the cost of dealing with the results of these experiments when they don’t fit.
By all means, experiment with new approaches, techniques and technologies — it is essential to remaining in touch with the market and the technology to avoid falling into an irreversible spiral of technical debt — just make sure that the problems you have and the pain points you need to prioritise actually require the newest and greatest. Good decision making and objective evaluation of alternatives trumps trend chasing every time.
4. Over-engineering
The pursuit of any ideal to the extreme only very rarely leads to an outcome that meets that same ideal.
The bane of any architect who thinks they are worth their salt is the pressing need to define the most elegant solution. In a software engineering culture where the very word elegance has taken on a meaning of its own — using small amounts of code to great effect — architects have taken this to heart looking to maximise reuse and increase modularity, sometimes at all costs.
At best this leads to greater complexity in the design of some parts of the architecture (and usually several times the complexity in implementation) with a payoff of simplicity at other levels. At worst it can lead to unintended coupling between unrelated components e.g. because they artificially share the same library or consume the same service purely in the name of reuse. The pursuit of any ideal to the extreme only very rarely leads to an outcome that meets that same ideal.
As architects or anyone playing an architectural role in the design and implementation of a software system, we need to be mindful of the value of simplicity. Simplicity is not the opposite of complexity and is not the same as simple, but rather is a goal for solving complex problems with grace and economy, aiming toward necessary elegance rather than most elegant — good is often better than perfect.
The King is Dead! Long live the King!
Software engineering and software architecture are continually being changed, evolved and refined but at the same time remain fundamentally the same as they always have been.
The original ideas and values still hold true today as they have in the past and will in the future. The core strategies for designing a well-architected system such as good isolation, loose coupling, high cohesion and designing for change will remain, but the mechanisms we use to implement them will continue to change. And while we need to pay attention to the areas that need to change in order to continue to advance and improve, we need to ensure not to put too much emphasis on the foretellers of doom, prophesying the death of the next current big thing. After all, death comes to us all, even to the technological and platform royalty of the day, but it’s just an opportunity to pass on the crown 👑.
Disclaimer:
All views expressed are my own and do not reflect the opinions of my employer or any other entity I am/have been associated with. Use at your own risk. Always ask your pharmacist :)
