In Greek mythology, Scylla and Charybdis represented the issue of navigating between two hazards: lethal rocks on one facet, a fearsome whirlpool on the opposite. Fashionable digital and IT professionals encounter the same dilemma: technical debt and sprawl make IT portfolios insecure and unsustainable, however processes and controls supposed to reduce debt and sprawl are themselves too typically value-destroying.
Typically primarily based on prolonged checklists, these evaluations have been the bane of software and product groups for a few years. Countless inquiries and types to fill out, ending too typically with seemingly arbitrary choices calling for vital alterations to structure and design, with ensuing delays in deploying worth to prospects and finish customers. The outcome has been years of low-intensity battle in IT organizations, and much an excessive amount of waste and re-work.
How did we get right here?
At first, there was the mainframe. This was an extremely constrained platform. You would have any sort of database so long as it was DB2 (or IMS for you actual oldtimers on the market). Authentication, authorization, transactional integrity, scaling, scheduling, logging, chargeback, knowledge persistence – many companies had been out there, many of the arduous questions had been already settled, and builders simply sat down and wrote their COBOL functions.
Then, distributed computing got here. Each challenge was customized engineered. I entered the workforce right now, and one in every of my earlier positions was with Andersen Consulting (later Accenture), the place we had been constructing Peoplesoft ERP options. Each challenge had a definite technical design. Would it not be Solar or Compaq or HP servers? HP/UX or Solaris? EMC or Hitachi storage? Oracle or DB2? Gross sales reps for the distributors would become involved and the decision-making was messy.
Then I moved into the enterprise world the place we had been simply as typically constructing our personal apps. All the infrastructure choices nonetheless needed to be made – the infrastructure group might need opinions, but when your challenge was sufficiently big or the CTO favored you, you can just about purchase no matter stack you wished – and on prime of that, you needed to clear up many extra issues. What was your safety strategy? Create your personal little record of customers and passwords in your app? (Quite common again within the day.) What about backup? Resilience? Scaling? Firewall? You needed to determine all of that out.
So with the customized engineering of technical stacks for each distributed software, basic structure wheels wanted to be re-invented each single time. Add in challenge, and later product workforce autonomy, and also you had a recipe for large variation and sprawl.
(A lot is alleged concerning the variations between challenge and product administration, however talking from expertise each types typically share a fierce autonomy and disdain for centralized requirements.)
Enter enterprise structure, and its detractors. Whereas EAs typically hear cries of “ivory tower,” “out of contact,” and so forth, such criticisms overlook the basic level: enterprises put structure in place to ship enterprise worth, similar to the worth of controlling the dangers that emerge from technical debt and sprawl: safety breaches, outages, unsustainable value constructions, and so forth. And so EA began to attempt to management the morass distributed computing had gotten itself into, by means of standardizing product decisions (Oracle and SQL Server are OK, however let’s sundown Informix) and reviewing design decisions submitted by challenge (later product) groups.
The EA evaluations began to embody a wider and wider array of questions. Prolonged consumption types and checklists grew to become typical, protecting all the things from software design patterns to persistence approaches to monitoring to backup and extra. The infamous “Structure Assessment Board” grew to become a requirement for all tasks. Assessment cycles grew to become extra protracted and painful, and quite a lot of enterprise organizations bought fed up and simply went shadow, at the least till their shadow methods failed, at which level central IT could be pressured to take over the mess. Agile groups advocating for autonomy fought the EAs to a standstill, however neither facet may “win” as a result of each had been and are wanted.
The answer began to emerge from the Agile and DevOps communities themselves. A key second was the IT Revolution Press publication of Staff Topologies by Skelton & Pais (disclosure: I used to be a developmental reviewer on TT). Their dialogue of workforce cognitive load, and recognition of the necessity for platform groups in addition to stream-aligned groups, was a watershed.
What’s a platform? Recently I merely outline it as a product that helps different merchandise. Within the context of digital methods, it represents a standardized providing “marketed” to groups who’re creating merchandise for finish shoppers, whether or not within the open market or inner to a big group.
Platforms cut back cognitive load by means of standardization and automation. They characterize “paved roads” enabling supply at pace, in comparison with the “gravel roads” of constructing and securing your personal stack from scratch. They’re opinionated, and cut back your decisions similar to the mainframe, however with a funds code, your builders will be growing software program with confidence in a short time and guess what? In a correctly executed platform technique, main structure and safety checks have occurred. That fifty-item guidelines diminishes by 90%. (There nonetheless could also be issues about grasp knowledge administration and some different matters). I’m speaking to many EA organizations who’re aligning carefully with the platform engineering workforce and making certain that as many as attainable of the standard EA checks are both baked into the platform structure, or executed as a part of automated QA within the related DevOps pipeline (I consider each main platform also needs to embody the automated mechanisms by which code is developed, constructed, and deployed). Automated testing, static and dynamic evaluation, SBOM checks, and extra can all be operationalized on this means.
There are different advantages to specializing in fewer platforms: permitting for smoother transition of workers between groups, concentrating on fewer venders and getting higher leverage, minimizing your assault floor, and more practical incident response as a result of your workers has a deeper information within the chosen platforms.
There is no such thing as a free lunch. A platform constrains your choices. Nonetheless, we nonetheless know learn how to construct stacks from scratch as properly, and we’ll proceed to do this in emergent areas like GenAI. And in the end constructing that stack turns into “undifferentiated heavy lifting,” with at finest an oblique connection to actual finish client worth.
It’s been an extended journey, from the mainframe by means of the bewildering number of distributed choices, and now again to one thing that once more is extra standardized. In spite of everything, the mainframe was and is nothing greater than the unique opinionated platform. At the very least these days you’ll have a couple of extra decisions.
For extra data see my report Speed up With Platform Engineering.