It's great to see this post I wrote years ago still being useful for people.
I agree with many here that the situation is arguably worse in many ways. However, along similar lines, I've been pleased to see a move away from cargo culting microservices (another topic I addressed in a separate post on that site).
To all those helping companies and teams improve performance, keep it up! There is hope!
I gave a lecture at the University of Toronto last year with a similar point to what the speaker was getting at. Namely, we should be focusing on augmenting human intelligence, not chasing some sort of future robot overlords. See https://adamdrake.com/novel-results-considered-harmful.html
This was the approach Douglas Englebart took in the lead up to the Mother of All Demos (MOAD) and it's unfortunate people have forgotten that bit of history.
There is so much that can be done to improve the lives of people if we simply focus on how to optimize a variety of things that have to happen every day. This doesn't mean we should forsake longer-term goals like fully autonomous vehicles or other things, but it does mean that some of those who are most capable of making a very real and significant impact in the lives of thousands of people are forsaking that opportunity.
There is also often the argument that such near-term thinking will lead us to a local optimum in terms of technology advancement, but I don't see that there has been significant evidence to support that claim. After all, Englebart's demo of version control and collaborative editing (among other things) was over 50 years ago and that didn't seem to stall technological progress.
Great! We (technologists) should be using machine learning approaches to practical real world problems, particularly those problems whose solutions allow humans to be more productive. That's what I mean by augmenting human intelligence/capabilities.
Most of the time, I've found a push to microservices within an organization to be due to some combination of:
1) Business is pressuring tech teams to deliver faster, and they cannot, so they blame current system (derogatory name: monolith) and present microservices as solution. Note, this is the same tired argument from years ago when people would refer to legacy systems/legacy code as the reason for not being able to deliver.
2) Inexperienced developers proposing microservices because they think it sounds much more fun than working on the system as it is currently designed.
3) Technical people trying to avoid addressing the lack of communication and leadership in the organization by implementing technical solutions. This is common in the case where tech teams end up trying to "do microservices" as a way to reduce merge conflicts or other such difficulties that are ultimately a problem of human interaction and lack of leadership. Technology does not solve these problems.
4) Inexperienced developers not understanding the immense costs of coordination/operations/administration that come along with a microservices architecture.
5) Some people read about microservices on the engineering blog of one of the major tech companies, and those people are unaware that such blogs are a recruiting tool of said company. Many (most?) of those posts are specifically designed to be interesting and present the company as doing groundbreaking stuff in order to increase inbound applicant demand and fill seats. Those posts should not be construed as architectural advice or _best practices_.
In the end, it's absolutely the case that a movement to microservices is something that should be evolutionary, and in direct need to technical requirements. For nearly every company out there, a horizontally-scaled monolith will be much simpler to maintain and extend than some web of services, each of which can be horizontally scaled on their own.
I also wrote https://adamdrake.com/enough-with-the-microservices.html as a way to communicate some of this, including some thoughts on when and how to structure a codebase (monolith) and when it might make sense to start moving towards microservices, etc. There are cases where it's reasonable (even advisable) to move towards microservices, but they are rare.
Microservices are a cult, but SOA is Amazon's conerstone. Most services are the right size to fit a team (so, not micro) and implement separation of concerns.
Some Amazon teams have multiple services of varying sizes. What Amazon and similar companies get right is by (generally) insisting on good engineering/business/regulatory reasons for splitting out services.
> Business is pressuring tech teams to deliver faster, and they cannot, so they blame current system
And are very often right about it. Delivering monoliths can require such amount of bureaucracy and needless coordination that it slows everything down. I've seen it.
> Inexperienced developers proposing microservices because they think it sounds much more fun than working on the system as it is currently designed.
It doesn't matter who proposes something, if its good idea, do it. And in my experience it is indeed more fun (in addition to other benefits).
> Technology does not solve these problems.
Using microservices is a matter of organisation, it has nothing to do with technology. It is non-technical solution for non-technical problem. Any effect it may have on a technology is secondary to main goal.
> In the end, it's absolutely the case that a movement to microservices is something that should be evolutionary
Nothing you said before contradicts that.
> and in direct need to technical requirements
How people are organized is not a technical requirement.
> For nearly every company out there, a horizontally-scaled monolith will be much simpler to maintain and extend than some web of services,
This sounds sanctimonious and rife with the exact same inexperience and overly brittle purist attitudes you are criticizing.
It really is true that huge monolith legacy systems might prevent dev teams focused on product growth from even being capable of doing their jobs, let alone meeting aggressive deadlines.
It doesn’t always mean microservices or heavy re-architecture is the right choice, but sometimes it absolutely is.
The places where I’ve seen the most value to pivoting away from existing monoliths often have benefited a lot from microservices.
I was part of a group that split a huge tangled mess of search engine and image processing services in a monorepo into separate smaller web services, and by further separating them into distinct repos per project, we could migrate things to new versions, convert some legacy Java services into Python to take advantage of machine learning tools that fundamentally do not exist in jvm languages, all in more careful, isolated ways that monorepo tooling just simply doesn’t support, and lots of other things that would not have been possible if we tried to steadily change portions while preserving their co-integration in a single large project that attempted to support modularization in ways that were simply just bad.
Your language seems to betray the fact that you personally associate the entire concept of microservices with being intrinsically dogmatic.
Typically only dogmatic people feel that way, in my experience. But either way, there’s nothing inherently dogmatic about a microservices approach.
To be fair, here is a direct quote from the parent:
»In the end, it's absolutely the case that a movement to microservices is something that should be evolutionary, and in direct need to technical requirements.«
I would argue that what you did is exactly that. Perhaps with the caveat that it should have been done earlier.
I'm not reading tha parent arguing that one should stick with a monorepo/monolith until the end of time, but rather providing a few thoughts around what might cause a push in applying microservices incorrectly.
Isn't the argument that is being made that you did things exactly right. That microservices are a great architecture to migrate to when you feel the need. But aren't a great to start a project.
I don’t see how you get that at all. The comment starts out expressly criticizing when organizations consider migrating to microservices from existing monolith projects.
The comment was arguing the use of microservices is far more common than the need for microservices. And based on your description it sounds like you guys were one of the few that had a need for microservices.
> " The comment was arguing the use of microservices is far more common than the need for microservices."
In re-reading the parent comment several times now and taking some time to reflect on it, I find that I am not able to agree with this interpretation of it.
As I understand it, the parent comment is taking issue with any type of reaction to a monolith in the direction of switching to microservices as a tactic to get rid of the blockage and tech debt. The comment does allow that some cases may support the use of microservices, but this secondary comment is so at odds with the sanctimonious tone sardonically criticizing people who want to migrate to SOA from a monolith, that I just do not find that phrasing to contribute much to my understanding of the comment. It seems clear to me that the comment means to harshly denigrate the idea of wanting to switch to SOA as a solution strategy in those cases, and the "concession" that sometimes it might be the right thing to do is tacked on, not really related to everything else.
I accept that we might just agree to disagree on the interpretation, but I still feel comfortable that my original interpretation is the most consistent with the available text of the comment and the context of it.
This is a great list. I've been reflecting on this drive for microservices in early stage companies for the last four years, and you hit all the major points.
One additional one I'll add is the marketing objectives of containerization and infrastructure companies.
And a good time to resurface Martin Fowler's Monolith First:
People can see that in a one day hackathon, the same bunch of people can produce more stuff than they do in a year otherwise. Why? Are they lazy? Did they use better tools?
My niece Shelly added address book integration to her hobby app in an afternoon, while drunk. WhyTF are we 640 man hours deep into "identity architecture coordination" meetings?!! Just do with Shelly did!
Those things don't make total sense, even to even the saltiest of developers. They know to expect it, but can't understand it. Neither can I, honestly. It's not surprising this gets so many people.
A lot of the hairy, abstract rabbit holes we climb into (whether organisation, like agile, or architectural, like microservices) are an attempt to solve the "100+100=6, wtf!" problem.
I guess that's the difference between proper Engineering and hacking something quickly and under self-inflicted pressure together. Your nieces app might work on her setup but not on others and may need almost a complete rewrite on an OS/API update. The properly engineered solution on the other hand might "just work" for years. I think key is to realise how much engineering is needed on what occasion. I find it disturbing when a simple app has highly sophisticated error handling unit tests as I would find it disturbing to find 30kLOC code bases with no unit tests, completely random spaghetti architecture and no explicit error handling.
I guess it depends on the company. But most of the time it seems bloated bureaucracy is a symptom of dysfunctional work relationships. When people know what to expect from each other and stuff "just works", then there is no need for any such thing and indeed you probably won't find it I think
Especially as I'm sitting here learning ASP.NET Core Identity on Pluralsight for hours and thinking about all the "identity architecture coordination meetings" I'm anticipating having on this next project.
> My niece Shelly added address book integration to her hobby app in an afternoon, while drunk.
We shouldn't downplay this achievement. That is impressive!
I agree with you that reality is very confusing. I think much suffering is caused by not fully embracing this fact. I don't mean to be defeatist. On the contrary, this great confusion presents a vast landscape for potential improvements.
I would attribute it to _fragile ego_. The confidence that's displayed right after reading/skimming a blog post & taking it AS-IS is baffling (not the first time I've seen it)
Mostly agree in full. But I think it's worth noting that microservices - or any technology for the matter - cannot fix a dysfunctional organization that's not clear about it's current and future business needs.
Does IT fail sometimes? Sure. But more often than not projects go sideways at the leadership / team (i.e., all stakeholders, not just IT) level. Blaming IT is a convenient narrative.
You make quite a lot of assumptions with 2 and 4. I was one of those inexperienced. Not as a developer but of the architecture of micro-services. As a technical lead I did not have the visibility of the cost and organizational effort into a micro-service architecture. I just started developing an architecture for our current project domain because we needed the flexibility of deploying these components separately.
But upon pitching the idea to an Organization Solution Architect VP he quickly stopped me and demonstrated me the cost of this effort and he did not have to demonstrate the effort because I went through similar challenges with in my team, and so expanding that effort into the entire organization would have been a massive undertaking.
So he did not shoot down the idea he just wants to take it down a notch and start with compartments, and not the entire organization.
I would just like to add that I consider there to be two types of microservices, one is developer focused, which is what most people are talking about when they say the word, but the other is operations focused, which is some sysadmins have finally been embracing.
When it comes to the sysadmin version, since you may be wondering, it mostly means decoupling entangled services into seperate, less centralized bins, bringing more resilency and quicker diagnosis timeframes when problems occur.
In my experience microservices have been prescribed as a sales bullet point instead of a software architecture decision.
Every time it's resulted in insane low traffic bottlenecks all over the place as services chatter away or separately need to look at he same file data so all request a copy etc.
Any architecture has tradeoffs and it's poor form to pick one before you've even described what the software is for.
Yeah, I definitely agree with these points. And I think quite a few of them come down to one simple thing:
There's a huge disconnect between what many developers wish they were doing and what many developers are doing. They wish they were at Google/Facebook/Amazon/whatever working on some complex greenfield project that'll change the world, they're instead working on CRUD apps for corporate clients, agencies and businesses with far less technical needs.
So their obsession with using microservices and modern JavaScript frameworks and complicated build processes and what not for everything comes down to them trying to turn B into A, even if the actual solutions they need don't actually require any of that complexity.
I don't believe it os reasonable to portray microservices as the result of incompetence and blame-shifting.
Microservices are actually a very basic and fundamental principle of software engineering: separation of concerns. If your system is extensive enough so that it covers multiple independent concerns and your team is large and already organized into teams focused on each concerns then it makes technical and organizational sense to divide the project into independent services.
They're subject to a clear limit, though. The more micro you make your services, the less they resemble operational units and the more they resemble primitives from which your actual system is built, sort of like an inner-platform effect. And then you have to debug interactions between microservices, with all the overhead that entails.
Separation of concerns is important, but how much separation do you need? Separate classes, definitely. Separate libraries...often. Completely separate services on different VMs with an api between them...if that is really what your situation requires, then sure, but I wouldn't make it the default option.
The mistake is when people think a buzzword is the new best practice without doing real analysis.
Is large is a key phrase here. I am seeing many times now that teams build more services then they have members. This only pushes separation of concern into the network layer when it could just as well be in a module or class level.
You can tightly couple services too, you know. I would say it is a good option to have but suffers a bit from too much popularity right now. People are using it for its own sake.
I find that, ironically, they are great for all of the things that lambda and google cloud functions make difficult, e.g. image processing, open cv, heavy calculation using third party libraries, etc. What we need is a lambda-like service that uses Docker files that run for a max of 540 seconds.
> Business is pressuring tech teams to deliver faster, and they cannot, so they blame current system (derogatory name: monolith)
They are not wrong though, monoliths cannot give you fast delivery. Fast delivery implies at least expressive dynamically typed languages with some resilience to bugs, which in turn requires limiting the scope of bugs and therefore decoupling and isolating everything as much as possible. This is very different architecture from monoliths. Microservices are a first step there, but of course not a substitute for lightweight isolated processes and supervision trees. Still, monoliths are definitely bad choices in every way possible if you can split them into isolated services.
I disagree. You can have fast delivery with monoliths - IME microservices provide better separation of concerns, which one the benefits could be faster delivery.
However your end user doesn’t interact with microservices, they interact with a product. “microservices” suggested as a delivery silver bullet tend to be ways managers try and mask the fact that they are trying to hire 9 women to make a baby in one month.
I typically see people try to push microservices as a "best practice" for improving the delivery rate of software teams. They carve the codebase into separate services, but still leave all of the persistence in place so that all services are communicating with the same DB(s). The result is a tangled web of interdependent services, plus additional tooling like Docker, Kubernetes, etc. that actually makes the teams even slower than they were before.
I'm glad to see us as a community getting more pragmatic about these topics, and realizing that _best practices_ are highly context dependent!
One thing I haven't seen in the comments, but have observed personally is the accounting implications and reasoning for these failed SAP projects.
Sometimes the project cannot be stopped immediately during development, even if it is already known that the project is a failure. The reason is that some companies capitalize the cost of the development over many years, and if they were to cancel the project they would have to book the entire amount as an expense in a given year, instead of depreciating it over multiple years.
I know of one company that worked on an SAP implementation for about 5 years, when it was only supposed to take 6 months or so, and the reason was that if they stopped the project they would have to book the entire cost as an expense in the current financial year. The exec team, who had financially-based bonuses, weren't keen on that.
Instead the project was put together to the extent possible so that it could be declared usable, and then promptly shelved. However, it was "completed" so in theory the sunk cost could have been spread out over multiple years.
You may want to re-submit that, since it didn't get any traction 38 days ago but might today, if this article and this comment get enough eyeballs.
I now hear actual data scientists complain that startups trying to hire them think they have "big data" problems when all they have are "data" problems.
No real questions, just that I love reading about this sort of optimization - Abrash's Black Book is a favourite that still gets pulled out every now and then. Thanks for the fun post!
It always astounds me these days when someone manages to release slow software for a desktop computer despite modern systems being orders of magnitude faster than the first desktop computers while often being no more responsive.
Edit: Also I'm amused by how much of a nerve this seems to have hit. I guess some people are defensive about their high performance computing approaches... :)
> I guess some people are defensive about their high performance computing approaches...
Fortunately, when articles about real HPC clusters (aka Supercomputers) make it to the front page, they need no defense, only an occasional explanation that what makes them (extra) special are the high-bandwidth, low-latency interconnects. (Those interconnects make a very strong effort at Fallacies of Distributed Computing numbers 2 and 3[1] and tend to neatly take care of the remaining 6).
To be fair, though, I don't think the claim for the more common distributed systems is that they're "high performance" so much as that they're scalable (and have other benefits of being multi-node like resilience).
[1] The bandwidth may well be indistinguishable from infinite if it exceeds local (e.g. CPU-memory, CPU-CPU, CPU-GPU) bandwidths. I don't think that's yet true for something like multi-socket Intel systems, but it might be possible with enough interface cards. I didn't look at the specs on those POWER9 chips.
I use it from time to time, but I certainly wouldn't consider myself an expert. When I need to do something beyond my current knowledge I search around a bit until I find the awk syntax I need to get the job done. Sorry I can't be more helpful!
I'll second the sibling comment about the original AWK book (The AWK Programming Language).
I'd also suggest just using it more. If you find yourself wanting to use grep, cut or sed (especially if you need more than one!) for a one-liner, try using awk instead, if only for the practice. Once you're accustomed to some of the idioms, simplicity, and built-in looping, it will feel like a more natural, casual text-processing tool that you'd reach for automatically.
Lastly, search for "cookbooks" or libraries or other collections of useful scripts to see how others have used the language, in more advanced ways. Personally, I don't necessarily find it as something to aspire to (i.e. use of the language for complex, multi-line programs) as much as a way to better understand some of the power of the quirkier features that may, otherwise, be less obvious.
One of my favorite non-obvious behaviors is bypassing the input/loop functionality by having the only pattern be BEGIN. This results in an output-only program that makes for a convenient way of playing around with things like printf or just language syntax features (or differences between awk/mawk/gawk), without having to redirect input from /dev/null or worrying about hitting control-D twice and exiting the shell (for those of us who abhor ignoreeof):
The main thesis is that there seem to be more and more companies out there solving interesting problems, which by itself is great, but they're bolting a lot of wording on top talking about AI. Most of this seems to be an attempt to differentiate themselves in the market and access funding and I find it incredibly dishonest.
The whole discussion around AI these days has become so tainted by scheisters trying to attract funding and attention that I actively try to distance myself from association.
Douglas Engelbart and others had this figured out 50 years ago (see: the Mother of All Demos). The AI hype is dangerous, since it will no doubt lead to the trough of disillusionment.
I'm not of the opinion that it is dangerous as an idea, but rather that the intentional deception I see on the AI topic is dangerous (not to mention unethical).
I can't think of a single example of an "AI" company I talked with who did not know that they were explicitly exaggerating their claims and capabilities for the express purpose of attracting funding and fueling the hype fire.
The solid tech startups I've seen are focused on the problem they are solving for customers rather than the tooling (AI).
Not to mention that AI as we define it does not actually exist yet. Anyone who uses that term is doing so dishonestly. Show me any intelligent code and I'll shut up. Until then I call BS on anything that's claims to be powered by AI.
I think you can reasonably call self driving cars AI without it being BS. Many things like translation may qualify based on how you frame the question, but driving is an open ended real world task.
Artificial Intelligence: the theory and development of computer systems able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages. [Google]
It's great to see this post I wrote years ago still being useful for people.
I agree with many here that the situation is arguably worse in many ways. However, along similar lines, I've been pleased to see a move away from cargo culting microservices (another topic I addressed in a separate post on that site).
To all those helping companies and teams improve performance, keep it up! There is hope!