As the owner of a digital product agency, this is really hard to read. It is such a shame that an agency would do this to you. I know you felt like they "did their best", but by setting you up with unrealistic expectations out of the gate they essentially guaranteed that everyone was going to walk away unhappy. Besides, when you go to an agency communication/transparency/team of experts is what you are paying for! You're paying for accountability! There really are good agencies out there that care about their customers and bend over backwards to deliver what they promise, but they are hard to find. For the size project you were looking at though, I do agree that a freelancer could have been a better option, but you'll run into some of the same challenges. Finding good freelancers can be just as hard as finding a good agency.
The agency shouldn't have taken the work to begin with -- the part where the lead admitted that he killed project governance entirely was a bit of a painful moment. Agencies are optimized to work at a specific scale, and it's risky for them to scale up or down for a specific project; in this case, their client fell through the cracks because they were using him as fillable hours and didn't ride herd on their designers. Considering they were working outside of the SoW, those were disputable invoices, but that's cold comfort when you don't have the free cash flow to take this to legal.
The difference in size between companies imply different operation trajectories. If you are their smallest customer, you are bound to be deprioritized and treated as second-class. If you are their biggest customer, you may well destroy them with procurement and governance processes.
Yeah, I don't think the agency is blameless here, but I also don't think they're malicious or dishonest. I think they just overestimated their ability to scale down their workflows to a project smaller than their typical gig.
Well they know they've done an awful job, you know it too, but they happily kept the $45K. If they were as honest as you say, they would have refunded some of it.
That's exactly what does not leave my mind! It would have been a question of honour for them to stomach it and deliver close to the original estimate.
Everything else leaves me with a sore feeling, that it, while eventually not a scam, at least a careless, profiteering way to handle this, on the back of the customer. I hate this agency :-)
We build and maintain a lot of Rails applications, with several of them almost a decade old. We have to keep them up to date, but Rails is stable and mature and this process is fairly easy. We come across a lot of web apps written in other ecosystems, using frameworks that were popular just a few years ago and have now been mostly abandoned. Or we get bit by an app that is a few years old and now has dependencies that are completely dead and core to the application. This isn’t the case with Rails. It is just as actively developed now as it was 10 years ago, and we are confident that it’ll be just as actively developed 10 years from now. Due to the fragmentation of these other ecosystems, even though the community as a whole may be larger, finding a stack that you can depend on for a long time can be a huge challenge.
> We come across a lot of web apps written in other ecosystems, using frameworks that were popular just a few years ago and have now been mostly abandoned
Interesting, can you give a few names? Just so that I'll know to reconsider if I need to use one of them.
This post isn’t about a single snapshot in time, this is about all of the people who didn’t evaluate MongoDB properly over the years and then were “burned” by it. I actually think it has received a bad rap due to people over the years not evaluating it properly. Sure, some of these things have changed (transactions very recently), but they all come with serious caveats and limitations that need to be explored... which was the main point of the article.
Hey Justin, thanks for writing the article! I agree with you that there was a failure of evaluation, and that MongoDB has matured quite a bit since the NoSQL explosion.
Sorry if my comment came across as a criticism of your post. It was more targeted at the long threads of people bashing MongoDB, thinking the caveats are still relevant/accurate today. Because they really aren't – MongoDB is a completely valid choice of database today.
ACID transactions were added over a year ago, and Mongoose is almost as old as MongoDB itself.
Author here... we talk a lot at our company about essential complexity and incidental/accidental complexity. Software is complex, absolutely. It is complex because business and reality are complex. I think that is inescapable. What we can try to escape though is the incidental complexity that we add to our systems while we are following trends or chasing down shiny new tools.
You get an outcome appropriate for the risk assumed. The capitalist business world values speed of development over reduction of risk.
That's why at many companies, whatever tools and languages allow you to get an MVP out the door before a competitor are not merely accepted but are often celebrated and lauded.
When risk really matters, such as the commercial flight industry, the pace moves much slower. It takes comparatively forever for new features to reach the market because the cost of mistakes is so significant (human life). If business behaved the way regulated commercial flight behaved, we wouldn't be having this conversation.
Pointing to open source or expressive languages as a source of complexity is, obviously to me, a red herring. The failure is our lack of rigorous engineering practices - those practices we gave up in the interest of beating our competitors to market.
You don't even have to look very far back in time to recall when software was expected to be correct upon delivery. Nowadays, particularly with online patching, we are totally accustomed to bugs and frequent fixes. Only a decade ago this was unheard of. The cost of reprinting manuals, burning new CDs (or gasp stacks of floppies) and shipping those out to all customers was so prohibitive that it was worth delaying release until all important features were in place and a full series of tests had been performed.
It's our behavior and expectations that got us where we are, not the tools. And probably there's no going back. This is the age of instant, so we'll do our best to adapt. Ironically, the human who used to have to BE the integration between his/her software tools is now the monitor making sure that the automated integrations are managed properly when an unexpected situation arises. And now we've built new layers of logging tools to help us spot these situations. But we're still stuck managing the processes...
> That's why at many companies, whatever tools and languages allow you to get an MVP out the door before a competitor are not merely accepted but are often celebrated and lauded.
The linked article has some points I agree strongly with, but I disagree in its approach to tools. An MVP that's out the door as soon as possible is unlikely to have the sort of incidental complexity tech debt that kills projects. It can't, because the focus was on shipping.
A quick prototype for an MVP is great, get stuff out the door as soon as possible. If your app needs some javascript, write it however lets you get it out the door the quickest - if that means importing someone else's big framework, fine.
I think the secret is more akin to keep _your own code_ as minimal as possible. Your infrastructure and stack should respect the M part of MVP just as much as the product should.
Cause I rarely see true MVPs. I see "ok in order to scale to a million users in the first year, we're going to need x, y, and z on top of...". And then you're not just importing someone else's JS code and writing your business logic on top. Now you're building several systems that have to work together correctly even as they get modified independently in the future, and so in the process you're encoding a bunch of assumptions about your business that the MVP hasn't actually proved out yet! And that's tech debt of the deadly sort.
We're far away from having tools that will prevent a dev from building something overly complicated if they really really want to. Instead, we have to prevent it by always asking the hard questions "do we really need this right now" and not being afraid to look like the fun police.
That's the catch. The MVP becomes the future product. We basically start with a "what can you get me by date X" codebase, and then we're forever in the "ok, what can you get me by Y" cycle. It's a long time before the builders get a real opportunity to go back and do things right.
If the MVP is successful, that success almost guarantees a continuation of the risky practices. If it's not successful, it probably means the jig is up and the code goes into /dev/null.
I don't have a lot of big team experience. But in my limited experience working with other devs, I've seen a mixture of the builders and the doers. But what I've seen in almost every case (except for the too-much-cash-flow .com cases, which are largely a thing of the past... perhaps replaced with VC funded frenzy behavior) is management that doesn't want to spend any more time on a feature than is "absolutely" necessary. And they have comparisons to offer - other companies that did the thing within a certain timeframe (unrealistic or MVPish).
It's true though, the tools we use aren't really project development tools; they're still mostly get-it-done tools. We probably would be well served with commercial (funded, supported) components and frameworks that devs work with. At least some of the obvious mistakes wouldn't be repeated.
But then we return to the realities of the modern business world. Project budgets are compared to competitors, and there's always some competitor that has outsourced a project for 1/5 of what we suggest it should take, or they've bought a COTS kit and "tailored" it for their needs. Basically, the management expectations have become quite unrealistic. Ultimately, it's easier to change jobs than to effect a change of understanding and behavior in a company. In a techie way, it's a race to the bottom.
The alternative is to build a side project that isn't total garbage. If enough of us do that, we reshape the expectations of users.
While I generally agree with you - do remember that Rich talks about "information-driven situated programs" - there are plenty of occasions when you don't do that. Like, when you're building a business. It could be a fatal flaw to think that your early prototype is a "situated program" - it is, in fact, just a quick hack meant to validate a concept. You start by not knowing enough about the market & your customers - so you shouldn't optimize for simplicity, you should optimize for time-to-value. New shiny tools might be good for this purpose (they're often shallow, but also they often deliver shiny&meaningful value fast).
Author here... I agree that developers often lose sight of the business problem being solved, but I think just stopping there is an over-simplification. It is very possible to build an application that serves the business problem adequately, but is grotesquely over-engineered or solves the problem in a ridiculous way. The world is built on software like that. My point was that we are introducing a lot of needless complexity to systems, all in the name of following trends or chasing the new shiny. And we are hurting ourselves by doing it.
Author here... yeah, we are loading a lot of crap, aren't we? We will do an audit and make sure we aren't loading a bunch of unnecessary stuff, but the whole site should be served up through a CDN, sorry it is loading slow for you.
The page triggers 50 HTTP queries and is 3.85MB, so a CDN is only going to do so much.
* the base page is light but has very high latency (250ms)
* there's an almost-empty CSS (style.css) next to one which seems a bit overwrought (screen.css, 140KB is a lot for a CSS file), oh plus normalize from cloudflare despite it apparently already being included in screen.css
* you're loading 9 different JS files from your CDN, plus some emoji crap from not your CDN
* typekit accounts for 1 JS query followed by fetching 12 different font files, at 15~30K each
* 14 different queries to "driftt.com" whatever the fuck that is though I expect some sort of analytics/tracking crap given
* tracking & analytics & ancillary service queries up the ass: hubspot (4), facebook (4), google analytics (2), drift.com (2), others (5)
* the Lato CSS from google fonts but apparently not the font itself, on the other hand you're loading "larsseit.otf" from your CDN, maybe pick just one?
It also works perfectly fine with all 3rd-party requests blocked (via uMatrix) for me. So other than the CSS none of what you mentioned is necessary to get a readable article.
50 could be "quite low" if most of the requests were for "content" images. TFA has under half a dozen images, as I've outlined in my comment most of the requests are for ancillary garbage.
> I was auditing a big uk ecommerce site at the weekend and the homepage had over 170 elements
See above, 170 is pretty high but assuming at least one image request per product[0] plus a few more for buttons & the like, having a large number of requests would be understandable for an ecommerce site.
ecommerce sites would also be a place where:
1. analytics is very, very understandable
2. the site would use a pretty generic system which could hinder specific optimisations
3. assuming the user will browse around, the site could preload various stuff, sacrificing some upfront performance for a nicer "inside" experience
All in all, I'd be much more understanding of that than 50 request on a blog post, even more so a blog post on software complexity. Incidentally, the amazon.com home page generates ~250 requests.
[0] because spriting for web pages remains a pain in the ass, doubly so for JPEG
I've felt like any project horribly out of date is a pain to get current, Rails is no exception to that. Sure, many Rails projects have a lot of dependencies, but so do other frameworks. Sure, part of your argument is static vs dynamic, but I'm not going to join you in that argument.<grin>
I considered speaking to that, but then I felt like I was delving into religious arguments. My framework is more efficient than your framework! I felt it would have drawn people away from my main point, which is to think deeply about the tools you choose, and everyone needs different tools.
