But on earth, we have seen now several instances where teams have moved from polyrepo setup to a monorepo. Although "monorepo vs polyrepo" is always a debated topic, and it's hard to scale a monorepo, large companies like Stripe, Canva, Cruise, Doordash have been able to manage monorepos by building strong tooling and automation to handle the scale.
We're excited to announce "The Hangar", a community created specifically for software engineers working in developer experience space to come together, and learn from each other.
The Hangar already boasts a community of dev-productivity teams/ dev-experience teams from leading companies like Figma, Cruise, Square/Block, Slack, and Netflix. The group features open discussions around best practices, useful tools, relevant industry news and career tips.
At The Hangar, we host monthly “off-the-record” zoom hangouts where you'll find detailed insights on topics such as migrating to a self-hosted CI system or setting up Bazel correctly. This is a space where vetted experienced professionals can exchange ideas, share hard-earned wisdom, troubleshoot issues, and ultimately help each other in their projects and careers.
I've also had terrible experience posting on SO lately. Even though I have been posting on it for several years, they keep flagging my questions. Plus none of the questions these days get answered without a bounty. So, now I have to wait over 2+ days to get an answer.
Dev productivity metrics have come a long way from the time when one use to measure the number of lines of code written or the PRs created. I think Dora metrics is still a step up from that as it focuses on measuring team productivity vs individuals, and provides some indicator of the true bottlenecks for productivity.
For everyone who complains about monorepos, remember some of the most forward thinking engineering companies like Google and FB also use monorepos. All the arguments that people make in favor of polyrepo are making so because of lack of strong tooling for monorepos. That's also why Google and FB would not have scaled if they were using GitHub / GitLab but had to build their own. Also Google's original source control was built on top of perforce!
The issue with building a custom monorepo system that can handle Google's and Facebook's scale is that it fails to scale down, even to moderately large project and organizational size. It's expensive (think at least 7 figures opex) and not what most people should be doing.
git, for all its issues (and I'm a git-hater), scales down to an individual coder and scales up (with a lot of hacks, the hacks being used varying depending on whether you're taking a poly or mono approach) to companies that employ thousands of developers.
Polyrepo scales to thousands coders without anyone even noticing, and that's the beauty of it.
Just look at the size of node_modules in an average project. You stand on the shoulders of thousands of other engineers and you don't even think about it. That's your polyrepo at work.
Now imagine that every time a dependency wants ship a new version, their maintainers have to update all of the dependents. That's your monorepo.
It is quite obvious which one is more scalable.
The only real benefit monorepo has is that every dependency is always at it's latest version. But the cost to achieve that... let's just say there's a reason you mostly hear about monorepos from Google.
For anyone who complains about dictatorships, remember some of the most resource-abundant countries are dictatorships.
Those companies should be one of last places to look for good software development practices. They have absolutely no incentive to recognize their own mistakes.
a) There is no correlation between mono/poly repos and your ability to scale. There are many examples of successful companies using either approach.
b) As a general rule people should be cautious about adopting approaches and technologies from Google, Meta without a clear understanding of why they need it. What works at their scale doesn't always apply to smaller teams.
> remember some of the most forward thinking engineering companies like Google and FB also use monorepos
As a counterpoint though I’d say that the issues Google and FB face, particularly in terms of the sheer scale of the work they’re doing, is pretty unique.
Google literally invents programming languages for domains it feels needs them, I’m not about to blindly follow that practise either.
They way at least FB is using a monorepo is very different than any kind of monorepo most people imagine. It's not just about tooling, git itself could never handle it. I am all about using a monorepo but Google and FB having one is not the an argument for it.
IIRC originally neither mercurial nor git would scale enough, but mercurial was much more willing to accept scalability-related patches, so long as they didn’t harm the more common small-scale use-cases. After a couple of years of submitting patches for moderate improvements, meta wanted to make some more controversial large-scale changes like dropping support for sequential commit numbers, and ended up hard-forking and breaking compatibility to do that. That incompatible-but-better-performing fork then stayed internal for a while, before recently being released as Sapling.
As a bonus, part of the rewrite of the internal storage engine involved creating a storage engine abstraction layer, which in turn made it easy to add Git as a backend :D
GitHub actions has support for this, not sure about Circle, but I think they do iirc. Still doesn't help if the hosted service is down. If your hosted trigger connection is down, no amount of instances is going to help
That's very interesting feedback. We certainly don't have a way to force simulate failure.
A related capability we are working on is to also rerun the identified flaky tests X times so they pass. This depends on the capabilities of the test runner, so it will work with specific ones first (cypress, pytest, etc). That way you still make sure that flaky tests pass instead of supressing.
Curious, how often do you see a flaky test in your system? In my past experience at one of the mid-size startups, we used to get a new flaky test almost on a weekly basis in a monorepo. We started the process of actually flagging them as ignored (we created a separate tag for flaky tests), but later realized that the backlog of fixing flaky test never came down.
In another case observed, devs just got used to rerunning the entire suite (the flakiness here was about 10-20%)
Disclaimer: I'm the co-founder of Aviator