Worse is better

https://www.dreamsongs.com/RiseOfWorseIsBetter.html

1 Great documentation.

2 Makes you realize how complex async rust is. I can’t believe how many subtle bugs are reported in this report, and how many more are potentially related…

3 I think DST can help spot these cancellation bugs

Honestly, I’d rather just be true to myself.

If you play the game too much, you risk ending up on teams full of “senior software architects” with 20 years of experience in event-driven microservices with TDD + CQRS + AI.

Or these days they’re probably vibe coding and writing RFCs with emojis.

Being true to myself has never paid a single bill or supported my addiction to food and shelter. I’m 51 and play the game with the best of them including the banal “thought leadership” bullshit.

Some thoughts:

Linkedin just deprecated Kafka in favor of Northguard [1]. Curious how this actually compares in practice. Is it worth the effort of building yet another distributed log? Is it worth the effort to mantain Kafka if new alternatives are popping? Jepsen found plenty of issues last year [2], so maybe Linkedin is in the right direction by building from scratch...

Distributed systems are tough to get right, and while it's exciting to see new approaches, the space is definitely getting harder to follow.

[1] https://www.linkedin.com/blog/engineering/infrastructure/int...

[2] https://jepsen.io/analyses/bufstream-0.1.0

Besides the implementation effort in building from scratch, it also opens you up to unknown unknowns, while the limitations of mature software are better defined. Just because Jepsen didn't test other alternatives doesn't mean they are free from issues.

Regarding the Kafka issues pointed out:

* 2 only affect you if you are using transactions (and I would be interested in knowing of alternatives that better handle exactly-once semantics);

* Consumer.close() is likely off the critical path of your applications (personally I never encountered these hangs);

* Aborted reads and torn transactions were mentioned as being fixed by KIP-890, of which the mentioned ongoing work was completed since that article was published: https://issues.apache.org/jira/browse/KAFKA-14402

I'm writing a distributed DB rn ; and all the algs and tech they are using, SWIM ; hash ring ; the questions they are asking themselves - are pretty much standard ; it is not easy but it's a great field with different depth ; lot of successful companies don't dive that deep because it's uncharted experimentation failure & document process that is very very expensive to justify in a traditional context. I don't see how I would sell what i'm doing rn without getting it right first - and there is no point in proposing a new technology if there is no operational guarantees - And tbh a lot of people who are using kafka or doing distributed work are mostly doing the equivalent of setting up kafa consumers and publishers and that is enough and *it is fine*.

I'm a big fan of https://nats.io/ as a lightweight alternative to Kafka. I think of it like Nomad vs. Kubernetes: you can get 80% of the functionality for the cost of 25% of the complexity.

It is impossible to know what Northguard can and cannot do since it's an internal products.

Padding in the post refers to memory alignment.

If we had an "array of structs" instead of "struct of arrays" it would be: string(8) + long(8) + int(4) + padding(4) = 24 bytes

Even better, don't discuss them, ever.

I use Apple Notes and Reminders for work.

- Reminders: I just have a few simple lists: TODO, WIP, and BLOCKED (for stuff I'm waiting on others for).

- Notes: I keep daily, weekly, and monthly notes. At the end of each day, I clean up my daily note and move anything important to the weekly one, and then do the same into monthly.

>>This organization corresponds nicely to a idealized human organization of bosses and workers - bosses say what is to be done, workers do stuff. Bosses do quality control and check that things get done, if not they fire people re-organize and tell other people to do the stuff. If they fail (the bosses) they get sacked etc. <<note I said, idealized organization, usually if projects fail the bosses get promoted and given more workers for their next project>>

We miss you Joe :)

He was one of my favorite humans; the few emails I exchanged with him were funny and insightful.

It's amazing how coherent Zig's fundamental building blocks are as a programming language, everything fits together like a puzzle.

This post reminds me of one of Andrew's early talks about the type system and the comptime... With the core building blocks, we can achieve elegant solutions without adding unnecessary syntax complexity.

In my opinion Zig is elegant except for one thing: cannot attach data to error.

https://github.com/ziglang/zig/issues/2647

You have to workaround by things like passing into a pointer to error object.

this is because attaching a payload requires asking the question of who and how the memory of such a payload is managed, and Zig the language never prescribes you into a particular answer to that question.

Yes also the function syntax is weird and ugly because functions aren't expressions.

What do you mean? pun fn f() { } ? How is this weird

perfectly valid to do const f = fn (...) ... {...}

> For example let's look at Microsoft pricing. Using a single macOS runner for 1 year costs 0.08 * 60 * 24 * 365 = 42 thousand USD. Meanwhile, for Zig's CI testing we purchased a beefy mac mini for $3000 two years ago.

Hard to explain this to some folks