I actually built my own immutable database which does support branching (see profile), so it seems like a huge miss that these ones don't. It's pretty much the main reason I would want an immutable database.
It appears that Datahike [0] is a Datomic workalike that supports branching. I haven’t tried it out myself (yet), but the documentation suggests it’s possible [1].
That said, I’m adding xitdb to the list of tech to try out. Thank you for building it!
The linked article points out that Datomic doesn't support branching from the past. It absolutely does support branching, and I've built entire test suites that way.
From a cursory glance, I'd say Datomic does exactly what the original parent article is discussing. It works great and it's super convenient.
If each "branch" is read only, it's not a branch at all. The entire idea of branching implies that you can make changes on one branch, then switch to another branch and make changes to it. They start from the same point and grow in different directions, as the metaphor of branches on a tree depicts.
I don't disagree with anything you've said here, I just don't see how it applies to the situation.
With `datomic.api/with`, you can apply new datoms and get back a new DB value. Repeat this process as many times as you want, in as many directions as you want, switching as you choose. You're building a tree of immutable DB values—seems clearly like branching to me.
If by "read only" you mean that they're not persisted to disk, then that's an important point, but it surely doesn't obviate the utility of the functionality. It's useful in a number of cases, and especially testing scenarios like the Xata article describes.
If you built an immutable database that persists the branches, that is very cool and sounds useful—kudos! That said, I also don't want to downplay the utility of what Datomic does; it's a major help to me.
Yes the article I linked mentioned d/with (speculative writes), and you are right that it is useful for testing -- but not much else, since it is purely in-memory. If you want to call that in-memory branching that's fine, I'll concede that.
My database supports persisted branching, but not just at the database level. You can "branch" (i.e., make a fast clone) data at any level, such as data for a specific user. Many production uses for this, not just testing, yet almost no database supports this. It uses the same HAMT algorithm that Clojure uses.
I really like it. API-first git repos without the limitations of a git service like github that are built primarily for humans. Looks like a competitor to code.storage by pierre.
Zig is a great choice. I spent the last three years working on my own git implementation in Zig (see my profile) and it's really the perfect language for this. It gives precise low level control and heavily emphasizes eliminating dependencies (like libc) which makes it perfect for web assembly.
What a banger of a release. The new `Io` interface was a huge breaking change for my project, but I made the transition. Zig seems to be pulling the same trick it pulled with allocators: just make it an explicit value that you pass around. Explicit allocators felt obviously right in retrospect, and so far this feels obviously right too.
The approach feels like a natural extension of Zig's philosophy about explicitness around low-level operations (no hidden control flow, no hidden memory allocations, etc.). Your function can be blocking? Communicate that through the function signature! Very in style for the language.
Clojure's immutable HAMTs are still a superpower nearly 20 years later. They've been copied in pretty much every language as a library (I did so myself in Zig) but what really makes it work well in Clojure is the fact that they're built into the language, so the entire ecosystem is built around them. Libraries that were made independently usually fit together like a glove because they are all just maps/vectors in -> maps/vectors out.
Yeah. I do wish there was something that was like Clojure with a TypeScript or Go-like nominal typing, but I do feel myself missing types a lot less with Clojure compared to other languages.
Type annotations mix poorly with s-expressions imo. Try an ML, which answers the same question of "How do we represent the lambda calculus as a programming language?"
There's already type annotations in Clojure and they look fine (though are a bit noisy). There are type algorithms that don't need annotations to provide strong static guarantees anyways, which is the important part (though I'm not sure you can do that with nominal types?). I think TypeScript and Go's syntaxes are a bad fit for s-expr but the idea probably isn't.
> Libraries that were made independently usually fit together like a glove because they are all just maps/vectors in -> maps/vectors out.
This is also the biggest weakness of Clojure (IMO). When everything is "just data", you spend a lot of time digging deep in libraries trying to figure out exactly what shape the data should take. Additionally, the shape of input data is almost never validated, so you spend lots of time debugging nasty type errors far from the place where bad data entered the program.
There have been some abortive attempts at solving this with things like spec, Prismatic Schema, etc, but nothing that has taken hold like TypeScript did with JS.
I'm still waiting for my dream language with the flexibility and immutability of Clojure, but without the pain points of an anything-goes attitude towards typing and data shape.
> When everything is "just data", you spend a lot of time digging deep in libraries trying to figure out exactly what shape the data should take
or that is a good spend of time where you familiarize yourself with said library (as they say, the documentation is the code!).
Usually the library is well written enough that you can browse through the source code and immediately see the pattern(s) or keys. The additional experimentation with the REPL means you can just play around and visually see the data in the repl.
A spec does similar (and it does make it easier to seek through the source to find it).
I've often got value from digging through libraries (in other languages), but I've almost always had the feeling that I'm not "doing it right", or that someone somewhere isn't "doing it right". Logically, the concept of encapsulation doesn't extend to meta-coding, but it feels like it should, by symmetry. It feels like I'm breaking encapsulation if I use the knowledge I gain from poking around when I code to the library.
I'm fumbling at the concept of a library surface not being self-describing but I suspect I lack some concepts; does this thought lead anywhere? Can anyone give me a clue?
I suppose if you examine the behaviour of a library, and code against that, then it is possible that the behaviour is unintentional and thus you end up being locked into a bug. This is most clear when the library is supposed to follow a standard (e.g., parsing some format), but is bugged and didn't do it right - and you code against that buggy behaviour.
However, that's an extreme case imho - you do that when you can't fix that library's bug or wrong behaviour.
But for things like key names and such, i dont think this applies - those key names are part of the library's api - and i often find that clojure libraries don't document them (or do but it's one of those auto-generated docs that dont mean anything).
I'm curious what you found inadequate with the existing solutions?
I remember when I started writing Clojure I'd use stuff like Records to encode the data-shape. Paired with Protocols they're quite powerful when the interface is more important than the shape. But in most situations the flexible data shapes are what make programs very easy to extend/evolve.
> When everything is "just data", you spend a lot of time ... trying to figure out ... shape the data ...
> you spend lots of time debugging nasty type errors far from the place where bad data entered the program.
I've been using Clojure for over a decade in various domains, different projects, diverse teams, etc., and quite honestly I don't even remember the last time it felt to me that way. Briefly, for a few weeks in the beginning perhaps it did. But at some point, maybe the REPL-driven workflow model internalized or something - I just don't ever feel like the way you describe. You're looking at the data as you build, so "far from where bad data entered" rarely happens - you watched it enter.
If anything, Clojure has spoiled me - I get annoyed having to dig through confusing type/data mismatch in some other languages, despite their sophisticated type systems. Uniform data structures mean your mental model transfers everywhere. You're not learning 50 bespoke APIs, you're learning map, filter, get, assoc. The real question is what failure modes you'd rather have. Clojure's tend to be runtime, but local and observable. Some type systems trade that for compile-time errors that are... also confusing, just differently. At the end of the day, sorry for a cliché, but it is a "skill issue". I can endlessly complain about my confusion with type systems, but a seasoned Haskell developer doesn't feel lost in types, just like I don't feel lost in Clojure without explicit type annotations.
You don't need team's approval of your choices for using a tool, a technique, an idea. A tool remains useful even if nobody else but only you understand its practicality. There are days when I deal with projects where some don't even know about Clojure's existence, but I still may use babashka to understand the data flow. Having a Clojure REPL around is immensely handy, slashing through data that flows down the APIs is the best - JSON is so lame and annoying to deal with - even LLMs often can't get the jq filter syntax right, or when they do, it looks terrifyingly cryptic and confusing. Or sometimes I'd fire up an nbb REPL with Playwright - just to reproduce some issue without having to manually click through things.
Yeah, they are so underappreciated - the practical differences in designing, delivering and maintaining software are real. Initially you see small differences "What's the point? I can write that in Python too... maybe it's not as delightful, but who cares?...", etc. Yet over time small annoyances accumulate and become an endless stream of headaches. I see it over and over again - I work on a team where we have codebases in different languages, and some services written in Clojure. Immutability by default is a game of a different league.
Didnt they move to CHAMP? Otherwise, that seems like a waste of resources. They are literally the same but CHAMPs are a little faster in just about every way.
I am still a bit disappointed that they didn't change to RRB trees or copy the scala vectors instead of the built in ones. Iirc the scala vectors are faster in general while providing a bunch of benefits (at the cost of code complexity though, but even a better RRB list implementation instead of the scala finger trees would allow for that).
I wrote an RRB tree implementation in c# just for fun [0], and while they are harder than the tries of clojure, the complexity is pretty well contained to some functions.
I think astronomers could measure the age of the universe in nano-Valhallas. Every year, it feels 50% closer to completion...
In all seriousness I'm happy with what Mr. Goetz and the team have done. Sealed interfaces (java 17) + exhaustive switch statements (java 21) means we now have union types in java! And instead of jumping on the async/await bandwagon we now have a more general solution that doesn't lead to API duplication (virtual threads). But Valhalla has been a veeery long time coming.
'Tis true. At the same time, Project Valhalla will be the most significant change to the JVM in a very long time, and probably its best chance to stay relevant in the future.
I actually built my own immutable database which does support branching (see profile), so it seems like a huge miss that these ones don't. It's pretty much the main reason I would want an immutable database.
reply