Neo4j first of all is very slow for vectors, so if performance is something that matters for your user experience they definitely aren't a viable option. This is probably why Neo4j themselves have released guides on how to build that middleman software I mentioned with Qdrant for viable performance.
Furthermore, the vectors is capped at 4k dimensions which although may be enough most of the time, is a problem for some of the users we've spoken to.
Also, they don't allow pre filtering which is a problem for a few people we've spoken to including Zep AI.
They are on the right track, but there are a lot of holes that we are hoping to fill :)
Edit: AND, it is super memory intensive. People have had problems using extremely small datasets and have had memory overflows.
Hey, want to correct some of your statements here. :-)
Neo4j's vector index uses Lucene's HNSW implementation. So, the performance of vector search is the same as that of Lucene. It's worth noting that performance suffers when configured without sufficient memory, like all HNSW vector indexes.
>> This is probably why Neo4j themselves have released guides on how to build that middleman software I mentioned with Qdrant for viable performance.
No, this is about supporting our customers. Combining graphs and vectors in a single database is the best solution for many users - integration brings convenience, consistency, and performance. But we also recognise that customers might already have invested in a dedicated vector database, need additional vector search features we don't support, or benefit from separating graph and vector resources. Generally, integrating well with the broader data ecosystem helps people succeed.
>> Furthermore, the vectors is capped at 4k dimensions
We occasionally get asked about support for 8k vectors. But so far, whenever I've followed up with users, there doesn't seem to be a case for them. At ~32kb per embedding, they're often not practical in production. Happy to hear about use cases I've missed.
>> Also, they don't allow pre filtering which is a problem for a few people we've spoken to including Zep AI.
We support pre- and post-filtering. We're currently implementing metadata filtering, which may be what you're referring to.
>> AND, it is super memory intensive.
It's no more memory-intensive than other similar implementations. I get that different approaches have different hardware requirements. But in all cases, a misconfigured system will perform poorly.
Neo4j performance is horrendous unless you have huge amounts of memory. I would wager that anyone who has used Neo4j for anything related to graphrag or used its vector features knows it’s not a great solution. Anyone can verify this quite easily.
Thanks for clearing up some of these points. Really pleased the post has reached the industry leaders, and I genuinely appreciate your response :)
With regards to the prefiltering, I was referring to filtering during the neighbor search in the HNSW. If you wanted 10 vectors, but with specific conditions, you'd have to retrieve surplus vectors and then perform the filter, hoping you were left with enough. Does that sound right? I suppose that is metadata filtering.
I should've been more specific about the memory issue. Not tying to slate you here, just that a lot of complaints I've read online were about issues with memory overflows using the vectors. But of course, a misconfigured system would definitely perform poorly :)
The fortunate thing about our vector DB, like I mentioned in the post, is that we store the HNSW on disk. So, it is much less intense on your memory. Similar thing to what turbo puffer has done.
With regard to the graph db, we mostly use our laptops to test it and haven't run into an issue with performance yet on any size dataset.
Partly because they're working with a monolith that I imagine is difficult to iterate on and it's written in Java. We've had the benefit of working on this in Rust which lets us get really nitty and gritty with different optimisations.
My friend who I worked on this with is putting together a technical blog on those graph optimisations so I'll link it here when he's done
This is definitely a problem we want to work on fixing quickly. We're currently planning an MCP tool that can traverse the graph and decide for itself at each step where to go to next. As opposed to having to generate actual text written queries.
I mentioned in another comment that you can provide a grammar with constrained decoding to force the LLM to generate tokens that comply with the grammar. This ensures that only valid syntactic constructs are produced.
General consensus is it's really slow, I like the concept of surreal though. Our first, and extremely bare bones, version of the graph db was 1-2 orders of magnitude faster than surreal (we haven't run benchmarks against surreal recently, but I'll put them here when we're done)
Congratulations on the launch! This is a very exciting space, and it's great to see your take on it.
Running fair benchmarks, not benchmarketing, is a significant effort and we recently put in this effort to make things as fair and transparent as possible across a range of databases.
We'd be very interested in seeing the benchmarks you'd run and how we compare :)
You can sacrifice many things for faster performance, such as security, consistency levels or referential integrity.
I'm genuinely curious to learn what design decisions you will make as you continue building the database. There are so many options, each with its pros and cons.
If you would like to have a chat where we can exchange ideas, happy to do that :)
That is just heresy though, am interested myself now and will run some proper benchmarks