Looks great and I thinks it’s a nice way to review vocab I learned some years ago now that I will be in Japan for vacation. Thanks!

However, it would be great if it was easier to select multiple vocabulary sets at once. Right now, I have to minimize many sets to get to let’s say set 20 and then select a maybe 20 to 25. That’s a lot of clicking here I would say. Maybe include a button that’s like select all sets of one level.

A second thing, I haven’t found is furigana or some pronunciation hint when doing the vocabulary test exercises. I know the meaning of the most of the words but it would be great to see also the pronunciation (maybe after clicking the right answer? Or as a tool tip). Or include a practicing mode for default word -> hiragana or something like that.

Otherwise, looks great I love the default font.

Sounds a bit like generative retrieval (e.g. this Google paper here: https://arxiv.org/abs/2202.06991)

Yeah, they share a similar intuition. I found that the difference is that PageIndex is more of a learning-free approach, more like how a human would do retrieval?

I love it

Maybe you could use: https://github.com/facebookresearch/nougat/tree/main or https://github.com/VikParuchuri/marker

Both are tools to convert pdfs into Latex or Markup with latex formulas. Maybe that helps

Reading the motivation for the second:

"Relying on autoregressive forward passes to generate text is slow and prone to hallucination/repetition. From the nougat paper: We observed [repetition] in 1.5% of pages in the test set, but the frequency increases for out-of-domain documents. In my anecdotal testing, repetitions happen on 5%+ of out-of-domain (non-arXiv) pages."

When these are fed in the next levels as inputs, isn't it even less surprising to get even more hallucinations/repetitions?

Here is the paper: https://arxiv.org/abs/2312.00752