FlyHash vs SimHash, one query at a time

Real rankings from the released agent-memory benchmark (6,000 stored memories, hashed with the project's own code). Pick a question and a bit budget k, then press Play to watch each method turn text into a code and rank memories with it.

Bit budget k

    Question asked of the agent's memory

    FlyHash

    2,000 units, each wired to a random tenth of the embedding. Keep only the k strongest.

      SimHash

      k random hyperplanes, one sign bit each. No pool, nothing is dropped.

      bit = 1bit = 0bits agree
        Average precision for this question at k =
        FlyHash
        SimHash
        1.00 means every true answer is ranked first. Answers sit at ranks for FlyHash and for SimHash.
        How typical is this?

        How the questions were chosen. The 12 questions are a fixed random sample of the 150 benchmark questions, not picked for flattering results. FlyHash does not win every one, and that is visible here. Twelve questions can't settle anything, so the paper's table (150 questions, 10 random seeds) is the evidence.

        What this does not show. It compares ranking quality only. At the same k, FlyHash's codes are larger and slower to compare than SimHash's, and an HNSW index beats both at this scale. See the paper's Limitations. The embeddings here are TF-IDF with SVD (256 dimensions), so the text matching is cruder than a modern sentence model.

        Source and data: github.com/mohanji/flyhash-agent-memory (scripts/make_demo_data.py regenerates the numbers on this page).

        Paper (arXiv:2611.05791) · Code & dataset · Zenodo archive