Greedy Hearth is a personal memory assistant built for Telegram. Xandrov Labs designed it to explore recall and retrieval as a product problem—not as a feature in a broader note-taking system. The core behavior is minimal: send voice notes, images, screenshots, files, or text to a Telegram bot; later, ask for any of it back. The architecture stays focused on two functionalities and nothing else.
Most second-brain products die of scope creep long before their retrieval systems are any good. They pile on note-taking, tagging, collaboration, and organization before solving the hard part: finding the right piece of information weeks later in a messy archive.
Recall over a multimodal archive is fundamentally a retrieval problem, not a prompt engineering problem. An LLM with access to all your data can synthesize answers, but that doesn't mean it can find the one screenshot, voice memo, or document you need on command.
Second-brain products typically fail by expanding features—adding note-taking, tagging, linking—before their retrieval is reliable. The outcome is a messy archive that even transformers can't search effectively. The actual hard part is narrow: fetching the right object from mixed media when a user asks for it, fast and accurate.
Built on Python 3.12 and FastAPI. The ingestion pipeline normalizes incoming data—voice, images, files, and text—into a ParadeDB store: Postgres with BM25 full-text search, pgvector embeddings, and trigram indices. Gemini handles multimodal understanding and entity extraction during ingestion. Cloudflare R2 stores raw objects, separating retrieval logic from file hosting. Inngest orchestrates async ingestion jobs, processing uploads without blocking Telegram responses.
Recall runs a multi-stage pipeline. Query understanding extracts intent and entities. Three retrievers execute in parallel—BM25 for keyword precision, pgvector for semantic similarity, trigram for typo resilience—then reciprocal-rank fusion combines their rankings. Re-ranking selects the strongest candidates, and Gemini synthesizes results into a natural response. Each retriever fails on different queries; fusion captures cases where single methods fall short.
Voice uploads are transcribed; images are understood via Gemini's vision API; documents are processed into embeddings. All objects—transcripts, images, summaries, embeddings—are stored in ParadeDB with multiple indices for redundant search. The async pipeline via Inngest ensures reliability: if ingestion of a large file stalls, it retries without blocking the Telegram bot. Each upload spawns parallel indexing jobs across all three retrievers.
Most second brains sprawl until their retrieval fails; this one locks scope until search works.
Ingestion and recall are operational. The system is in build against a stated benchmark with clear acceptance thresholds: recall@10 ≥ 90%, precision@5 ≥ 80%, text recall p95 under 3 seconds, 100 mixed messages under $1 inference cost. The team is tracking toward these targets. Status is transparent: this is a production experiment with defined goalposts, not a finished product.
No client names, user data, or benchmark queries are disclosed. The specific test set and raw inference costs are kept internal to protect competitive positioning in multimodal retrieval. The engineering decisions and stack are public; the calibration data and performance benchmarks are not, pending release.