Supporting Malhar

We're thrilled to announce a grant of INR 3 lakhs to Malhar, a fast fuzzy text search engine developed by Anubhav Nain.

Malhar started as a side-project to explore how fuzzy text search could be made better. Over the past year, it has evolved into a robust and minimal text search engine built from scratch in Nim and Python.

Why make a new text search engine?

While lots of excellent text-search engines and libraries already exist, they fall short in certain scenarios. The initial idea of Malhar was to improve performance for some of those shortcomings.

• Fuzzy Search: Handles real-world queries with typos and even phonetic misspellings!
• Speed: Malhar processes most queries on a 2.2M recipe dataset in under 5 milliseconds, with no caching.
• Lightweight: Designed to produce much smaller search indices—you can run it even on a Raspberry Pi or Android device.
• Cross-platform: Runs on any platform which has a C compiler.
• Minimal: Zero 3rd party code, fully written from scratch, readable, hackable, portable!

The author tested this on a ~2.2 Million recipes dataset. You can read some of the stats from this experiment here.

Roadmap

The project’s roadmap for the next year includes:

• Improvements to the existing codebase
• Experiments with disk-backed storage
• A tokenizer to test on languages other than English
• Experiments with different datasets like HackerNews to compare performance with Algolia
• Document processing pipeline / NLP

"Malhar started as an experiment to improve (text) search experience for a personal project in form of an improved fuzzy-search and predictable resources' (latency/RAM) requirements. Since text-search is a complex and highly subjective affair, engine aims to be very flexible as in by providing out-of-the-box search suggestions, in-word token matching, tunable tokenizer to fit domain-specific datasets!_

Current version has been written from scratch, depending upon only Nim's standard library. Python Extensions are also provided to quickly add a robust search functionality for almost any unstructured heap of data a user may have access to!"
-Anubhav, Creator, Malhar

This grant is made possible thanks to our Industry Partners who help us run the FOSS United Grants program. If you are an organisation that wants to give back to the FOSS Community, please reach out to us at foundation[at]fossunited[dot]org.

If you’re an Indian FOSS project looking for a grant, please drop us an email at grants[at]fossunited[dot]org. Find information about previous grantees at https://fossunited.org/grants.