Python writes it, Rust runs it: The architecture behind zero-recomputation data pipelines

Here's a (common) workflow problem I keep running into: after every standup call, dev-rel collab meeting, or community planning session, there's always a pile of actionable items and TODOs I need to track. Sometimes I jot them down in a hackmd.io document, sometimes a Google doc, sometimes just in the chat itself. There's no consistency, and when I need to reference what was decided a few meetings ago or track what's actually been completed, it's a mess. Being involved in SciPy India community organizing myself, this hits too close to home - volunteer assignments scattered across docs, speaker commitments buried in email threads, sponsorship follow-ups lost in Slack. Having a knowledge graph that updates incrementally with each meeting would actually be incredibly useful.

This talk walks through building a solution for exactly this using CocoIndex, which handles the complexity of keeping derived data continuously in sync with changing sources - change detection, incremental updates, ghost data cleanup, schema evolution. If you've never worked with knowledge graphs, you'll see how they transform scattered text into something queryable - "which volunteers haven't confirmed their shifts?" or "what speaker logistics are still pending?". The live demo extracts structured data (people, tasks, relationships) from meeting notes using LLMs and loads it into Neo4j, and you'll see incremental updates working in real time. For those building pipelines, this is about the materialized view pattern - treating your graph as a derived store that stays in sync with your sources. After the demo, we'll dig into the architecture underneath: how a Rust engine handles all of this while you write pure Python.

Under the hood, this is enabled by PyO3, which lets Rust expose native bindings as idiomatic Python objects - the same approach used by Polars, Pydantic v2, and DataFusion. The Rust layer manages concurrent processing, state tracking, and batching; you just define transformations in Python. For those curious about systems architecture, we'll dig into why this Rust-Python interop pattern is becoming the default for performance-critical Python tools.