What I learned fixing a bug and adding a feature to Celery.

This talk is about two problems the speaker hit while using Celery, and the two upstream contributions that came out of them — one a bug fix, one a feature. Celery is used all over the Python world, so both are problems other people can run into.

The bug — a memory leak: workers using ETA/countdown tasks slowly grew in memory until they were killed and restarted. The talk walks through how it was tracked down — what the heap showed, how ETA tasks get prefetched and held, and why the unacknowledged ones were never released — and the fix that went upstream.

The feature — a missing default: Celery sets worker concurrency from the host CPU count, which is wrong inside a container, because it ignores the cgroup CFS quota and starts too many workers. The talk covers the cpuset-vs-CFS-quota gap, why os.cpu_count() misleads in a container, and the cgroup-aware default proposed as an enhancement.

The real point isn't the two problems. Neither needed prior knowledge of Celery's internals — both needed the willingness to open the source and read it. This talk is about getting comfortable debugging into a project you didn't write, and turning a local fix into something the whole community gets.

Outline: the two problems and why they matter (3m) · the memory leak, end to end (10m) · the concurrency default, end to end (10m) · reading into source and deciding when to upstream (5m) · takeaways and Q&A (2m).

Two Celery issues you can check in your own projects: ETA-task memory growth, and worker over-provisioning in containers.

How to debug into the source of a library you depend on, instead of treating it as a black box.

What it takes to turn a local fix into an upstream contribution — a bug fix and a feature — in a project this widely used.