Understanding Query Execution: Lessons from Thanos Internals

Querying is what gives a proper meaning to the data we collect. Irrespective of whether it is a RDMS or time-series database, query execution is always the core part. This talk will walk through the internals of how thanos queries the promql query expression in its distributed promql-engine execution.

The talk will be about the how the whole process of query execution takes place by correlating it with Thanos distributed promql-engine.

Understanding what happens during logical, physical and execution plan. What optimization are made to make query faster by leveraging various Go language functionalities.

Being LFX mentee at Thanos starting this year, I've been working contributing to its promql query engine along with its various components. Learning many interesting things about building a scalable and distributed system, I felt a need to share them with FOSS community.

Flow of my talk:

• Introduction to normal database query execution.

• Giving a very little brief about Prometheus, Thanos and it's promQL engine.

• Deep dive into how query execution works in Thanos query engine.

• Understanding role of parser and querying AST(Abstract Syntax Tree)

• Preparing Logical plan and optimization being made in physical plan. Understanding execution plan.

• How Thanos leverages Go's functionalities to make queries faster.

• Bit walkthrough of open source codebase of Thanos promql engine

• Ending with how contributing to open source has benefited me.

Attendee will be able to get clear view of how query execution internals takes place. Through this session, one might get a little familiar with Thanos promql engine codebase. Attendee will be able to learn various use cases that Go language provides. Learning low-level stuff is always beneficial.