Understanding GStreamer application development

GStreamer is an open-source, cross-platform multimedia framework for building applications that process audio and video. Its modular design provides reusable components called elements that can be connected to build pipelines for a wide range of multimedia applications, including playback, recording, transcoding, media processing, streaming, and real-time communication.

In the first part of the workshop, we will introduce the core concepts needed to understand and work with GStreamer. We will explore how elements are connected, how data flows through a pipeline, and how capabilities, pads, bins, and plugins affect pipeline behavior. Using simple examples, we will understand pipelines for common multimedia tasks and examine how to inspect and debug them when they do not behave as expected.

Expected runtime: 30-40 mins

The second part of the workshop will then dive into how to write a GStreamer application using the GStreamer libraries and tools. While simple pipelines via gst-launch-1.0 command line tool can be useful for quick testing , we will cover some of the API GStreamer provides for building more complex pipelines, for example, handling a media file with multiple streams.

Familiarity with a programming language will be required. This part of the workshop will use Rust. While Rust expertise is not needed, being able to read and understand Rust code would be expected.

Expected runtime: 60-75 mins

By the end of the talk, attendees will be able to:

1. Explain the basic architecture of a GStreamer pipeline.
2. Identify the roles of elements, pads, caps, bins, and plugins.
3. Construct simple pipelines using gst-launch-1.0.
4. Understand how media data flows between elements.
5. Modify simple test pipelines from the command line for playback, conversion, recording, and streaming use cases.
6. Understand the basics of diagnosing linking, format-negotiation, and runtime errors.
7. Understand some of the most basic GStreamer APIs and how to use them
8. Build basic applications using a programming language of their choice