The Hardware Behind AI: Why GPUs Are Taking Over

GPUs seem to be everywhere today. Every new AI model needs them, companies are spending billions of dollars building them, and governments are even regulating who can buy them. Yet if you ask many software developers what makes a GPU special, the answer is often something like: "It's basically a faster CPU." but its completely wrong.

In this talk, we'll build an intuition for how CPUs and GPUs are designed and why they excel at very different kinds of work. We'll start with the CPU and look at some of the tricks that make it so good at running everyday software: instruction pipelining, caching, branch prediction, and other techniques that help finish a single task as quickly as possible.

Then we'll look at the GPU, which solves a very different problem. Instead of making one task run faster, a GPU is built to keep thousands of simpler cores busy at the same time. We'll see why this design makes GPUs incredibly powerful for some workloads and surprisingly bad for others. In fact, many programs run better on a CPU, and nobody would want to run an operating system on a GPU.

Once we understand that difference, the reason AI loves GPUs becomes much clearer. Most machine learning workloads are dominated by huge numbers of similar calculations that can be performed in parallel, making them a natural fit for GPU architecture.

Because this is IndiaFOSS, we'll finish by looking beyond the hardware itself and exploring the open-source software ecosystem around modern accelerators. We'll discuss projects such as ROCm, OpenCL, SYCL, Triton, and llama.cpp, the challenges of building an open GPU software stack in a CUDA-dominated world, and the many opportunities for developers to contribute.

1) Understand why a GPU is fundamentally different from a CPU, and why "a GPU is just a faster CPU" is a misconception.

2) Build an intuition for why modern AI workloads map so effectively to GPU architectures.

3) Discover the open-source GPU ecosystem, including ROCm, OpenCL, SYCL, Triton, and llama.cpp and learn where developers can contribute.