Notes from a Maintainer - Lakshmipathi Ganapathi, Webminal

How it started

All of this started more than 20 years ago, in college, burning Fedora Core 2 onto six CDs and swapping discs into the drive at midnight to dual boot with Windows. That is how I fell into the GNU/Linux addiction and somewhere along the way I picked up the tag of "the Linux guy" among my friends. In my MCA final year I got campus recruited as the top pick out of 180 students, purely on my Linux skills.

Then came a long, frustrating gap before my joining date. Going to a city like Chennai or Bangalore to look for other jobs was not an option for me because of money, so I decided to build a FOSS project instead, hoping it might help. All I had was a desktop with a Celeron processor and 512MB of RAM. Six months later, I had giis-ext3, a file undelete tool. I learned much later that I could have done it in a couple of weeks using existing libraries, but as a student with very limited internet access, that was not how it went. The nearest cafe was around 20km away, I paid 20 rupees an hour and I carried whatever I downloaded back on floppy disks. So I am perfectly okay with those six months. I uploaded the tool to SourceForge and started sending it along with my resume.

Once I joined work, the job itself did not feel challenging enough, but the open source world had already pulled me in. Getting an email or a comment from someone on the other side of the planet only pumped up the addiction. At the peak of Facebook's popularity, I was on a site called LinuxForums.org, LFo for short and now gone, almost every day answering user questions. This was before Stack Overflow even existed.

The award that changed things

In 2008, my ext3 undelete tool won the FOSSIndia award and got featured in Linux For You magazine. You can see that bit of my story here. giis was written from scratch, without ext2lib, because at the time I did not even know ext2lib existed. That award was one of the turning points of my journey. It told me the work mattered to people other than me.

The friends I met on a forum

LFo is also where I met two people who shaped everything that came after.

One was Elija, whose real name is Richard, from England. He contributed to my file system projects like giis and giis-ext4. The other was Freston, whose real name is Wybrand Wlohman, from the Netherlands. He helped me with the networking part of minili, my 3MB bootable Linux. Both of them gave me their time and effort for free, on projects that had no money and no promise of anything in return.

The idea behind Webminal

Most of my work was on Windows and only a few projects let me touch Linux. Friends kept coming to me with the same question, "how do I get started with Linux commands, our servers are all Linux now."

That question, along with my own frustration of not being able to use Linux whenever I wanted, slowly turned into an idea.

The idea was simple. I was sitting at my Windows machine at work, wanting to learn Linux. What if I could open a browser, practice on a real Linux terminal, gain some confidence and then rotate my chair to the actual Linux machine next to me and start working with zero fear?

That was the whole point. Zero fear of the command line. So the interface had to feel exactly like a real terminal. No "Run" button. No "Execute now" button. Just type the command and get the response, like a real system.

Building it with people I had never met

I shared the web terminal idea with Freston and Elija. Freston was in the Netherlands, Elija (Richard) was in England and I was in India. That is how three people from three countries started building a project for other users. Richard built the frontend and we shipped the first version, webminal-v1. After that Richard had to step away because of the workload from his regular job.

Wybrand (Freston) and I kept going. The way we worked still makes me smile. Both of us would SSH into the same server, open a shared screen session and just chat inside the terminal while we coded. We even worked through New Year's Eve. I still remember Freston and me SSHing into the server a few hours before midnight, both of us at our keyboards while the rest of the world was out celebrating.

Here is something I still find funny. Until 2015, Freston and I had never seen each other's face. Not even on Skype. Everything happened over SSH and a screen session.

Buying a server with a stranger

Until then we were running everything from Wybrand's home system. In 2011 we decided to buy a proper server for the project and split the cost.

Now think about this. I was about to send money to someone I had never seen face to face, never even heard his voice. Friends could not believe I trusted someone online like that. My answer was simple, I had known him for more than four years on the forum. Still, sending around INR 25,000 to a person in the Netherlands felt a little crazy. I did it anyway. And we got our server.

Then the users asked for root

Webminal users started asking for root access. Obviously we could not just hand it out, since it was a shared server with other users' home directories sitting on it.

So we kept exploring. OpenVZ, then UML, later Docker when it arrived, then EC2 spot instances. Over the years we added and removed a lot of things. Through all of it, User Mode Linux stayed. We still use UML today to give each user a real kernel with real block devices, so a student can run fdisk, LVM, RAID and mkfs without any risk to the host machine.

One server that refuses to die

Webminal runs on a single CentOS box with 8GB RAM. No Kubernetes, no microservices, no auto scaling. One server since 2011. It has survived a datacenter fire in 2021 where we lost 150k user accounts, multiple power outages in the Netherlands and the day in 2017 when a Spanish tech blog sent us 10,000 users in a single day.

Along the way the infrastructure moved through a long road. Linode, DigitalOcean, AWS, GCP, OVH, IBM Cloud and back to Linode. Full circle.

How one small server keeps all this running is a story in itself. Python 2.7, Flask, Shellinabox from 2005, User Mode Linux from 2001 for the root labs and a little eBPF counting the commands as they scroll on the homepage. Every conference talk would tell you this stack is wrong, yet it has served 500,000 users for 15 years. I wrote up that technical side separately, if you want the deep dive: 15 years, one server, 8GB RAM and 500k users.

I pay the server bill from my savings. Honestly, I have spent more on this project than on a lot of personal and family things.Around 500,000 people have typed their first ls on Webminal. Some of them are sysadmins now. Some run their own servers. As long as it helps even one student, it will keep running.

The other projects along the way

Webminal is the one that stayed alive the longest, but it grew out of a habit of building small tools and I never stopped working on storage and file systems. Over the years I also contributed to Linux file system tools and BTRFS and to kernel file system features like case-insensitive testing. A few interesting projects :

• giis and giis-ext4, file undelete tools for ext3 and ext4.
• extcarve, a semantic file carving tool for forensic recovery.
• dduper, a fast way to dedupe BTRFS files, along with patches to btrfs-progs and xfstests.
• minili, the 3.8MB Linux build that Freston helped me network.
• DistroQA, automated ISO image testing using containers.

You can find the full list at giis.co.in/foss.html and on github.com/lakshmipathi.

Advice to my younger self

If I could tell my younger self one thing, it would be this. It is much easier to contribute to an existing FOSS project than to start one from scratch as your first project. You learn a lot and you gain visibility quickly, by joining something that already has people and momentum. Starting from zero, the way I did with giis, works too, but it is the harder road.

---

This blog post is part of the "Notes from a Maintainer" series, an effort to highlight the stories of the people behind India's digital commons ecosystem. Read other posts here