SoberAI: A Lighthouse-Style Audit Tool for AI Agents — Building It, Open-Sourcing It, Learning From It

Most websites today have zero infrastructure to handle the agents that visit them. AI crawlers fetch pages they can't parse, agentic browsers stumble on JavaScript heavy SPAs, and llms.txt adoption is patchy. Generative Engine Optimization (GEO) the discipline of making your site readable to LLMs and agents is becoming what SEO was in 2008: critical, poorly understood, and almost entirely without tooling.

I built SoberAI to fix that gap. Think Lighthouse, but instead of measuring Core Web Vitals, it measures AI readiness: structured-data coverage, llms.txt presence and quality, semantic HTML hygiene, robots.txt agent rules, content extractability, and JS-rendering dependency. It runs as a CLI, a hosted scan, and a GitHub Action.

This talk is the inception-to-growth story of building SoberAI in the open:

• The gap I noticed: Auditing GoDaddy-scale customer sites, I kept finding the same issues sites that looked perfect to humans were invisible to GPTBot, ClaudeBot, and PerplexityBot. No off-the-shelf tool measured this.

• Architecture choices: Why I picked a plugin-based audit pipeline over a monolithic scanner, how I avoided reinventing Lighthouse's Chrome-driver layer, and where I deliberately broke from Lighthouse conventions.

• Hard tradeoffs: Static parsing vs. headless rendering. Heuristics vs. LLM-as-judge. CLI-first vs. dashboard-first. The wrong call I made and then reversed.

• The honest scoreboard: I'll show audit results from multiple real sites and where SoberAI gets it wrong. Counter-evidence included.

• Going open-source: Why MIT instead of AGPL. Why GitHub Actions before SaaS. The contribution paths that worked (good first issues, audit-rule plugins) and the ones that didn't (RFC threads).

• A working mental model of GEO and what "AI readiness" actually means measurably

• A free tool you can run on your own site before leaving the venue

• The architectural decisions and tradeoffs in building Lighthouse-style developer tools

• Concrete contribution paths into SoberAI (audit rules, integrations, language support)

• Honest data on what AI readiness looks like across Indian web properties