Vectorizing JavaScript: Universal Functions, Automated Packages

What if you could use powerful mathematical functions on any kind of data - from simple numbers to complex multi-dimensional arrays - with just a single line of code? This is the magic of universal functions, a core feature of numerical libraries like NumPy and, in the JavaScript world, stdlib.

Here is an example of one such Universal function.

But this magic comes at a cost. Behind every function like abs or sin lies not one, but multiple specialized low-level kernels to handle different data types and precision levels. Managing over 300 of these kernels manually is a Herculean, error-prone task that stifles innovation and scalability.

For example, multiple kernels for the abs can be found here.

In this talk, we’ll discover how we tackled this challenge head-on by using an automated package generation approach for stdlib. We'll journey from the foundational scalar kernels all the way up to the high-level functions you use in your code, and reveal how we:

• Tamed the complexity by creating a central metadata database to act as the single source of truth for all our mathematical functions.

• Built a scaffolding system that auto-generates complete, production-ready npm packages; complete with tests, benchmarks, and docs - from a set of smart templates.

• Engineered a self-updating system that uses cron jobs to dynamically track changes, ensuring our ecosystem is always in sync.

The result? A faster, more reliable, and truly decomposable library where developers can consume exactly what they need, and contributors can add new functions without the manual grunt work. Join us to see how automation and smart architecture are paving the way for the future of scientific computing in JavaScript. You'll leave with a clear blueprint for managing complexity in large-scale open-source projects.

• Universal Functions Aren't Simple: A single function like abs requires many specialized kernels (real, complex, different precisions) to work efficiently.

• Automate Package Creation: We built a system that auto-generates complete npm packages (code, tests, docs) from templates and a metadata database.

• JSON Databases are the Engine: A central store of function metadata and policies powers everything, from code generation to type casting rules.

• Self-Healing Systems: Cron jobs automatically detect changes in core kernels and update the entire package ecosystem, ensuring it's always in sync.