Varunayan and VarunayanR: Scalable analysis of climate data across data stores, languages and geographies

Getting climate data in this climate-change-aware world should be easy. Datasets like ECMWF's ERA5 provide granular climate data for the entire planet over the past 100 years or more. Our own Indian Meteorological Department provides gridded rainfall and temperature data for 120+ years. Yet, the journey from accessing this data to drawing insights is not straightforward.

Here are some typical questions that one would be involved with:

1. I want temperature and humidity for one district, to correlate its impact on health outcomes.
2. I want a daily or monthly mean over a custom region of interest, say, rainfall over IITB campus and how it has changed over the years.
3. I want to understand how similar or different the estimates from ERA5 and IMD are.

In this session, we'll demo the duo of varunayan and varunayanR,  two open-source packages written in Python and R, respectively.

Both varunayan and varunayanR are ‘paths’ (ayan) for obtaining climate data (‘varun’) that start with raw global grids and produce analysis-ready tables in a single command. At the core, Varunayan implements utility functions to work efficiently with netCDF files from ERA5 and gridded data files from IMD. They allow the user to ‘attach’ geometry to the data, enabling seamless visualisation at the district, state, and national levels. The Python and R versions share the same API, so users can split work between the two languages without learning two tools. Here is an example of using the varunayan API for visualising climate change in India over the past 80 years.

What makes it unique?

1. One tool for the whole pipeline: query, download, clip to region, aggregate, and tidy table. Other libraries each do one step and leave you to put them together.
2. Correct area-weighted aggregation over irregular boundaries, avoids bias from partial grid cells.
3. Heat-stress indices built in: heat index, wet-bulb, WBGT, humidex, UTCI, with risk categories.
4. Access four datasets through one interface: ERA5, IMD gridded data for India, HadEX3 climate extremes, and CRU TS observations. No other package combines all four.
5. Same workflow in Python and R, so mixed teams learn it once.
6. Few general climate packages read IMD data at all.