Research Corner with Damien Irving etc. Given their large user base, these libraries are usually written and supported by large companies( e. g. Continuum Analytics supports bokeh and holoviews / geoviews), large institutions( e. g. the Met Office supports iris, cartopy and geoviews) or the wider PyData community( e. g. pandas, xarray). Within each sub-discipline of weather and climate science, individuals and research groups take these libraries and apply them to their specific data analysis tasks. Increasingly, these individuals and groups are formally packaging and releasing their code for use within their community. For instance, Andrew Dawson( an atmospheric scientist at Oxford) does a lot of EOF analysis and manipulation of wind data, so he has released his eofs( Dawson, 2016a) and windspharm( Dawson, 2016b) libraries( which are able to handle data arrays from numpy, iris or xarray). Similarly, a group at the Atmospheric Radiation Measurement( ARM) Climate Research Facility have released their Python ARM Radar Toolkit( Py-ART; Helmus & Collis, 2016) for analysing weather radar data, and a similar story is true for metpy. It would be impossible to list all the sub-discipline-specific libraries in this article, but the Python in the Atmospheric and Ocean Sciences( PyAOS; http:// pyaos. johnny-lin. com /) community is an excellent resource if you’ re trying to find out what’ s available in your area of research.
26
BAMOS March 2017
Research Corner with Damien Irving etc. Given their large user base, these libraries are usually written and supported by large companies( e. g. Continuum Analytics supports bokeh and holoviews / geoviews), large institutions( e. g. the Met Office supports iris, cartopy and geoviews) or the wider PyData community( e. g. pandas, xarray). Within each sub-discipline of weather and climate science, individuals and research groups take these libraries and apply them to their specific data analysis tasks. Increasingly, these individuals and groups are formally packaging and releasing their code for use within their community. For instance, Andrew Dawson( an atmospheric scientist at Oxford) does a lot of EOF analysis and manipulation of wind data, so he has released his eofs( Dawson, 2016a) and windspharm( Dawson, 2016b) libraries( which are able to handle data arrays from numpy, iris or xarray). Similarly, a group at the Atmospheric Radiation Measurement( ARM) Climate Research Facility have released their Python ARM Radar Toolkit( Py-ART; Helmus & Collis, 2016) for analysing weather radar data, and a similar story is true for metpy. It would be impossible to list all the sub-discipline-specific libraries in this article, but the Python in the Atmospheric and Ocean Sciences( PyAOS; http:// pyaos. johnny-lin. com /) community is an excellent resource if you’ re trying to find out what’ s available in your area of research.
Installing the stack
While the default Python package installer( pip) is great at installing libraries that are written purely in Python, many scientific / number crunching libraries are written( at least partly) in faster languages like C, because speed is important when data arrays get really large. Since pip doesn’ t install dependencies like the core C or netCDF libraries, getting all your favourite scientific Python libraries working together used to be problematic( to say the least). To help people through this installation nightmare, Continuum Analytics have released a package manager called conda, which is able to handle non-Python dependencies. The documentation for almost all modern scientific Python packages will suggest that you use conda for installation.
Navigating the stack
All of the additional libraries discussed in this article essentially exist to hide the complexity of the core libraries( in software engineering this is known as abstraction). For instance, iris was built to hide some of the complexity of netCDF4, numpy and matplotlib, while geoviews was built to hide some of the complexity of iris, cartopy and bokeh. So if you want to start exploring your data, start at the top right of the stack and move your way down and left as required. If geoviews doesn’ t have quite the right functions for a particular plot that you want to create, drop down a level and use some iris and cartopy functions. If iris doesn’ t have any functions for a statistical procedure that you want to apply, go back down another level and use scipy. By starting at the top right and working your way back, you’ ll ensure that you never re-invent the wheel. Nothing would be more heartbreaking than spending hours writing your own function( using netCDF4) for extracting the metadata contained within a netCDF file, for instance, only to find that iris automatically keeps this information upon reading a file. In this way, a solid working knowledge of the scientific Python stack can save you a lot of time and effort.
References
Dawson, A.,( 2016a). eofs: A Library for EOF Analysis of Meteorological, Oceanographic, and Climate Data. Journal of Open Research Software. 4( 1), e14. DOI: http:// doi. org / 10.5334 / jors. 122
Dawson, A.,( 2016b). Windspharm: A High-Level Library for Global Wind Field Computations Using Spherical Harmonics. Journal of Open Research Software. 4( 1), e31. DOI: http:// doi. org / 10.5334 / jors. 129
Helmus, J. J. & Collis, S. M.,( 2016). The Python ARM Radar Toolkit( Py-ART), a Library for Working with Weather Radar Data in the Python Programming Language. Journal of Open Research Software. 4( 1), e25. DOI: http:// doi. org / 10.5334 / jors. 119