Research Corner with Damien Irving etc . Given their large user base , these libraries are usually written and supported by large companies ( e . g . Continuum Analytics supports bokeh and holoviews / geoviews ), large institutions ( e . g . the Met Office supports iris , cartopy and geoviews ) or the wider PyData community ( e . g . pandas , xarray ). Within each sub-discipline of weather and climate science , individuals and research groups take these libraries and apply them to their specific data analysis tasks . Increasingly , these individuals and groups are formally packaging and releasing their code for use within their community . For instance , Andrew Dawson ( an atmospheric scientist at Oxford ) does a lot of EOF analysis and manipulation of wind data , so he has released his eofs ( Dawson , 2016a ) and windspharm ( Dawson , 2016b ) libraries ( which are able to handle data arrays from numpy , iris or xarray ). Similarly , a group at the Atmospheric Radiation Measurement ( ARM ) Climate Research Facility have released their Python ARM Radar Toolkit ( Py-ART ; Helmus & Collis , 2016 ) for analysing weather radar data , and a similar story is true for metpy . It would be impossible to list all the sub-discipline-specific libraries in this article , but the Python in the Atmospheric and Ocean Sciences ( PyAOS ; http :// pyaos . johnny-lin . com /) community is an excellent resource if you ’ re trying to find out what ’ s available in your area of research .
26
BAMOS March 2017
Research Corner with Damien Irving etc . Given their large user base , these libraries are usually written and supported by large companies ( e . g . Continuum Analytics supports bokeh and holoviews / geoviews ), large institutions ( e . g . the Met Office supports iris , cartopy and geoviews ) or the wider PyData community ( e . g . pandas , xarray ). Within each sub-discipline of weather and climate science , individuals and research groups take these libraries and apply them to their specific data analysis tasks . Increasingly , these individuals and groups are formally packaging and releasing their code for use within their community . For instance , Andrew Dawson ( an atmospheric scientist at Oxford ) does a lot of EOF analysis and manipulation of wind data , so he has released his eofs ( Dawson , 2016a ) and windspharm ( Dawson , 2016b ) libraries ( which are able to handle data arrays from numpy , iris or xarray ). Similarly , a group at the Atmospheric Radiation Measurement ( ARM ) Climate Research Facility have released their Python ARM Radar Toolkit ( Py-ART ; Helmus & Collis , 2016 ) for analysing weather radar data , and a similar story is true for metpy . It would be impossible to list all the sub-discipline-specific libraries in this article , but the Python in the Atmospheric and Ocean Sciences ( PyAOS ; http :// pyaos . johnny-lin . com /) community is an excellent resource if you ’ re trying to find out what ’ s available in your area of research .
Installing the stack
While the default Python package installer ( pip ) is great at installing libraries that are written purely in Python , many scientific / number crunching libraries are written ( at least partly ) in faster languages like C , because speed is important when data arrays get really large . Since pip doesn ’ t install dependencies like the core C or netCDF libraries , getting all your favourite scientific Python libraries working together used to be problematic ( to say the least ). To help people through this installation nightmare , Continuum Analytics have released a package manager called conda , which is able to handle non-Python dependencies . The documentation for almost all modern scientific Python packages will suggest that you use conda for installation .
Navigating the stack
All of the additional libraries discussed in this article essentially exist to hide the complexity of the core libraries ( in software engineering this is known as abstraction ). For instance , iris was built to hide some of the complexity of netCDF4 , numpy and matplotlib , while geoviews was built to hide some of the complexity of iris , cartopy and bokeh . So if you want to start exploring your data , start at the top right of the stack and move your way down and left as required . If geoviews doesn ’ t have quite the right functions for a particular plot that you want to create , drop down a level and use some iris and cartopy functions . If iris doesn ’ t have any functions for a statistical procedure that you want to apply , go back down another level and use scipy . By starting at the top right and working your way back , you ’ ll ensure that you never re-invent the wheel . Nothing would be more heartbreaking than spending hours writing your own function ( using netCDF4 ) for extracting the metadata contained within a netCDF file , for instance , only to find that iris automatically keeps this information upon reading a file . In this way , a solid working knowledge of the scientific Python stack can save you a lot of time and effort .
References
Dawson , A ., ( 2016a ). eofs : A Library for EOF Analysis of Meteorological , Oceanographic , and Climate Data . Journal of Open Research Software . 4 ( 1 ), e14 . DOI : http :// doi . org / 10.5334 / jors . 122
Dawson , A ., ( 2016b ). Windspharm : A High-Level Library for Global Wind Field Computations Using Spherical Harmonics . Journal of Open Research Software . 4 ( 1 ), e31 . DOI : http :// doi . org / 10.5334 / jors . 129
Helmus , J . J . & Collis , S . M ., ( 2016 ). The Python ARM Radar Toolkit ( Py-ART ), a Library for Working with Weather Radar Data in the Python Programming Language . Journal of Open Research Software . 4 ( 1 ), e25 . DOI : http :// doi . org / 10.5334 / jors . 119