Research Corner with Damien Irving
36
BAMOS Dec 2017
Research Corner with Damien Irving
Best practices for scientific software
Code written by a research scientist typically lies somewhere on a continuum ranging from “ scientific code ” that was simply hacked together for individual use ( e . g . to produce a figure for a journal paper ) to “ scientific software ” that has been formally packaged and released for use by the wider community . I ’ ve written at length in previous issues about the best practices that apply to the scientific code end of the spectrum , so for this article I wanted to turn my attention to scientific software . In other words , what ’ s involved in turning scientific code into something that anyone can use ? My attempt at answering this question is based on my experiences as an Associate Editor with the Journal of Open Research Software . I ’ m focusing on Python since ( a ) most new scientific software in the weather / ocean / climate sciences is written in that language , and ( b ) it ’ s the language I ’ m most familiar with .
Hosting
First off , you ’ ll need to create a repository on a site like GitHub or Bitbucket to host your ( version controlled ) software . As well as providing the means to make your code available to the community , these sites have features that help with things like community discussion and software release management . One of the first things you ’ ll need to include in your repository is a software license . Jake VanderPlas has an excellent post on why you need a license and how to pick one .
Packaging / Installation
If you want people to use your software , you need to make it as easy as possible for them to install it . In Python , this means packaging the code in such a way that it can be made available via the Python Package Index ( PyPI ). If your code and all the libraries it depends on are written purely in Python , then this is all you need to do . People will simply be able to “ pip install ” your software from the command line .
If your software has non-Python dependencies ( e . g . netCDF libraries ), then it ’ s a good idea to make sure that it can also be installed via conda . Using recipes that developers ( i . e . you , in this case ) submit to conda-forge , this popular package manager installs software and all its dependencies at once .
Documentation
While it might seem like the documentation pages for your favourite Python libraries were painstakingly typed by hand , much of what appears on the web has been compiled by software that automatically takes the information from the docstrings in your code and formats it nicely . In most cases , people use Sphinx to generate the documentation and Read the Docs to publish it ( here ’ s a nice description of that whole process ).
Of course , while this saves a lot of time and effort in documenting the precise details of each function in your code library ( i . e . the API documentation ), you still need to write some narrative documentation that describes how to use your package ( with examples ). Expect to spend a significant fraction ( easily 25 %) of the time you spend writing code on writing documentation .
Assistance
In providing assistance to users , software projects will typically use a combination of encouraging people to submit issues on their GitHub / Bitbucket page ( for technical questions that will possibly require a change to the code ) and platforms like Google Groups and / or Gitter ( a chat client provided by GitLab ) for more general questions about how to use the software . The bonus of these platforms is that anyone can view the questions and answers , not just the lead developers of the software . This means that random people from the community can chime in with answers ( reducing your workload ) and it also helps reduce the incidence of getting the same question from many people .
Testing
If you want users ( and your future self ) to trust that your code actually works , you ’ ll need to develop a suite of tests using one of the many testing libraries available in Python . You can then use a platform like Travis CI to automatically run those tests each