Developing Software for a Scientific Community: Some Challenges and Solutions

Introduction

Many scientific software projects intended for a broad scientific community succeed in that they make a significant contribution to the science. Many, however, fail. Some of these fail for scientific reasons (the underlying science was imperfectly understood), or because of coding problems (for example, an inappropriate choice of implementation language). Another less obvious cause of failure is the differences in the behaviour, knowledge, values, assumptions and goals between three different groups of people involved in such projects. These three groups are scientists; scientific end-user developers, that is to say, scientists who are developing software for their own use or for that of their close colleagues; and professional software developers, to whom the science is just another user domain.

In writing this chapter, we draw heavily on the field studies conducted by the first author, an academic, in a variety of scientific settings, and on the many years’ experience developing scientific software of the second author, a professional software developer.

Our aims in writing this chapter are:-

• •

  To articulate some specific challenges facing scientific software developers. These challenges have their origins either in the culture of scientific end-user development or in the nature of science itself.

• •

  To suggest ways in which these challenges might be addressed.

In what follows, we shall firstly articulate the behaviour, knowledge, values, assumptions and goals that characterize much scientific end-user development and then discuss the challenges which these characteristics pose when the context of the development is broadened. We then go on to discuss which development approaches, methods/techniques and tools might be useful in scientific software development, and, equally importantly, identify some which will not. Finally, we discuss how this identification of effective ways of supporting scientific software development can be progressed.

Throughout this paper, we stress the importance of context. A couple of examples give a flavour of this importance:

• •

  A particular tool which is useful in a commercial development context might not be so useful in a scientific;

• •

  Assumptions which are perfectly justified in a setting where a scientist is developing software for himself/herself to explore a particular scientific question might not be justified in other development settings.

This emphasis on the importance of context means that it is difficult to set any hard-and-fast rules along the lines of ‘scientific software developers should apply this testing technique to their software’. We hope rather that this chapter might provide the means by which you might recognise the challenges in your particular development context, and suggest some ways by which you might address such challenges.