Quality Assurance for Climate Codes

THELMA 2014
Edge Hill University are partners in the Quality Assurance for Climate Codes (QACC) project, which forms a significant phase of project work for the Data Analysis and Representation group. The predictions of climate change are made by computer simulations.  The models of the Earth’s climate are based on expert meteorological understanding and may be assumed to be sound.  However, the computer programs are large and complex, with long histories of development and many authors.  There is a risk that they contain programming errors. The existence of errors could cause underestimation or overestimation of the predicted changes, and could undermine the credibility of the studies which make the predictions.  This becomes an issue as government and military agencies use such models to make critical decisions to drive policy.

The Software Quality Assurance team at Edge Hill Computing Department, led by Dr Mark Anderson, is working in collaboration with John Collins and Brian Farrimond of SimCon Ltd (http://www.simcon.uk.com ) to identify issues in climate models and develop strategies for the software development processes of the models to overcome and avoid errors which are discovered.

The project team has access to the SimCon software tools which automate the detection of errors in large programs such as climate models. The tools can verify the complete absence of many classes of error.  The University also has a small cluster computer which may be used to run climate prediction programs with test data. We are using the tools and hardware to:

  • analyse the major climate prediction programs and to report on the quality of implementation;
  • run the programs on the high performance machine with test data, and to investigate the effect of correcting any errors discovered.

Thee project outcome will be directly relevant to any organisation developing large Fortran models particularly in the fields of science and engineering where Fortran continues to be the predominant language used.

Climate change is predicted to accelerate far beyond currently observable effects during the next few decades.  Very large sums of money are being spent to reduce carbon emissions and to convert a significant proportion of the UK national power generation to renewable energy sources and to nuclear power.

The predictions of climate change are made by computer simulations.  The models of the Earth’s climate are based on expert meteorological understanding and may be assumed to be sound.  However, the computer programs are large and complex, with long histories of development and many authors.  There is a risk that they contain significant programming errors. The existence of errors could cause underestimation or overestimation of the predicted changes, and could undermine the credibility of the studies which make the predictions.  Most of the programs are publicly available in the academic world and the errors will eventually be found.

The project is expected to have considerable impact in the climate modelling world. Preliminary investigations of Fortran code used in weather forecasting (WRF Model) have revealed a number of issues, either related to the code or the compilers used to build the code, and it is reasonable to assume that most, if not all, climate models would face the same issues.  It is possible that these issues may call into question the validity of conclusions drawn from the models which are the main source of predictions about global warming. It is also possible that we may able to demonstrate that the code is free from many classes of error or modify the code systematically so that this is indeed the case.

Publications

  • Collins, J., Farrimond, B., Anderson, M., Owens, D., Bayliss, D. and Gill, D. “Automated Quality Assurance Analysis: WRF – a case study”.  Accepted for publication in Proceedings of 5th International Conference on Computer Engineering and Technology (ICCET), Vancouver, Canada, 13-14 April 2013
  • Owens, D. and Anderson, M. “A Generic Framework for Automated Quality Assurance of Software Models: Supporting Languages of Multiple Paradigms”.  Accepted for publication in Proceedings of 5th International Conference on Computer Engineering and Technology (ICCET), Vancouver, Canada, 13-14 April 2013

Submitted Papers

  • Collins, J., Farrimond, B., Flower, D., Anderson, M. and Gill, D. “Removal of Numerical Drift in Parallel Debugging: WRF – a case study”.  Submitted to Automated Software Engineering, Springer-Verlag, June 2012
  • Collins, J., Farrimond, B., Flower, D., Anderson, M. and Gill, D. “The Removal of Numerical Drift from Scientific Models: A Case Study using WRF”.  Submitted to 22nd Australasian Software Engineering Conference (ASWEC 2013), Melbourne, June 2013

Presentations/Posters

  • Collins, J., Farrimond, B., Anderson, M. “Analysis of the WRF Program.” Presented at 13th Annual WRF Users Workshop, Boulder, CO. 25-29 June 2012.
  • Collins, J., Farrimond, B., Anderson, M., Owens, D., Bayliss, D. “Systematic Errors in Climate Models Consequent on the Design of the Fortran Language: Detection and Correction”.  To be presented at the 4th WGNE Workshop on systematic errors in weather and climate models, UK Met Office, Exeter, UK, 15-19 April 2013

 

Last updated on Last updated on Was this page helpful? Yes NoThanks for your feedback!Please tell us more:
Share