RealClimate logo


Technical Note: Sorry for any recent performance issues. We are working on it.

Labels for climate data

Filed under: — rasmus @ 24 April 2014

metadata-fig “These results are quite strange”, my colleague told me. He analysed some of the recent climate model results from an experiment known by the cryptic name ‘CMIP5‘. It turned out that the results were ok, but we had made an error when reading and processing the model output. The particular climate model that initially gave the strange results had used a different calendar set-up to the previous models we had examined.

In fact, the models used to compute the results in CMIP5 use several different calendars: Gregorian, idealistic 360-day, or assuming no leap years. These differences do not really affect the model results, however, they are important to take into account in further analysis.

Just to make things more complicated, model results and data often come with different time units (such as counting hours since 0001-01-01 00:00:0.0) and physical units (precipitation m/day or kg m/s; temperature: Kelvin, Fahrenheit, or Centigrade). Different countries use different decimal delimiters: point or comma. And missing values are sometimes represented as a blank space, some unrealistic number (-999), or ‘NA’ (not available) if the data is provided as ASCII files. No recorded rainfall is often represented by either 0 or the ASCII character ‘.’.

For station data, the numbers are often ordered differently, either as rows or columns, and with a few lines in the beginning (the header) with various amount of description. There are almost as many ways to store data as there are groups providing data. Great!

Murphy’s law combined with typically different formats imply that reading data and testing takes time. Different scripts must be written for each data portal. The time it takes to read data can in principle be reduced to seconds, given appropriate means to do so (and the risk of making mistakes eliminated). Some data portals provide codes such as Fortran programs, but using Fortran for data analysis is no longer very efficient.

We are not done with the formats. There are more aspects to data, analyses, and model results. A proper and unambiguous description of the data is always needed so that people know exactly what they are looking at. I think this will become more important with new efforts devoted to the World Meteorological Organisation’s (WMO) global framework on climate services (GFCS).

Data description is known as ‘meta-data‘, telling what a variable represents, what units, the location, time, the method used to record or compute, and its quality.

It is important to distinguish measurements from model results. The quality of data is given by error bars, whereas the reliability of model result can be described by various skill scores, depending on their nature.

There is a large range of possibilities for describing methods and skill scores, and my guess is that there is no less diversity than we see in data formats used in different portals. This diversity is also found in empirical-statistical downscaling.

A new challenge is that the volume of climate model results has grown almost explosively. How do make sense out of all these results and all the data? If the results come with proper meta-data, it may be possible to apply further statistical analysis to sort, categorise, identify links (regression), or apply geo-statistics.

Meta-data with a controlled vocabulary can help keep track of results and avoid ambiguities. It is also easier to design common analytical and visualisation methods for data which have a standard format. There are already some tools for visualisation and analysis such as Ferret and GrADS, however, mainly for gridded data.

Standardised meta-data also allows easy comparisons between same type of results from different research communities, or different types of results, e.g. by the means of experimental design (Thorarinsdottir et. al. 2014). Such statistical analysis may make it possible to say whether certain choices lead to different results, if they are tagged with the different schemes employed in the models. This type of analysis makes use of certain key words, based on a set of commonly agreed terms.

Similar terms, however, may mean different things to different communities, such as ‘model’, ‘prediction’, ‘non-stationarity’, ‘validation’, and ‘skill’. I have seen how misinterpretation of such concepts has lead to confusion, particularly among people who don’t think that climate change is a problem.

There have been recent efforts to establish controlled vocabularies, e.g. through EUPORIAS and a project called downscaling metadata, and a new breed of concepts has entered climate research, such as COG and CIM.

There are further coordinated initiatives addressing standards for meta-data, data formats, and controlled vocabularies. Perhaps most notable are the Earth System Grid Federation (ESGF), the coupled model inter-comparison project (CMIP), and the coordinated regional downscaling experiment (CORDEX). The data format used by climate models, netCDFCF‘, is a good start, at least for model results on longitude-latitude grids. However, these initiatives don’t yet offer explanations of validation methods, skill scores, modelling details.

Validation, definitions and meta-data have been discussed in a research project called ‘SPECS‘ (that explores the possibility for seasonal-to-decadal prediction) because it is important to understand the implications and limitations of its forecasts. There is also another project called VALUE that addresses the question of validation strategies and skill scores for downscaling methods.

Many climate models have undergone thorough evaluation, but this is not apparent unless one reads chapter 9 on model evaluation in the latest IPCC report (AR5). Even in this report, a systematic summary of the different evaluation schemes and skill scores is sometimes lacking, with an exception of a summary of spatial correlation between model results and analyses.

The information about model skill would be more readily accessible if the results were tagged with the type of tests used to verify the results, and the test results (skill scores). An extra bonus is that a common practice of including a quality stamp describing validation may enhance the visibility of the evaluation aspect. To make such labelling effective, they should use well-defined terms and glossaries.

There is more than gridded results from a regional climate model. What about quantities such as return values, probabilities, storm tracks, number of freezing events, intense rainfall events, start of a rainy season, wet-day frequency, extremes, or droughts? The larger society needs information in a range of different formats, provided by climate services. Statistical analysis and empirical-statistical downscaling provide information in untraditional ways, as well as improved quantification of uncertainty (Katz et al., 2013).

Another important piece of information is the process history to make the results traceable and in principle replicable. The history is important for both the science community and for use in climate services.

unlabelmedicne
One analogy to proper meta-data is to provide a label on climate information in a similar way to labels on medicine.

In summary,there has been much progress on climate data formats and standards, but I think we can go even further and become even more efficient by extending this work.

Update: Also see related Climate Informatics: Human Experts and the End-to-End System


References

  1. T. Thorarinsdottir, J. Sillmann, and R. Benestad, "Studying Statistical Methodology in Climate Research", Eos, Transactions American Geophysical Union, vol. 95, pp. 129-129, 2014. http://dx.doi.org/10.1002/2014EO150008
  2. R.W. Katz, P.F. Craigmile, P. Guttorp, M. Haran, B. Sansó, and M.L. Stein, "Uncertainty analysis in climate change assessments", Nature Climate change, vol. 3, pp. 769-771, 2013. http://dx.doi.org/10.1038/nclimate1980

12 Responses to “Labels for climate data”

  1. 1
    SecularAnimist says:

    Thank you. This is the kind of informative “look inside” climate science article that RealClimate does very well, perhaps uniquely well.

  2. 2
    Dikran Marsupial says:

    That is the good thing about standards, there are always so many to choose from.

  3. 3
    Mankoff says:

    One class of tools I’ve found helpful to deal with this are the units packages. These are available in all the popular data analysis languages (MATLAB, R, Python), and many even support uncertainty. For example, a precip variable can be created with the code

    precip = nc.load["precip"] * uu.mm/uu.day

    And then later easily convert with precip.to(uu.m/uu.year)

    Alternatively

    book_length = (20. * ureg.centimeter).plus_minus(2.)

    I use the Pint library for Python http://pint.readthedocs.org/en/latest/

  4. 4
    sidd says:

    ” … using Fortran for data analysis is no longer very efficient.”

    fortran isn’t quite dead yet. I am biased by my training, but for some recent analysis i was doing, fortran still beat R,Matlab,Octave, Mathematica and the like by orders of magnitude in running time. And if i am running on a beowulf, fortran is my weapon of choice every time for numerics (distributed text processing is another matter …). The most important thing is that the numeric libraries for fortran are very old, very well debugged, and their failure modes are well known. It is now possible to import most of these libraries into environments such as R, so we sorta can have the best of both worlds in some situations.

    but, as i said, i am biased.

    and the flip side is that i can develop and debug a program in the latter group of environments an order of magnitude faster, which is more important in a great many cases.

    sidd

  5. 5
    Sanele Ncube 14295068 says:

    So technically there are different climate calenders?

  6. 6
    Ed Davies says:

    “… temperature: Kelvin, Fahrenheit, or Centigrade”

    What, not Celsius?

    Of coures, Centigrade was very like Celsius but it had a slightly different definition (in terms of the freezing and boiling points of water rather than, via Kelivn, absolute zero and the triple point of water) but mostly it’s just an old name for the scale which isn’t much used these days except in countries which are particularly cantankerous in their use of units (the US and, to a lesser extent, the UK and a few other English speaking countries) – a large part of the problem in the first place.

  7. 7
    Chris Dudley says:

    One of the strengths of IDL has been its adaptability in IO along with its back compatibility with FORTRAN formatting statements. The licensing fee is a drawback though.

  8. 8
    John Mashey says:

    Fortran: Any language that gets widespread use lasts … a long time, see:
    Languages, Levels, Libraries, and Longevity in ACM Queue.

    In the 1970s, at Bell Labs we went through nightmares trying to get all the various operations support systems (400) to use similar coding for data.

  9. 9
    GlenFergus says:

    Maybe you should also talk to those palaeo people Rasmus. Have a look here: http://gergs.net/2014/03/earth-temperature/. Just 8 sources, but a total of 4 different base dates, 3 different reference intervals, 3 different (and not always obvious) spatial coverages … and, of course, 8 completely different data formats.

    One big plea. If you’re going to publish a csv of you data, by all means use fixed column widths, but please please pad them with a tab character instead of spaces. Makes data import much simpler in many environment; not everyone is writing READ(3,8H(10F5.2)) … does that still work?

  10. 10
    prokaryotes says:

    Is the meta-data discussed, also considered an extra layer for a common modeling infrastructure, or a standalone framework? And how relevant is the Earth System Modeling Framework, or more recently known as ESPC? Thanks.

  11. 11

    A semantic web organized via an ontology is a laudable goal. This removes much of the ambiguity on search terms and on definitions and when tied into a logic engine, you can start to do interesting things. I used the JPL SWEET ontology on my semantic web server: http://entroplet.com/

    Like all good things, the more effort you put into the organization, the more payback you can get. It does take a lot of discipline, which is why these things are still not as popular as free-form search and organization such as Google.

  12. 12
    ezra abrams says:

    two classics in this vein

    1
    http://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/

    2
    also, you probably know the 1st hubble scope had a multimillion dollar piece of glass, the primary mirror, that had spherical aberration

    well, while they were building this piece of glass, there was an old time telescope maker – the kind of guy who would grind glass by hand
    And he built, out of cardboard and such, a test jig, and he told PerkinElmer, you multi milldollar mirror has 1o of spherical aberration
    So PE build a multimillion dollar test rig which said the mirror was fine…


Switch to our mobile site