Surely you’re also familiar with similar efforts like Old Weather at http://www.oldweather.org which employs volunteers to transcribe naval weather records. They’ve made a game out of it, with ranks, etc. to keep interest up.
If you would like to experience the thrill of looking at some fascinating historical documents in search of weather data, have a look at the website oldweather.org where a community of several thousand volunteer scientists are transcribing Royal Navy ships’ logbooks from the first world war. Those logs record not only the weather, but also the story of the ships; and provide new insights into historical events, as well as vital information on historical weather and climate.
I am not a “legion follower” of the blog that shall not be named, but I do follow it. I also think this is an excellent post, as there is great value in rescuing old data. I am an active participant in one of the named rescue projects. What are you doing to help?
Hi John, Tim and Philip, Oldweather looks like a great project and got a lot of publicity over here – it was on (Canadian) national radio a while ago. There are other similar projects going on in several places, some of which should be listed on the ACRE site. Another one is the IEDRO organization. Let’s hope we can get as much data rescued as possible!
I’m sure the legion followers of the “best science blog” will be be around any minute to step right up an volunteer their time in support of this valuable citizen-science project
Would their help be accepted?
[Response: It would be. And on that note, end of sniping (all sides) before it goes further please. Vicky's given us a nice springboard to discuss this--and other actual or possible--citizen contributions to climate science. Let's use it.--Jim]
I’m happily doing my small part with the Solomon Islands data.
I leave it to other eyes with better patience than I possess to work with the much more challenging German radiosonde stuff – and those ships’ logs are very demanding. But it is fun to see the journey progress. It’s like super slo-mo GPS.
There is a related project on rescuing pre-1900 instrumental data and documentary evidence of weather variations in Australia called SEARCH (South-East Australian Recent Climate History http://climatehistory.com.au/ ). Volunteers are welcome to assist and can find more information through the OzDocs web site http://www.ozdocs.climatehistory.com.au/ .
Join as a volunteer and find out more about the weather in the Antipodes.
When Anthony Watts does this sort of thing checking for accuracy, he is ridiculed as a lunatic by this site. What gives?
[Response: Watts is never ridiculed as a lunatic for doing 'this sort of thing'. He is ridiculed as a lunatic for overstating and mispresenting what his research shows, for making things up, for accusing scientists of fraud and/or stupidity, and for generally being an ass. Don't conflate the possibilty that Watts sometimes does intelligent things with our criticism of his stupid things.--eric]
One potent source of historic climate information that I believe has yet to be appropriately compiled are the observations of Native Alaskans, particularly those who live in communities that still practice a subsistence culture closely entwined with the environment. Here in Juneau, Alaska I have sat in on state climate hearings and heard testimony such as:
“The marrow in moose bones has changed consistency – it is runnier than it used to be since they are eating different vegetation.”
“The bees are bigger than they used to be.” (Interestingly from numerous different Arctic coast villagers. I believe they mean the same species is getting bigger because in my experience they distinguish between new types of insects/birds/fish.)
“The birds are confused: they are singing different songs than they used to and they do not understand why spring is happening so much earlier.”
And dozens more astute observations of how the ecosystem is changing beyond their long experience of observing natural historic variability.
11 EverettRowdy: Those are interesting sayings, but I have no idea how to make them into data. Do you? If somebody had measured the viscosity of marrow in moose bones, how would you relate that to temperature?
Edward. It looks as though that info would need to be recorded as an oral history of some kind. With a menu or checklist of issues or questions to be addressed in particular ways.
It wouldn’t finish up as reliable as the meticulously recorded cherry blossom festival dates from Japan, but it could inform further enquiries or corroborate info from other sources. Converting to data? If you don’t record it at all, there’s no chance.
Record what you can as best you can. Then make the most of what you’ve got. Even if the material finishes up only as interesting footnotes, it would have been worth the effort. And it would inform other historical and archaeological endeavours regardless. (I’m encouraged by the memory of hearing a woman talk about her research into the history of police horses in one Australian state. It finished up taking her into some really interesting political and other history. And all because of beautiful grey horses.)
Nice writing! (The guest post, not the handwriting.)
[Response: :) It's not uncommon, in my experience with land survey notes and other officially transcribed historical information, to find penmanship that is so "nice" (i.e. excessively elaborate/ornate) that you actually can't read it! These and other issues make it a sort of constant challenge--but well worth it in the bigger scheme--Jim]
As part of the Surface Temperatures Initiative (surfacetemperatures.org, blog at surfacetemperatures.blogspot.com) there is a major incipient effort to create a single repository of land meteorological records. The aim is to create a truly global holding with strong version control and provenance tracking, where possible back to the original measurement. This will substantially increase their fundamental scientific value. The effort aims to build upon and augment ongoing activities and pull through to a data holding in a timely manner. We know there is much more data than exists in current public holdings. The databank will consist of a number of distinct stages:
Stage 0: raw data in hard copy or image form
Stage 1: data keyed in native format
Stage 2: data converted to common format
Stage 3: data merged so only a single version for each station
Below stage 3 there may exist multiple data versions for a given station. The databank would also hold all available metadata. More details are on the initiative website.
I was wondering if anyone has contemplated the large database of temperatures from personal weather stations, ie http://www.weatherunderground.com. Some of the PWS have been running for quite some time. Would be an interesting project some day to see how much effect the UHI really has.
Great post! A couple of follow up questions.
Are there any school lesson plans that have student volunteers helping to transcribe the data? Seems like a great opportunity to integrate history and science in the classroom.
Are there any projects looking at the maritime logs of European voyages through the South Atlantic?
Everett at 9:18:
I agree that there is an enormous amount of knowledge of the past and present held by native peoples–or any people who spend much time outdoors observing their environment. The difficulty is in obtaining, standardizing and calibrating it so that it is useful. Sometimes this dictates that it has to remain a relatively disparate collection of stories and anecdotes–which can still be very useful, if less succinctly quantified.
Jim@20: ACRE has helped coordinate a number of small scale ed/outreach activities for high school and undergraduate students. There is an example described at this link that would be fairly easy to duplicate:
For those interested in the relationship of climate to American Indian life in the late 1700s and 1800s, see the biographical and environmental works of my friend Jack Nisbet on David Thompson and David Douglas. Both were remarkable men, diligent researchers, cartographers, and collectors, who linked the relationships between daily reports of weather and climate with the native indigenous peoples.
Rocco@23: Yes, the verification software is already being pursued as an option. Also expansion of oldweather.org type approaches. There are literally millions of images in the NOAA Foreign Data Library – just one single finite holding. So there is more than enough data to pursue multiple avenues to get it digitized. Key is coordination (to avoid redundancies) and pull through to a databank (to ensure actual use).
Many scientist and organization has been collecting so many climate data from the past until present day and I agree that effort will give us something important. But there still so much variable that can’t analyze, so collecting the data and analyze it to become useful information would be a great effort.
Richard Hendricks at 19: The Urban Heat Island effect is well known, as a local meteorological effect. Places the size of London are warmer than the surrounding countryside. You could get this from published records by comparing sites marked as urban, with sites marked as rural. Its roughly a degree C or thereabouts. I think Tamino at Open Mind has done this in the past. What both rural and urban sites have, of course, is the same anomaly (rise in recent temperatures wrt the same baseline), so that the UHI has nothing to do with *global* warming.
At some point ‘tho we will reach a level of understanding of cause, effect and resulting trends to say that we know enough. I believe in fact that we are already there.
We know that the ‘big’ parameter of GHG concentration drives all the rest. At or below 280 ppm CO2 the balmy living conditions of the old Holocene will continue, while anywhere near of above our present 390 ppm CO2 things will get worse with a rapidity that is (at least) proportional to the divergence from pre-industrial levels.
We already have the Necessary and Sufficient evidence to show the truth of the matter. There is nothing that we need to know or understand that we do not already know and understand. Continuing as we are the weather will be more extreme, the rainfall heavier, the droughts deeper, the winds stronger, the seas more acidic, the oceans will rise until the ice is gone. More evidence will not undo these knowns.
To check the progress of our climate-related efforts all we need to do is continue to monitor CO2 levels – up is bad, down is good.
I do not mean to denigrate the efforts of the scientific community in their ongoing research and analysis in any way – without them we would be traveling blind into disaster. But they have ‘removed the scales from our eyes’ and any additional information does little to improve our state of acceptance or denial of our unhappy circumstances.
The difficulty now lies not in providing more detailed or precise information telling us essentially the same thing, but in finding a way to get we-the-people all behind sensible concerted efforts to minimise the scale of the disaster (by reducing emissions if we can) while simultaneously vigorously preparing our families and our communities for survival in a post-oil world battered by what ever climate-driven miseries our past foolishness will bestow upon us and our children.
We know enough. All that is left is that we act in terms of the knowledge that we have today.
What a refreshing return to the disinterested tradition of amateur science at its Victorian best- and what a remarkable contrast to efforts to rewrite the instrumental record to polemic ends.
One hopes the project will inspire parallel efforts to retrieve and upload the
meteorological data of observers in Imperial Russia and the chaotic early decades of the 20th century, not just to improve temperature and precipitation data bases, but retrieve accounts of seasonal tundra thaw limits and fire emissions in the great Eurasian boreal forest.
Great post Vicky,
I have been following the travails of a colleague who is trying to extend Icelandic temperature records back into the 17th century. It is truly daunting. Simple things like how the thermometer was placed (was it on the outside of a tar-painted house?), at what time was the observation taken etc.
Keep up the good work!
Comment by Halldór Björnsson — 10 Apr 2011 @ 7:57 AM
Nigel Williams @30.
You live in a nice ordered world or at least believe in it. Where do you hang out?
I live in a chaotic place where we don’t know doodly squat about anything, Every time we look we just unearth another can of worms. Cause and effect is nice but we don’t understand the basics. Just a pile of unproven theories based upon other unproven theory upon theory.
Must be very comfortable and reassuring place to live in your world.
[Response: The reality of the situation lies somewhere between your and Nigel's extremes. We do not know everything we need to (though we do know enough to take action, which was his main point), nor do we know nothing. Also, theory not = speculation, as I'm sure you know--Jim]
I’ve found voice recognition software can be quite good for entering data provided it knows you are dealing with numbers.
I’ve also found that there is a lot of data from the ‘measured’ era which is not properly computerised or generally available. I was in Malawi last year and was given good quality data for rainfall and temperature for stations which had several decades of measurements but which are not in the GHCN data base. I am currently working with a data base of met data from central Africa and whilst it currently stops in 2000 there are stations here with more than 100 years of data for which the GHCN data base stops before that date. Until these data and other comparable sources are explored we will not have the BEST possible data for climate analysis.
Ron @ 34: I’ve tried voice recognition before without much success (it took longer to correct the errors than to type the data in directly), but will give it another try. It could certainly be a useful tool.
Comment by Heikki Tuomenvirta — 11 Apr 2011 @ 3:14 PM
@34 Ron, v. interesting – I also do some work in Central Africa (e.g. Lewis et al. 2009, Nature, 457, 1003-1006), and would really like to see some long climate records from there, as I didn’t know they exist. Perhaps you could get in touch (via my U. Leed email address) ?
(as an aside I’ve been collating old forestry records from the tropics to better understrand these forests’ role in the global carbon cycle – similarly lots of tedious typing of old records, but fascinating too.)
Weather data is nothing compared to fisheries data, which is truly a moving target even on the decadal level. But if you dig deep enough, and gather enough, and are unrelenting you can dig out good, reliable data, which for current purposes only needs to be within two orders of magnitude of reliability (meaning there are no swordfish left right off Portland, Maine). Often it is the ancillary parts of these records that are the most useful and informative.
Allow me to make a generalized observation: it’s good to get this data, but it’s almost exclusively records of temperature. The dew points are going up and up, maybe more distinctly and larger than increases in temperature as such. We should see more study, commentary, and publicizing of dew point trends. Thanks.
Neil @41 Dew point data, alas, is even messier than temperature data. This includes the modern stuff. The automated dew points taken at U.S. airports are subject to instrumental drift and local conditions (e.g. near an irrigated corn field, such as KSDA in July) It was up to me as an NWS forecaster to guess whether it was accurate, and request a fix if it wasn’t. Fixing dew point equipment is generally not a high priority with airport authorities.
When a dew point of 86 was reported in eastern Iowa during the July 1995 heat wave that killed a lot of people in Chicago, the NWS actually got an observer to check it out with a sling psychrometer. That time, the instrument was right.
Vicky @ 36. I found the best way to use voice recognition was to select the numbers only option (in Dragon naturally speaking) and produce a CSV file. It meant saying ‘comma’ a lot but generally worked.
Neil #41. Some, and I do mean some, of the locations do have dew points, wind direction, wind strength, rainfall – some have little more than max, min, current temp for whatever the daily time of recording is.
But it is all useful. When the whole lot is digitised and incorporated into a coherent picture, we may not know a great deal more than we already do about global or ocean-wide climate. But a neater, tighter picture of smaller regions may be useful for some purposes.
Titus @33, I do live in a pretty ‘nice place’ thank you. As far as we are able we are comfortable and reassured that we are making the best preparations that we can for the coming ‘interesting times’. As I’ve said; we know enough to act.
Our haven is located on the null-point between climate-forced increasing and declining agricultural production, rainfall and (initially at least) temperature will be bearable; it is clear of direct threat from melting of all the ice there is; in an area still rich enough in mine-by-hand minerals; established decent soils; established transition-minded community.
The hand weaving courses are behind us, the spinning wheels spin and the loom is live. The solar cookers and ovens are working like a charm, and thousands of food-seed balls are waiting on the turf for the winter rains.
And we don’t need to take a geiger-counter with us to go shopping for food.
Even if all these assets are somehow taken away from us, our thinking and our spirits are already well prepared.
Yes; acknowledging that there is always the potential for a Black Swan event, we are pretty content, but by no means complacent. Its the tired army that wins the battle. Hope things are going as well for all, but each to his journey, eh.
I love hearing about the adventures of folks taking cryospheric measurements, oceanographic sampling, and biological/ecological/phenological fieldwork. By contrast, this stuff may be less ‘Indiana Jones-ish’–but it’s an intellectual adventure nonetheless.
It’s equally unacknowledged by those who imagine that AGW science is ‘climate modeling all the way down.’
(As if you could model usefully without tons of ‘obs’ to work from!)
Neil@41, John@46,adelady@44, there were a few who tried to take dewpoints or other hygrometer readings in the 19th and even 18th centuries, but they’re not as common, and as John said, they would be difficult to check for quality control, especially in isolated diaries. On the other hand, almost all observers recorded weather descriptions. I’ve started to be interested in these as much as in the measurements, on the grounds that, by and large, we can be reasonably certain we can interpret “rain”,”snow”, “cloudy” or “showers”! More esoteric descriptions do pop up occasionally. Counts of precipitation days, thunder, ratio of snow to rain, dates of frost in fall or snow melt in spring, and so on can be quite robust measures.
Vicky, I’ve stuck pretty well exclusively to early 20th century records which by and large are pretty well kept (apart from a bit of idiosyncratic handwriting.) They seem to stick pretty well to SSW, calm, NE and other standard descriptions for wind at least.
What many people forget is that this was an endeavour, by and large, of colonial, seafaring nations and their more adventurous private companies, as well as conscientious individuals. They were dedicated to keeping good records to maintain the best schedules and courses for ships and to seek out good areas for agriculture, mining and forestry.
I also think we overlook what was the most common ‘scientific’ interest of those times. The simple recording (and a bit of classification) of botanical, geographical, zoological, meteorological and astronomical observations rather than the theoretical, analytical and explanatory approaches we now focus on. We tend to treat observations as items to be fed into our work. In earlier times, those records were seen as intrinsically valuable in their own right.
Vicky, in addition to the phenological projects, another similar application and potential source of skilled volunteers, is the transcription of cemetary records. These are done for mapping and maintenance by public works departments, and are used in geneological research. People who do that are likely pretty good at reading old penmanship and there is software that might be adaptable to your purpose. Just a thought, hope it is useful.
BTW Nigel is right, we know enough to act, but it is also true that we should always strive to fill gaps in time and space, so I find this a worthy project. Not to say we should not also be out in the streets demanding meaningful action on climate by our recalcitrant (USA) government.
Comment by Scientific American — 13 Apr 2011 @ 11:03 AM
You’re making a great job trying to rescue our historical weather past.
Quebec ministry is doing big efforts to rescue historical data but also metadata. We are doing many efforts to type up, validate and analyse theses data that stayed in dusty boxes. Last years, 4 people typed historical weather data. It took 6 months to type up 7 000 000 data. We will homogeneize data, calculate statistics and evaluate climate changes with them.
Beer is a terrible terrible beverage! Saccharomyces Cerevisiea produces CO2…
But what the heck let’s enjoy it while we still can!
A Biologist’s St. Patrick’s Day Song… http://www.youtube.com/watch?v=oax3cUFsBSw