A CENTRAL thesis of global warming is that temperatures will keep going up, and up. They did from about 1960 to 2002 at many places around Australia, but not at all of them. At Ruthergen, a wine-growing region of north-eastern Victoria, temperatures have been fairly steady since a Stevenson screen was erected and temperatures first recorded back in November 1912.
Dr Bill Johnston asks other climate scientists to consider the evidence for Rutherglen
The Bureau of Meteorology includes Rutherglen in a network of stations that it uses to report on regional and national temperatures. But it doesn’t report on the temperature values as recorded at Rutherglen. It first remodels them. Through this process the mean annual minium temperature trend is changed from cooling of 0.35 degree celsius per century to warming of 1.73 degree celsius per century. That’s a pretty large change in both magnitude and direction.
Since this was first reported in The Australian by Graham Lloyd there have been accusations that people like myself are into conspiracy theories, and more. But we really would just like some answers.
No one is more curious than Bill Johnston. He is disappointed that Andy Pitman and other Australian climate experts seem to be avoiding discussion of the temperature data for Rutherglen. Out of curiosity, and also in an attempt to generate some scientific interest amongst his collegues, Dr Johnston has undertaken the following analysis. It’s a bit technical. But hopefully not too technical for the experts.
Rutherglen - A brief overview by Bill Johnston
I examined the data graphically using a CuSum curve (cumulated deviations from the grand mean). There were 4 turning points, in 1923, 1957, 1964 and 1975. Two of those could have been independently-documented climate shifts (~1923 and ~1975).
I analysed the annual minimum temperature data, down-loaded from BoM using shift-detection software (CPA from variation.com) and sequential t-tests (STARS) (see: http://www.beringclimate.noaa.gov/regimes/rodionov_var.pdf) allowing for autocorrelation; and assuming all data were in-range (Huber’s H = 5).
CPA, which is based on bootstrapping, is not so good, when there are numerous changepoints. However, it detected shifts in 1924 and 1937 (P = 99% and 82% respectively). I did not follow-through fine-tuning the methodology, as I’d normally do to ensure no additional steps.
STARS detected regime changes in 1924 (P = 0.0007), 1958 (P = 6.07E-06) and 1965 P = 3.85E-06). This was inconsistent with the published metadata. The ACORN catalogue mentions no station moves. The steps were not documented climate-change years.
STARS settings are indicated on the graphic, so anyone can obtain the STARS Excel add-in, and data and re-do the analysis and check my interpretation. (Years indicated are start-years for the new regime.)
Of the step-changes, the 1965 shift was the most influential. Ignoring the intervening ‘hump’ the difference in mean level between 1957 and 1965 was 0.06 degree C.
Because it occurs in the middle of the dataset, such a step change could be trend-determining. If the hump alone was deducted, by shifting the data down by the difference, the 1965 shift may not be detectable. Trend however, would be little altered, and would still be no different to zero-trend.
The hump could have been a data-fill; a temporary move or a change in screen-aspect or instruments; but that is all a guess.
The shift down in 1924 of 0.77 degree C is consistent with a possible station move. However, based on Metadata, no such move happened, which was the point made by Jennifer Marohasy, which in-turn was based on the best information available from the BoM ACORN catalogue.
If we go back to BoM’s climate data on line, there was a post office record (082038; 1903-1925) as well as a viticulture research record (082085; 1903-1927). I’ve not compared these, but at an annual scale, they should behave in parallel (same trend, different intercepts).
(A graphical way of checking is to difference the datasets, and cumulatively sum the result, to see if the CuSum of the difference goes ‘off-line”. (Add a to b, then the answer, to c and so on – it takes 2 calculation columns, one for the difference, the other for the CuSum, and data needs to be closed-up on missing pairs). I think visual comparisons; using line-graphs are difficult to interpret. (Also some people have trouble with discerning colours.) Comparing regression lines may also introduce the possibility of spurious trends in either or both of the datasets being compared.
It so happens that in my research collaboration days, our team met time-about in the historic and interesting viticulture laboratory, which became a meeting room within the Rutherglen Research complex.
It seems likely that, in order to produce the long ACORN record; between 1924 and 1927, the viticulture record was merged with the Rutherglen Research record, and that BoM ignored or did not detect that particular stitch.
It could also have been an inadvertent error; the sort of thing that happens when someplace morphs into something else. (It could have been done locally for example.) (I don’t know when Rutherglen became an agricultural research centre.)
Irrespective of all the argy-bargy, the important points are:
1. The truth is in the data, not necessarily the metadata.
2. Without leaving Excel, small (less than 1 degree C) enduring shifts in the mean level of a data-stream, typical of undocumented changes, can be detected. However, for any time-series iterative testing using contrasting tests, and interpretive skill is required (at the 95% level there is a 5% chance of NOT detecting a shift that is real).
3. Most importantly, in the case of Rutherglen (and Deniliquin; RAAF Amberley; Longreach; Nobbys Head, Moruya PS and elsewhere) when abrupt-shifts are allowed for (deducted sign-intact), it is often the case residual trends are not different to zero-trend. (It can also be the case that the data are useless; Eucla may be in that category!)
In other words, for many places, naïve mean-squares trends may be spurious. They are due to shifts either in the measurements or in the climate.
In Rutherglen’s case, trend between 1924 and 1957 (34 years), and between 1958 and 2013 (49 years) were no different to (bootstrapped) zero trend (P <0.05). (Least squares trends were 0.007 and –0.003 oC/yr.) This further supports the view that we have a non-trending data stream, containing a small step. According to CPA and STARS there were no significant (P <0.05) changes in variability. With all step-changes removed, there was no trend. There are four important lessons. 1. Graphical and statistical evaluation on a site-by-site basis is essential before any inter-station comparisons, or inter-station homogenizations are undertaken. This is formally called exploratory data analysis (EDA). My endless analysis of data leads me to conclude that many series are not fit-for-purpose; or that it is simply not possible to detect valid trends against background noise. (Valid trends are trends ex-steps. For least-squares regression to be valid, data MUST be homogeneous.) 2. There can be changes within data that are undocumented (like Rutherglen and Amberley); likewise there can be documented changes that don’t result in data inhomogeneties (Eucla possibly being a case in point). 3. It is inappropriate for the Bureau having made a mistake in respect of supporting comments made by Professor David Karoly, who has a long history of being biased; to pull in heavy artillery from another, potentially equally-biased UNSW group, in an attempt to reclaim the high-ground. 4. There is an answer. I’ve put forward a transparent approach. Having said that I’m quite willing to be proved wrong at the statistical confidence level of P =0.05. The situation has become a bit absurd.
The Bureau needs to show unambiguously ‘true’ or ‘real’ uncontestable positive trends in minimum temperature data that don’t rely on their homogenisation procedures.
It is really that simple.
If BoM cannot do that, they have lost the statistical debate and need to fess-up or develop yet another trend-setting dataset.
Cheers,
Dr Bill Johnston
Former NSW Natural Resources Research Scientist
Note that I am a data analyst, not an expert mathematical statistician. I would not be regarded as an expert in the sense of arguing a statistical case with BoM, for example. That does not invalidate my analysis; it is simply a disclaimer. And if you want disclaimers, look no further than most BoM reports.
[…] colleague, Bill Johnston, has forensically examined the temperature data generated by the weather station at Rutherglen. Dr Johnston is of the opinion that it is possible […]
[…] original minimum temperature series for Rutherglen for discontinuities and found none. Agronomist Bill Johnston has run his own very fine-scaled ‘ruler’ over the same dataset and found step-changes that could be attributable to shifts in the climate, […]
Detecting shifts.
There is a process involved in detecting step-changes (inhomogeneties).
I start with annual data before looking at monthly anomalies (data with monthly grand-means (GM) deducted.
Missing data needs to be filled. If that is not possible, I close-up the data, so there are no gaps. This may introduce some degree of statistical error; so it must be mentioned that data were closed-up on missing values and leave it to the reader to work out the effect of that. Later, its important to check if missing data corresponds to a step; they could signal for example, a station move or new instrument.
Start off with a linear time-plot; look for possible sudden shifts. If shifts are not evident, they probably won’t exist. Plot a regression line and look at variability – the magnitude of ups and downs should look the same along the series. It is also a guide to see if data runs off-trend.
I generally then plot a CuSum (cumulated deviations from the dataset grand mean (GM)).
(CuSum’s for strongly trending data can be misleading. Y values less than the x-centroid, tend to be less than the GM; those >centroid, more than GM; giving the CuSum a ‘U’ shape about the centroid. If this is observed, detrend the data first; i.e. work with residuals.)
A trend-unaffected CuSum, will have one or 2 sharp “V’-shaped inflections. If it has 3 or 4, suspect the data are randomly-varying about the GM; so there is unlikely to be significant change-points.
(A problem with many series is that the 1947 climate shift occurred ~half-way along datasets starting in 1910. An inhomogeneity about that time, or a bit later, could be real; but it will be ‘V’ shaped, not ‘U’ shaped.)
Note the time (year) of suspected changepoints.
Change point analyzer (CPA)
I then use CPA, set usually to detect changes at 50% CI; but to only tabulate change-points where P>90%. CPA produces a CuSum curve overlaid by detected inflections; also a line-plot, showing calculated shifts; and a table of detected changes. Increasing the rigor by increasing the CI from 50% to 80%, for instance, eliminates nuisance or possibly spurious changes. The tradeoff is between detection and inclusion in the table; and the result must be sensible against expectations.
CPA loses-out if there are more than a few change-points and it does not detect well near the ends of a dataset. I use 10,000 bootstraps with replacement; and CuSum estimates. There are other options. It is important to look at documentation relating to handling autocorrelation.
CPA does not provide digital output; but jpg’s which are a bit klunky. I paste them into Excel.
STARS.
Like CPA, STARS also needs to used iteratively. Input is columns in a single worksheet. I usually do my analysis one variable at a time; but you can do 12 for instance. The risk is you end with many changes, none optimised for any particular variable. Missing data or text within a column will cause a runtime error; so remember, for calculated data, to past as values.
I usually presume all the data are real, so set Huber’s H = 4 or 5. I also assume data are autocorrelated, and use the ‘IPN4’ option; STARS will work out necessary compensations for the t-test.
There are several settings Target probability p sets a detection parameter; l, sets a target regime ‘length’ in whatever the time units are. l is a weighted parameter; the default is 10 time units.
The aim of iterative testing is to detect only changes that are meaningful; while ensuring that we don’t fail to detect changes that may be real at the scale we are interested in. (The Type I vs. Type II error conundrum!)
It is achieved by holding p constant while trailing various settings of l or vice versa.
Short l and low p (say 5-years; and 0.5) will detect many spurious and non-meaningful changes (low significances); whereas 5-years and 0.01 will detect significant, short-term shifts, if they exist.
Increasing l to say, 10 or 50, for annual data will detect decadal-scale shifts (if they exist) and if p is set appropriately. A high p (such as 0.3, and l = 20), may detect shifts with a range of t-test P levels; so it’s best to try a range of p and examine the output.
Shifts in the output should be significant at some level of choosing (P<0.05 for instance); and the residuals should be step-free. Remember we are not step-hunting; we are using the tool to investigate data.
An appreciation of the shape of the data, gained from the CuSum (or CPS’s CuSum and other visual clues); together with STARS is a useful diagnostic combination for detecting inhomogeneties.
It is also useful for conducting a ‘blind’ analysis, independent of metadata.
Cheers,
Bill
Bill, thanks re RHtestv4
>"....which led me to believe the principles behind it were in general use by BoM."
It would and it's a start, I suspect that package has been adapted by BoM for their PM variant rather than code from scratch. Just a hunch so will look into this over time. BoM has adapted Menne & Williams (2009) for their initial homogenization (M&W09) so makes sense that they probably haven't reinvented the wheel for adjustments (PM95) either (but see below).
>"Happy to be wrong;"
Well, at this stage you're not yet necessarily wrong as above.
>"....somewhere deep within the ACORN docs. is reference to percentile matching."
Actually it's very explicit. I've laid out the provenance of BoM's adoption of M&W09 and PM95 at the JoNova post 'Hiding something' #38.2 here:
http://joannenova.com.au/2014/08/hiding-something-bom-throws-out-bourkes-hot-historic-data-changes-long-cooling-trend-to-warming/#comment-1554309
TR049, page 65 pdf,
7.6 Evaluation of different adjustment methods
“Based on this evaluation, the PM95 method [7.4] with 10 neighbours was selected for use in most cases. Details of the implementation are given in the following section.”
So having decided on the PM95 method, they (BoM) probably went looking for some pre-written software because it's a pre-existing method. That looks to me what Ashcroft, Karoly and Gergis have done but adopting QM by default. I would have thought BoM assessed RHtestv4 for their use but I'm not convinced they actually do use it but they may very well have adapted it.
In the following comment at JN (#38.2.1) I've provided a definition of Percentile Matching from its use in health policy (it's a statistical modeling technigue involving “parameter estimates”, with "no guarantee that an estimate will be unique"), and a tutorial from the Casualty Actuarial Society.
>"Steps in Amberley min were 1972 and 1980. I have not paid much notice to max lately;"
OK good to know. I didn't think there would be a very large step in Max without a corresponding step in Min. Amberley Max was investigated by Marohasy et al:
‘Modelling Australian and global temperatures: what’s wrong? Bourke and Amberley as case studies’
https://climatelab.com.au/wp-content/uploads/2011/08/Changing_Temperature_Data.pdf
Has the BoM done the simple check of reversing the years of data for each station to demonstrate their adjustment algorithm at least operates in a symmetrical manner?
511KEV:
Point 3: That is precisely what I am currently doing. With 100% of the sites for which the Bureau has released the identities of reference stations i.e. only Rutherglen. Stay tuned.
"However, I think it would be unreasonable to expect BOM to jump every time you, or Jennifer, or The Australian get hot under the collar."
Is making substantial criticism or expressing well-founded doubt the same as getting hot under the collar? BoM should respond. If they feel they don't have to "jump" then they should take a day or two. Then they should respond.
I think Jen and Bill know about their freedoms to blog, correspond and write papers. Guidance on excel and stats is found in seconds on the web, though I don't doubt that some educational suggestions may be valuable. We can all use some of that!
However Bill and others shouldn't be diverted from this specific urgent matter of the Rutherglen temps and the justification for what Jen calls "a pretty large change in both magnitude and direction". That ball has to be kept in play.
And when we speak of change in direction, there is something all together too predictable and familiar about the direction taken: namely, from cooler to hotter. That's getting to be a sub-optimal look, to put it nicely.
BoM really needs to clear this up. They might think they don't have to. They do have to.
Bill said:
I’m not saying that observers were slack, its just that historically, well-before the CO2 clamor, they had no idea that 50 to 100 years after their observations were written down; 1 or 2 generations later in some cases, our BoM would be going over their figures trying to prove something the data were not collected to prove.
Yes, and given that the claimed rate of warming is 1C or less, and the adjustments are of a similar order of magnitude, then further it is claimed that such a trend supports the climate models which predict catastrophic, CO2-caused warming, we are getting into Alice In Wonderland territory.