OVER the last year, John Abbot and I have worked on a new method for forecasting rainfall based on the use of artificial intelligence.
We have benchmarked our forecasts against output from the Australian Bureau of Meteorology’s General Circulation Model (POAMA 1.5). Our model gives a more accurate forecast for 16 of the 17 sites in Queensland that have the highest quality rainfall data. Our methodology is detailed in a new paper that will soon be published in the international journal Advances in Atmospheric Sciences.[1]
Artificial neural networks can find existing complex relationships and patterns repeated in rainfall data. The neural network software that we run on our standard laptop computer, can perform millions of calculations very quickly, and in this way find natural patterns that repeat themselves.
In contrast, the Bureau uses general circulation models; the same models used to predict global warming. These models attempt to forecast seasonal and monthly rainfall by working from a particular theory of climate. Our system is radically different in that we let the model find the patterns. We let the model find the natural cycles and then forecast forward up to 9 months.
That our system works, proves that there are patterns - natural recurrent cycles - in historical rainfall data.
Contrary to some of the claims of some climate sceptics: the natural hydrological cycle is not totally chaotic.
Contrary to the claims of most warmists: these natural rainfall cycles have not been perturbed by the current elevated levels of carbon dioxide.
What we did is tune one of the most advanced off-the-shelf software packages for pattern recognition (Synapse, Peltarion) into the natural patterns and recurrent cycles in historic rainfall data.
We found a combination of historic rainfall and temperature data combined with historic data for three climate indices (Southern Oscillation Index, Pacific Decadal Oscillation and Nino 3.4) gave the most accurate forecasts. Inclusion of solar irradiance and sunspot number did not enhance the performance of our model. Inputting carbon dioxide concentrations doesn’t improve our model.
We believe we have barely scratched the surface in terms of potential for rainfall forecasting using this method for eastern Australia and are looking for significant additional funding to progress this work.
The work has so far been wholly funded by the B. Macfie Family Foundation and supported by the Centre for Plant and Water Science at Central Queensland University.
*****
[1] Email me at jennifermarohasy at jennifermarohasy.com if you would like an advance copy of our paper:
John Abbot and Jennifer Marohasy
Application of Artificial Neural Networks to Rainfall Forecasting in Queensland, Australia
Advances in Atmospheric Sciences, Volume 29, Number 4, Pages 717-730.
FOLLOWING IS SOME RECENT MEDIA:
Notebook neural network beats supercomputer model for predicting rain
By Stephen Withers. May 28, 2012
Researchers at Rockhampton-based CQUniversity have come up with what appears to be a better model for predicting rainfall in Queensland.
The Australian Bureau of Meteorology makes its forecasts from a mathematical model of the atmosphere (known as POAMA) that run on supercomputers.
But a pair of researchers at CQUniversity have applied neural network technology to the task and claim this gives better rainfall forecasts - and runs on an ordinary notebook computer.
The basic idea of a neural network is a collection of connected nodes that are modelled after the neurons in a brain. Like the biological structure on which it is modelled, a neural network is capable of 'learning'…
*****
Software offers brain on the rain
Angus Thompson. May 30, 2012
ARTIFICIAL intelligence software has outperformed the Bureau of Meteorology in predicting long-term rainfall - and could soon be used for daily forecasts.
The neural networking technology proved more accurate in forecasting weather events in 16 out of 17 Queensland locations, including rainfall that led to the Wivenhoe Dam bursting, researchers from Central Queensland University found…
http://www.heraldsun.com.au/ipad/software-offers-brain-on-the-rain/story-fn6bfkm6-1226372995011
*******
Weather Forecast System Developed
By Austin King, May 30, 2012-05-30
Dr Abbot combined his interest in artificial intelligence for pattern recognition with Dr Marohasy's knowledge of climate science to come up with a neural network system that predicts monthly rainfall from three to nine months in advance.
Their system is the focus of a study about to be published in the international journal Advances in Atmospheric Sciences.
They decided to put their heads together just after the January 2011 flood that isolated Rockhampton from the rest of Central Queensland.
The method is based on pattern analysis accepting that there are patterns like short- and long-term cycles evident in rainfall data…
http://www.themorningbulletin.com.au/story/2012/05/30/weather-forecasting-system-developed/
******
Original Media Release
Laptop beats supercomputers at forecasting rainfall - CQU
28 May 2012
Laptop beats supercomputers at forecasting rainfall, thanks to artificial intelligence approach
CQUniversity researchers - including one who honed his skills through share market trading - have been able to forecast seasonal rainfall more accurately than the Met Bureau by using artificial intelligence for pattern analysis.
A journal, Advances in Atmospheric Sciences, is about to publish a new paper by CQUniversity researchers Dr John Abbot and Dr Jennifer Marohasy. The paper shows the application of artificial intelligence to monthly and seasonal rainfall forecasting in Queensland, Australia, and compares their forecasts with forecasts by the Australian Bureau of Meteorology's super computers.
In March 2009 the Australian government ordered two new supercomputers ostensibly to ensure Australia
is at the forefront of international weather forecasting and climate modelling.
The super computers run a general circulation model known as POAMA that is a mathematical representation of general atmospheric circulation patterns. In the West, attempts to improve rainfall forecasts from general circulation models have focused on improving our basic understanding of weather processes, most recently with a focus on the role of carbon dioxide as a greenhouse gas.
In other parts of the world (notably China, India, Iran), governments have also funded research into artificial neural networks for rainfall forecasting. This radically different method is based on pattern analysis accepting that there are patterns, for example short and longer-term cycles, evident in rainfall data. Neural networks, based on artificial intelligence, have the ability to consider large numbers of climate indices (eg. El Nino, Indian Ocean Dipole) and other inputs (eg. temperature, cosmic ray flux) simultaneously and make predictions independently of any understanding of, for example, the hydrological cycle.
Following the devastating floods of January 2011, with three-quarters of Queensland declared a disaster zone, CQ-based researchers Dr Abbot and Dr Marohasy combined their respective interests in artificial intelligence for pattern recognition with climate science to see if they could forecast the weather at least as well as the Australian Bureau of Meteorology.
In their first attempt at optimising their dynamic stand-alone time-delay recurrent neural network (TDRNN) they found that patterns within rainfall data alone could provide a forecast.
Their optimal model combined current and lagged rainfall, temperatures, Southern Oscillation Index, Pacific Decadal Oscillation and Nino 3.4 reflected in the highest Pearson correlation coefficient and lowest root mean squared error value (RMSE). This model was applied to 20 sites across Queensland generating monthly rainfall predictions three months in advance.
The Australian Bureau of Meteorology provided output data from POAMA enabling a direct comparison of the ability of the two models to forecast seasonal rainfall: the general circulation model (POAMA) developed from a theory of climate and run on a super computer with a large staff versus the neural network prototype based on pattern recognition theory and run off a laptop in a small office.
Outputs, as monthly rainfall forecasts three months in advance for the period 1993 to 2009, were compared with observed rainfall data using time-series plots, RMSE and Pearson correlation coefficients. The comparison showed the prototype neural network achieved a lower RMSE for 16 of the 17 sites compared, meaning it gave a better forecast for 16 of the 17 sites.
Dr Abbot, who honed his skills with neural networks through share market trading, considers the prototype design for rainfall forecasting very preliminary, with potential for significant improvement and application for anywhere in eastern Australia with at least 100 years of high quality historical rainfall data.
The findings have significant application to everyone affected by the weather but particularly for agriculture and mining with scheduling of mine activities in CQ significantly impacted by wet days. ENDS
Details and PICs via http://uninews.cqu.edu.au/UniNews/viewStory.do?story=9483
Luke-I am a stockowner dependent on the weather and the BOM with all the radar gear is improving marginally at predicting up to a week ahead. You say-
I bet if this paper was written by someone from BoM or CSIRO you’d all be at it hammer and tong. ............
These people pick up their pay and never have to be right and most of the time they aren't! The CO2 scam has devalued science so badly that I will never believe another thing to come out of the csiro.
Roger Stone has had a blank cheque and a wonderful career but I would sooner toss a coin and read my stars than put money out on his predictions. An improvement @16 of 17 spots for beginners on a tiny budget must be spooking the likes of you Luke -I assume that you have your nose in the taxpayers trough. hence all the time trolling on here.
But Debbie why would want any input - you won't even speculate how you'd like to use the information or what would be useful. I tried to engage you on terciles after I said how would you use Jen's RMSEs or indeed her forecast (and wasn't a shot at Jen - just a question). . You've spurned attempts by Bazza and myself to engage with you. "Not interested in packaging" as you phrased it. Which is strangely typical - lots of opinions - then try to engage - and find the audience wasn't really interested after all.
Thanks for the link Cohenite,
And Luke.....comment 19 at least demonstrated some much needed humility about the veracity and reliability of this work.
Seriously....us humble landholders are more than just a tad over the dictating and would prefer some serious input into where climate research funding goes and for what useful purpose.
I particularly liked your comment Cohenite about CBA.
Dear dear Cohenite could do well to read comments 8 and 19 from Saint Warwick's most excellent informative site.
Well done Jen; I haven't time right now to read your paper in detail, or luke's detailed bloviations, but will do so soon.
BOM has no standing at all for making predictions, about anything, over any time scale:
http://www.warwickhughes.com/blog/?p=931
So, perhaps it is not doing your paper justice to use BOM as the benchmark; POAMA or ACORN, they are all duds.
So you have not REFUTED the paper? Merely suggested improvements?
Do you agree with the central tenet that it is worth investigating alternative methods without an obessive focus on CO2?
The scientific method of review is to provide the paper to expert reviewers who, over a period of time, usually months, will consider the paper, seek supplementary information and clarification, discuss contentious issues, and seek changes, clear it for publication, and further scientific discourse.
The political method of review is for automatic gainsay by taking a contrary position, and attempting to declare the discussion closed, and that any further discussion after this arbitary point in time is useless. The poposition is thus REFUTED and any further discussion diluted by reference to the spurious "science" approved by St Luke.
Its not hard to see which method you apply.
I mean it took you what? A few hours to REFUTE the paper. Amazing!
The bottom line is it is worth investing alternative methods outside of the CO2 or any other current "approved" paradigm.
You use "science" as a smokescreen to hide you are pushing a political wheelbarrow.
Jen has challenged your orthodoxy from first principles, and you dont like that. You just dont like it because it may get other to people to think outside the box you want them to think within.
St Luke of the Bulging Intellect, who forsake common sense and manners, for the love of his cause.
No Luke,
I haven't confused it at all.
I'm commenting on potential use/usefulness which is the way Jen and Abbot seem to be looking at it too.
That 'usefulness' judgement does tend to draw a comparison with the exponentially increasing number of contradictory AGW modelling.....from my humble landholding perspective....especially when it's clear that huge amounts of political attention and hence climate research funding has gone in that direction.
How would I use the system?
I suspect in a similar manner to the way I use the ones available now.
I tend to usually go to to BoM and YR and compare those 2....sometimes go further afield if those 2 are completely contradictory....which happens more often than we would expect.
Both of them help but I don't find either particularly reliable until about 36 hours beforehand.
They don't pretend they are either....they speculate on known factors as well...and give probability figures.
I also commit what you and Bazza would likely consider a grave sin and factor in what I have learnt from the 'old timers' (including indigenous) who seem to know how to read signs in the behaviour of the natural environment (plant and animal)
Those are very often (but not always) a better earlier indication of impending changes but not necessarily any better indication of amounts of rain or duration of rain events.
If Jen and Abbot's work help to make the speculative stuff capable of 'less error' then it has no argument from me.
And your last question.
I saw most of what you wrote as 'defensive' rather than 'postive'....but maybe that's my fault.
However....the suggested tests against indices and other well known systems was not defensive but was rather obvious advice (but nonetheless good advice).
Easy Luke use it for stream flow prediction for water yield and fire season potential - currently the statisical forecastes and POAMA 2.4 aren't all that great at providing data for either.
0.01% attempt at a contribution. It's an improvement.
I never said the science was settled. That's why it's interesting.
Alas you're way off the mark - as you've confused AGW stuff with seasonal forecasting. Not even warm sp. It's a diversionary issue here.
If you noticed sp - I have suggested that the authors could (1) compare to other well known used every day systems (2) improve how farm users might receive the information (3) suggested a few other indices to try too - especially if they're going for all eastern Australia
Have you thought how you would use the system?
You should read what I wrote - you don't see these suggestions as a positive contribution?
Luke?
Debbie won't even speculate how she'd use a forecast?
I speculate all the time Luke.
Weather forecasts are more important in my industry than most.
If Jen and Abbot have opened the doors to something more useful then I'm happy for it to be added to what we already know.
I hope it does prove to be more useful.
Why are you being so defensive and critical?
I can't see anything inherently suspicious or counter productive here. They're not pretending that they have invented something entirely new. They are just looking at it in a different manner and also taking advantage of the ever growing source of reliable historical data that is available paricularly on the eastern seaboard.