Forecasting accuracy called in to question (again)

Forecasting accuracy has been called into question again in 2020.  Economists, climate scientists, and epidemiologists (and probably others) have been tarred with the same brush and their expert modelling described as having a history of failure.

vanuatu 194asf

The period since mid-2018 has been a challenging one for forecasting accuracy and a review is timely.  Volume 1 of my book “Forecasting: the essential skills” was written over the period 2013 to 2018.  I found many flaws in the practice of economic forecasting and some holes in the models for forecasting weather and climate.  I found that the accuracy of political polling was in decline.  There are, of course, some success stories and also improvements in weather forecasting.

Through case studies and reviews of performance I have identified a set of skills which are essential to improving forecasting accuracy.

Volume 2 is now in preparation.  Two chapters are now available: a review of recent criticism and an evaluation of forecasting accuracy as perceived by the court of public opinion.  A September 2020 survey of the Australian general public measured perceptions of accuracy in the fields of economic, weather, and climate forecasting.  Perceptions are reasonably consistent with the reviews I have conducted and not as bad as suggested by the most strident critics – whose views are reflected in a small minority of the general public.

Future chapters will review the poor performance of forecasts over the period 2018 to 2020 and identify reasons for the shortcomings in economic and political forecasts in Australia.  There will also be a chapter on the performance of demographic forecasting upon which economic forecasts and infrastructure plans depend.

Volume 1 and the first two chapters of Volume 2 are available at foreseechange.com.au.  This is important information for both decision makers who rely on forecasts and forecasters who wish to improve their skills.
Charlie Nelson

 

Futurescape: the outlook for Australia in 2021

cnwordcloud

 

It is just over six months since the Pandemic started and resulting restrictions plunged Australia and much of the world into economic recession.  Government income support is starting to taper off, risking an extended period of economic growth.  Second waves of the Pandemic have emerged in many countries, so Australia’s borders are likely to be closed until well into 2021.

It is time to look forward, to make an assessment of what the next year holds for Australia.  In order to do that we have consulted with the experts – the people at the centre of the economy and the Pandemic.  The general public.  They know about employment prospects, their financial outlook, and how they plan to use their money.  Their expectations about the Pandemic and other factors such as property prices influence their behaviour.

We have been tapping into the collective wisdom of the general public, the Wisdom of the Masses, for over 15 years and found it to be predictive and insightful.

Our September 2020 survey suggests that there will be no economic “snap back”, although there will be some growth opportunities.  The detailed evidence is contained in our report.

Charlie Nelson

 

A boom in first home buyers in Australia

house4sale

A unique alignment of favourable conditions has created a boom in first home buyers in Australia.  Based on official data and proprietary tracking surveys, this boom is predicted to continue well into 2021.  This will help to lift Australia’s economy out of recession.

Details are available in a foreseechange report.

 

Charlie Nelson

planets042002

 

Criticism of forecasting accuracy

Adam Creighton has criticised the accuracy of modelling used in forecasting, conflating the performance of pandemic, climate, and economic forecasting. He is an economics editor of The Australian newspaper and his article on June 17 2020 was headed “Coronavirus: Inflated pandemic estimates weaken climate forecasts”.

Here are some excerpts from his article.

“so spectacularly bad was expert modelling of the spread and lethality of the coronavirus, faith in all modelling must surely suffer”.

“Why trust the experts to forecast the climate decades into the future when they were so wrong about a disease related to the common cold?”

“Climate modelling was struggling even before the pandemic, given the planet has warmed about half as much as forecast by the first Intergovernmental Panel on Climate Change report in back 1990”.

“It’s remarkable we put so much faith in expert models, given their history of failure. The Club of Rome in 1972 notoriously forecast that growth would collapse as the world’s resources ran out, ignoring human ingenuity and the shale revolution”.

“Financial models failed to account for — indeed they probably facilitated — the global financial crisis. And as almost every utterance by a central bank governor since has reminded us, economists struggle to know what happened last month, let alone forecast the impact of a policy change tomorrow”.

He has cited modelling from the 1970’s, the 1990’s, a failure to predict the global financial crisis in 2008-09, and modelling of the Pandemic in 2020 – which is still spreading and claiming lives. He identifies only 4 “failures” in the last 50 years!

There are many forecasting errors throughout recent decades, which lead one to question why he selected these particular four to demonise.

Many other forecasting failures are documented elsewhere in my book Forecasting: the essential skills. One standout is political leaders in USA, UK, Australia, and Spain predicting that Iraq had weapons of mass destruction, despite UN arms inspectors on the ground never having found any. The experts got it right and the politicians got it wrong (perhaps wantonly). The cost of this failed prediction was thousands of US armed forces lives, tens of thousands of civilian lives, and arguably a power vacuum filled by ISIS. The financial cost was in the trillions of dollars, as documented by Joseph Stiglitz and Linda Bilmes in the 2008 book “The three trillion dollar war”.

The Australian newspaper’s proprietor, Rupert Murdoch, predicted that the invasion of Iraq would result in the price of oil falling from over $US30 a barrel to $US20 a barrel. He said that the whole world would benefit from cheaper oil. This was presumably a judgemental forecast which was at least as inaccurate as any modelling. In the event, the price never fell below $28 and was over $34 in February2004. The price soared to $74 by mid-2006 and to $134 by mid-2008.

In fact the end of cheap oil was predicted, by modelling. In 1956, M. King Hubbert, a geologist with Shell Oil, correctly predicted that the rate of oil production from the lower 48 American states would peak in 1969. His prediction was based on the observation that oil production from any large region starts to fall when about half the crude is depleted. The output from a region as a whole follows a bell shaped curve. Hubbert extrapolated this to output from whole countries. Campbell and Laherrere extrapolated this pattern to world production (“The End of Cheap Oil”, Scientific American, March 1998). Their modelling was very accurate! More recently, shale oil has increased supply which has contributed to a fall in prices – but not sustainably to $20 a barrel.

There is a common element to the four cases cited by Creighton – perceived damage to economic growth if the forecasts are acted on, in the cases of the Club of Rome Limits to Growth, Climate change, and the COVID-19 pandemic. The reason that the failure of economists to predict the GFC is included may be that economic recession in the USA was not avoided by appropriate policy which could have implemented if recession was predicted. This would have been a self-defeating prophecy!

In the case of the Club of Rome limits to growth, the dire predictions did not occur (yet) because of action, such as the green revolution, in response to the predictions. A self-defeating prophecy.

Similarly in the case of climate change predictions: some actions in response have been taken. Not enough action to slow the rate of growth of carbon dioxide in the atmosphere, but enough to stop an acceleration of it.

At least Creighton acknowledges there has been global warming – half as much as predicted is his claim – so the climate model predictions were directionally accurate. The models used by climate scientists have improved since the 1990’s and will continue to improve in the future as more is learned about feedback mechanisms and other climate effects.

The predictions of epidemiologists concerning COVID-19 also resulted in action to reduce the rate of spread, which did have severe economic consequences. This pandemic has a long way to run yet, with global cases and deaths still increasing rapidly. Hopefully their predictions were also a self-defeating prophecy.

The failure of the vast majority of economic forecasters to predict the global financial crisis was consistent with their track record – they fail to predict turning points.

In the case of the impact of the COVID-19 pandemic in Australia, I was shocked at the scenarios. Australia’s Deputy Chief Medical Officer Paul Kelly said that the number of infections would be in the range 20% to 60% of the population. Deaths would range from 50,000 to 150,000 (The Age 17 March 2020). Australian economist Warwick McKibbin estimated that almost 100,000 Australians could die from COVID-19 (the range was 21,000 to 96,000). This is based on modelling seven different scenarios, building on the experience of the SARS outbreak in 2003 and Spanish Flu in 1918 (Australian Financial Review, 3 March 2020). The media reporting did not mention the specific assumptions used by the experts.

As at 4 August 2020, the number of confirmed infections in Australia was 18,730 and the number of deaths was 232. The accuracy of the forecasts was abysmal. To some degree, this is attributable to the policy response – a shutdown of travel, workplaces, and schools combined with social distancing. It might be argued that without the dire forecasts, politicians may never have agreed to the policies which were effective but which were known to cause an economic recession. Self-defeating prophecy.

There is, however, a second wave occurring concentrated in Victoria. The number of active cases has surged from 129 on 14 June to 6,755 on 4 August.

The actual number of infections is likely to be much higher than the confirmed number – one estimate made on the basis of analysis of blood test from pregnant women and pathology test for other purposes puts the figure at 500,000 (Australian Financial Review, July 22 2020.  If this figure, representing 2% of the population, is accurate it is far lower than the 20% to 60% predicted.

Had Australia experienced the same death rate as Sweden, for example, there would have been 15,000 deaths instead of 232.  Future analysis will ascertain the relative impacts of government policies, population density, inherent population characteristics such as vitamin D levels, and other factors on death rates by country, but Australia does appear to have fared exceptionally well so far.

Suggesting that poor forecasting performance in the case of the pandemic experts automatically casts doubt on the forecasting performance of climate experts is misguided.  The forces at play are very different.

In the case of the pandemic, the characteristics of the virus, interactions with the human body, and the policy response are the key drivers of the outcome.  Early on in the pandemic these were all unknown and so scenarios were needed to crystallise the range of outcomes.  In the case of Australia, a fourth, lower, scenario was not provided initially.  Later, more detailed, modelling guided the policy response of social distancing which has proven to be effective.

In the case of climate change, it is the behaviour of carbon dioxide molecules and the responses of the atmosphere and oceans which is modelled.  The behaviour of carbon dioxide molecules, and those of other greenhouse gasses, is well known and proven empirically.  There are uncertainties in the responses of the atmosphere and oceans – for example, some types of cloud reflect incoming sunlight into space, while other types prevent heat radiation from Earth from escaping into space.  To handle these uncertainties, and the range of responses in terms of reducing emissions, scenarios are constructed to span the range of likely outcome.

I am no staunch defender of current practices in modelling.  Throughout my book I have criticised forecasts based on modelling.   My aim, however, is to contribute to improved modelling and forecasting accuracy.  Creighton, on the other hand demonises modelling without offering any solution to the problem of forecasting inaccuracy.

Creighton did not question why the pandemic itself was not predicted.  Pandemics are rare, but do they occur randomly?  Or is there something about our interaction with nature which is making coronavirus outbreaks more common?

Previous coronavirus outbreaks have included MERS (Middle East respiratory syndrome) in 2012 and SARS (Severe acute respiratory syndrome) in 2003.  While it has been known since the 1960’s that the common cold is caused by a coronavirus, there have been no recorded deadly coronavirus outbreaks until SARS.

While influenza pandemics are rare, are coronavirus pandemics becoming more frequent given three within 20 years?  If so, why?  Could it be due to the clearing of forests, for example.

We need more experts analysing this problem, not a pandemic of mistrust in experts and modelling based on misunderstanding.

Creighton’s article, and similar opinions emanating from some prominent politicians and conservative commentators, present a challenge to expert professionals involved in modelling and forecasting.  The challenge is twofold: improve accuracy and build trust in the court of public opinion.

Charlie Nelson

 
Charlie Nelson

 

A year of shocking forecasting inaccuracy

clouds0315 026 50

There have been huge forecasting errors in 2020.  While this year may be an extreme case, we are experiencing very poor accuracy about one year in ten.  Even in other years, forecasting accuracy can be poor.  How then should business and government plan?

Forecasts for the number of deaths attributable to COVID-19 were very pessimistic.

Australia’s Deputy Chief Medical Officer Paul Kelly said that the number of infections would be in the range 20% to 60% of the population.  Deaths would range from 50,000 to 150,000 (The Age 17 March 2020).

Australian economist Warwick McKibbin  estimated that almost 100,000 Australians could die from COVID-19 (the range is 21,000 to 96,000).  This is based on modelling seven different scenarios, building on the experience of the SARS outbreak in 2003 and Spanish Flu in 1918 (Australian Financial Review, 3 March 2020).

As at 16 April, the number of deaths attributable to COVID-19 is 63 and the daily number of new deaths is declining.  The number of new confirmed cases is declining, despite increased testing.  On current trends, and assuming continued adherence to restrictions such as social distancing, there will be no active cases by the end of May.

The lowest of the predicted number of deaths, 21,000, is likely to be at least 210 times too high.  The highest of the predicted number of deaths, 150,000, is likely to be 1,500 times too high

These are dreadfully inaccurate forecasts.  They may not have allowed for how Australia would respond to the pandemic – if so, they should have stated this assumption (or the reporters should have asked about the assumptions).

Economic growth forecasts were not much better.

On February 6 2020, the Reserve Bank of Australia’s forecasts were for GDP growth of 2.75% and an unemployment rate of 5.0%.  The 90% confidence interval on their GDP growth forecast included +5%, but not 0% or lower!

A survey of economic forecasters by The Age, published on 1 February 2020, produced an average predicted GDP growth of 2.18% (range 1.0 to 3.3) and an average predicted unemployment rate of  5.23% (range  4.8 to 6.4).

On March 18, the governor of the Reserve Bank said “While it was not possible to provide an updated set of forecasts for the economy given the fluidity of the situation, it was likely that Australia would experience a very material contraction in economic activity, which would spread across the March and June quarters and potentially longer. The size of the fall in economic activity would depend on the extent of the social distancing requirements, and potential lockdowns, put in place to contain the virus”.

Australia will almost certainly be in recession and the unemployment rate is likely to exceed 10%.

We only have to think back to 2008 and 2009 for another example of very poor economic forecasting.  In 2008, the Reserve Bank was lifting interest rates until March, seemingly oblivious to the imminent global financial crisis.  By September 2008 they were cutting rates as fast as they could.  In May 2009, the federal budget predicted a recession.  So too did the Reserve Bank and 99% of economic forecasters.  The recession never arrived!

Economic forecasting seems to experience very bad economic forecasting errors at least once every decade.

With frequent large forecasting errors and an increasingly turbulent world, how can planners prepare for the future?

They could look at risk assessments, such as that produced by the World Economic Forum (WEF) at the start of each year – it is an assessment of long term risks.  At the start of 2020, the WEF Global Risks Report, prepared in partnership with Marsh & McLennan and Zurich Insurance Group, rated a set of 30 risks on the basis of impact and likelihood.  Infectious disease was rated 27th out of 30 on the likelihood scale and 9th on impact.  No doubt many planners would not include infectious disease at the centre of their plans.  The WEF most likely risk was extreme weather followed by climate action failure.  The highest rated risk in terms of impact was climate action failure.

Within weeks of the release of this report, an infectious disease has had devastating impacts on lives and economies with long term repercussions.  Events which are considered unlikely do happen!  Accordingly, there is a need for a well-rehearsed plan ready for implementation to minimise the impacts of such events.

Scenarios are a valuable input to developing robust plans in the face of uncertainty.  Scenarios are plausible futures, based on a range of assumptions.  While scenarios can map out an envelope of future possibilities, they also provide a basis for developing plans which are resilient across all plausible futures.

A frequent refrain from some business leaders is a call for certainty, especially from governments.  There is no such thing as certainty, as shown by the global financial crisis and the COVID-19 pandemic!  Even government policy must change in the face of uncertainty

Tracking surveys of Australian consumer expectations about the year ahead, conducted by foreseechange, showed that in November 2019 the average estimated likelihood of a major disease outbreak in the year ahead was 42% (the Wisdom of the Masses).  This is a perceived risk that people would expect governments to manage.  The Wisdom of the Masses also indicated economic downturn and a rise in unemployment.

The general public has information that is important in the development of scenarios.  The Wisdom of the Masses should complement information from experts in the development of scenarios.

Future posts will provide more information on scenarios and the Wisdom of the Masses.

Charlie Nelson

 

 

A change in the expectations and concerns of the Australian general public

hhcev0919

There has been an abrupt shift in consumer psychology which has implications for government policy, the Reserve Bank, and business decision makers.  Economic pessimism has increased and the level of belief in climate change has lifted.

These changes seem to be at odds with the federal government’s “sticking to our policy” mantra.  The heightened expectation of a rise in unemployment is inconsistent with the Reserve Bank’s hope for a decline in the unemployment rate to 4.5%.  Other shifts in consumer psychology are more positive and provide an opportunity to boost consumer spending growth.

The level of belief in imminent climate change in late 2019 is the second-highest recorded and is slightly higher than in 2007, when John Howard lost his seat in parliament and his government lost office.

The federal government and many businesses need to take more decisive action on climate change to satisfy voter (and customer) expectations.

For several years, prominent Australian economist Ross Garnaut has warned of “the great Australian complacency” which has significantly slowed Australia’s economic growth rate.  That description clearly also applies to the issue of climate change.

As the new decade dawns, more Australians are experiencing the costs of these policy complacencies.

Our tracking survey update was in field in November and early December 2019.

A summary report is available at foreseechange.com.au.

Charlie Nelson

 

The collapse of the Australian consumer: a remedy

bourkest

Symptoms of a collapse in consumer spending include four successive quarters of declining per capita retail sales volume, to the September 2019 quarter.  Declining new vehicle sales to private buyers for nearly two years to October provides confirmation of the breadth of the slump.

The recent cuts in interest rates, boosted personal income tax returns, and lower income tax rates have had no measurable impact on consumer spending so far.

Causes of this damaging slump are easily identified: low income growth and a fall in residential property prices.  There are, however, several causes of low income growth and these are identified in this report.

My report identifies the characteristics of those consumers who are most willing to spend.  These have been deserted by recent monetary and fiscal policy and should be the targets of immediate stimulus.  The forms this remedy should take are described and an implementation strategy is outlined.

My report uses proprietary consumer tracking data which measures consumer willingness to spend by demographic.

Industries exposed to consumer spending have an interest in understanding how to stimulate spending and recommending our strategy proposals to the Treasurer and the Governor of the Reserve Bank before they make any more mistakes.  Industries which would benefit include retailers and their suppliers, service industries, media companies, and media agencies.

My report is available at www.foreseechange.com.au

 

What will Australian consumers do with their tax cuts?

gw 028 50

Most will not be spent initially, only 25% to 30%.  This is based on modelling past data and tracking surveys, conducted since 2003 by foreseechange, of how consumers allocate discretionary funds.

More will be used for debt repayment.  When consumers repay debt, some may become more willing to spend subsequently.  But very low income growth in Australia will limit the ability to spend.

Even more will be used for saving.  Much of this is precautionary or for other long-term saving, such as retirement.  Quite a lot will be saved by young adults towards the purchase of residential property.

One of the main purposes for saving is for a holiday, mostly overseas.  Tax cuts put towards saving for a holiday will eventually be spent, but much of it will be spent overseas.

The overall impact on consumer spending may be just noticeable and may hold the economy up a little for the next 6 to 12 months.  Beyond then, we will be back to weak consumer spending growth and a weak economy unless we have policies to significantly boost productivity.

The glass may appear to be more than half-full for a short time, but the content will soon be consumed.

Details are available at foreseechange.com.au.

Charlie Nelson

 


style="display:block"
data-ad-client="ca-pub-7514722518174380"
data-ad-slot="1379185305"
data-ad-format="auto">

 

How accurate are betting markets?

They were horribly wrong on the 2019 Australian federal election outcome.  Now I have come across an article from my files concerning Sportsbet’s odds on Australian interest rates in 2018.  This was not a good guide!

In the Australian Financial Review of 22 September 2017, Philip Baker noted that Sportsbet had two interest rate increases in 2018 as the $2.80 favourite, while just one increase was priced at $2.88.  No change was paying $3.60.  In fact, interest rates did not change in 2018 and the Reserve Bank cut interest rates in June 2019.

It is thought that betting markets are accurate because people with inside knowledge gamble and win from the great unwashed who also gamble.  But insiders are not always right!

More in my book “Forecasting: the essential skills

Charlie Nelson

 

 

Australia’s economy: official forecasts catch up with reality


black swan 50

In my August 2018 post, I noted a suite of indicators that Australia’s economy was slowing significantly.  It is only in May 2019 that official forecasts have been revised downwards to reflect he long-evident facts.

The Reserve Bank of Australia have announced their downwards revision on the eve of Australia’s federal election on 18 May 2019.

Those who read my August 2018 post will have had plenty of time to prepare for this slowdown.

My latest report on Australia’s economic outlook is sobering, but essential, reading.

Charlie Nelson