fb

South Africa’s use of COVID-19 modelling has been deeply flawed. Here’s why

By: Seán Mfundza Muller, University of Johannesburg

President Ramaphosa’s government is easing the lockdown because of unsustainable economic costs.
Getty Images

Seán Mfundza Muller, University of Johannesburg

When President Cyril Ramaphosa announced the decision to implement an initial 21-day national lockdown in response to the threat posed by the COVID-19 pandemic, he referred to “modelling” on which the decision was based. A media report a few days later based on leaked information claimed that the government had been told that “a slow and inadequate response by government to the outbreak could result in anywhere between 87,900 and 351,000 deaths”. These estimates, the report said, were based on, respectively, population infection rates of 10% to 40%.

In late April, the chair of the health minister’s advisory committee sub-committee on public health referred to the early models used by the government as “back-of-the-envelope calculations”, saying they were “flawed and illogical and made wild assumptions”.

These assertions have been impossible to fully assess. This is because no official information on the modelling has ever been released – despite its apparently critical role.

A briefing by the chair of the health minister’s advisory committee in mid-April sketched some basic details of what the government’s health advisors believed about the likely peak and timing of the epidemic. But no details were given on expected infections, hospital admissions or deaths.

A spokesperson for the presidency said that government was withholding such numbers “to avoid panic”.

Finally, towards the end of May the health minister hosted an engagement between journalists and some of the modellers government was relying on. It then started releasing details of the models and projections.

The predictions of these models for an “optimistic scenario” are that the vast majority of the population will be infected, there will be a peak of 8 million infections in mid-August and in total there will be 40,000 deaths.

To understand the significance of these – and the previous numbers – it is useful to consider more broadly what models are and how they are being used in the current context.

What models are and how they are used

A theoretical model – whether in epidemiology, economics or even physics – is a simplified representation of how the modeller thinks the world works.

To produce estimates or forecasts of how things might play out in the real world, such models need to make assumptions about the strength of relationships between different variables. Those assumptions reflect some combination of the modeller’s beliefs, knowledge and available evidence.

To put it differently: modelling is sophisticated guesswork. Where models have been successfully used across different contexts and time periods we can have more confidence in their accuracy and reliability.

But models, especially outside sciences like physics, are almost always wrong to some degree. What matters for decision-making is that they are “right enough”. In the current situation, the difference between predicting 35,000 and 40,000 deaths probably won’t change policy decisions, but 5,000 or 500,000 instead of 40,000 might.

In the case of South Africa’s COVID-19 response, available information indicates that epidemiological models have played two main roles.

First, they have provided predictions of the possible scale of death and illness relative to health system capacity, as well as how this is expected to play out over time.

Second, they have been used to assess the success and effects of the government’s intervention strategies.

There are reasons to believe that there have been significant failures in both cases, in the modelling itself and especially in the way that it has been used.

In recent weeks, the government and its advisors have been keen to emphasise the uncertainty of the modelling predictions. From a methodological point of view, that is the responsible stance. But it’s too little too late.

Modelling COVID-19 is challenging in general, but there are at least four additional reasons to be cautious about our COVID-19 models.

Reasons for caution

First, certain key characteristics of SARS-CoV-2 remain unknown and the subject of debate among medical experts.

Second, unlike some countries, South Africa does not have detailed data on the dynamics of social interactions and the models presented so far do not use household survey data as a proxy. Nuanced questions therefore aren’t addressed. For example, most cases early on in the epidemic appeared to have been relatively wealthy travellers. But there was no way to model the consequences of domestic workers being exposed by their employers and thereby infecting others in their (poorer) communities. So the structure of South Africa’s models is high level and does not account for country-specific factors.

Third, the values for the parameters of the models (representing the strength of relationships between different factors) are being taken from evidence in other countries. They may not actually be the same in South Africa.

Finally, the unsystematic nature of aspects of the government’s approach to testing, such as through its community screening programme, makes it much harder to infer the effects of its interventions.

Unclear objectives

There is little reason to believe that government had anything other than good intentions. Nevertheless, the consequences of its lack of sophistication in using evidence and expertise may burden an entire generation of South Africans.

A major problem linked to the combination of excessive confidence and secrecy is that the government’s strategy was never clear: although it referred to “flattening the curve” it never stated what its specific objectives were. In the terms of the most influential modelling-based advice during the pandemic, was its strategy “suppression” or “mitigation”?

The government and its advisors have made much of the fact that the lockdown probably delayed the peak of the epidemic. But there is no evidence so far that this was worth the cost – since most of the population is expected to be infected anyway.

One key claim is that the lockdown bought the country time to prepare the health system.

The Imperial model defined the primary objective of “flattening the curve” as reducing ICU admissions below the number of critical care beds. On that dimension, the government’s own modellers predict a peak of 20,000 critical cases in the optimistic scenario and only about 4,000 ICU beds with little increase from the pre-lockdown numbers. By this definition, it has failed dismally.

There appears to have been more success with securing supplies of personal protective equipment, quarantine locations, overflow beds and some ventilators. But there is also little evidence that many of those small gains could not have been achieved without such a costly lockdown.

Given this, it is concerning that many academics and commentators have praised the success of government’s strategy. This has included the Academy of Sciences, which has asserted that “strong, science-based governmental leadership has saved many lives, for which South Africa can be thankful”.

This is entirely unsubstantiated.

First, the full toll of the epidemic will be experienced over time and so it is possible to have fewer deaths at the outset due to a policy intervention being exceeded by a larger number of deaths later because of the limitations of that same policy intervention.

Second, the only way to substantiate such claims would be to use models of different scenarios. But we’ve seen that the early models were not credible and the subsequent ones are subject to a great deal of uncertainty. It seems that the government and some of its advisors want to have the best of both worlds: they want to use dramatically incorrect predictions by early models to claim success of their interventions. This is misleading and does not meet the most basic standards by which academics in quantitative disciplines establish causal effects of policy interventions.

In an earlier article, I noted that “if the current lockdown fails to drastically curb transmission, which is possible, it would layer one disaster on another … the country may exhaust various resources by the time the potentially more dangerous winter period arrives”.

This appears to be the situation in which South Africa finds itself.The Conversation

Seán Mfundza Muller, Senior Lecturer in Economics, Research Associate at the Public and Environmental Economics Research Centre (PEERC) and Visiting Fellow at the Johannesburg Institute of Advanced Study (JIAS), University of Johannesburg

This article is republished from The Conversation under a Creative Commons license. Read the original article.

    Don’t miss out on the latest local news, interviews and competitions.

    Don’t miss out on the latest local news, interviews and competitions.

    Receive the latest news

    Subscribe To Our Newsletter

    THE KAYA 959 APP NOW AVAILABLE

    DOWNLOAD YOURS NOW

    Copyright Notice

    1. COPYRIGHT

    1.1 The contents of this Website, including but not limited to its compilation and arrangement, is the exclusive property of Kaya 959, alternatively the suppliers of content to Kaya 959, and accordingly remain protected by South African and International Copyright and Trademark laws.

    1.2 Any person accessing this Website, may not, save for downloading one copy for their personal computers and solely for their private and non-commercial use :

    1.2.1 Copy, disseminate, distribute, advertise, publish, adapt, modify or in any way reproduce the contents of this website for commercial purposes, unless this notice and any disclaimer attached thereto is published in its entirety, or unless the permission of Kaya 959 is obtained in writing.

    Privacy Policy

    THIS PRIVACY STATEMENT FORMS PART OF KAYA 959’S TERMS OF USE POLICY. IF YOU DO NOT AGREE WITH ANY TERM OF THIS PRIVACY STATEMENT, YOU MUST CEASE YOUR ACCESS OF THIS WEBSITE IMMEDIATELY. 

    POPIA ActTo promote the protection of personal information processed by public and private bodies; to introduce certain conditions so as to establish minimum requirements for the processing of personal information; to provide for the establishment of an Information Regulator to exercise certain powers and to perform certain duties and functions in terms of this Act and the Promotion of Access to Information Act, 2000; to provide for the issuing of codes of conduct; to provide for the rights of persons regarding unsolicited electronic communications and automated decision making; to regulate the flow of personal information across the borders of the Republic; and to provide for matters connected therewith.

    RECOGNISING THAT—

    • section 14 of the Constitution of the Republic of South Africa, 1996, provides that everyone has the right to privacy;
    • the right to privacy includes a right to protection against the unlawful collection, retention, dissemination and use of personal information;
    • the State must respect, protect, promote and fulfil the rights in the Bill of Rights;

    AND BEARING IN MIND THAT—

    • consonant with the constitutional values of democracy and openness, the need for economic and social progress, within the framework of the information society, requires the removal of unnecessary impediments to the free flow of information, including personal information;

    AND IN ORDER TO—

    • regulate, in harmony with international standards, the processing of personal information by public and private bodies in a manner that gives effect to the right to privacy subject to justifiable limitations that are aimed at protecting other rights and important interests,
    1. Definitions and Interpretation

    1.1.“Personal Information” means information relating to an identifiable, living, natural person and where it is applicable, identifiable, existing juristic person, including all information as defined in the Protection of Personal Information Act 4 of 2013. 

    1.2  Parliament assented to POPIA on 19 November 2013. The commencement date of section 1Part A of Chapter 5section 112 and section 113 was 11 April 2014. The commencement date of the other sections was 1 July 2020 (with the exception of section 110 and 114(4). The President of South Africa has proclaimed the POPI commencement date to be 1 July 2020.

     
    1.3. “Processing” means the creation, generation, communication, storage, destruction of personal information as more fully defined in the Protection of Personal Information Act 4 of 2013.  

    1.4. “You” or the “user” means any person who accesses and browses this website for any purpose. 

    1.4. “Website” means the website of the KAYA 959 at URL www.kaya959.co.za or such other URL as KAYA 959 may choose from time to time.   

    1. Status and Amendments

    2.1. KAYA 959 respects your privacy. This privacy policy statement sets out KAYA 959’s information gathering and dissemination practices in respect of the Website. 

    2.2. This Privacy Policy governs the processing of personal information provided to KAYA 959 through your use of the Website. 

    2.3. Please note that, due to legal and other developments, KAYA 959 may amend these terms and conditions from time to time.  

    1. Processing of Personal Information

    3.1. By providing your personal information to KAYA 959 you acknowledge that it has been collected directly from you and consent to its processing by KAYA 959. 

    3.2. Where you submit Personal Information (such as name, address, telephone number and email address) via the website (e.g. through completing any online form) the following principles are observed in the processing of that information: 

    3.2.1. KAYA 959 will only collect personal information for a purpose consistent with the purpose for which it is required. The specific purpose for which information is 
    collected will be apparent from the context in which it is requested. 

    3.2.2. KAYA 959 will only process personal information in a manner that is adequate, relevant and not excessive in the context of the purpose for which it is processed. 

    3.2.3. Personal information will only be processed for a purpose compatible with that for which it was collected, unless you have agreed to an alternative purpose in writing or KAYA 959 is permitted in terms of national legislation of general application dealing primarily with the protection of personal information. 

    3.2.4. KAYA 959 will keep records of all personal Information collected and the specific purpose for which it was collected for a period of 1 (one) year from the date on which it was last used. 

    3.2.5. KAYA 959 will not disclose any personal information relating to you to any third party unless your prior written agreement is obtained or KAYA 959 is required to do so by law. 

    3.2.6. If personal information is released with your consent KAYA 959 will retain a record of the information released, the third party to which it was released, the reason for the release and the date of release, for a period of 1 (one) year from the date on which it was last used. 

    3.2.7. KAYA 959 will destroy or delete any personal information that is no longer needed by KAYA 959 for the purpose it was initially collected, or subsequently processed. 

    3.3. Note that, as permitted by the Electronic Communications and Transactions Act 25 of 2002, KAYA 959 may use personal information collected to compile profiles for statistical purposes. No information contained in the profiles or statistics will be able to be linked to any specific user.    

    1. Collection of anonymous data

    4.1. KAYA 959 may use standard technology to collect information about the use of this website. This technology is not able to identify individual users but simply allows KAYA 959 to collect statistics. 

    4.2. KAYA 959 may utilise temporary or session cookies to keep track of users’ browsing habits. A cookie is a small file that is placed on your hard drive in order to keep a record of your interaction with this website and facilitate user convenience. 

    4.2.1. Cookies by themselves will not be used to identify users personally but may be used to compile identified statistics relating to use of services offered or to provide KAYA 959 with feedback on the performance of this website. 

    4.2.2. The following classes of information may be collected in respect of users who have enabled cookies: 

    4.2.2.1. The browser software used; 

    4.2.2.2. IP address; 

    4.2.2.3. Date and time of activities while visiting the website; 

    4.2.2.4. URLs of internal pages visited; and 

    4.2.2.5. referrers. 

    4.3. If you do not wish cookies to be employed to customize your interaction with this website it is possible to alter the manner in which your browser handles cookies. Please note that, if this is done, certain services on this website may not be available. 

    1. Security

    5.1. KAYA 959 takes reasonable measures to ensure the security and integrity of information submitted to or collected by this website, but cannot under any circumstances be held liable for any loss or other damage sustained by you as a result of unlawful access to or dissemination of any personal information by a third party. 

    1. Links to other websites

    6.1. KAYA 959 has no control over and accepts no responsibility for the privacy practices of any third party websites to which hyperlinks may have been provided and KAYA 959 strongly recommends that you review the privacy policy of any website you visit before using it further. 

    1. Queries

    7.1. If you have any queries about this privacy policy please contact us by emailing [email protected] 

    Competition Terms and Conditions

    • The competitions are open to all persons over the age of 18 years; except directors, partners, employees, agents, service providers, and consultants of Kaya 959, the sponsor and all its subsidiaries and its holding company, if any, as well as all spouses, life partners, parents, children, siblings, business partners and associates of such persons.

    • The outcome of the competition is subject to the decision of the judge/presenter, whose decision is final and no negotiation will be entered into thereafter. Neither Kaya 959, sponsors nor their agents will be held responsible or answerable to any dispute arising from the competition or prize awards.

    • Participants/listeners enter or take part in competitions at their own risk and Kaya 959 bears no responsibility for any loss, damage or harm suffered as a result of participation in any of Kaya 959 competition.

    • One listener is entitled to winning one prize in a period of 3 months. Kaya 959 reserves the right not to award a prize if the listener has won a prize prior during the 3 month window period. This also applies to listeners who provide family or friend’s contact details.

    • Kaya 959 reserves the right to redistribute all unclaimed prizes if not claimed after 3 months after being given away On Air or on the website.

    • Prizes are not transferable and may not be exchanged for cash.

    • Finalists will forfeit their participation in the competition if they fail to attend the draws.

    • The competition will run during the period advertised on Kaya 959; entries received outside of the competition period will not be considered for the competition draw.

    • Kaya 959 and their sponsors reserve the right to cancel, modify or amend the competition at any time if deemed necessary in their opinion, or if circumstances arise outside of their control.

    • By entering the competition, entrants agree to accept these rules and to be bound by them.