HDR Launches New Customizable COVID-19 Coronavirus Operational Risk Report

 customizable coronavirus (covid-19) operational risk REPORT

Providing daily, weekly, or Twice-weekly Probabilistic Forecasts tailored to inform your operational decisions for mitigating a rapidly-changing threat

,Hubbard Research

STAY AHEAD OF THE CORONAVIRUS CURVE: THE GROWING IMPACT ON YOUR ORGANIZATION’S OPERATIONAL RISK

The COVID-19 coronavirus pandemic that is currently spreading across the globe threatens not just the health of every person, but the health of every organization. From lower productivity to higher insurance rates, slower and more irregular supply chains, and other potential disruptions, the threat of critical disruption to key operations is real. And as the number of cases grows, so does the operational risk for an organization.

This is the coronavirus curve, and an organization needs to know where the curve is so they can stay ahead of it.

With our innovative Coronavirus Operational Risk Report, we provide the means for organizations to:

Z
Understand and quantify specifically how the pandemic poses a threat
Z
Forecast the spread of coronavirus cases beyond what is currently available to the public
Z
Provide timely interpretation of breaking news as it pertains to operational risk
Z
Make strategic decisions about mitigating operational risk

Reports can be customized to your organization’s specific needs, including: timeframe, delivery method, scope, subject matter, and more. The insight we deliver can be easily incorporated into your current decision-making process, even if you don’t already use statistical modeling. And, we don’t use risk matrices, “high-medium-low” ratings, weighted scoring, heat maps, or other pseudo-quantitative methods that introduce more error than they eliminate.

the DELIVERABLE

how our customizable reports deliver the insight you need

HDR has conducted probabilistic analysis on the spread of this virus using our industry-leading Applied Information Economics (AIE) methodology that incorporates probabilistic modeling to create actionable projections. The projections so far have tracked well with subsequently observed outcomes.

For example, we forecasted a 90% confidence interval for the number of total cases on the quarantined Diamond Princess cruise ship prior to the tests being conducted.  The figure below shows the forecast made 12 days before the tests were given and the outcome after test results were known. Note that the actual result was well within the forecasted range, and this result was repeated on multiple forecasts.

Reports are customizable by:

  • Frequency: Receive updates daily, weekly, or twice-weekly
  • Location: Choose the domestic or international locations you want to monitor and forecast
  • Timeframe: Determine how far out from the present you want to forecast (i.e. 7 days, 14 days, 30 days)
  • Delivery method: Tailor the content and method of the deliverables to what your decision-makers prefer
  • Optional consulting: Work with us to develop a more complete probabilistic model that can measure more variables and make additional forecasts to further quantify and mitigate operational risk

The result: actionable intelligence beyond what organizations have now through more accurate and more timely forecasts.

Only measuring the right things in the right way, and building a decision model to help inform better decisions, can move your organization ahead of the coronavirus curve.

Contact Us

schedule your consultation

15 + 14 =

How to Measure Anything Book Sales Have Exceeded the 100,000 Mark

How to Measure Anything Book Sales Have Exceeded the 100,000 Mark

Over 12 years ago, on August 3, 2007, the first edition of How to Measure Anything was published.  Since then, Doug Hubbard has written two more editions that now have sold over 100,000 copies in that series alone – copies purchased by customers ranging from university professors, scientists, and government officials to Fortune 500 executives, Silicon Valley visionaries, and global leaders in virtually every industry under the sun.

The premise of How to Measure Anything remains true all these years later: that anything can be measured, and if you can measure it – and it matters – then you can make better decisions. These measurement challenges that we’ve overcome over the past 12 years have included:

  • the most likely rate of infection for COVID-19
  • drought resilience in the Horn of Africa
  • the value of “innovation” and “organizational development”
  • the risk of a mine flooding in Canada
  • the value of roads and schools in Haiti
  • the risk and return of developing drugs, medical devices and artificial organs
  • the value and risks of new businesses
  • the value of restoring the Kubuqi Desert in Inner Mongolia
  • the value and risks of upgrading a major electrical grid
  • near and long-term reputation damage from data breaches

In addition to learning how to better understand critical measurement concepts by reading How to Measure Anything, thousand of customers from all over the world have obtained proven, industry-leading quantitative training via our series of online webinars and in-person seminars. This includes the ever-growing line of How to Measure Anything-inspired webinars in key areas like:

  • Cybersecurity
  • Project Management
  • Innovation
  • Public Policy and Social Impact
  • Risk Management

More are on the way. (Do you have an area you’d like us to cover in our training? Tell us here.)

Through this series, they’ve become more calibrated and have learned how to calibrate others in their organizations. They’ve learned how to build their own Monte Carlo simulations in Excel from scratch – no special software needed. They’ve learned how to move away from pseudo-quantitative methods like risk matrices and heat maps in cybersecurity. And they’ve learned how to figure out what’s worth measuring by performing value of information calculations. These are just a few examples of the practical skills and takeaways our customers have received since How to Measure Anything was first published.

We’re also planning on taking quantitative training to the next level in an exciting and ground-breaking development that will be announced later this year.

The era of measurement – the pursuit of modern science – began in 1687 with Newton’s Principia. Finding better ways to create better measurements is the logical next step in the evolution we’ve seen over the past four centuries, and How to Measure Anything will continue to do its part as the world continues further down an uncertain path. Thanks for being a part of the journey. We look forward to what’s to come and hope you continue on with us.

 

Learn how to start measuring variables the right way – and create better outcomes – with our two-hour Introduction to Applied Information Economics: The Need for Better Measurements webinar. $100 – limited seating.

 

RECOMMENDED READING:

TROJAN HORSE: HOW A PHENOMENON CALLED MEASUREMENT INVERSION CAN MASSIVELY COST YOUR COMPANY

Measuring the wrong variables is a Trojan horse that infiltrates virtually every organization. This phenomenon has a real cost that can be measured – and avoided.

TWO WAYS YOU CAN USE SMALL SAMPLE SIZES TO MEASURE ANYTHING

A measurement isn’t useless if the sample size is small. You can actually use small sample sizes to learn something useful about anything – and use that insight to make better decisions.

Five Data Points Can Clinch a Business Case

Any decision can be made better through better measurements – and as these three examples show, just five data points can tip the scales in a business decision.

 

Diamond Princess: A Data Science Solution To the COVID-19 Test Kit Shortage

Rarely does an opportunity present itself where statistics can so immediately and with so much impact solve a pressing real world problem. Shortly before publishing this post, we reached out and talked to the Japanese Ministry of Health and they indicated that they planned to test everybody on board. Whether they go ahead with that plan or not, the point of the article remains valid – doing a random sample (it could have been done 6 days ago) would be an efficient (both in terms of time and the limitation on test kits) way to reduce uncertainty on the total population of positive COVID-19 cases.

Situation: There are 3,600 passengers and crew aboard the Diamond Princess cruise ship docked off the coast of Yokohama, Japan. A significant percentage of them are infected with coronavirus, but it is unknown how many. Currently, it would be difficult to quickly test all 3,600 passengers and crew for the coronavirus, especially since there is limited availability of test kits, according to Japanese authorities.

Solution: To get a much better estimate of total cases, only a fraction of the passengers need to be tested at first. This could lead to important time savings in crucial decisions facing the Japanese Ministry of Health and the Diamond Princess cruise ship.

Despite the total number of 3600 passengers and crew, you only need to test a couple hundred to narrow the range meaningfully on the number of total cases; the results of those tests could have immediate and important consequences. A crucial tenet of Applied Information Economics (AIE) is that measurements are generally more useful (or have a higher ROI) when they are connected to a decision. The important decisions in this situation aren’t going to be based so much on a small difference (e.g. whether there are 200 or 220 additional cases). The important decisions will be made based on whether there are 50 or 500 additional cases. This level of uncertainty reduction could be achieved quickly with a smaller random sample of the whole population.

A crucial point is that these tests need to be selected randomly – thus far the samples have been taken from suspected cases (people with symptoms or contact with known cases). However, a random selection of 200 would allow us to apply an inverse beta distribution to the results and produce a relatively tight 90% confidence interval on total cases in the remaining 3,400 people.

Recommendation 1: Randomly test 70 staff and 130 passengers, and then use a beta distribution to project the total in the remaining population. The reason to break out staff and passengers is that they are likely to have different proportions of infection. Because the staff on the Diamond Princess continues to commingle (i.e. eat and berth together), their rate of infection is likely higher.

Recommendation 2: Once the new range for total infections is obtained with the random sample, prepare local hospitals for case load. Among other things, the ministry will know if there are enough isolation units available, and if not make other arrangements (i.e. dedicate one hospital to COVID-19 patients).

Recommendation 3: If the infection level of the staff is greater than a threshold (15-20%), then other arrangements should be made for serving passengers. Identifying the healthy staff would become a priority in this case.

At this moment, there is a wide range for the 90% confidence interval of current level of infections (https://hubbardresearch.com/cruise-ship-coronavirus-infected-passengers-in-hundreds/). People on board are worried about if they are sick or not, regardless of whether they have symptoms. If test results come back that indicate that a very small percentage of asymptomatic people tested positive, this would reassure those on board who are asymptomatic. Additionally, the percentage of asymptomatic cases that test positive could have broad global implications for detection and planning for we could back out percentage of cases that test positive while asymptomatic.

This is a unique opportunity to understand the disease that could help the effort to contain it worldwide!

 

Learn how to start measuring variables the right way – and create better outcomes – with our two-hour Introduction to Applied Information Economics: The Need for Better Measurements webinar. $100 – limited seating.

 

RECOMMENDED READING:

THE TOTAL NUMBER OF CORONAVIRUS-INFECTED CRUISE SHIP PASSENGERS WILL BE IN THE HUNDREDS [UPDATE]

Authorities have revealed the results of the first round of COVID-19 tests. But, a more impactful question remains: How many passengers will ultimately become infected?

THE DIAMOND PRINCESS QUARANTINE: USING A BETA DISTRIBUTION TO PREDICT INITIAL 2019 CORONAVIRUS INFECTIONS

The Diamond Princess cruise ship is currently under quarantine due to COVID-19. Here, we share lessons learned from applying our quantitative method.

Five Data Points Can Clinch a Business Case

 

Any decision can be made better through better measurements – and as these three examples show, just five data points can tip the scales in a business decision.

Watch Doug’s Talk at Cybersecurity Risk Seminar Hosted by Defense Acquisition University and NavalX

On February 5, 2020, Doug Hubbard participated in a panel about cybersecurity risk hosted by Defense Acquisition University and NavalX. You can watch him help acquisition professionals see cybersecurity in a whole new way in the video clip below.

 

Learn how to build better defenses against cyber attack – and reduce cybersecurity risk – with our two-hour How to Measure Anything in Cybersecurity Risk webinar. $150 – limited seating.

 

RECOMMENDED READING:

TROJAN HORSE: HOW A PHENOMENON CALLED MEASUREMENT INVERSION CAN MASSIVELY COST YOUR COMPANY

Measuring the wrong variables is a Trojan horse that infiltrates virtually every organization. This phenomenon has a real cost that can be measured – and avoided.

TWO WAYS YOU CAN USE SMALL SAMPLE SIZES TO MEASURE ANYTHING

A measurement isn’t useless if the sample size is small. You can actually use small sample sizes to learn something useful about anything – and use that insight to make better decisions.

Five Data Points Can Clinch a Business Case

Any decision can be made better through better measurements – and as these three examples show, just five data points can tip the scales in a business decision.

 

The Total Number of Coronavirus-Infected Cruise Ship Passengers Will Be in the Hundreds

Update: A new batch of test results was released on 2/15/2020, bringing the total number so far to 355 confirmed infections. The cruise ship has another week left in quarantine, and officials expect more cases to come – in line with our estimates (see below).

The Diamond Princess, a cruise ship carrying 3,700 passengers and crew, has been in quarantine ever since a passenger from Hong Kong fell ill with the 2019 novel coronavirus (2019 n-CoV). 

On 2/6/2020, we estimated that the total number of positive test results from the first 273 samples would be between 35 and 66. Today, 2/7/2020, the testing was completed and reported – and the number of samples that tested positive (61) fell within our estimated range (see Figure 1 below).

But, a more impactful question remains: How many passengers will ultimately become infected?

Today, 2/7/2020, we estimate the total number of coronavirus infections on the Diamond Princess as 150 to 850. We’ll explain the methodology we used to make this estimate, but the salient point is this: If the “hidden” population of the virus is large as we predict it holds important implications for policy makers (and the general public).

First, we’ll revisit our estimate from earlier this week. This estimate was made using a Beta distribution on the first 102 tests and applying it to the remaining 171 tests. (Figure 1).

Using a probabilistic model, we now give a 90% confidence interval for the total number of coronavirus infections on the Diamond Princess on 2/3/2020 as 150 to 550. Because infections could continue to occur between people in the same cabins after the quarantine, our 90% confidence interval for total infections before the quarantine is released is 150 to 850.

It is virtually impossible that the initial 61 are the only 2019 n-CoV cases on board the Diamond Princess. Why?

The answer is found in our article from 2/3/2020 – this disease has a long incubation period when people are asymptomatic but potentially infectious. This means there is likely a large group of people who are asymptomatic and not in the group of 273 but who do have the virus (Figure 2).

Figure 2: Projected date of first symptoms of 2019 n-CoV on Diamond Princess, given no infections after the 2/3/2020 quarantine. (No y-axis value is given since a wide range of total infections is possible).

Put another way, if one or two individuals could start a chain reaction on 1/21/2020 that created 61 symptomatic passengers by 2/2 – 12 days later – how many people would those symptomatic passengers have infected in the days before the quarantine? The answer to this question lies in knowing the average and distribution of the incubation period.

Using the Incubation Period to Estimate Asymptomatic Cases

The mean incubation period has been estimated in the New England Journal of Medicine as 5.2 days, with the 95th percentile of the distribution at 12.5 days. We are slightly troubled taking the values from these early cases as canonical (the same case study indicated a doubling time of 7.4 days which is clearly inconsistent with cases multiplying from 41 on 1/1/2020 to 32,000+ on 2/6/2020).

However, this is the best evidence based estimate we have, and it seems consistent with other (more recent) anecdotal cases where exposure and first symptom times are well known (such as cases 19-22 in Singapore). The NEJM incubation estimate is well described by a lognormal function with a mean of 1.425 and a standard deviation of 0.67 (Figure 3).

Figure 3: The incubation time described by the recent NEJM article is well modeled by a lognormal curve (mu=1.425 and sigma = 0.67

The Utility of Probabilistic Models

Given that 61 people were symptomatic with the coronavirus by 2/2 or 2/3, how do we calculate the total number infected before the quarantine? We do this with a Monte Carlo model – using the incubation time described above and solve for what the doubling interval would have to be to produce 56 to 61 symptomatic people by 2/2 or 2/3. It matters a great deal if the original group of 273 was selected on 2/2 to be tested or 2/3. This can be seen visually in Figure 2 (the proportion of the curve up to 2/2 is much smaller than to 2/3).

The best fit if the group was selected on 2/2 would be a doubling time of 1.33 days, implying 530 people would have been infected by the quarantine. If the group was selected on 2/3, the best fit doubling time would drop to 1.48 – implying a best guess of 275 people infected prior to the quarantine.

Additional Sources of Uncertainty

Note that 275 to 530 does not represent our 90% confidence interval. We would place about a 60% chance of certainty on that range. However, there are significant additional sources of uncertainty:

  1. Most importantly, the incubation could be significantly different than what the early evidence indicated. This could make the range go up or down.
  2. We don’t know the passenger from Hong Kong was the only one who had 2019 n-CoV. Other passengers could have had it or gotten infected early on in other ports. If so, this would make the range go down.
  3. If passengers on board are sharing rooms, there is the possibility for further infections after the quarantine started. If that occurs, this would make the final infection number go up.

The best-case scenario is that the tested group was symptomatic on 2/3/2020, that there was another source of coronavirus on board by 1/21/2020, that the true incubation is shorter, and that there were relatively few infections after the quarantine. In this case, the best estimate of infections would be around 150.

The worst-case scenario is if the tested group was symptomatic by 2/2/2020, there were no other cases of 2019 n-CoV on the ship on 1/21/2020, the true incubation period is longer, and there were significant infections of family members after the quarantine. In this case, the best estimate of infections would be around 850.

Therefore, our 90% confidence interval for all infected passengers before the ship is released, assuming the quarantine is maintained, is between 150 and 850.

Drawing the Right Conclusions

Here, it’s important to provide a non-quantitative side-note. The people who are sick or quarantined on the ship are not numbers on a page – they are real people who are facing a difficult trial. My thoughts go out to these people, and my gratitude to the cruise operators and all the medical and support staff that are helping in this situation. May they be given strength and not fear their situation; may the sick passengers be comforted by the knowledge that Japan has great medical services and there are hopeful signs regarding treatment.

Additionally, it is important that other passengers draw the correct conclusions if additional cases are found in the coming days. Panic is not the best response. In fact, part of the motivation to write this article is to point out that we should all assume there will be reports of additional cases of people who were infected before the quarantine. This is crucial, and my hope is that this message can find its way to the passengers to prevent further unnecessary worry.

Macro Implications

There are potentially valuable implications that come out of this extremely unfortunate situation. This may provide a stark illustration of the hidden population of 2019 n-CoV – although only 61 people were symptomatic on 2/3/2020, there were likely hundreds already infected. This estimate, if correct, is vital to the understanding of policymakers and how the outbreak needs to be handled.

Also, I believe that we should avoid condemning the quarantine as ineffective and should not unnecessarily scapegoat the Diamond Princess staff or Carnival policies. In no way do I intend this to be “apologist” – rather let’s draw the correct conclusions from the data and not give in to emotional but incorrect reasoning.

More importantly, if the “hidden” population of asymptomatic infected people is significantly larger than the population of known cases, then the idea of “best practice” currently adopted in countries other than China may need to change. At this point, we aren’t willing to speculate what those changes might be, but it is worth starting to think about. We will learn a lot by the rate of infection revealed on the cruise ship over the next 7-10 days. Roughly 90% of those infected on or before 2/2/2020 should be symptomatic by 2/10/2020, so we should have a good idea of the total number within a week.

 

Learn how to start measuring variables the right way – and create better outcomes – with our two-hour Introduction to Applied Information Economics: The Need for Better Measurements webinar. $100 – limited seating.

 

RECOMMENDED READING:

TROJAN HORSE: HOW A PHENOMENON CALLED MEASUREMENT INVERSION CAN MASSIVELY COST YOUR COMPANY

Measuring the wrong variables is a Trojan horse that infiltrates virtually every organization. This phenomenon has a real cost that can be measured – and avoided.

TWO WAYS YOU CAN USE SMALL SAMPLE SIZES TO MEASURE ANYTHING

A measurement isn’t useless if the sample size is small. You can actually use small sample sizes to learn something useful about anything – and use that insight to make better decisions.

Five Data Points Can Clinch a Business Case

Any decision can be made better through better measurements – and as these three examples show, just five data points can tip the scales in a business decision.

 

The Diamond Princess Quarantine: Using a Beta Distribution to Predict Initial 2019 Coronavirus Infections

The Diamond Princess cruise ship is currently under quarantine while 271 passengers are being tested (as of 2/5/2020) for the 2019 novel coronavirus (2019 n-CoV). Concern about infection arose when a prior passenger from Hong Kong who was on board the ship from 1/20 to 1/25 was later found to be infected. As a result, the ship was delayed and then quarantined off the port of Yokohama to test a group of 271 passengers who either had symptoms of 2019 n-CoV or had significant contact with the original case from Hong Kong. On Wednesday (2/5) 10 out of 31 tests had come back positive from a suspected 271 people. By Thursday, 20 out of 102 tests had come back positive. This is a real world application where we can test the utility of a Beta distribution to predict an outcome – we shouldn’t be surprised if another 15-46 people test positive for the coronavirus out of the remaining tests. 

A Beta distribution for the first 31 tests would have an alpha of 11 (10+1) and a beta of 22 (21+1); the 90% confidence interval for the proportion using the first sample is 21%-47% (Figure 1). The second group of tests has an alpha of 11 (10+1) and a beta of 62 (61+1); the 90% confidence interval for the proportion given this sample is 9%-22% (Figure 1). Based on these results, we suspect that they tested the more likely cases first, and the remaining 169 are more likely to resemble the second sample in likelihood of infection. However, if the first two samples were randomly selected then we would use the beta distribution of all 102 initial cases (alpha = 21, beta = 83) with a 90% C.I. of 14 to 27%.

coronavirus estimates

Figure 1: Difference in distributions between the first and second sets of test results

Since we don’t know which is accurate, we’ll use 9-27% for our 90% confidence interval, which gives us an estimate of 35 to 66 total of this group of 271 will test positive for the coronavirus (Figure 2). This would imply that an additional 15 to 46 positive results will come back from the remaining 169 tests.

Figure 2: Predictions for the lower bound and upper bound of test results for the 271 suspected cases aboard the Diamond Princess

 

Drawing the Right Conclusions

There is another chapter to this story however. Whether the original group has 35 or 66 cases, these will not be the only 2019 n-Cov cases on board the Diamond Princess cruise ship, and it is crucial that policy makers understand why. The ship has nearly 3,700 people trapped on board, and the infection spread uninhibited for at least seven days. The correct conclusion is that the incubation period is long, and the doubling rate is short – therefore when these initial 271 passengers were selected and tested, there already existed another group of people who were infected and not symptomatic. This is important for three reasons:

  1. Don’t blame the quarantine. As additional cases are found over the next 10 days, it would be incorrect to assume that quarantining people to their rooms failed. That was the correct move and will prevent additional infections and serious illness.

  2. The “hidden population” of 2019 n-Cov is a crucial aspect of understanding this disease. This was the main point in the post published on Monday – that the undercount of this disease will hamper attempts to control the spread because of the long incubation period and asymptomatic cases. If policy-makers can draw the right conclusions from the Diamond Princess experience, it could dramatically help in the effort to slow or stop the spread of the disease.

  3. It is likely that additional cases may have gotten off the ship between 1/20 and 2/2, and those passengers should be alerted and local health officials made aware of the risk.

We will publish a follow up article to this once the test results for the 271 passengers are completed. Our initial estimates are that at least 100 people will test positive before the quarantine is released. We estimate that even if quarantine efforts prove perfectly successful, the Diamond Princess will likely have over 100 people aboard the ship test positive for 2019 n-Cov before the quarantine is released.

 

Learn how to start measuring variables the right way – and create better outcomes – with our two-hour Introduction to Applied Information Economics: The Need for Better Measurements webinar. $100 – limited seating.

 

RECOMMENDED READING:

TROJAN HORSE: HOW A PHENOMENON CALLED MEASUREMENT INVERSION CAN MASSIVELY COST YOUR COMPANY

Measuring the wrong variables is a Trojan horse that infiltrates virtually every organization. This phenomenon has a real cost that can be measured – and avoided.

TWO WAYS YOU CAN USE SMALL SAMPLE SIZES TO MEASURE ANYTHING

A measurement isn’t useless if the sample size is small. You can actually use small sample sizes to learn something useful about anything – and use that insight to make better decisions.

Five Data Points Can Clinch a Business Case

Any decision can be made better through better measurements – and as these three examples show, just five data points can tip the scales in a business decision.