PEU COVID-19 Science Makes The Lancet

Would citizens be concerned if greedy math whizzes made healthcare treatment decisions?   The financial world already experienced this change.  Business Insider described Wall Street's quants:

“We do it with maths,” he says. “We buy stock market data and we analyse it. It’s like weather forecasting."

“Some of the guys who come from pure science and maths backgrounds are used to solving a problem and it works,” Patrick Boyle says. “They think they can find a formula that will perfectly describe how the market moves. That is the philosopher’s stone – it is utterly impossible.” The danger is that in only seeing numbers and patterns the human dimension is forgotten.

“I was working with the best of the best,” he says. “My bank employed the brightest engineers, chemists and scientists – and we were all working together to get richer. The chemical and physics and health industries are worse off because of what we do because I tell you this: if there was a pay bonus structure similar to what we had in the City for curing cancer, we’d have found a cure for cancer.”

Or COVID-19?  A recent study published in The Lancet concluded:

"Covid-19 patients who received the malaria drug were dying at higher rates and experiencing more heart-related complications than other virus patients. The large observational study analysed data from nearly 15,000 patients with Covid-19 who received the drug alone or in combination with antibiotics, comparing this data with 81,000 controls who did not receive the drug."

It came under fire from researchers across the globe.

This "observational study" was a retrospective review of medical records, not the result of a double blind clinical study, the gold standard for studying medication efficacy.  Data came from Surgisphere, which described itself in a press release:

About Surgisphere
The Surgisphere Corporation, founded in 2007, is creating a seismic transformation in healthcare, so that the world can become a healthier place. For medical organizations who need to measure and increase performance with precision, QuartzClinical fulfills on the promise of advanced business intelligence and connects knowledge from data in ways that empower your entire organization to make better decisions every day at every level.

The February 2019 press release stated:

QuartzClinical announced the availability of a new suite of sophisticated machine learning-powered data warehousing and clinical registry tools.

These new tools permit 24/7/365 access to mission-critical data in a secure, cloud-based environment. The data warehouse system uses machine learning and sophisticated artificial intelligence to rapidly correct errors in structured and unstructured data, thus minimizing the cost and time associated with data entry. This system is fully integrated into the QuartzClinical cloud-based healthcare data analytics platform, thus allowing a hospital to fully leverage a smart business intelligence solution within days – not months.

The new clinical registry tools can be used by any healthcare entity, including hospitals, clinics, pharmaceutical companies, and device manufacturers. These tools use a secure, cloud-based architecture to store data in a fully HIPAA-compliant manner. Machine learning and artificial intelligence algorithms power the data entry systems, thus enabling rapid acquisition of key clinical metrics with minimal effort. These new tools are expected to replace traditional data entry systems that rely upon nurses, data entry staff, and technicians. The result is higher quality data that is timelier and more comprehensive, thereby solving many of the issues faced by large database research studies.

 Surgisphere subsidiary Quartz Clinical said the following on its website:

The precision of QuartzClinical provides hyper-accurate predictive models using machine learning. Flexible enough to be applied to a single procedure, this model takes into account dozens of different variables and provides an overall accuracy of 81%. Imagine having a prediction engine that can be applied at the individual patient level to determine the chance that they will be readmitted. Extremely flexible and scalable, the model is self-learning and individualized to the particular hospital system, so the results are always customized.

Using QuartzClinical can guide individual patient care and directly impact the quality of that care while reducing costs.

The Lancet publication is not the result of a controlled experiment but a review of patient medical records.  How did they get access to 96,000 records so quickly?  Surgisphere clarified in their response to questions about their work:

The sophistication of the data retrieval requires that we link directly with the Electronic Health Records (EHRs) of our collaborating hospitals, and all information is transferred in a deidentified manner. Thus, these demands require that we work exclusively with healthcare institutions that utilize well established EHRs. 

Having worked on both the hospital and outpatient side of healthcare I know the sole use of historical medical records to conduct clinical research is highly questionable.  Electronic records can sometimes push around inadequate information faster.

It's an "after this, therefore because of this" exercise vs. an experiment, whereby an intervention is made and the impact assessed, immediately and over time. 

The question of causation vs. correlation arises.  Consider the Super Bowl stock market indicator:

The Super Bowl indicator is a theory wherein we can predict the stock market’s year end closing price based on which conference wins the Super Bowl. The theory claims that if the NFC team wins the stock market will finish the year higher, and if the AFC team wins the market will finish lower. Most of the traders I know are highly logical and analytical and are quick to dismiss the theory as hokum and of course they are right. I think. Oddly the Super Bowl indicator has an 80% success rate.

Surgisphere's Quartz Clinical "provides an overall accuracy of 81%," beating the Super Bowl indicator by 1%.

I'm not sure I want my healthcare decisions driven by the same math that fuels Wall Street's short term bets, especially if the underlying data may be suspect.  I have seen outstanding healthcare professionals perform time saving workarounds that bypassed large sections of an EHR system.  I learned which sections were unreliable and where to look for useful information on patients.

Operational definitions are critical for accurate data collection.  One might consider the founding of a company to be a specific date, say date of incorporation.  Consider Surgisphere's own representations of their founding.

.
Data mining found three dates ranging from 1998 to 2007 to 2008.  Texas has an incorporation date of June 2012 and Illinois records show April 2016.  Which date would Surgisphere enter into the electronic corporate record?

PEU Report found no evidence that Surgisphere is owned by a private equity firm, the normal focus of this blog.  However, I fear science is being distorted in the pursuit of massive profits, a distinct feature of the greed and leverage boys.

Update 5-31-20:  ZeroHedge ran a piece asking why "the Lancet study failed to test HQC with zinc."  The Lancet study did not test any drugs.  It data mined clinical records for information on their effectiveness.  A better representation might be "the Lancet study failed to retrospective assess the use of HQC in combination with zinc for hospitalized patients in facilities with contracts with Surgisphere subsidiary Quartz Clinical."

Update 6-3-20:   The Guardian ran a story questioning Surgisphere's data and its founder.  How could bad data mining be interpreted as a scientific experiment?

Update 6-4-20:  Three researchers retracted the study published in The Lancet.

Update 6-7-20:  New questions arise as to how Surgisphere got its data.  Bloomberg picked up the banner.

Update 7-11-20:  A study physician said this about his error, "I did not do enough to ensure that the data source was appropriate for this use."

Disclosure:  PEUReport followed Peak Prosperity's Chris Martenson and his COVID-19 coverage since late January and am grateful for his many contributions..