Cost Impact Model of a Novel Multi-mRNA Host Response Assay for Diagnosis and Risk Assessment of Acute Respiratory Tract Infections and Sepsis in the Emergency Department

Background Early identification of acute infections and sepsis remains an unmet medical need. While early detection and initiation of treatment reduces mortality, inappropriate treatment leads to adverse events and the development of antimicrobial resistance. Current diagnostic and prognostic solutions, including procalcitonin, lack required accuracy. A novel blood-based host response test, HostDx™ Sepsis by Inflammatix, Inc., assesses the likelihood of a bacterial infection, the likelihood of a viral infection, and the severity of the condition. Objectives We estimated the economic impact of adopting HostDx Sepsis testing among patients with suspected acute respiratory tract infection (ARTI) in the emergency department (ED). Methods Our cost impact model estimated costs for adult ED patients with suspected ARTI under the standard of care versus with the adoption of HostDx Sepsis from the perspective of US payers. Included costs were those assumed to be associated with an episode of sepsis diagnosis, management, and treatment. Projected accuracies for test predictions, disease prevalence, and clinical parameters was derived from patient-level meta-analysis data of randomized trials, supplemented with published performance data for HostDx Sepsis. One-way sensitivity analysis was performed on key input parameters. Results Compared to standard of care including procalcitonin, the superior test characteristics of HostDx Sepsis resulted in an average cost savings of approximately US$1974 per patient (−31.3%) exclusive of the cost of HostDx Sepsis. Reductions in hospital days (−0.80 days, −36.7%), antibiotic days (−1.49 days, −29.5%), and percent 30-day mortality (−1.67%, −13.64%) were driven by HostDx Sepsis providing fewer “noninformative” moderate risk predictions and more “certain” low- or high-risk predictions compared to standard of care, especially for patients who were not severely ill. These results were robust to changes in key parameters, including disease prevalence. Conclusions Our model shows substantial savings associated with introduction of HostDx Sepsis among patients with ARTIs in EDs. These results need confirmation in interventional trials.

JOURNAL OF HEALTH ECONOMICS AND OUTCOMES RESEARCH per year. 9 EDs account for 500 000 sepsis cases per year, meaning roughly two-thirds of all septic patients entering the hospital through the ED. 9 Hospital mortality in all patients suspected of sepsis is 4.1%, rising to 28% in those with septic shock. 10 The incidence of sepsis continues to increase with the aging population, causing an increase in annual mortality rates. Repeated findings have shown that patients with bacterial septic shock have a 7% to 8% increase in mortality for each hour of delay in antibiotics administration, highlighting the need for early and accurate diagnosis and treatment. 11,12 Acute respiratory tract infections (ARTIs) are one of the leading causes of adult hospitalizations, sepsis, and death worldwide, and are also associated with the overutilization of antibiotics. 13 ARTIs account for 10% of all ambulatory visits and 44% of all antibiotic prescriptions in the US. 14 Although about 65% of ARTIs are viral, 15,16 antibiotics are prescribed in more than 60% of cases, 15,17 contributing to overuse of antibiotics and increasing antibiotic resistance. [18][19][20][21][22][23] Antibiotic treatment is associate with a substantial rate of adverse events. 24,25 Both the Surviving Sepsis Campaign guidelines and the Center for Medicare & Medicaid Services (CMS) Sepsis Core Measure (SEP-1) bundle can almost entirely be reduced to two main treatments: (1) source control and antimicrobial therapy to fight an underlying infection and (2) supportive care to maintain physiology. In general, the components can also be split into two separate but complementary axes, namely: (1) whether there is an infection and (2) how severe the underlying condition is. These two separate questions have distinct, but linked, clinical actions. For example, a patient with an acute bacterial infection needs antibiotics. However, the antibiotics chosen for a non-severe acute infection may be narrow oral antibiotics, whereas a patient with a greater risk of organ dysfunction/sepsis may need broadspectrum parenteral antibiotics. Similarly, patients with non-severe acute infections may not require admission while higher risk cases could be kept in the ED for observation or admitted to a general or ICU ward. Other than lactate, almost all in vitro diagnostics for sepsis primarily lead to clinical actions regarding whether and how to properly attain source control and treat with antimicrobials. Modern guidelines often focus on initiating antibiotics within 60 minutes of presentation, and as a result, only diagnostics with a turnaround time substantially shorter than 60 minutes are useful in initial workflow.
Current laboratory tools are inadequate for the diagnosis and prognosis of patients with ARTI and/or sepsis. In particular, culturing bacterial or viral pathogens cannot rule out an infection outside the sampled tissue. Molecular pathogen panels for the detection of respiratory infections are fast and sensitive, but (1) they are confounded by the presence of contaminant and commensal organisms, 26 (2) they can only detect a defined number of pathogens, 27 and (3) are expensive as screening tests.
Procalcitonin is the biomarker most extensively studied for its diagnostic and prognostic abilities. In protocol-driven studies, procalcitonin testing during infection treatment reduces antibiotics days and length of stay. [28][29][30] In addition, the most recent long-term patient-level study by Schuetz et al. found a relative reduction in mortality rate of 1% (absolute reduction of 9%) for acute respiratory infection patients when care was guided with procalcitonin. 13 However, procalcitonin can also be elevated in patients with non-infectious conditions, such as heatstroke, trauma, surgery, and others. 31,32 Because of these limitations and concerns regarding test accuracy limitations, procalcitonin often does not change clinician behavior. 29,33 For instance, a recent large multicenter US study showed no change in prescribing behavior in an intention-to-treat analysis. 34 HostDx™ Sepsis (Inflammatix, Inc., Burlingame, CA) is a novel, blood-based 29 target host mRNA test with an advanced proprietary algorithm to inform on (a) the likelihood of a bacterial infection, (b) the likelihood of a viral infection and (c) the severity of the condition (likelihood of mortality). [35][36][37] The test is designed for use in emergency rooms, urgent care clinics, and inpatient settings.
To conduct an economic evaluation of HostDx Sepsis testing versus standard of care we built a cost impact model based on published data. Our results show substantial savings associated with HostDx Sepsis protocols of ARTI across common US treatment settings mainly by direct reduction of unnecessary hospitalization. These results are robust to changes in key parameters, and the savings can be achieved without any negative impact on treatment outcomes.

METHODS
We built a deterministic decision-analytic cost impact model to estimate costs for 1000 adult patients with suspected ARTIs presenting to the ED under two scenarios: (1) a base case scenario with standard of care using procalcitonin and (2) a scenario where procalcitonin is replaced with HostDx Sepsis for each patient upon initial presentation. For each scenario, our model estimated episode of care costs, which encompasses the costs of treatment, hospitalization, outpatient visits, and medications. All costs were estimated from the perspective of US payers. In addition to costs, the primary model estimated outcomes including hospital length of stay (LOS), days of antibiotic utilization (antibiotic days), and 30-day mortality.

HostDx Sepsis
HostDx Sepsis is a novel, blood-based host response test that measures expression of 29 human host mRNAs and interprets them with an advanced proprietary machine learning algorithm to output the likelihoods of bacterial infection, viral infection, and disease severity (30-day mortality). Each result is provided as a numerical score that falls into one of four interpretation bands (very unlikely, unlikely, possible, very likely). Test performance for the three result readouts has been published based on analysis of heterogeneous patient cohorts. [35][36][37] Potential Diagnostic Results and Suggested Clinical Actions To simplify our model, only three likelihood bands (low, moderate, and high) were modeled for the risk of a bacterial infection and the risk of 30-day mortality, and only two bands (low or high) were modeled for the viral readout. Two bands were used for the viral diagnostic because we only had a binary action to assign (test and treat if positive). Combining the three individual readouts for bacterial, viral and mortality risks there were a total of 3*2*3=18 potential combinations of bacterialviral-mortality risk predictions. We then assigned each combination of predictions a clinical action appropriate for ARTI patients in the ED based on assumptions informed by input from key opinion leaders (Figure 1). Critically, we assumed that physicians would always follow these patient management actions for the given standard of care or HostDx Sepsis results, regardless of whether the ground truth aligned with the predictions or not.

Modeling Outcomes
There are two ground truth states (true or false) for each of the three risk areas, so 2*2*2=8 ground truth states overall (eg, a patient could in fact have a bacterial infection, no viral infection, and a high risk of 30-day mortality). Thus for each of the 18 possible risk prediction combinations there are a total of 8*18=144 possible prediction-ground truth combinations (Figure 2a). We generated clinical outcomes for each prediction-ground truth combination based on clinical data from literature and assumptions (Table 1). For example, when a bacterial infection patient with high risk of 30-day mortality had low-risk predictions for bacterial, viral, and mortality (and so was discharged home without antibiotics), the clinical outcome was a 7-day hospital readmission. Supplementary Material provides a complete list of outcomes for each prediction-ground truth combination.

Costs
The costs included in the model were those assumed to be associated with an episode of sepsis diagnosis, management and treatment. Costs for each component of suggested clinical actions and clinical outcomes (eg, IV antibiotics, blood culture, hospital readmission) were estimated from literature or assumptions ( Table 2). We then calculated the perpatient costs of each prediction-ground truth combination. Since the cost of HostDx Sepsis has not been established as of today, we did not include the cost of HostDx Sepsis into our model.

Placement of Patients into Prediction-Ground Truth Combination Groups
The performance (AUROCs) of standard of care and HostDx Sepsis tests were derived from clinical studies. We assumed AUROCs for bacterial, viral, and mortality prediction of 0.8, 0.8, and 0.78 in the base case, and 0.85, 0.9, and 0.88 in the HostDx Sepsis case (Table 1).
We built models to test how improved accuracy may be used to make a correct decision in a greater number of patients. To do this, we first simulated ideal receiver operating characteristic curves at the stated AUROCs and broke the curves into bands at preset target likelihood ratios (low-band LR 0.1, high-band LR 10) which roughly correspond to a low-band sensitivity of 93% to 95% and a high-band specificity of 95% to 97%. The same likelihood cutoffs/targets were used for each of the bacterial, viral, and mortality scores. Because the interpretation bands used preset LR targets, for a higher AUROC, more patients were assigned to actionable "high risk" or "low risk" prediction bands. In other words, a more accurate test placed more patients into the "correct" actionable band.
Taking the probability of a case being in a given prediction band, we then multiplied through assumed case prevalence (Table 1) to arrive at expected patient assignments for each of the 144 prediction-truth combinations. This was done for the AUROC assumptions of the base case, and then again for the AUROC assumptions of the HostDx Sepsis test. All of the predictive modeling was accomplished with custom code written in R.

Cost Calculations and Sensitivity Analysis
The per-patient cost for each prediction-ground truth combination was multiplied by the number of assigned patients to yield final estimates of the total expected costs and clinical outcomes for both the base and HostDx Sepsis scenarios. Finally, we conducted extensive one-way deterministic sensitivity analyses to characterize the robustness of the model on key parameters including diagnostic accuracy, prevalence, cost, and clinical outcomes. With the exception of the custom R script written to place patients into prediction-ground truth combination groups, all aspects of the model was coded in Microsoft Excel.

Overall Outcomes
We built a model of diagnostic and prognostic testing of ARTI patients in an ED as described for a standard-of-care (with procalcitonin) base case and a case with the introduction of HostDx Sepsis. Projected accuracies for test predictions, disease prevalence, and clinical parameters were derived from published data and supplemented with internal independent data from Inflammatix. We assumed the perspective of a payer and modeled costs for a cohort of simulated ED patients at risk for ARTI. On average, using HostDx Sepsis resulted in 0.8 fewer hospital days, 1.5 fewer days on antibiotics, a 1.7% absolute reduction in 30day mortality, and an expected cost savings of US$1974 per patient compared to standard of care, exclusive of the cost of HostDx Sepsis ( Table 3). These reductions correspond to 36.7%, 29.5%, 13.64%, and 31.3% reductions in hospital days, antibiotic days, 30-day mortality and per-patient costs, respectively. For a cohort of 1000 at-risk patients, the potential savings are approximately US$2 million.

Outcomes by Infection Status and Mortality Ground Truth
Further segmenting these results by ground truth patient characteristics ( Table 4) shows the greatest cost savings are generated from a reduction of hospital days among non-severe patients. A substantial reduction in antibiotics days was found for nearly all patient categories. The biggest reductions in antibiotic days was observed for patients that were nonbacterial, non-severe, and either virally infected or non-virally infected. Of interest, even patients that were bacterial but non-severe showed marked reductions in antibiotic days, likely because treatment duration was reduced. An overall increase in ICU days (coupled with mortality reduction) is projected for severe patients, which can be explained by earlier appropriate ICU admissions leading to lower mortality.

Outcomes by All Possible Base Case/HostDx Sepsis Predictions and Ground Truth Combinations
Next, we segmented estimated costs and number of patients by all 144 possible combinations of prediction and ground truth for the base and HostDx Sepsis cases (Figure 3). The superior performance (higher modeled AUROCs) of HostDx Sepsis pushed relatively more patients into the "actionable" bands at LR 0.1 or LR 10 (low or high, respectively). Aggregate cost savings were driven largely by HostDx Sepsis reassigning patients from "noninformative" predictions of moderate risk for bacterial infections and mortality to actionable predictions of low or high risk.

Sensitivity Analysis
We performed one-way deterministic sensitivity analysis on key input parameters. Clinical outcomes and cost parameters tested were varied by 20% in each direction. Test accuracies and prevalence parameters were varied up and down using different ranges based on literature and assumptions (Figure 4).
The HostDx Sepsis scenario was dominant as it generated cost savings for all one-way sensitivity analysis scenarios. The net cost impact results were most sensitive to daily hospital ward costs and hospital ward LOS. A 20% increase in either parameter resulted in a net savings decline of roughly US$400. The next most influential parameters (in descending order) were hospital ICU LOS, ICU cost per day, antibiotic costs and ICU LOS after rehospitalization. Changes in these parameters resulted in minor effects on the overall findings (mostly less than +/-US$100).

DISCUSSION
In this study we constructed a cost impact model to estimate the cost impact associated with introducing HostDx Sepsis, a novel blood-based host response test that assesses the likelihood of a bacterial infection, the likelihood of a viral infection, and the severity of the condition. The key finding of this study is that introducing HostDx Sepsis results in net expected savings of approximately US$2000 per suspected ARTI patient in the ED, exclusive of the cost of HostDx Sepsis.
Our study showed that the superior performance characteristics of HostDx Sepsis compared to standard of care with procalcitonin allows for a 36.7% reduction in hospital LOS. HostDx Sepsis accomplishes this by reducing the proportion of patients with uncertain "moderate risk" predictions, thus allowing for patients to receive the appropriate level of care to reduce unnecessary days in hospital, freeing up hospital resources. This decrease in hospital LOS was the primary driver of the approximately US$2000 in cost savings per patient we calculated. Considering the average hospital sees 750 ARTI patients in the ED each year, 38  Additionally, our model estimates that HostDx Sepsis will result in a substantial reduction in antibiotic treatment days. While this reduction may not drive down overall costs from the perspective of the US payer, it can empower hospitals in their goals toward antimicrobial stewardship. Infections caused by resistant bacteria can lead to up to two-fold higher rates of adverse outcomes compared with similar infections caused by susceptible strains. 39 Reducing the amount of antimicrobial treatment will also reduce the rate of serious adverse effects and Clostridium difficile infection among ARTI patients, potentially resulting in added cost savings to payers that are not included in our model.

Limitations
The study has several limitations. First, the model is a simulation based on the differential probability of certain events at each band or test outcome, and the bands are set at high required stringency (LR=0.1 and LR=10 for low band and high band, respectively). Calculations of expected costs are based on mean values obtained from the literature, and for each parameter there is uncertainty. In the sensitivity analysis, we attempt to assess the importance of this uncertainty by simulating model outcomes under a variety of alternative (but plausible) levels of key input parameters. However, this sensitivity analysis may not encompass all the clinical scenarios observed in practice. Second, the model lacks real-world data. Prospective clinical trials for HostDx Sepsis are currently being conducted and will allow us to update the model with real-world data. Also, while the actual HostDx Sepsis test will split the component scores into four risk bands (very low, low, moderate, and high), only two to three are modeled here, for simplicity in describing clinical actions. In addition, treatment assumptions for each of our prediction-ground truth combinations were based on primarily on key-opinion-leader input, with some level of confirmation from published literature. Again, as is the case with the aforementioned limitation, reliance on these data sources may not accurately reflect all of the clinical scenarios observed in practice. Surprisingly, our model showed an overall increase in ICU days across all patients. This is due to a limitation in how we simulated ICU transfers given the 30-day mortality risk AUROC data we used for the standard of care arm. As built, the model underestimates the number of patients that would be "ruled-in" as high-severity patients needing ICU care under standard of care. Regardless of this technical limitation, we showed that HostDx Sepsis can lead to (appropriate) increases in ICU days for severe patients who truly needed the care, leading to decreased mortality. Moreover, the overall decrease in hospital days more than made up for the slight overall increase in ICU days, generating cost savings overall. Future models with real-world data will be able to overcome these technical limitations.
For both the base case with procalcitonin and HostDx Sepsis, our model assumes that physicians will have trust in test results and follow all treatment guidelines for standard of care with procalcitonin and HostDx Sepsis results. In real clinical practice, we expect that physicians will not always adhere to treatment guidelines. For example, in the ProACT clinical trial of procalcitonin for lower respiratory tract infections, Huang et al. showed that physicians deviated from procalcitonin guidelines in 72.9% of patients despite defined trainings. 40 Finally, this model only addressed ARTIs. The HostDx Sepsis test is designed to work across other infection types including abdominal, urinary tract, and skin and soft tissue infections. The current model does not consider the potential benefits of the test's routine use in these indications.

CONCLUSION
Patients with suspected ARTI are a high clinical burden. Current standard of care lacks rapid, accurate diagnostic tools, and is characterized by high rates of unnecessary antibiotic prescribing, high hospital resource utilization, and economic inefficiencies. The novel HostDx Sepsis test is estimated to substantially reduce costs, improve treatment and triaging decision-making, and help hospitals achieve antimicrobial stewardship goals. Further studies, including interventional studies, are necessary to confirm these results. One-way deterministic sensitivity analysis was performed on key input variables. Most clinical outcomes and cost parameters tested were varied by 20% in each direction. Test accuracies and prevalence parameters were varied up and down based on ranges derived from literature. Red and blue bars indicate the net cost impact if the model was rerun with high-level and low-level estimates of the corresponding parameter, respectively.