Value of clinical laboratory test for early prediction of mortality in patients with COVID-19: the BGM score

Introduction

The SARS-CoV-2 virus emerged in the last quarter of 2019 in Wuhan, the capital of Hubei province of China. The disease caused by SARS-CoV-2 virus, named COVID-19 by the World Health Organization, has spread rapidly and globally causing a pandemic with unprecedented clinical, humanitarian, and economic repercussions (1,2). In the absence of reliable data on worldwide seroprevalence, the number of confirmed infections and deaths exceed 17.8 million and 680,000, respectively. The first studies that analyzed the clinical complications associated with this disease were from China. In this area, many of the patients had mild to moderate symptoms (80%), about 14% had a severe disease course (dyspnea, O₂ saturation ≤93%, pulmonary infiltrates), and about 6% presented with critical progression characterized by respiratory failure, septic shock, and/or multiorgan failure (3). The data accumulated so far from more than 10,000 patients in the European Union and in the New York City area show that among the confirmed cases, 30% required admission and 4% required care in intensive care units (ICUs) due to their critical condition (4,5). In turn, it was observed that mortality is particularly high in the subgroup of patients with advanced age and preexisting comorbidities, among which obesity, hypertension, and diabetes are frequently found (6). It is noteworthy that patients without these associated comorbidities can also present with a critical or severe course of the disease. Therefore, the search for early biomarkers to assess the severity of the pathology and its clinical progression is currently necessary to rationalize the use of hospital resources in ICUs and reduce the mortality associated with COVID-19.

Several clinical laboratory markers, such as lymphocyte (LYMPH) count, lactate dehydrogenase (LDH), and D-dimer (DD), are altered in patients with COVID-19 (7). Other studies have shown significant differences in the concentration of cytokines in blood (interleukin [IL]-6, tumor necrosis factor [TNF]-γ, IL-8, IL-2R) among patients who have required ICU admission and patients who do not (8). In turn, infection biomarkers such as C-reactive protein (CRP), procalcitonin (PCT), and ferritin (FER) increased significantly with the severity of the disease (8). However, despite the research efforts made in the field of laboratory tests, reliable algorithms with early prognostic value have not yet been generated to establish the risk of future complications in patients infected with SARS-CoV-2.

The lack of accurate early prognostic algorithms based on central laboratory testing for COVID-19 has spurred researchers to direct their efforts toward the use of omics tools in the search for potential biomarkers. In a first study published by Shen et al (9), the combination of proteomics and metabolomics allowed the identification of a panel of 22 proteins and 7 metabolites with predictive power to differentiate mild vs. severe COVID-19 with 94% accuracy. A second study published in Cell Systems (10) showed that European researchers identified 27 differentially expressed proteomics biomarkers associated with different grades of COVID-19 severity in hospitalized patients. Despite these positive advances, the technical complexity of these omics tools and their high cost limit their applicability in the clinical arena.

In the context of the vast scope of the SARS-CoV-2 crisis and until we achieve sufficient immunization coverage of the population, we believe that the search for new early prognostic strategies must prioritize their applicability and accessibility to most health care centers. In this line, the objective of our study was to generate predictive algorithms for early stratification of patients with COVID-19 who may be at the risk of developing severe complications. To this aim, we designed a retrospective cross-sectional single-center study in which we evaluated different predictive algorithms for mortality considering demographic factors, clinical factors, and standard laboratory tests usually present in most central clinical laboratories.

Materials and methods

Statistical analysis

Categorical variables were expressed as numbers and percentages and compared using the Chi-square test. Continuous variables were expressed as median and interquartile range (IQR) and were compared by the Mann-Whitney-Wilcoxon test.

The strength of the relationship between the laboratory parameters was assessed using the Pearson or Spearman correlation coefficients. The multivariate statistical analyses conducted were logistic regression (LR) (11), classification tree (CT) through the CART algorithm (12), and artificial neural network (NNet) (13). Missing data were imputed via bagged tree models (11), and the data set was then split into a training and test set. The optimal parameter for each model was determined in the training set, calculating the best averaged predictive performance after 10-fold cross-validation. Additionally, to the previous multivariate models, we generated a computational method, called the BGM score, which provides the survival probability of a patient with COVID-19 considering the variables age, CREA, DD, CRP, PLT, and TNI. We modulated the survival probability of the BGM score as a probabilistic event depending on the survival probability given by the LR (P_s(LR)), the CT (P_s(CT)), and the NNet (P_s(NNet)) models generated from our data set. Further, the P_s(LR), P_s(CT), and P_s(NNet) were multiplied by their corresponding model accuracies (Ac_(LR), Ac_(CT), and Ac_(NNet); respectively), giving the following equation for the BGM score survival probability: P_s(BGM) = (Ac_(LR) × P_s(LR)) ∩ (Ac_(CT) × P_s(CT)) ∩ (Ac_(NNet) × P_s(NNet)). Additionally, the following constraints were applied to the P_s(BGM) to incorporate the best predictive features of the LR, CT, and NNet models:

Sensitivity, specificity, positive predictive value, negative predictive value, kappa, total accuracy, and balanced accuracy ([sensitivity+specificity]/2) were calculated for each model considering only the test set (99 cases). All the statistical analyses were performed using public libraries from the Comprehensive R Archive Network (CRAN; http://CRAN.R-project.org) rooted in the open-source statistical computing environment R, version 3.6 (http://www.R-project.org/). A p-value <0.05 was considered statistically significant.

We wrote an interactive web application using the R Shiny package (14) that implements the four multivariate models that we generated in our study. This web application can be used to calculate the survival probability for a patient with COVID-19 and is freely available at the link “https://bgm-hoc.shinyapps.io/Shiny_covid_clinic/.”

Results

Five hundred subjects with a confirmed diagnosis for COVID-19 formed the study population. Overall, the median age of the patients was 64 years, 42.6% were female, and patients were discharged within a median of 10 days. The most common comorbid conditions were hypertension (44.2%), dyslipidemia (31.2%), and diabetes mellitus (18.8%). Among the patients recruited, 23.4% required ICU, and 19.4% died during follow-up. The demographic, clinical, and laboratory results of the patients corresponding to the first 48 hours after admission are summarized in Tables I and II.

We evaluated a panel of 12 biochemical, 5 hematological, and 3 coagulation biomarkers for each patient. As shown in Figure 1, we detected the presence of high significant correlations in AST-ALT (r = 0.9, p < 0.001) and WBC-NEU (r = 0.8, p < 0.001). To avoid the presence of multicollinearity bias in multivariate analysis, we excluded the variables AST and NEU for future calculations. We included the rest of the biochemical, demographic, and clinical variables in the multivariate LR model that we designed considering the occurrence of death as a response variable and that we generated performing 10-fold cross-validation. Among all the explanatory variables included initially in the model (variables shown in Tabs. I and II) only age, CREA, DD, CRP, PLT, and TNI were early independent predictors of mortality after admission (Tab. III), according to the model selection rule based on the Akaike information criterion (AIC). This multivariate model yielded a sensitivity of 47.3%, a specificity of 98.7%, a kappa of 0.56, and a balanced accuracy of 0.73 for identifying patients with a high risk of mortality. We obtained these performance characteristics using a validation set of 99 cases that were not used for training the model. Despite the high negative predictive value of the LR model (0.9), we observed a low sensitivity that suggests that the model was sensitive to class imbalance.

To improve our classification performance without downsampling, we next generated two additional multivariate models based on a different paradigm of categorization, CTs with the CART algorithm (the implementation in R is called rpart) and NNets. Considering all the variables from Tables I and II, the CART CT yielded TNI (cutoff value of 44 ng/L), age (cutoff value of 79 years), DD (cutoff value of 700 ng/mL), and CRP (cutoff value of 15 mg/dL) as the most potent early predictors for stratifying patients with a high vs. low risk of mortality (Fig. 2). These cutoff values were similar to their reference intervals implemented for clinical diagnosis (TNI < 45.2 ng/L, DD < 500 ng/mL, and CRP < 1 mg/dL). This model outperformed the LR model with a sensitivity of 68.4%, a specificity of 92.5%, a kappa of 0.61, and a balanced accuracy of 0.80 for identifying patients with a high risk of mortality in our validation set. Despite this performance comparison, one common characteristic in these models was that both yielded clinical laboratory measurements as the most powerful predictors of mortality in patients with COVID-19.

In general, the outputs of LR and CT are intuitive and easy to implement as predictive algorithms in the clinical setting. However, NNets are black boxes regarding the contribution of the explanatory variables to the output of the response variable. Therefore, we have to limit the selection of variables to generate manageable NNet models applicable to most clinical settings regardless of their limitations in their laboratory tests portfolio. In this context, and with the intention intending of improving our LR model, we generated an artificial NNet including only the variables that remained as early independent predictors of mortality in the LR model: age, CREA, DD, CRP, PLT, and TNI. Supplemental figure 1 shows the optimal architecture of the neural model that was obtained after 10-fold cross-validation. The NNet model yielded a sensitivity of 66.7%, a specificity of 92.3%, a kappa of 0.54, and a balanced accuracy of 0.79 for identifying patients with a high risk of mortality in our validation set. This performance was comparable to that achieved by our previous CT algorithm.

The three algorithms we generated can be divided into two groups considering their sensitivity and specificity. The model with the highest specificity was LR, while the CT and NNet models presented lower specificity but a higher sensitivity. These differences in predictive accuracy led to our developing a new hybrid model in combination with the LR, CT, and NNet models to incorporate the best predictive features of each. As described in the Material and Methods, our model, called the BGM score, calculates a survival probability for patients with COVID-19 by multiplying the survival probabilities of the three previous models corrected by their accuracies. We assessed the BGM score performance in terms of prediction accuracy over the validation set, yielding a sensitivity of 73.7%, a specificity of 96.5%, a kappa of 0.74, and a balanced accuracy of 0.85 for the prediction of COVID-19 patients who died. Figure 3 shows the statistical comparison of the accuracies of the four models, where it can be seen that the BGM score model significantly outperformed all the other models. Our hybrid model corroborates the prognostic value of the clinical laboratory tests for patients with COVID-19.

A web application with the implementation of the BGM score model is available at “https://bgm-hoc.shinyapps.io/Shiny_covid_clinic/”.

Discussion

This retrospective study identified risk factors for death in hospitalized patients with COVID-19. Older age, lower LYMPH and PLT in addition to higher CREA, DD, CRP, and TNI were independent risk factors for death among patients. Taking this into account, we developed a predictive algorithm for mortality (BGM score) considering standard laboratory tests usually present in most central clinical laboratories.

Concerning biochemical, hematological, and coagulation parameters, our findings are in consonance with those previously described. For instance, a recent published work by Sisó-Almirall et al (15) revealed that LDH, DD, and CRP were the most important laboratory parameters significantly associated with adverse outcomes, evaluated as death or ICU admission.

Alterations in coagulation parameters, in particular high DD level and low PLT, have been linked with severe COVID-19 patients (16,17). These disorders reflect the hypercoagulable state present in poor prognosis, which could promote microthrombosis in the lungs, as well as in other organs (18).

Elevated TNI levels are frequent in patients with COVID-19 and have been significantly associated with fatal outcomes. Several mechanisms may explain this phenomenon: viral myocarditis, cytokine-driven myocardial damage, microangiopathy, and unmasked coronary artery disease. SARS-CoV-2 uses angiotensin-converting enzyme 2 (ACE2) as its entry receptor and subsequently downregulates ACE2 expression. This mechanism may complicate the clinical course mediated through inflammatory response, endothelial dysfunction, and microvascular damage (19).

Early monitoring of immunological biomarkers is an important basis to guide treatment strategies in COVID-19. In this study, CRP was the only immunological biomarker assessed significantly related to mortality. Recently, a meta-analysis including 16 independent studies highlighted the importance of CRP as a possible biomarker for mortality due to COVID-19 infection (20). Furthermore, the study by Wang (21) showed that CRP levels were positively correlated with lung lesion and disease severity in the early stage of COVID-19.

Lymphopenia is a common feature in patients with COVID-19. Significant decreases in T-cell counts have been observed in patients with severe disease (22). Up to now, the underlying mechanisms leading to the observed lymphopenia are little known and better understanding will provide insight into better management of such patients (23).

A high serum CREA level is a frequently observed complication in nonsurvivor inpatients (24). It has been described that around 20% of patients admitted to an ICU require renal replacement therapy 15 days after illness onset (25).

Previous studies reported comorbidities to be one of the most important risk factors associated with increased disease severity (6,7,26). Likewise, our study reported a significant association between mortality and some of the collected comorbidities, including diabetes, dyslipidemia, hypertension, heart failure, and ischemic heart disease. Despite these differences found between survivors and nonsurvivors, none of our prediction models did include any comorbidity since the clinical laboratory measurements were stronger predictors of mortality in patients with COVID-19.

Age was the only nonlaboratory-related variable associated with symptom aggravation in our study. This is in accordance with previous publications reporting age to be the most important predictor of death in patients with COVID-19 (27). Immunosenescence is defined as the declined ability of elderly patients to react properly upon infection, to initiate and maintain an adequate protective immune response, and to develop immunological memory (28). Thus, the severity of viral infections (e.g., influenza, respiratory syncytial virus) is notably increased among older adults compared to younger individuals, and more acute and long-term sequelae often develop as a result (29,30).

Other studies have been published using machine learning models to predict the mortality in COVID-19 patients, and some of them were included in systematic reviews and meta-analysis (31). Our study presents some common points with these publications since the predictors used in the BGM score were also identified to be relevant predictors of mortality in other models, supporting their significant association with adverse patient outcomes. One of our study’s strengths is that we have used a relatively large sample size compared with the research items cited in the meta-analysis, including a substantial number of nonsurvivors. Also, our final algorithm, the BGM score, only includes a small number of simple laboratory measurements, which makes our model easy to implement in the routine clinical practice. It’s noteworthy that all the variables of our study were collected in the first 48 hours of admission. Hence, our model can provide an early detection of patients at high risk of death, favoring early interventions.

Our study has several limitations. First, it was a retrospective single-center study, which may lead to biased results. Second, we recruited only patients with moderate or severe-stage disease and not asymptomatic or mild-stage disease. Therefore, prospective multicenter studies including patients with various stages of disease are warranted to confirm the reliability of the BGM score model.

The effects the pandemic is causing on medical resources worldwide highlight the need to develop early predictor models capable of detecting which patients can be managed safely at district hospital or can benefit from domiciliary hospitalization programs and which ones will need intensive care. Therefore, identifying risk factors at presentation that predict the likelihood of disease progression will be useful to: (1) increase the therapeutic effect in patients with a risk of higher disease progression and (2) reduce the mean hospitalization time in patients not at risk.

To conclude, we have developed an easy-to-use model comprising biochemical, hematological, and coagulation parameters presented in most clinical laboratories able to predict the survival probability of a patient with COVID-19 with high accuracy. This web tool may support clinicians in managing this infectious disease and providing focused interventions to patients with COVID-19 at a higher risk of death.

Disclosures

Financial support: This work was supported by grants to MM-R (PID2019-105502RB-100) and to WJ (RTI2018-094734-B-C21) from Dirección General de Investigación Científica y Técnica and Agència de Gestió d’Ajuts Universitaris i de Recerca (SGR 2017/2019). The work was cofinanced by FEDER of European Union. The Centro de Investigación Biomédica en Red de Enfermedades Hepáticas y Digestivas (CIBERehd) is funded by Instituto de Salud Carlos III.

Conflict of interest: The authors have no financial relationships to disclose relevant to this study.