Advertisement

Risk Prediction for Acute Kidney Injury in Patients Hospitalized With COVID-19

Open AccessPublished:April 07, 2022DOI:https://doi.org/10.1016/j.xkme.2022.100463

      Rationale & Objective

      Acute kidney injury (AKI) is common in patients hospitalized with COVID-19, but validated, predictive models for AKI are lacking. We aimed to develop the best predictive model for AKI in hospitalized patients with coronavirus disease 2019 and assess its performance over time with the emergence of vaccines and the Delta variant.

      Study Design

      Longitudinal cohort study.

      Setting & Participants

      Hospitalized patients with a positive severe acute respiratory syndrome coronavirus 2 polymerase chain reaction result between March 1, 2020, and August 20, 2021 at 19 hospitals in Texas.

      Exposures

      Comorbid conditions, baseline laboratory data, inflammatory biomarkers.

      Outcomes

      AKI defined by KDIGO (Kidney Disease: Improving Global Outcomes) creatinine criteria.

      Analytical Approach

      Three nested models for AKI were built in a development cohort and validated in 2 out-of-time cohorts. Model discrimination and calibration measures were compared among cohorts to assess performance over time.

      Results

      Of 10,034 patients, 5,676, 2,917, and 1,441 were in the development, validation 1, and validation 2 cohorts, respectively, of whom 776 (13.7%), 368 (12.6%), and 179 (12.4%) developed AKI, respectively (P = 0.26). Patients in the validation cohort 2 had fewer comorbid conditions and were younger than those in the development cohort or validation cohort 1 (mean age, 54 ± 16.8 years vs 61.4 ± 17.5 and 61.7 ± 17.3 years, respectively, P < 0.001). The validation cohort 2 had higher median high-sensitivity C-reactive protein level (81.7 mg/L) versus the development cohort (74.5 mg/L; P < 0.01) and higher median ferritin level (696 ng/mL) versus both the development cohort (444 ng/mL) and validation cohort 1 (496 ng/mL; P < 0.001). The final model, which added high-sensitivity C-reactive protein, ferritin, and D-dimer levels, had an area under the curve of 0.781 (95% CI, 0.763-0.799). Compared with the development cohort, discrimination by area under the curve (validation 1: 0.785 [0.760-0.810], P = 0.79, and validation 2: 0.754 [0.716-0.795], P = 0.53) and calibration by estimated calibration index (validation 1: 0.116 [0.041-0.281], P = 0.11, and validation 2: 0.081 [0.045-0.295], P = 0.11) showed stable performance over time.

      Limitations

      Potential billing and coding bias.

      Conclusions

      We developed and externally validated a model to accurately predict AKI in patients with coronavirus disease 2019. The performance of the model withstood changes in practice patterns and virus variants.

      Index Words

      As the COVID-19 pandemic evolved, acute kidney injury (AKI) has remained a common complication in patients hospitalized with COVID-19. We developed models to predict the occurrence of AKI in patients hospitalized with COVID-19. We built models using patient characteristics, comorbid conditions, and laboratory values. The model with the most accurate prediction included patient demographic characteristics, comorbid conditions, and inflammatory blood tests acquired on hospital admission. We also validated that the model remained effective as the pandemic progressed and the Delta variant emerged. The model was compared between the initial cohort and 2 cohorts of patients hospitalized later in the pandemic as the Delta variant emerged. We found that the model was stable and could predict AKI.
      The coronavirus disease 2019 (COVID-19) may have a multiorgan involvement, and acute kidney injury (AKI) is a known sequela. Potential mechanisms include acute tubular injury because of hypotension, intravascular volume depletion leading to prerenal injury, and acute interstitial nephritis from antibiotics and antiviral treatments. However, COVID-19–specific mechanisms are also likely to be involved in kidney injury, as evidenced by case reports and postmortem autopsies.
      • Braun F.
      • Lütgehetmann M.
      • Pfefferle S.
      • et al.
      SARS-CoV-2 renal tropism associates with acute kidney injury.
      ,
      • Hassler L.
      • Reyes F.
      • Sparks M.A.
      • Welling P.
      • Batlle D.
      Evidence for and against direct kidney infection by SARS-CoV-2 in patients with COVID-19.
      Observational studies that controlled for typical AKI risk factors have shown that COVID-19 is associated with AKI, suggesting that part of the pathogenesis may be specific to the virus itself.
      • Moledina D.G.
      • Simonov M.
      • Yamamoto Y.
      • et al.
      The association of COVID-19 with acute kidney injury independent of severity of illness: a multicenter cohort study.
      COVID-19–related mechanisms may include kidney ischemia from hypercoagulability, inflammation-related kidney injury in the setting of cytokine storm, and cardiorenal syndrome.
      • McAdams M.
      • Ostrosky-Frid M.
      • Rajora N.
      • Hedayati S.S.
      Effect of COVID-19 on kidney disease incidence and management.
      Older age; male sex; and a history of diabetes mellitus, chronic kidney disease (CKD), and hypertension are associated with AKI in patients with COVID-19.
      • Gupta S.
      • Coca S.G.
      • Chan L.
      • et al.
      AKI treated with renal replacement therapy in critically ill patients with COVID-19.
      • Bowe B.
      • Cai M.
      • Xie Y.
      • Gibson A.K.
      • Maddukuri G.
      • Al-Aly Z.
      Acute kidney injury in a national cohort of hospitalized US veterans with COVID-19.
      • Chan L.
      • Chaudhary K.
      • Saha A.
      • et al.
      AKI in hospitalized patients with COVID-19.
      • Hirsch J.S.
      • Ng J.H.
      • Ross D.W.
      • et al.
      Acute kidney injury in patients hospitalized with COVID-19.
      • Fisher M.
      • Neugarten J.
      • Bellin E.
      • et al.
      AKI in hospitalized patients with and without COVID-19: a comparison study.
      • Pelayo J.
      • Lo K.B.
      • Bhargav R.
      • et al.
      Clinical characteristics and outcomes of community- and hospital-acquired acute kidney injury with COVID-19 in a US inner city hospital system.
      However, predictive models with temporal external validation to prognosticate the development of AKI in the setting of hospitalization for COVID-19 are lacking. This is especially pertinent given the evolution of the pandemic over time because of changes in clinical management, vaccination, and emergence of new variants.
      With the rollout of COVID-19 vaccines, cases began to decline in late 2020 and early 2021. However, the emergence of the Delta variant in March 2021 led to a subsequent rebound in cases. Delta soon became the dominant variant, with 70% and 98% of US COVID-19 cases attributable to the variant by July 1, 2021 and by August 1, 2021, respectively.

      Delta Variant Report. Alaa Abdel Latif, Julia L. Mullen, Manar Alkuzweny, Ginger Tsueng, Marco Cano, Emily Haag, Jerry Zhou, Mark Zeller, Emory Hufbauer, Nate Matteson, Chunlei Wu, Kristian G. Andersen, Andrew I. Su, Karthik Gangavarapu, Laura D. Hughes, and the Center for Viral Systems Biology. outbreak.info. Available at https://outbreak.info/situation-reports/delta?pango=Delta&loc=USA&selected=USA. Accessed 29 October 2021

      The United States saw a rapid increase in hospitalizations because of a marked increase in transmissibility and number of breakthrough infections with the Delta variant compared to the original strains.
      • Brown C.M.
      • Vostok J.
      • Johnson H.
      • et al.
      Outbreak of SARS-CoV-2 infections, including COVID-19 vaccine breakthrough infections, associated with large public gatherings - Barnstable County, Massachusetts, July 2021.
      ,
      • Hagan L.M.
      • McCormick D.W.
      • Lee C.
      • et al.
      Outbreak of SARS-CoV-2 B.1.617.2 (Delta) variant infections among incarcerated persons in a federal prison - Texas, July-August 2021.
      Increasing evidence suggests that infection with the Delta variant leads to more severe illness in unvaccinated individuals
      • Hagan L.M.
      • McCormick D.W.
      • Lee C.
      • et al.
      Outbreak of SARS-CoV-2 B.1.617.2 (Delta) variant infections among incarcerated persons in a federal prison - Texas, July-August 2021.
      • Sheikh A.
      • McMenamin J.
      • Taylor B.
      • Robertson C.
      Public Health Scotland and the EAVE II Collaborators. SARS-CoV-2 Delta VOC in Scotland: demographics, risk of hospital admission, and vaccine effectiveness.
      • Ong S.W.X.
      • Chiew C.J.
      • Ang L.W.
      • et al.
      Clinical and virological features of SARS-CoV-2 variants of concern: a retrospective cohort study comparing B.1.1.7 (Alpha), B.1.315 (Beta), and B.1.617.2 (Delta). Clin Infect Dis.
      and an increased risk of hospitalization compared with the previously prominent variants.
      • Twohig K.A.
      • Nyberg T.
      • Zaidi A.
      • et al.
      Hospital admission and emergency care attendance risk for SARS-CoV-2 delta (B.1.617.2) compared with Alpha (B.1.1.7) variants of concern: a cohort study.
      ,
      • Bager P.
      • Wohlfahrt J.
      • Rasmussen M.
      • Albertsen M.
      • Krause T.G.
      Hospitalisation associated with SARS-CoV-2 delta variant in Denmark.
      It is unclear, however, what effect the Delta variant has on AKI pathogenesis, incidence, and outcomes.
      We, therefore, aimed to (1) derive a prognostic model to predict the risk of AKI using granular demographic, clinical, and laboratory variables, including inflammatory biomarkers, during an index hospitalization with COVID-19; (2) externally evaluate the model’s predictive performance based on discrimination and calibration with out-of-time cohorts; and (3) assess data shifts in the pandemic because of varying treatments and emergence of the Delta variant.

      Methods

      Study Design and Participants

      We conducted a longitudinal study using the University of Texas Southwestern (UTSW) COVID-19 Registry Collaborative database that contains granular demographic, clinical, and laboratory data on all COVID-19–related hospital admissions at 19 hospitals in North Texas, including UTSW Clements University Hospital and 18 hospitals in the Texas Health Resources Health System. Ethics approval and written informed consent were not required because the research was declared exempt by the institutional review board at UTSW as nonhuman research using existing data from the COVID-19 registry. For the development cohort, we included all individuals aged 18 years or older with a positive SARS-CoV-2 polymerase chain reaction test admitted between March 1, 2020, and January 1, 2021. The first hospital admission after the first positive polymerase chain reaction or up to 10 days prior was used. Only the first hospital admission was used if an individual had multiple admissions within the timeframe. We excluded the following patients: (1) those admitted for observation, (2) those without a COVID-19-associated billing diagnosis, (3) those with end-stage kidney disease or kidney transplantation, (4) those without comorbid condition information or a serum creatinine measurement during hospitalization, and (5) those in whom AKI was already present on admission (Fig S1). Data were collected for the entire hospitalization or a minimum of 30 days. Two out-of-time validation cohorts were used. The validation cohort 1 included patients with a positive SARS-CoV-2 polymerase chain reaction result admitted between January 2, 2021, and June 30, 2021, whereas the validation cohort 2 included those admitted between July 1, 2021, and August 20, 2021. The same exclusion criteria were applied to the validation cohorts. Data for July and August 2021 were grouped together because the COVID-19 Delta variant was the predominant strain at both UTSW and in Texas at-large based on variant testing.

      Delta Variant Report. Alaa Abdel Latif, Julia L. Mullen, Manar Alkuzweny, Ginger Tsueng, Marco Cano, Emily Haag, Jerry Zhou, Mark Zeller, Emory Hufbauer, Nate Matteson, Chunlei Wu, Kristian G. Andersen, Andrew I. Su, Karthik Gangavarapu, Laura D. Hughes, and the Center for Viral Systems Biology. outbreak.info. Available at https://outbreak.info/situation-reports/delta?selected=USA_US-TX&loc=USA&loc=Worldwide&loc=USA_US-TX. Accessed 29 October 2021

      Clinical and Laboratory Variables

      Demographic variables, past medical history, and medications before admission were extracted from the electronic health records (Epic Systems). We collected treatments during hospitalization, including medications and mechanical ventilation, intensive care unit use, and kidney replacement therapy. We obtained laboratory variables at admission and during hospitalization. Comorbid conditions were considered present if the condition was listed as active on the patient’s problem list, medical history, or hospital problem and/or used 2 or more times (to improve specificity) as an encounter or billing diagnosis. Computable condition definitions were expressed as intentional (rule-based) value sets defined using standard Systemized Nomenclature of Medicine-Clinical Terms (SNOMED-CT) nomenclature as previously described (125 condition definitions are included as an online Supplement to this reference).
      • Willett D.L.
      • Kannan V.
      • Chu L.
      • et al.
      SNOMED CT concept hierarchies for sharing definitions of clinical conditions using electronic health record data.
      Computable definitions of laboratory variables and inflammatory biomarkers were expressed using an internally vetted value set field within the Epic electronic health record. This field (“Common Name”) is used to determine which laboratory results can be displayed together for clinical viewing within the electronic health record to meet pathology and laboratory accreditation requirements.
      Weekly sampling was performed for COVID-19 variant testing from August 2020 to August 2021 in patients from the UTSW Medical Center. All COVID-19–positive specimens with a cycle threshold value <35 by real-time polymerase chain reaction (approximately half of the positive samples) were sent for Sanger sequencing to determine the variant of coronavirus (Fig 1).
      Figure thumbnail gr1
      Figure 1COVID-19 variant proportions over time by week at the University of Texas Southwestern Medical Center.

      AKI Definitions

      A detailed prespecified algorithm was used to define AKI according to standard KDIGO (Kidney Disease: Improving Global Outcomes) criteria: stage 1—increase in serum creatinine level by 0.3 mg/dL within 48 hours or 1.5-1.9 times increase from the baseline; stage 2—2.0- 2.9 times increase; and stage 3—≥3.0 times increase or initiation of renal replacement therapy (Fig S2).
      • Khwaja A.
      KDIGO clinical practice guidelines for acute kidney injury.
      Urine output criteria were not used, given the variability in documentation. Patients were stratified according to the highest AKI stage attained during hospitalization. AKI was categorized as present on admission if the creatinine level at admission was higher than that at baseline and met one of the AKI stage criteria. We defined “baseline creatinine” by adapting and modifying a published algorithm, based on the availability or absence of prior serum creatinine values (Fig S3).
      • Hirsch J.S.
      • Ng J.H.
      • Ross D.W.
      • et al.
      Acute kidney injury in patients hospitalized with COVID-19.

      Outcome Measures

      The prespecified primary outcome was in-hospital development of AKI that was not present on admission.

      Statistical Analysis

      Categorical data were presented as percentages and continuous data as means ± standard deviation (SD) or median (25th, 75th percentile). To compare categorical variables between patients with and without AKI, Fisher exact or χ2 tests were used as indicated. For continuous variables, Mann-Whitney U test was applied. For comparisons among the development, validation 1, and validation 2 data sets, continuous variables were compared using the nonparametric analysis of variance (Kruskal-Wallis test) followed by Dunn’s test. Categorical variables were compared using the Pearson’s χ2 test. Pairwise comparisons between data sets were performed if the overall test was significant. P values were adjusted with the Holm’s method. For comparison across AKI stages, Cochran-Armitage trend test was used for categorical variables and Jonckheere-Terpstra test was used for continuous variables.
      To build a predictive model for AKI, we assessed a group of patients without AKI on admission and excluded the treatments and laboratory tests performed after the development of AKI while hospitalized. Prespecified covariates selected based on clinical relevance and biological plausibility were age, sex, race, ethnicity, smoking, hypertension, CKD, diabetes mellitus, coronary artery disease (CAD), congestive heart failure (CHF), use of an angiotensin-converting enzyme inhibitor (ACEI) or angiotensin receptor blocker (ARB), initial levels of white blood cell (WBC) count, high-sensitivity C-reactive protein (hs-CRP) level, hemoglobin level, ferritin level, and D-dimer level. Lactate dehydrogenase level was not included because of 26% missing values. Nested multivariable logistic models were built on top of a base model 1 that included demographic characteristics and comorbid conditions (age, sex, race, ethnicity, smoking, hypertension, diabetes mellitus, CKD, CAD, and CHF), as well as ACEI/ARB use. Model 2 added admission laboratory values commonly obtained in the setting of COVID-19—WBC count, hs-CRP level, and hemoglobin level—to model 1. Model 3 added additional inflammatory biomarkers, ferritin and D-dimer, to model 2. Nested models were compared with likelihood ratio tests after adding each biomarker group. A separate missing category for categorical variables was created, and complete case analysis was used for modeling. The estimated effect was reported by adjusted odds ratios with their 95% confidence intervals. We assessed the models for multicollinearity using variance inflation factors, where a variance inflation factor >10 suggested multicollinearity.
      • Neter J.
      • Wasserman W.
      • Kutner M.H.
      Applied Linear Regression Models.
      No multicollinearity was found by variance inflation factor analysis. A mathematic model was derived to compute the log odds of AKI that could be transformed to predict the probability of AKI. The variable importance was assessed by the absolute value of the z statistic for each variable in the models.
      • Kuhn M.
      Building predictive models in R using the caret package.
      The performance of the prognostic models was internally validated by 10-fold stratified cross-validation and externally validated with out-of-time validation over 2 intervals. Model discrimination was measured using areas under the receiver operating characteristic curve. Optimal predicted probability cut-offs were determined by Youden’s index from receiver operating characteristic analyses. Sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio were reported as an average (± SD) across 10-fold cross-validation. Model calibration was evaluated with calibration curves and estimated calibration index,
      • Van Hoorde K.
      • Van Huffel S.
      • Timmerman D.
      • Bourne T.
      • Van Calster B.
      A spline-based tool to assess and visualize the calibration of multiclass risk predictions.
      which is the mean squared difference between predicted AKI probabilities and estimated observed probabilities based on the flexible calibration curves fitted by locally weighted scatterplot smoothing method.
      • Van Hoorde K.
      • Van Huffel S.
      • Timmerman D.
      • Bourne T.
      • Van Calster B.
      A spline-based tool to assess and visualize the calibration of multiclass risk predictions.
      • Davis S.E.
      • Lasko T.A.
      • Chen G.
      • Siew E.D.
      • Matheny M.E.
      Calibration drift in regression and machine learning models for acute kidney injury.
      • Van Calster B.
      • Nieboer D.
      • Vergouwe Y.
      • De Cock B.
      • Pencina M.J.
      • Steyerberg E.W.
      A calibration hierarchy for risk models was defined: from utopia to empirical data.
      Bootstrap percentile-based confidence intervals were obtained from 2,000 bootstrap samples. Data shift was measured with population stability index (PSI), which summarizes the difference between 2 statistical distributions.
      • Karakoulas G.
      Empirical validation of retail credit-scoring models.
      A PSI < 0.10 indicated no significant changes, PSI between 0.10 and 0.25 indicated a small change that needs to be assessed, and PSI > 0.25 suggested a significant change.
      • Siddiqi N.
      Intelligent Credit Scoring: Building and Implementing Better Credit Risk Scorecards.

      Results

      Baseline Characteristics

      The development cohort, validation cohort 1, and validation cohort 2 contained 5,676 individuals, 2,917 individuals, and 1,441 individuals, respectively. Demographic characteristics, comorbid conditions, laboratory values, and clinical outcomes of the 3 cohorts are shown in Table 1. Comparisons by presence and stage of AKI in the development cohort are reported in Table S1. In the development cohort, 2,812 (49.5%) were men, 844 (15.8%) were African American, and 1,657 (30.2%) were Hispanic. The development cohort contained more Hispanic patients than the validation cohorts, driven by a peak in COVID-19 cases among the Hispanic population in the early summer of 2020. Patients in the validation cohort 2 were younger than patients in the development cohort or validation cohort 1; the mean ages were 54 ± 16.8 years versus 61.4 ± 17.5 years and 61.7 ± 17.3 years, respectively (adjusted P < 0.001). The patients in the validation cohort 2 had lower rates of hypertension, diabetes mellitus, CAD, CHF, and CKD than those in the development cohort and validation cohort 1. The patients in the validation cohort 2 also had higher hs-CRP levels (adjusted P < 0.01) than those in the development cohort and higher levels of ferritin than those in the development cohort and validation cohort 1 (adjusted P < 0.001).
      Table 1Baseline Characteristics and Outcomes of the Development and Validation Cohorts With Population Stability Indices of Variables in the Validation Cohorts
      Development

      March-December 2020

      N = 5,676
      Validation 1

      January-June, 2021

      N = 2,917
      Validation 2

      July-August 2021

      N = 1,441
      Overall

      P Value
      Baseline characteristic, n (%)
      AKI incidence776 (13.7%)368 (12.6%)179 (12.4%)0.26
      Age, y, mean ± SD61.4 ± 17.5
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      61.7 ± 17.3
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      54.0 ± 16.8<0.001
      Male sex2,812 (49.5%)1448 (49.6%)725 (50.3%)0.87
      Hispanic ethnicity1,657 (30.2%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      ,
      P<0.001: Pairwise comparison between the development cohort and validation cohort 1.
      704 (24.7%)
      P<0.05: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      310 (21.9%)<0.001
      African American race844 (15.8%)418 (14.9%)240 (17.0%)0.20
      Smoker (smoked at any time)1,913 (34.1%)1037 (36.0%)
      P<0.01: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      446 (31.3%)0.008
      Hypertension3,380 (59.5%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      1747 (59.9%)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      602 (41.8%)<0.001
      Diabetes mellitus1,967 (34.7%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      996 (34.1%)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      344 (23.9%)<0.001
      CKD1,238 (21.8%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      ,
      P<0.05: Pairwise comparison between the development cohort and validation cohort 1.
      570 (19.5%)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      212 (14.7%)<0.001
      CAD689 (12.1%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      368 (12.6%)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      106 (7.4%)<0.001
      CHF545 (9.6%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      292 (10.0%)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      77 (5.3%)<0.001
      ACEI/ARB (on presentation)2,195 (38.7%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      1192 (40.9%)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      372 (25.8%)<0.001
      Laboratory variable, median (IQR)
      WBC count, initial, K/μL (median (IQR))6.8 (5.1-9.3)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      ,
      P<0.001: Pairwise comparison between the development cohort and validation cohort 1.
      7.2 (5.2-9.8)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      6.5 (4.9-8.8)<0.001
      hs-CRP level, initial, mg/L (median (IQR))74.5 (32.0-135.4)
      P<0.01: Pairwise comparison between the development cohort and validation cohort 2.
      78.9 (34.2-141.1)81.7 (43.2-137.0)0.004
      Hemoglobin level, initial, g/dL (median (IQR))13.4 (12.0-14.6)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      13.4 (12.1-14.6)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      13.9 (12.5-15.0)<0.001
      Ferritin level, initial, ng/mL (median (IQR))444.0 (205.1-961.1)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      ,
      P<0.001: Pairwise comparison between the development cohort and validation cohort 1.
      495.8 (237.0-1068.0)
      P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      696.0 (298.2-1511.0)<0.001
      D-dimer level, initial, mg/L (median (IQR))0.9 (0.6-1.7)
      P<0.01Pairwise comparison between the development cohort and validation cohort 1.
      1.0 (0.6-1.8)
      P<0.01: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      0.9 (0.6-1.6)<0.001
      Outcome, n (%)
      ICU895 (15.8%)
      P<0.001: Pairwise comparison between the development cohort and validation cohort 1.
      369 (12.6%)193 (13.4%)<0.001
      Mechanical ventilation427 (7.5%)184 (6.3%)105 (7.3%)0.11
      In-hospital mortality within 30 d387 (6.8%)185 (6.3%)107 (7.4%)0.40
      CRRT for AKI33 (0.6%)10 (0.3%)10 (0.7%)0.23
      HD for AKI126 (2.2%)60 (2.1%)20 (1.4%)0.14
      CRRT and HD for AKI7 (0.1%)5 (0.2%)1 (0.1%)0.67
      CRRT/HD for AKI152 (2.7%)65 (2.2%)29 (2.0%)0.23
      PSI
      AKI0.0010.001
      Age0.0030.191
      Male sex00
      Hispanic ethnicity0.0190.045
      African American race0.0080.037
      Smoker (smoked at any time)0.0020.003
      Hypertension00.128
      Diabetes mellitus00.056
      CKD0.0030.034
      CAD00.027
      CHF00.027
      ACEI/ARB (on presentation)0.0020.076
      WBC count, initial, K/μL0.0120.017
      hs-CRP level, initial, mg/L0.0090.037
      Hemoglobin level, initial, g/dL0.0040.061
      Ferritin level, initial, ng/mL0.0180.135
      D-dimer level, initial, mg/L0.0080.016
      Note: Percentages for categorical variables were obtained using the total number of patients with available variable as the denominator. For overall P value, continuous variables were compared using nonparametric analysis of variance (Kruskal-Wallis test), followed by Dunn’s test. Categorical variables were compared using Pearson’s χ2 test. Pairwise comparisons among data sets were performed if an overall test was significant. P values were adjusted with the Holm’s method. PSI summarizes the difference between the development and validation cohorts. A PSI < 0.10 indicates no significant change, PSI between 0.10 and 0.25 indicates a small change, and PSI > 0.25 indicates a significant change.
      • Siddiqi N.
      Intelligent Credit Scoring: Building and Implementing Better Credit Risk Scorecards.
      Abbreviations: ACEI/ARB, angiotensin-converting enzyme inhibitors/angiotensin II receptor blocker; AKI, acute kidney injury; CAD, coronary artery disease; CHF, congestive heart failure; CKD, chronic kidney disease; CRRT, continuous renal replacement therapy; HD, hemodialysis; hs-CRP, high-sensitivity C-reactive protein; ICU, intensive care unit; IQR, interquartile range; PSI, population stability index; SD, standard deviation; WBC, white blood cell.
      a P<0.001: Pairwise comparison between the development cohort and validation cohort 2.
      b P<0.001: Pairwise comparison between the development cohort and validation cohort 1.
      c P<0.05: Pairwise comparison between the development cohort and validation cohort 1.
      d P<0.01: Pairwise comparison between the development cohort and validation cohort 2.
      e P<0.01Pairwise comparison between the development cohort and validation cohort 1.
      f P<0.001: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      g P<0.05: Pairwise comparison between the validation cohort 1 and validation cohort 2.
      h P<0.01: Pairwise comparison between the validation cohort 1 and validation cohort 2.

      Outcomes

      The outcomes of the 3 cohorts are illustrated in Table 1. The outcomes of the development cohort grouped by presence and stage of AKI are shown in Table S1. There were 776 (13.7%) patients in the development cohort, 368 (12.6%) in validation cohort 1, and 179 (12.4%) in validation cohort 2 who developed AKI. There was no difference in AKI rates among the cohorts. The rates of mechanical ventilation, in-hospital mortality, and kidney replacement therapy were also similar among the 3 cohorts. The rate of intensive care unit admission was significantly higher in the development cohort than in the validation cohort 1 (adjusted P < 0.001). Age, hypertension, and ferritin level were found to show a small change over time in the validation cohort 2 (0.10 < PSI < 0.25) (Table 1).

      COVID-19 Variant Testing

      Figure 1 shows the results of weekly testing for COVID-19 variants at UTSW from August 2020 to August 2021. The Delta variant was first seen in April 2021 but did not become the dominant variant until the week of June 27, 2021, when it comprised 77% of the total COVID-19 cases. The Delta variant made up 97% of all samples by the week of July 11, 2021 and 100% by August 8, 2021.

      Predictive Models for AKI During Hospitalization

      Receiver operating characteristic curves for nested models for the development cohort are illustrated in Fig 2A. Model 1, the base model (age, sex, race, ethnicity, smoking status, underlying hypertension, diabetes mellitus, CKD, CAD, CHF, and ACEI/ARB use at baseline), had an area under the curve (AUC) (95% confidence interval) of 0.753 (0.734-0.771). Model 2 (which added admission WBC, hs-CRP, and hemoglobin to model 1), had a higher AUC of 0.764 (95% confidence interval, 0.746-0.783) compared with model 1 (likelihood ratio test adjusted P < 0.001). Model 3, which contained all variables in model 2 plus admission ferritin and D-dimer levels, had the highest AUC of 0.781 (0.763-0.799) (adjusted P < 0.001 for comparison with model 2). Each subsequent model showed a statistically significant improvement with the likelihood ratio test. The adjusted odds ratios for all variables included in the models are shown in Table 2. The metrics used to compare the performance of the AKI predictive models from internal validation are shown in Table S2. Variable importance for each model showed that the most important variable for all 3 models was the history of CKD; the second most important variable was hs-CRP level for model 2 and ferritin level for model 3 (Fig S4). Model 3 had a sensitivity of 0.69 (SD, 0.10), specificity of 0.76 (SD, 0.08), negative predictive value of 0.94 (SD, 0.01), positive predictive value of 0.32 (SD, 0.04), positive likelihood ratio 3.02 (SD, 0.62), and negative likelihood ratio of 0.40 (SD, 0.09) when using Youden’s index as the probability cut off for AKI.
      Figure thumbnail gr2
      Figure 2Receiver operating characteristic curves for nested acute kidney injury models for (A) development cohort, (B) validation cohort 1, and (C) validation cohort 2. Model 1 contains age, sex, race, ethnicity, smoking status, hypertension, diabetes mellitus, chronic kidney disease, coronary artery disease, congestive heart failure, and angiotensin-converting enzyme inhibitor/angiotensin receptor blocker use. Model 2 contains all variables in model 1 plus initial white blood cell count, high-sensitivity C-reactive protein level, and hemoglobin level. Model 3 contains all variables in model 2 plus initial ferritin and D-dimer levels. Abbreviation: AUC, area under the curve.
      Table 2Predictive Models for AKI in the Development Cohort
      N = 5,676Model 1Model 2Model 3
      aOR (95%CI)P ValueaOR (95%CI)P ValueaOR (95%CI)P Value
      Age, per year1.011 (1.005-1.017)<0.0011.010 (1.004-1.016)0.0011.009 (1.003-1.015)0.006
      Male sex1.369 (1.160-1.616)<0.0011.382 (1.159-1.649)<0.0011.214 (1.012-1.457)0.04
      Hispanic1.467 (1.192-1.805)<0.0011.368 (1.108-1.688)0.0041.373 (1.101-1.700)0.004
      African American1.281 (1.005-1.632)0.051.234 (0.965-1.577)0.091.114 (0.867-1.433)0.40
      Smoker0.922 (0.772-1.101)0.370.932 (0.779-1.115)0.440.976 (0.814-1.170)0.79
      Hypertension1.450 (1.157-1.817)0.0011.454 (1.158-1.828)0.0011.448 (1.149-1.825)0.002
      Diabetes1.448 (1.220-1.719)<0.0011.416 (1.191-1.685)<0.0011.518 (1.272-1.811)<0.001
      CKD4.766 (3.994-5.687)<0.0014.588 (3.829-5.499)<0.0014.288 (3.567-5.154)<0.001
      CAD1.065 (0.844-1.342)0.601.070 (0.846-1.355)0.571.110 (0.874-1.409)0.39
      CHF1.244 (0.979-1.579)0.071.256 (0.984-1.602)0.071.296 (1.013-1.659)0.04
      ACEI/ARB (on admission)0.976 (0.816-1.168)0.791.013 (0.844-1.215)0.891.070 (0.889-1.289)0.47
      WBC count, per K/µL increase1.010 (1.001-1.018)0.031.009 (1.001-1.018)0.04
      hs-CRP level, per mg/L increase1.004 (1.003-1.004)<0.0011.003 (1.002-1.004)<0.001
      Hemoglobin level, per g/dL increase0.928 (0.889-0.968)<0.0010.938 (0.899-0.980)0.004
      Ferritin level, per ng/mL increase1.0002 (1.0001-1.0003)<0.001
      D-dimer level, per mg/L increase1.055 (1.034-1.077)<0.001
      Note: Model 1 contains age, sex, race, ethnicity, smoking status, hypertension, diabetes mellitus, CKD, CAD, CHF, and ACEI/ARB use. Model 2 contains all variables in model 1 plus initial WBC count, hs-CRP level, and hemoglobin level. Model 3 contains all variables in model 2 plus initial ferritin and D-dimer levels. P values obtained using Wald’s test. Those with AKI present on admission and those with missing information on comorbid conditions or continuous variables were excluded.
      Abbreviations: ACEI/ARB, angiotensin-converting enzyme inhibitor/angiotensin receptor blocker; aOR, adjusted odds ratio; CAD, coronary artery disease; CHF, chronic heart failure; CI, confidence interval; CKD, chronic kidney disease; hs-CRP, high-sensitivity C-reactive protein; WBC, white blood cell.
      The following is the resultant formula for the AKI prediction model (model 3) that computes the log odds (logit) of AKI:Log odds of AKI = −3.4524 + 0.0087 (Age) + 0.1942 (Male∗) + 0.3171 (Hispanic∗) + 0.4016 (Hispanic missing#) + 0.1082 (African American∗) + 0.1111 (African American missing#) – 0.0248 (Smoker∗) + 0.5228 (Smoker missing#) + 0.3701 (Hypertension∗) + 0.4171 (Diabetes∗) + 1.4558 (CKD∗) + 0.1043 (CAD∗) + 0.2595 (CHF∗) + 0.068 (ACEI_ARB∗) + 0.009 (WBC) − 0.0636 (Hemoglobin) + 0.0025 (CRP) + 0.0002 (Ferritin) + 0.0536 (D-dimer)
      where ∗ indicates for male insert 1, for female insert 0; for Hispanic ethnicity insert 1, for non-Hispanic ethnicity insert 0; for African American race insert 1, for non-African American race insert 0; for smoker insert 1, for nonsmoker insert 0; for patients treated with ACEI or ARB or with hypertension, diabetes, CKD, CAD, or CHF insert 1 in the corresponding term, otherwise insert 0 and # indicates for missing Hispanic ethnicity, missing African American race, missing smoker, insert 1.
      The log odds of AKI can be transformed into the predicted probability of developing AKI with the following formula:
      Predicted AKI probability = 1/[1 + e(−1Logit AKI)]

      Model Evaluation Over Time

      Model discrimination was measured using AUC. Receiver operating characteristic curves for the nested AKI prediction models in validation cohorts 1 and 2 are illustrated in Fig 2B and C. The adjusted P values comparing the AUC for each model of the development cohort to both validation cohorts were all not significant, indicating stable discrimination over time (Table 3).
      Table 3Discrimination and Calibration Measures in the Development Cohort, Validation Cohort 1, and Validation Cohort 2
      Development Cohort

      March-December 2020
      Validation Cohort 1

      January-June 2021
      Validation Cohort 2

      July-August 2021
      Development vs Validation 1

      Adjusted P
      Development vs Validation 2

      Adjusted P
      Validation 1 vs Validation 2

      Adjusted P
      Model 1
      DiscriminationAUC0.7530.7570.740
      (95% CI)
      Bootstrap percentile-based confidence interval was obtained from 2,000 bootstrap samples for all CIs.
      (0.734-0.771)(0.730-0.784)(0.699-0.781)1.001.001.00
      CalibrationECI0.0050.0590.115
      (95% CI)
      Bootstrap percentile-based confidence interval was obtained from 2,000 bootstrap samples for all CIs.
      (0.004-0.043)(0.019-0.179)(0.070-0.340)0.11<0.0010.24
      Model 2
      DiscriminationAUC0.7640.7750.732
      (95% CI)
      Bootstrap percentile-based confidence interval was obtained from 2,000 bootstrap samples for all CIs.
      (0.746- 0.783)(0.749-0.800)(0.690-0.774)0.520.330.29
      CalibrationECI0.0040.0380.144
      (95% CI)
      Bootstrap percentile-based confidence interval was obtained from 2,000 bootstrap samples for all CIs.
      (0.003-0.046)(0.016-0.133)(0.083-0.356)0.13<0.0010.12
      Model 3
      DiscriminationAUC0.7810.7850.754
      (95% CI)
      Bootstrap percentile-based confidence interval was obtained from 2,000 bootstrap samples for all CIs.
      (0.763-0.799)(0.760-0.810)(0.716-0.795)0.790.530.53
      CalibrationECI0.0200.1160.081
      (95% CI)
      Bootstrap percentile-based confidence interval was obtained from 2,000 bootstrap samples for all CIs.
      (0.006-0.073)(0.041-0.281)(0.045-0.295)0.110.110.93
      Note: P value was adjusted with Holm’s method. Model 1 contains age, sex, race, ethnicity, smoking status, hypertension, diabetes mellitus, chronic kidney disease, coronary artery disease, congestive heart failure, and angiotensin-converting enzyme inhibitor/angiotensin receptor blocker use. Model 2 contains all variables in model 1 plus initial white blood cell count, high-sensitivity C-reactive protein level, and hemoglobin level. Model 3 contains all variables in model 2 plus initial ferritin and D-dimer levels.
      Abbreviations: AUC, area under the curve; CI, confidence interval; ECI, estimated calibration index.
      a Bootstrap percentile-based confidence interval was obtained from 2,000 bootstrap samples for all CIs.
      The calibration of all models for all cohorts is shown by calibration plots (Fig S5). The calibration plots for model 3 showed agreement between the predicted probabilities of AKI and the empirical probability, which aligned close to the diagonal reference line, suggesting good calibration. When comparing the development cohort to validation cohort 2, there was a statistically significant decrease in the calibration in both models 1 and 2. Model 3 showed no statistically significant changes in the estimated calibration index when the groups were compared, suggesting stable calibration over time (Table 3).

      Discussion

      To our knowledge, we report the first internally and externally out-of-time validated AKI prediction model in patients hospitalized with COVID-19. We generated 3 models for the development of AKI in a contemporary cohort of nearly 6,000 patients across 19 hospitals. Our models, which included clinically available demographic characteristics, comorbid conditions, and inflammatory biomarkers routinely obtained during hospitalization with COVID-19, were validated in 2 cohorts (a total of 4,358 patients hospitalized later in the pandemic) to account for emerging virus variants and increasing vaccination rates. Our final model maintained stable discrimination and calibration despite these developments in the pandemic, supporting the clinical utility of this model.
      As the pandemic evolved, so did the reports of AKI incidence. Initial reports from China showed rates of AKI ranging from 0%-15%.
      • Cheng Y.
      • Luo R.
      • Wang K.
      • et al.
      Kidney disease is associated with in-hospital death of patients with COVID-19.
      • Guan W.J.
      • Ni Z.Y.
      • Hu Y.
      • et al.
      Clinical characteristics of coronavirus disease 2019 in China.
      • Guo T.
      • Fan Y.
      • Chen M.
      • et al.
      Cardiovascular implications of fatal outcomes of patients with coronavirus disease 2019 (COVID-19).
      • Huang C.
      • Wang Y.
      • Li X.
      • et al.
      Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China.
      • Lei S.
      • Jiang F.
      • Su W.
      • et al.
      Clinical characteristics and outcomes of patients undergoing surgeries during the incubation period of COVID-19 infection.
      • Pei G.
      • Zhang Z.
      • Peng J.
      • et al.
      Renal involvement and early prognosis in patients with COVID-19 pneumonia.
      • Shi S.
      • Qin M.
      • Shen B.
      • et al.
      Association of cardiac injury with mortality in hospitalized patients with COVID-19 in Wuhan, China.
      • Wang D.
      • Hu B.
      • Hu C.
      • et al.
      Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China.
      • Wang L.
      • Li X.
      • Chen H.
      • et al.
      Coronavirus disease 19 infection does not result in acute kidney injury: an analysis of 116 hospitalized patients from Wuhan, China.
      • Zhang G.
      • Hu C.
      • Luo L.
      • et al.
      Clinical features and short-term outcomes of 221 patients with COVID-19 in Wuhan, China.
      • Zhang X.
      • Cai H.
      • Hu J.
      • et al.
      Epidemiological, clinical characteristics of cases of SARS-CoV-2 infection with abnormal imaging findings.
      • Zhou F.
      • Yu T.
      • Du R.
      • et al.
      Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.
      Later studies from the United States reported higher but inconsistent incidence of AKI, ranging from 14%-69%.
      • Chan L.
      • Chaudhary K.
      • Saha A.
      • et al.
      AKI in hospitalized patients with COVID-19.
      • Hirsch J.S.
      • Ng J.H.
      • Ross D.W.
      • et al.
      Acute kidney injury in patients hospitalized with COVID-19.
      • Fisher M.
      • Neugarten J.
      • Bellin E.
      • et al.
      AKI in hospitalized patients with and without COVID-19: a comparison study.
      • Pelayo J.
      • Lo K.B.
      • Bhargav R.
      • et al.
      Clinical characteristics and outcomes of community- and hospital-acquired acute kidney injury with COVID-19 in a US inner city hospital system.
      ,
      • Aggarwal S.
      • Garcia-Telles N.
      • Aggarwal G.
      • Lavie C.
      • Lippi G.
      • Henry B.M.
      Clinical features, laboratory characteristics, and outcomes of patients hospitalized with coronavirus disease 2019 (COVID-19): early report from the United States.
      • Argenziano M.G.
      • Bruce S.L.
      • Slater C.L.
      • et al.
      Characterization and clinical course of 1000 patients with coronavirus disease 2019 in New York: retrospective case series.
      • Price-Haywood E.G.
      • Burton J.
      • Fort D.
      • Seoane L.
      Hospitalization and mortality among Black patients and White patients with Covid-19.
      • Richardson S.
      • Hirsch J.S.
      • Narasimhan M.
      • et al.
      Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area.
      • Mohamed M.M.B.
      • Lukitsch I.
      • Torres-Ortiz A.E.
      • et al.
      Acute kidney injury associated with coronavirus disease 2019 in urban New Orleans.
      This variability may be because of differences in definitions and demographic characteristics between cohorts. For example, not all studies included patients requiring intensive care. Although most studies defined AKI according to KDIGO guidelines, methods for determining baseline creatinine levels differed. Changes in the predominant circulating variants over time may affect predilection for AKI. Practice patterns also evolved, with changes in available evidence to support the use of corticosteroids
      • Horby P.
      • Lim W.S.
      • et al.
      RECOVERY Collaborative Group
      Dexamethasone in hospitalized patients with Covid-19.
      and remdesivir,
      • Beigel J.H.
      • Tomashek K.M.
      • Dodd L.E.
      • et al.
      Remdesivir for the treatment of Covid-19 - final report.
      which may have affected AKI incidence. The emergence and adoption of vaccines likely also contributed to shifts in the populations most at risk of severe illness and AKI.
      Predictive models for AKI in the setting of hospitalization with COVID-19 have been reported in smaller samples but have not been temporally validated. Fisher et al.
      • Fisher M.
      • Neugarten J.
      • Bellin E.
      • et al.
      AKI in hospitalized patients with and without COVID-19: a comparison study.
      developed a predictive model for stages 2-3 AKI that included respiratory rate, WBC count, neutrophil/lymphocyte ratio, and lactate dehydrogenase level. This model did not exclude AKI present on admission; thus, it is possible that some of these variables may reflect collinearity with pre-existing AKI as opposed to the prediction of AKI development during hospitalization. To address this limitation in the existing literature, we excluded patients with AKI present on admission to determine risk factors before AKI onset. Our findings extend prior knowledge by developing a comprehensive predictive model and resultant formula for AKI to help prognosticate AKI in patients hospitalized with COVID-19.
      Well-calibrated clinical models that maintain their ability to accurately predict clinical risk over time aid providers in decision making to improve patient outcomes, prioritize how resources are allocated, and decrease health care costs.
      • Davis S.E.
      • Lasko T.A.
      • Chen G.
      • Siew E.D.
      • Matheny M.E.
      Calibration drift in regression and machine learning models for acute kidney injury.
      The development of treatment strategies including vaccination and the emergence of the more contagious Delta variant led to the evolution of the pandemic, highlighting the importance of models that are resilient over time. Although our findings showed data shift in independent variables in our 3 models, discrimination remained stable. The calibration of our models deteriorated over time mainly in the validation cohort 2, which corresponded to the increase in the Delta variant. However, our final, most extensive model, which included ferritin and D-dimer levels, showed stable calibration. Patients hospitalized with COVID-19 when the Delta variant was predominant were younger, with fewer comorbid conditions, and higher markers of inflammation than those patients admitted earlier in the pandemic. This is further illustrated by the higher PSI values reflecting unstable change in variables for age, history of hypertension, and ferritin level for the validation cohort 2. To date, limited information is available on differences in demographic characteristics, comorbid conditions, and laboratory abnormalities in patients with the Delta variant versus previous variants. The inclusion of inflammatory markers, specifically ferritin, not only improved the performance of the model but also maintained stability in discrimination and calibration despite a shifting patient population.
      This study has several strengths. We studied a more contemporary COVID-19 cohort than previously reported. Our study included 10,000 patients across multiple hospitals in a large metropolitan area, enrolling a diverse sample to adequately power the analyses. The variables used are commonly available among patients hospitalized with COVID-19. We also note several limitations. The comorbid conditions identified by billing or coding data may not be completely accurate. Missing values may not have been missing at random, but there were few such patients excluded from the study, so this likely had little impact on the generalizability of our findings. Changes in practice patterns and the effect of vaccination on the population at risk of severe infection may have affected the results. We addressed this limitation by validating our model using 2 cohorts later in the pandemic. Finally, although this tool could be very useful to identify the most vulnerable patients who could develop AKI, it may not be as useful with the emergence of new variants of the virus with significantly different behavior. Model monitoring would be needed over time to continue to evaluate performance as the pandemic further evolves.
      In conclusion, using demographic characteristics, comorbid conditions, admission laboratory values, and markers of inflammation, we developed and validated predictive models for the development of AKI during hospitalization with COVID-19 and demonstrated that our final model was robust to evolving changes in the pandemic from 2020 to 2021 and during the Delta variant surge.

      Article Information

      Authors’ Full Names and Academic Degrees

      Meredith C. McAdams, MD, Pin Xu, PhD, MS, Sameh N. Saleh, MD, Michael Li, BA, Mauricio Ostrosky-Frid, MD, L. Parker Gregg, MD, MSCS, Duwayne L. Willett, MD, Ferdinand Velasco, MD, Christoph U. Lehmann, MD, and S. Susan Hedayati, MD, MHSc.

      Authors’ Contributions

      Research idea and study design: MCM, MOF, DLW, FV, CUL, SSH; data acquisition; DLW, FV; data analysis/interpretation: MCM, PX, SNS, ML, DLW, CUL, SSH; statistical analysis: PX, SSH; supervision and mentorship: LPG, CUL, SSH. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work by ensuring that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved.

      Support

      Dr. McAdams is supported by training grant 5T32DK007257-38 from the National Institutes of Diabetes and Digestive and Kidney Diseases (NIDDK). Dr. Gregg is supported by a VA CSR&D Career Development Award (IK2CX002368). Dr. Hedayati is supported by the Yin Quan-Yuen Distinguished Professorship in Nephrology at the University of Texas Southwestern Medical Center, Dallas, Texas. This work was also supported in part by the Houston VA Health Services Research & Development Center for Innovations grant (CIN13-413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIDDK or the VA.

      Financial Disclosure

      The authors declare that they have no relevant financial interests.

      Acknowledgements

      This study used the University of Texas Southwestern COVID-19 Registry Collaborative data source. We acknowledge the following people for their efforts in pulling data for this research: from the University of Texas Southwestern Medical Informatics group, Mereeja Varghese, Clarie Wang, and Ki Lai; and from the Texas Health Resources Medical Informatics group, Chaitanya Katterapalli, Sohal Sukhraj, and Andrew Masica. We extend a special thanks to Jeffrey SoRelle from Genomics and Molecular Pathology at UTSW for providing information on the COVID-19 variants.

      Peer Review

      Received January 24, 2022 as a submission to the expedited consideration track with 2 external peer reviews. Direct editorial input from the Statistical Editor and the Editor-in-Chief. Accepted in revised form February 14, 2022.

      Supplementary Material

      References

        • Braun F.
        • Lütgehetmann M.
        • Pfefferle S.
        • et al.
        SARS-CoV-2 renal tropism associates with acute kidney injury.
        Lancet. 2020; 396: 597-598
        • Hassler L.
        • Reyes F.
        • Sparks M.A.
        • Welling P.
        • Batlle D.
        Evidence for and against direct kidney infection by SARS-CoV-2 in patients with COVID-19.
        Clin J Am Soc Nephrol. 2021; 16: 1755-1765
        • Moledina D.G.
        • Simonov M.
        • Yamamoto Y.
        • et al.
        The association of COVID-19 with acute kidney injury independent of severity of illness: a multicenter cohort study.
        Am J Kidney Dis. 2021; 77: 490-499.e1
        • McAdams M.
        • Ostrosky-Frid M.
        • Rajora N.
        • Hedayati S.S.
        Effect of COVID-19 on kidney disease incidence and management.
        Kidney360. 2020; 2: 141-153
        • Gupta S.
        • Coca S.G.
        • Chan L.
        • et al.
        AKI treated with renal replacement therapy in critically ill patients with COVID-19.
        J Am Soc Nephrol. 2021; 32: 161-176
        • Bowe B.
        • Cai M.
        • Xie Y.
        • Gibson A.K.
        • Maddukuri G.
        • Al-Aly Z.
        Acute kidney injury in a national cohort of hospitalized US veterans with COVID-19.
        Clin J Am Soc Nephrol. 2020; 16: 14-25
        • Chan L.
        • Chaudhary K.
        • Saha A.
        • et al.
        AKI in hospitalized patients with COVID-19.
        J Am Soc Nephrol. 2021; 32: 151-160
        • Hirsch J.S.
        • Ng J.H.
        • Ross D.W.
        • et al.
        Acute kidney injury in patients hospitalized with COVID-19.
        Kidney Int. 2020; 98: 209-218
        • Fisher M.
        • Neugarten J.
        • Bellin E.
        • et al.
        AKI in hospitalized patients with and without COVID-19: a comparison study.
        J Am Soc Nephrol. 2020; 31: 2145-2157
        • Pelayo J.
        • Lo K.B.
        • Bhargav R.
        • et al.
        Clinical characteristics and outcomes of community- and hospital-acquired acute kidney injury with COVID-19 in a US inner city hospital system.
        Cardiorenal Med. 2020; 10: 223-231
      1. Delta Variant Report. Alaa Abdel Latif, Julia L. Mullen, Manar Alkuzweny, Ginger Tsueng, Marco Cano, Emily Haag, Jerry Zhou, Mark Zeller, Emory Hufbauer, Nate Matteson, Chunlei Wu, Kristian G. Andersen, Andrew I. Su, Karthik Gangavarapu, Laura D. Hughes, and the Center for Viral Systems Biology. outbreak.info. Available at https://outbreak.info/situation-reports/delta?pango=Delta&loc=USA&selected=USA. Accessed 29 October 2021

        • Brown C.M.
        • Vostok J.
        • Johnson H.
        • et al.
        Outbreak of SARS-CoV-2 infections, including COVID-19 vaccine breakthrough infections, associated with large public gatherings - Barnstable County, Massachusetts, July 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 1059-1062
        • Hagan L.M.
        • McCormick D.W.
        • Lee C.
        • et al.
        Outbreak of SARS-CoV-2 B.1.617.2 (Delta) variant infections among incarcerated persons in a federal prison - Texas, July-August 2021.
        MMWR Morb Mortal Wkly Rep. 2021; 70: 1349-1354
        • Sheikh A.
        • McMenamin J.
        • Taylor B.
        • Robertson C.
        Public Health Scotland and the EAVE II Collaborators. SARS-CoV-2 Delta VOC in Scotland: demographics, risk of hospital admission, and vaccine effectiveness.
        Lancet. 2021; 397: 2461-2462
        • Ong S.W.X.
        • Chiew C.J.
        • Ang L.W.
        • et al.
        Clinical and virological features of SARS-CoV-2 variants of concern: a retrospective cohort study comparing B.1.1.7 (Alpha), B.1.315 (Beta), and B.1.617.2 (Delta). Clin Infect Dis.
        (Published online August 23, 2021)
        • Twohig K.A.
        • Nyberg T.
        • Zaidi A.
        • et al.
        Hospital admission and emergency care attendance risk for SARS-CoV-2 delta (B.1.617.2) compared with Alpha (B.1.1.7) variants of concern: a cohort study.
        Lancet Infect Dis. 2022; 22: 35-42
        • Bager P.
        • Wohlfahrt J.
        • Rasmussen M.
        • Albertsen M.
        • Krause T.G.
        Hospitalisation associated with SARS-CoV-2 delta variant in Denmark.
        Lancet Infect Dis. 2021; 21: 1351
      2. Delta Variant Report. Alaa Abdel Latif, Julia L. Mullen, Manar Alkuzweny, Ginger Tsueng, Marco Cano, Emily Haag, Jerry Zhou, Mark Zeller, Emory Hufbauer, Nate Matteson, Chunlei Wu, Kristian G. Andersen, Andrew I. Su, Karthik Gangavarapu, Laura D. Hughes, and the Center for Viral Systems Biology. outbreak.info. Available at https://outbreak.info/situation-reports/delta?selected=USA_US-TX&loc=USA&loc=Worldwide&loc=USA_US-TX. Accessed 29 October 2021

        • Willett D.L.
        • Kannan V.
        • Chu L.
        • et al.
        SNOMED CT concept hierarchies for sharing definitions of clinical conditions using electronic health record data.
        Appl Clin Inform. 2018; 9: 667-682
        • Khwaja A.
        KDIGO clinical practice guidelines for acute kidney injury.
        Nephron Clin Pract. 2012; 120: c179-c184
        • Neter J.
        • Wasserman W.
        • Kutner M.H.
        Applied Linear Regression Models.
        Irwin, 1989
        • Kuhn M.
        Building predictive models in R using the caret package.
        J Stat Softw. 2008; 28: 1-26
        • Van Hoorde K.
        • Van Huffel S.
        • Timmerman D.
        • Bourne T.
        • Van Calster B.
        A spline-based tool to assess and visualize the calibration of multiclass risk predictions.
        J Biomed Inform. 2015; 54: 283-293
        • Davis S.E.
        • Lasko T.A.
        • Chen G.
        • Siew E.D.
        • Matheny M.E.
        Calibration drift in regression and machine learning models for acute kidney injury.
        J Am Med Inform Assoc. 2017; 24: 1052-1061
        • Van Calster B.
        • Nieboer D.
        • Vergouwe Y.
        • De Cock B.
        • Pencina M.J.
        • Steyerberg E.W.
        A calibration hierarchy for risk models was defined: from utopia to empirical data.
        J Clin Epidemiol. 2016; 74: 167-176
        • Karakoulas G.
        Empirical validation of retail credit-scoring models.
        RMA J. 2004; 87: 56-60
        • Siddiqi N.
        Intelligent Credit Scoring: Building and Implementing Better Credit Risk Scorecards.
        2nd ed. Wiley, 2017
        • Cheng Y.
        • Luo R.
        • Wang K.
        • et al.
        Kidney disease is associated with in-hospital death of patients with COVID-19.
        Kidney Int. 2020; 97: 829-838
        • Guan W.J.
        • Ni Z.Y.
        • Hu Y.
        • et al.
        Clinical characteristics of coronavirus disease 2019 in China.
        N Engl J Med. 2020; 382: 1708-1720
        • Guo T.
        • Fan Y.
        • Chen M.
        • et al.
        Cardiovascular implications of fatal outcomes of patients with coronavirus disease 2019 (COVID-19).
        JAMA Cardiol. 2020; 5: 811-818
        • Huang C.
        • Wang Y.
        • Li X.
        • et al.
        Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China.
        Lancet. 2020; 395: 497-506
        • Lei S.
        • Jiang F.
        • Su W.
        • et al.
        Clinical characteristics and outcomes of patients undergoing surgeries during the incubation period of COVID-19 infection.
        EClinicalMedicine. 2020; 21100331
        • Pei G.
        • Zhang Z.
        • Peng J.
        • et al.
        Renal involvement and early prognosis in patients with COVID-19 pneumonia.
        J Am Soc Nephrol. 2020; 31: 1157-1165
        • Shi S.
        • Qin M.
        • Shen B.
        • et al.
        Association of cardiac injury with mortality in hospitalized patients with COVID-19 in Wuhan, China.
        JAMA Cardiol. 2020; 5: 802-810
        • Wang D.
        • Hu B.
        • Hu C.
        • et al.
        Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus-infected pneumonia in Wuhan, China.
        JAMA. 2020; 323: 1061-1069
        • Wang L.
        • Li X.
        • Chen H.
        • et al.
        Coronavirus disease 19 infection does not result in acute kidney injury: an analysis of 116 hospitalized patients from Wuhan, China.
        Am J Nephrol. 2020; 51: 343-348
        • Zhang G.
        • Hu C.
        • Luo L.
        • et al.
        Clinical features and short-term outcomes of 221 patients with COVID-19 in Wuhan, China.
        J Clin Virol. 2020; 127104364
        • Zhang X.
        • Cai H.
        • Hu J.
        • et al.
        Epidemiological, clinical characteristics of cases of SARS-CoV-2 infection with abnormal imaging findings.
        Int J Infect Dis. 2020; 94: 81-87
        • Zhou F.
        • Yu T.
        • Du R.
        • et al.
        Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study.
        Lancet. 2020; 395: 1054-1062
        • Aggarwal S.
        • Garcia-Telles N.
        • Aggarwal G.
        • Lavie C.
        • Lippi G.
        • Henry B.M.
        Clinical features, laboratory characteristics, and outcomes of patients hospitalized with coronavirus disease 2019 (COVID-19): early report from the United States.
        Diagnosis (Berl). 2020; 7: 91-96
        • Argenziano M.G.
        • Bruce S.L.
        • Slater C.L.
        • et al.
        Characterization and clinical course of 1000 patients with coronavirus disease 2019 in New York: retrospective case series.
        BMJ. 2020; 369: m1996
        • Price-Haywood E.G.
        • Burton J.
        • Fort D.
        • Seoane L.
        Hospitalization and mortality among Black patients and White patients with Covid-19.
        N Engl J Med. 2020; 382: 2534-2543
        • Richardson S.
        • Hirsch J.S.
        • Narasimhan M.
        • et al.
        Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area.
        JAMA. 2020; 323: 2052-2059
        • Mohamed M.M.B.
        • Lukitsch I.
        • Torres-Ortiz A.E.
        • et al.
        Acute kidney injury associated with coronavirus disease 2019 in urban New Orleans.
        Kidney360. 2020; 1: 614-622
        • Horby P.
        • Lim W.S.
        • et al.
        • RECOVERY Collaborative Group
        Dexamethasone in hospitalized patients with Covid-19.
        N Engl J Med. 2021; 384: 693-704
        • Beigel J.H.
        • Tomashek K.M.
        • Dodd L.E.
        • et al.
        Remdesivir for the treatment of Covid-19 - final report.
        N Engl J Med. 2020; 383: 1813-1826