Performance of the “Tropical Intensive Care Score” in the intensive care units of Kinshasa

Introduction: Several scores are used to assess prognosis in intensive care units (ICU). The Tropical Intensive Care Score (tropICS) has been proposed as an alternative in low income countries. Our objective was to assess the performance of tropICS in a few ICUs in Kinshasa. Methods: This was a multicenter cohort over the period 01/03 to 04/02/2021. The performance of "tropICS" was evaluated by analysis of the area under the ROC curve and calibration with the Lemeshow-Hosmer test. Results: A total of 249 patients with a mean age of 54 years with a sex ratio of 1.9 men to 1 woman were selected in four ICUs in Kinshasa. Medical (89%), surgical (8%) and traumatic (3%) conditions were the causes of admissions, with an average length of stay of 4 (2 to 7) days. The death rate was 38.2%. After analysis of the ROC curve, a tropICS value ≥ 3.8 predicted mortality with a sensitivity of 92.6%, a specificity of 77.9%, good discrimination with an area under the ROC curve of 0.85 (CI 0.80 - 0.90) and a poor calibration of with p < 0.05. Conclusion: tropICS is a simple and powerful tool for identifying high-risk patients and can be used in ICUs in Kinshasa.


Introduction
Intensive Care Unit (ICU) General Severity Scores estimate expected mortality based on patient characteristics.They facilitate the adjustment of ICU patients in research, interinstitutional benchmarking and critical care quality improvement assessments [1].Their use is influenced by their complexity, format, and feasibility [2].
However, these scores are rarely used in low-and middle-income countries [3,4] due to the complexity of their use, the unavailability of the variables of interest, the epidemiology of the pathologies and the variation in the management of ICU patients [5,6].
In addition, attempts to improve research methods and the quality of intensive care in low-income countries may be hampered by the lack of context-specific risk prediction models.
Some severity scores have been validated in low-and middle-income countries [7], but their use is limited by their complexity and the low statistical power of the studies.
In 2014, Iteke et al. who carried out a study in the intensive care unit of the Panzi general referral hospital in Bukavu, recommended the development of severity scores adapted to the socio-economic realities of countries with limited resources, because the cost necessary to use severity scores in Western countries limited its use [8].
In order to overcome these obstacles, some authors have developed generalist severity scores for low-and middle-income countries.In Tanzania (2015) Baker et al. showed an association between altered physiological parameters at admission and mortality [9].Watters et al. (1989) in Zambia, had developed the "Clinical Sickness Score (CSS)" [10].Riviello et al. (2016), in Rwanda, developed the "Rwanda Mortality Probability Model (R-MPM)" [11].
It was in 2017 that Haniffa et al. developed the "Tropical Intensive Care Score (tropICS)" [12], which is the first general gravity score developed and validated from 21 ICUs from 4 different countries low-and middle-income, with a sizeable sample of 3855 patients.
In the Democratic Republic of Congo (DRC), the use and studies evaluating the performance of severity scores in ICUs are rare.To help

Population and sampling
The study included all patients aged 16 years or older hospitalized in the departments selected, regardless of the reason for admission.Patients or their families who refused to participate at the study and those who missed one or more variables necessary to calculate the "tropICS" the first 24 hours of admission were excluded.Sampling was exhaustive with consecutive recruitment of patients admitted to the ICU during the study period.

Collection of data
We designed data collection forms sheets including all the variables needed to calculate the "tropICS": systolic blood pressure, respiratory rate, Glasgow score, urgent surgery or not, urea, hemoglobin.In addition to the "troICS" variables, we also looked for: age according to the date of last birthday and grouped into two categories: under 60 and over 60; sex; standard of living (considered high if the patient was able to pay the full cost of care and low if not); origin (in-hospital or out-of-hospital transfer).Data were collected prospectively from admission to ICU discharge.To calculate the mortality risk prediction, we used the nomogram of "tropICS" as shown in Figure 1.
To calculate the prediction of the risk of mortality according to "tropICS": we used a rule that we aligned vertically on the above graph to obtain the value of the intersection between the covariant and the score axis, then repeated this process for all variables of the score, thus calculated the individual score by the summation, and finally used the rule again to find the intersection value of the total cumulative score and the axis of the risk of mortality.

Data analysis
The data were entered using Microsoft Excel 2013 and analysed using SPSS 22.0 software.Quantitative variables were compared using the t-student test and qualitative variables using the Chi2 test.A value of p < 0.05 was considered the threshold of statistical significance.The performance of "tropICS" was evaluated by: -discrimination: the ability to predict surviv↓al or death below the area of the ROC curve; -calibration: evaluation of the performance of the "tropICS" throughout the range of its severity, with the Lemeshow-Hosmer test.

Regulatory and ethical aspects
The rules of confidentiality and ethics were respected in accordance with the Helsinki protocol.Written informed consent was obtained from each patient or their legal representative before being recruited into the study.The research protocol was approved by the Ethics Committee of the School of Public Health of the University of Kinshasa under number: ESP/CE/036/2021.The heads of the hospitals and departments concerned had given their consent.We have no conflict of interest in this study.The study group consisted of 273 patients.From this total, we excluded 24 patients because we did not have all the values of the variables necessary for the calculation of the "tropICS", which gave us a sample of 249 patients.

Socio-demographic characteristics and reasons for admission
Table 1 presents the socio-demographic characteristics and reasons for admission of patients.
The majority of patients were men with a sex ratio of 1.9 male to female, with 38.4% of deaths in women and 38% in men with no significant difference (p = 0.959).The average age of all patients was 54.8 ± 18.4 years, it was 53.5 ± 18.4 in survivors and 54.4 ± 18.5 in deceased without significant difference.Patients aged under 60 had a mortality of 37.1% and those aged 60 and over had a mortality of 39.3% with no significant difference (p = 0.722).Mortality was 26.9% in patients with a high standard of living versus 76.8% in those with a low standard of living with a significant difference (p <0.001).Patients transferred from other health structures had a higher mortality (76.9% against 33.6% in those transferred intrahospital with a significant difference (p <0.001).Patients who had a comorbidity represented 95.58% and had a mortality of 39.1%, but not different (p = 0.163) from

The variables of the "tropICS"
Table 2 presents the "tropICS" variables Patients operated in emergency represented 21.7% with a mortality of 66.7% while those operated without emergency context represented 78.3% with a statistically significant mortality of 30.3% (p <0.001).Patients with a Glasgow score ≤8/15 with 96% mortality, those with a Glasgow score between 9 and 13/15 accounted for 39.8%, those with a Glasgow score of 14 to 15/15 accounted for 50.2% with a mortality of 20% with a significant difference (p<0.001).The average systolic blood pressure was 131.9 ± 33.5 for all patients, 134.7 ± 30 for survivors and 124.4 ± 38.4 for deceased without significant difference (p = 0.099).Mean respiratory rate was 24.9±8.4 for all patients, 22.6±5.4 for survivors and 28.7±10.7 for deceased with a significant difference (p < = 0.001).The mean hemoglobin level was 10.9±2.5 for all patients, 11.4±2.3 for survivors and 10.1±2.4 for deceased with significant difference (p < 0.001).The mean uraemia was 43.5±71.2for all patients, 27.7±42.1 for survivors and 68.9±97.1 for deceased with significant difference (p < 0.001 ).The "tropICS" had an average value of 4.3 ± 2.8 in all patients, 2.9 ± 2.3 in survivors and higher, 6.4 ± 2.2 in deceased with a difference significant p < 0.001).

Table 3 presents the sensitivity and specificity of "tropICS"
The optimal threshold value for the occurrence of death in our study corresponded to "tropICS" ≥3.8.The application of "tropICS" on the outcome of patients in our study showed that the prediction of mortality was 122 patients versus 127 surviving patients.Among the 122 patients whose mortality was expected according to the "tropiCS", we actually observed 88 patients who died, giving a sensibility (Se) of 92.6%.The non-occurrence of death, which was expected in 127 patients, was actually observed in 120 patients, ie a specificity of 77.9%.

Discrimination of "tropICS"
Figure 3 presents the ROC curve of "tropICS" in IS patients in our study.
The area under the ROC curve has a large area of 0.85 with 95% CI of 0.80-0.90.

Calibration of the "tropICS"
From Table 4, we can deduce that the "tropICS" analyzed in its rating subclasses demonstrates a significant difference between the predicted and induced outcome, with the "p" value less than 0.05.This reflects a poor calibration of the "tropICS" in our study.

Discussion
Our study validates the performance of a simplified prognostic model designed for use in ICUs in low-income countries.The results of this study show a good discrimination of the "tropICS" with the area under the ROC curve at 0.85 95% CI (0.80-0.90); however the calibration turned out to be poor with a value of Khi2 = 26.205(p<0.05).The study by Haniffa (2017) [12], had evaluated the discrimination of the "tropICS" adequate with the area under the ROC curve at 0.76 95% CI (0.74-0.79), a good calibration with the value of Khi2 = 0.19 (p> 0.05).This is not surprising, however, as several well-established prediction models, when validated externally, have shown poor calibration [13], including: • APACHE II and III, SAPS II, in a multicenter study from the South of England in 2003 by Dieter H. Beck [14]; • for all APACHE II, SAPS II, SAPS 3 and MPM II versions, in a systematic review of articles by Haniffa (2017) in low-and middle-income countries [15], where adequate calibration has not been reported than in 59% and this for readjusted scores in certain cases [16].
Several reasons could explain this, notably the limitations of the Hosmer-Lemeshow test itself, such as high sensibility to sample size [17,18].Other reasons could include differences in the composition of cases [19].
The poor calibration in our study can be justified by the fact of the size of our small sample compared to that of Haniffa et al., but also we do not exclude a possible disparity of the pathologies, the management, and the context of ICUs, which may have some differences from those used in the development of "tropICS" by Haniffa et al.In addition, it was found that in our study blood pressure did not show a significant difference between the groups of surviving and non-surviving patients.The notion of urgent surgery or not, does not necessarily imply the performance of the act.Indeed, an urgent case may not be operated in our context.The urea level as a "tropICS" variable may not reflect a real risk for the patient because there is too much bias, for example: dehydrated patient, certain parenteral foods, liver failure.Generalist severity scores in ICU need to be properly calibrated before they can be used for quality improvement initiatives [20].
Models that exhibit poor calibration but have good discriminatory ability may be of benefit if their intended use is to identify high-risk patients and / or group patients in a randomized controlled trial or for some other useful purpose [20,21] .For a severity score to have good discrimination and calibration, it should be regularly calibrated for the current population to which it is applied.

Conclusion
At the end of this study, we can retain that the "TropICS" has a good discrimination, although its calibration is still poor.Thus, the "tropICS" can be used in ICU / Kinshasa to identify patients at high risk of death upon admission, and categorization of patients according to severity for different analyzes.
A study including a larger number of patients, with evaluation of other variables in the context of ICUs in Kinshasa is necessary, in order to improve the calibration of the "tropICS".

Table 3 .
"tropICS" contingency table and the actual outcome of patients leaving the IS