Validation of RIFLE, AKIN, and a modified AKIN definition (“… : Medicine

1 Introduction

Acute kidney injury (AKI) is a common problem in patients admitted to the intensive care unit (ICU).[1–3] It is associated with a prolonged ICU and hospital stay. Necessarily, this results in higher costs for the health care system.[4] Furthermore, patients suffering from AKI have higher rates of short- and long-term mortality.[5,6]

Early and standardized diagnosis of AKI is important for clinical, epidemiological, and scientific purposes. A pre-requisite is the comparison of incidences and outcome of AKI in different hospitals and countries. Appropriate diagnosis and classification of AKI is likely to improve the prognosis by early treatment. With regard to a plethora of definitions, in 2004, the Acute Dialysis Quality Initiative (ADQI) Group introduced the RIFLE (Risk, Injury, Failure, Loss of kidney function, End-stage kidney disease) classification.[7] In a pragmatic approach, it is based on changes in serum creatinine and urinary output. These criteria are low-cost and readily available even in smaller hospitals. In order to improve the sensitivity regarding prognosis and mortality, it was modified in 2007. The new AKIN (Acute Kidney Injury Network) classification does not further distinguish between loss of kidney function and end-stage renal disease. Furthermore, patients with an absolute increase in serum creatinine of less than 50%, but at least 0.3 mg/dL are classified as “AKIN stage 1.” By contrast, these changes in creatinine would be classified as normal according to RIFLE classification.[8]

RIFLE and AKIN were developed by large consensus conferences. Nevertheless, the serum-creatinine cut-offs are in a way arbitrary. They were not derived from a prospective study. Consequently, this required ex-post validation. AKIN and RIFLE were compared in large multicenter studies with heterogeneous study populations, as well as in smaller studies focusing on special groups of patients.[1,9–13] Some of these validation studies are restricted to elective surgery patients, where baseline creatinine values are usually available. However, in a general ICU setting with a more heterogeneous population, mostly emergency admissions, baseline creatinine values are frequently not known. According to AKIN, pre-existing renal function has to be assumed as normal, if no baseline values are available. Although this is an unambiguous definition, it obviously results in a misclassification of patients with unknown pre-existing chronic renal failure as AKI (“AKIN type I error”). On the contrary, this categorization fails to detect AKI in patients with low baseline creatinine values (“AKIN type II error”). For example, an increase from an unknown baseline value of 0.6 to 1.0 mg/dL would not be classified as AKIN stage 1. We hypothesized that the time course of creatinine during the ICU-stay might allow for an improved ex-post AKIN-classification in a substantial number of patients with an unknown baseline serum creatinine.

Therefore, it was the aim of this study to evaluate RIFLE, AKIN, and a modified AKIN-classification in a heterogeneous group of general ICU patients. In addition to the concordance of the different definitions, we compared their predictive capacities regarding ICU, 28-day mortality, and the requirement of renal replacement therapy (RRT).

2 Material and methods

2.1 Study design

This was a single-center analysis of patients’ medical records in an 8-bed university hospital general ICU. We included all patients admitted over a 1-year period. Patients discharged and readmitted within 4 days were analyzed as one individual case and the data of the 2 admissions were combined. Patients with readmission more than 25 days after discharge were analyzed as 2 admissions. All readmissions between 4 and 25 days after discharge were excluded from the analysis, as renal function might not have been able to fully recover within this period. Chronic renal replacement and a history of renal transplantation were further exclusion criteria. Due to the strictly retrospective design of the study, informed consent was not required.

2.2 Data acquisition

According to a structured protocol, the following data were extracted from the patients’ electronic hospital record: patient characteristics (age, gender, main diagnosis), admission and discharge date to/from hospital and ICU, readmission date to hospital and ICU, and date of death. To identify the type, start, and end of RRT, data taken from the patients’ charts were analyzed. We extracted all serum creatinine values determined in the ICU (at least 1 daily routine measurement according to the local ICU standard) and on the normal ward (maximum and discharge levels) from the electronic hospital information system. Determinations of serum creatinine were performed in the certified central laboratory facility of the hospital (Institut für Klinische Chemie und Pathobiochemie, Klinikum rechts der Isar der Technischen Universität München, München, Germany). Data were entered in an anonymized database, which were then used for the analysis procedure.

2.3

RIFLE

and AKIN definitions

The stages of the RIFLE score were determined analyzing maximum serum creatinine within 7 days after admission to the ICU (Table 1). For better comparability with the AKIN criteria, we did not include stages “loss of kidney function” and “end-stage renal disease.” Furthermore, we did not classify RIFLE criteria based on the glomerular filtration rate.[7] AKIN stages of AKI were classified according to maximum serum creatinine within 48 hours after the admission to the ICU (Table 1).[8] Urine output was not analyzed for the determination of RIFLE or AKIN criteria.

Table 1:

RIFLE

and AKIN definition—diagnostic criteria regarding serum creatinine.

2.4 Concordance of definitions and prediction of mortality and renal replacement therapy

In order to evaluate whether the 2 definitions were comparable regarding their grading of severity, we determined and compared the incidence of the RIFLE-stages “risk,” “injury,” and “failure” with the AKIN-stages 1, 2, and 3. Concordance of RIFLE and AKIN was defined as classification of “risk” – AKIN stage 1, “injury” – stage 2, and “failure” – stage 3. The ability of the 2 scoring systems to predict ICU and 28-day mortality was assessed by calculating mortality rates for every stage of AKI. The mortality rates were derived from the patients’ medical records. The patient was excluded from the 28-day mortality analysis, if information about the outcome over the complete 28-day-period was not available. In a receiver operating characteristics (ROCs) analysis, we compared the prediction of 28-day mortality by RIFLE, AKIN, and “non-staged” serum creatinine values (maximum serum creatinine and serum creatinine on admission).

In addition, we analyzed the capacity of the different stages of the 2 definitions to predict the requirement of RRT during the ICU stay. Again, we compared RIFLE and AKIN definitions with “non-staged” serum creatinine values using ROC analysis.

2.5 Modification of AKIN definition for prediction of ICU mortality (“backward classification”)

The AKIN definition stages AKI according to a rise in serum creatinine compared with the baseline creatinine. Lack of baseline serum creatinine values is a common problem in the ICU. For this scenario, the AKIN definition recommends assuming physiological values (up to 1.3 mg/dL for a male and up to 1.1 mg/dL for a female patient). This seems not appropriate for critically ill patients, who have a high prevalence of elevated baseline values for several reasons. These elevations can be related to true acute renal impairment, stable chronic renal impairment, and “acute on chronic” kidney impairment. On admission, it is uncomplicated to assume normal previous creatinine values in case of missing pre-ICU values, as suggested by AKIN. However, interpretation of subsequent creatinine values might help to improve classification. To reduce the rate of “type I” and “type II” AKIN errors, we analyzed all creatinine values of these patients until discharge from the ICU. In case of recovery with decreasing values, the final stable creatinine value was defined “ex-post” as the baseline creatinine. Using this baseline value, we calculated a “backward AKIN” score. For example, a patient admitted with a creatinine of 1.5 mg/dL without documented baseline creatinine values and recovering with a final creatinine of 0.7 mg/dL during the ICU stay was classified as modified AKIN 2, as the serum creatinine value decreased by factor 2. The original AKIN-classification would have been AKIN stage 1. We compared this modification of the AKIN score to the original score by ROC analysis of sensitivity and specificity regarding the prediction of ICU and 28-day mortality.

2.6 Statistical analysis

Calculations were performed with SPSS Statistics (Version 24.0; IBM, NY). Dichotomous variables are presented as number (percentage of the corresponding population) and continuous parameters as mean ± standard deviation. Differences between groups regarding dichotomous values were compared using binary logistic regression analysis. In order to evaluate the performance of the 2 classifications compared with serum creatinine, we performed ROC analysis calculating areas under the curve (AUC) with ICU, 28-day mortality, and the need for RRT as primary outcome.

3 Results

3.1 Patient characteristics

In total, 371 patients were admitted to the general ICU at our university hospital in 2010. Of these 371 admissions, 48 cases were readmissions to the ICU. We analyzed 19 of these 48 cases according to the criteria mentioned above, resulting in a total of 342 cases meeting the inclusion criteria. After the exclusion of 21 patients, 321 cases were included in the final analysis (Fig. 1).

Age was 62 ± 16 years. The length of ICU stay was 9.1 ± 15.3 days. Of the 321 patients, 144 (44.9%) were female. In accordance with a predominantly medical, noncardiac focus of our ICU, the most common diagnoses were pneumonia and/or acute respiratory distress syndrome (53 cases; 16.5%), liver failure (50; 15.6%), and gastrointestinal bleeding (40; 12.5%).

3.2 Concordance of definitions

Of the 321 patients, 104 (32.4%) developed AKI according to the RIFLE criteria, whereas 124 of the 321 patients (38.6%) had AKI according to the AKIN criteria. The concordance of both classifications with staging in a comparable group regarding severity of AKI was 86.8% of the patients (Table 2, marked with bold letters). When using the AKIN definition, 22 (6.9%) patients were classified as AKI level 1 that would have not been detected by the RIFLE criteria (Table 2).

Table 2:

Incidence of acute kidney injury according to

RIFLE

and AKIN.

3.3 Prediction of mortality

Complete data for 28-day mortality were available for 212 (66.0%) patients and resulted in a mortality of 100/212 (47.2%). The ICU mortality was 70/321 (21.8%). AKI according to RIFLE (RIFLE criteria R, I, or F) was associated with increased ICU mortality [45/104 (43.3%) vs 25/217 (11.5%); odds ratio: 5.9; 95% confidence interval (95% CI): 3.3–10.3; P < .001] and increased 28-day mortality [61/86 (70.9%) vs 39/126 (31.1%); odds ratio 5.4; 95% CI 3.0–9.9; P < .001] compared with patients without AKI. Similarly, AKI according to AKIN was significantly associated with mortality [ICU mortality: 52/124 (50.0%) vs 18/197 (9.1%); odds ratio 7.2; 95% CI 3.9–13.1; P < .001, 28-day mortality: 71/101 (70.3%) vs 29/111 (26.1%); odds ratio 6.7; 95% CI 3.7–12.2; P < .001].

ICU and 28-day mortality increased with the stage of AKI up to 23/42 (ICU, 54.8%) and 29/36 (28-day, 80.5%) in RIFLE stage F and 16/34 (ICU, 47.1%) and 22/28 (28-day, 78.6%) in AKIN stage 3 (Fig. 2).

Figure 2:

Mortality according to

RIFLE

(Risk, Injury, Faliure, Loss of kidney function, and End-stage kidney disease)/AKIN (Acute Kidney Injury Network) level.

Comparing RIFLE and AKIN stages with “non-staged” serum creatinine values regarding prediction of 28-day mortality, the largest ROC-AUC was found (Fig. 3) for maximum serum creatinine [area under the curve (AUC) 0.76; 95% CI 0.69–0.82], followed by AKIN classification (AUC 0.73; 0.66–0.80), RIFLE classification (AUC 0.71; 0.63–0.78), and serum creatinine on admission (AUC 0.69; 0.61–0.76).

Figure 3:

ROC curves for AKIN (Acute Kidney Injury Network),

RIFLE

(Risk, Injury, Faliure, Loss of kidney function, and End-stage kidney disease), maximum serum creatinine, and serum creatinine on admission predicting 28-day mortality (P < .001 for all parameters).

3.4 Analysis of patients with a history of chronic renal failure

Of the 321 patients, 53 (16.5%) had a history of chronic renal failure. In these patients, the risk of additional AKI according to RIFLE [26/53 (49%) vs 78/268 (29%); odds ratio: 2.3; 95% CI: 1.3–4.3; P = .005] and AKIN criteria [35/53 (66%) vs 89/268 (33%), odds ratio 3.9; 95% CI: 2.1–7.3; P = .0001] was increased. Chronic renal failure was associated with elevated 28-day mortality [27/42 (64%) vs 64/159 (40%), odds ratio 2.4; 95% CI: 1.2–4.8; P = .015] and ICU-mortality [18/53 (34%) vs 52/268 (19%), odds ratio 2.1; 95% CI: 1.1–4.1; P = .021]. ICU-mortality rates of patients with acute-on-chronic renal failure were comparable to those with “acute only” renal failure in both definitions [RIFLE-ICU-mortality acute-on-chronic 11/26 (42%) vs “acute only” 34/78 (44%), odds ratio 1.05; 95% CI: 0.43–2.59; P = .91; AKIN-ICU-mortality acute-on-chronic 13/35 (37%) vs “acute only” 39/89 (44%), odds ratio 1.32; 95% CI: 0.59–2.95; P = .63].

3.5 Prediction of renal replacement therapy

Of the 321 patients, 65 (20.2%) required RRT during the ICU stay. Both definitions were able to predict the need for dialysis. Patients with RIFLE criteria “R,” “I,” or “F” had an increased requirement of hemodialysis [52/104 (50.0%) vs 13/217 (6.0%); odds ratio 15.7; 95% CI: 8.0–30.1; P < .001]. The same effect could be seen in patients classified as AKIN 1, 2, or 3 [54/124 (43.5%) vs 11/197 (5.8%); odds ratio 13.0; 95% CI: 6.5–26.4; P < .001]. The highest prevalence of RRT was found in RIFLE stage F (71.4%) and AKIN stage 3 (73.5%).

ROC analysis (Fig. 4) showed a larger ROC-AUC for maximum serum creatinine (AUC 0.92, 95% CI 0.87–0.95) and serum creatinine on admission (AUC 0.85, 95% CI 0.79–0.90) than for the RIFLE (AUC 0.83, 95% CI 0.77–0.89) or the AKIN (AUC 0.83, 95% CI 0.76–0.89) classification.

Figure 4:

ROC curves for AKIN (Acute Kidney Injury Network),

RIFLE

(Risk, Injury, Faliure, Loss of kidney function, and End-stage kidney disease), maximum serum creatinine, and serum creatinine on admission predicting need for dialysis (P < .001 for all parameters).

3.6 Modification of the AKIN score for prediction of ICU mortality (“backward classification”)

In 32 patients, modification of the AKIN score by assessment of baseline creatinine values with a backward approach resulted in a higher AKIN score than without backward analysis (Table 3). The final backward classification could be determined after a mean of 9.1 days (95% CI 7.4–10.7).

Table 3:

Incidence of acute kidney injury according to conventional and backward AKIN.

The AUCs regarding 28-day and ICU mortality were slightly higher for the backward AKIN than the conventional analysis (AUC 28-day mortality: conventional AKIN 0.73, backward AKIN 0.76; AUC ICU mortality: conventional AKIN 0.74, backward AKIN 0.76).

4 Discussion

This study demonstrated a great concordance between the RIFLE and AKIN definitions of AKI in general ICU patients. In 87% of all cases, the 2 definitions classified the patients in concordant groups regarding the severity of AKI. There was a strong association between the grade of AKI and need for dialysis, ICU, and 28-day mortality. However, unclassified serum creatinine values were slightly superior in the prediction of mortality and requirement of RRT compared with RIFLE and AKIN. Subgroup analysis of patients with chronic kidney disease revealed increased rates of AKI and mortality. Our pragmatic backward modification of the AKIN score to “reconstruct” an unknown baseline creatinine increased the number of patients with a positive score.

Several studies comparing RIFLE and AKIN in ICU patients are in line with our finding of good concordance.[1,9,10,12,14,15] The AKIN definition showed a higher sensitivity, as—compared with RIFLE—it detected an additional 22 (6.9%) patients at risk of acute injury. This higher sensitivity was an original intention of the AKIN.[8] This might be explained by the definition that an increase in serum creatinine of only 0.3 mg/dL results in classification as AKIN stage 1. To classify the patient in RIFLE group “risk,” a minimum increase in creatinine of 50% is required. On the contrary, the RIFLE definition ranked 18 patients higher than AKIN. The increased observation period of 7 days of the RIFLE-score compared with 48 hours might explain this finding.

Both definitions were able to predict the requirement of dialysis as well as ICU and 28-day mortality. This is consistent with several studies particularly in cardiac surgery.[11,16,17] Interestingly, “non-staged” absolute creatinine values, such as maximum serum creatinine levels, were comparable regarding the prediction of mortality and even better in the prediction of the need for dialysis. This might in part be explained by the potential bias that we did not include creatinine clearance and urine output in our analysis. Our classification is based solely on serum creatinine values. Although analysis of urine output might have improved the staging of renal impairment, we did not include urinary output in this study due to the well-known lack of practicability in daily routine, even in the ICU-setting.[18] Absolute serum creatinine levels remain a main factor for the decision to perform renal replacement. This might explain the good performance of maximum serum creatinine. Its association with mortality has been known for a long time and resulted in the integration of maximum serum creatinine levels in several scores predicting ICU mortality.[19] In recent studies, creatinine values were superior to definitions such as AKIN and RIFLE or other biomarkers.[7,8,20,21] However, maximum serum creatinine is not suitable for the use as a prospective predictor of mortality and acute renal failure, as in practice it is unclear when the patient reaches his/her maximum creatinine value.

Unknown baseline values are a major shortcoming of the RIFLE and AKIN definitions. Baseline serum creatinine values are frequently not available, in particular in nonsurgical patients. Therefore, an increase might not be recognized and the patient not appropriately classified. In order to solve this problem and to further improve prediction of renal failure and mortality, the KDIGO-guidelines were revised in 2012. The new definition recommends estimation of baseline serum creatinine using the modification of diet in renal disease (MDRD) formula and the observation period of serum creatinine is extended to 7 days.[22] However, according to Bernardi et al,[23] age-adjusted calculation of baseline creatinine using the MDRD formula was not able to calculate correct baseline values in a large number of patients. Furthermore, many patients in a general ICU have chronic renal impairment before admission. Therefore, this method might be misleading in these patients, as it might overestimate the rate of acute renal impairment. However, as demonstrated by the APACHE-II score allocating twice the number of points for acutely compared to chronically increased creatinine values, acute renal impairment has a more pronounced impact on the short-term prognosis than chronic renal failure.[19]

Therefore, we did not apply the 2012 AKIN definition to our patients, but derived a backward AKIN score by analysis of serum creatinine over time in case renal function improves over time. This backward classification resulted in a substantial number of patients that could be classified “ex post.” The predictive capacities for 28-day and ICU mortality were slightly improved compared with the conventional AKIN score. A substantial number of patients with AKI but without additional chronic renal impairment were more sensitively staged. For example, a male patient admitted with a serum creatinine of 1.4 mg/dL would have been classified as “no AKI” using the conventional AKIN definition, based on the assumption of a normal baseline creatinine value. However, in case of recovery to a serum creatinine of 0.8 mg/dL during the ICU-stay, this patient would be correctly identified as “AKI” by application of the backward AKIN definition. It is likely that these cases increased the incidence of AKI in our modified AKIN score. An obvious limitation regarding this backward modification is the fact that calculations can only be performed when the patient is already recovering and therefore staging might be considered as too late. However, a more sensitive detection of acute renal impairment could facilitate treatment and prevention of further renal injury. For example, the ICU stay with extended hemodynamic monitoring can be prolonged.

With regard to practical applications, our study demonstrated that a substantial number of patients with unknown baseline serum creatinine might be more appropriately classified by an ex-post modification of the AKIN classification. An analysis of the time course of serum creatinine might also help to better classify patients without baseline creatinine levels but stably elevated creatinine levels during the ICU stay. As per definition, ex-post modifications have no use at the time of the ICU admission. However, during the ICU stay, they may improve the allocation of patients to therapeutic approaches for acute or chronic renal impairment. Furthermore, a modified classification of patients without baseline values might also improve the comparison of different studies, including patients with elevated baseline creatinine.

Subgroup analysis of the patients with chronic renal failure revealed an increased rate of “acute on chronic” kidney injury as well as overall and ICU mortality, compared with patients without chronic renal impairment.[24] Surprisingly, the outcome in patients with acute-on-chronic and “acute only” renal failure did not differ. This is in line with findings of a study by Pan et al.[25]

It might be considered as a strength of this study that the prevalence of AKI was high in our patient population. Furthermore, we analyzed unselected patients admitted to a general, noncardiac ICU. This is a patient population, which is generally under-represented in studies regarding acute renal failure. The severity of illness in our patients is represented by high 28-day and ICU mortality rates, especially in patients with AKI. These rates seem high, but are in line with studies investigating comparable patients who are treated on an ICU.[12,14,26] In contrast to patients at a regular ward, in most ICU patients, AKI is accompanied by failure of another organ system, resulting in increased mortality rates. It might be considered as a limitation that we did not determine severity of illness scores for further characterization of our patients. Despite the unselected group of patients, this study has the limitation of a single-center approach. Another limitation of our study is the lack of use of other biomarkers of renal failure such as NGAL, Cystatin C, or Nephrocheck. Although these biomarkers might further improve staging of acute renal impairment, they are expensive and not readily available in every hospital. Some of these parameters were available for the patients, but they were not part of clinical routine in our ICU during the analysis period. Regarding the general applicability of AKIN and RIFLE, we focused on serum creatinine, which is widely available.

The retrospective approach is another limitation of our study. Finally, we analyzed a limited number of patients’ medical records. Therefore, this study must be considered as exploratory. Our findings have to be confirmed in a larger prospective confirmatory trial.

5 Conclusion

In critically ill patients in a general ICU, RIFLE and AKIN definitions had a high concordance in staging the severity of AKI. There was a strong relationship between the staging of AKI and the need for dialysis, ICU-, and 28-day mortality. Unclassified serum creatinine values were slightly superior in the prediction of mortality and requirement of RRT. Chronic kidney disease was associated with increased rates of additional AKI and mortality. Backward modification of the AKIN score in patients without baseline creatinine graded 32 patients with a higher score.

Acknowledgment

We are indebted to all doctors and nurses caring for the patients on ward and in the ICU.

Author contributions

Conceptualization: Wolfgang Huber, Jan Schneider.

Formal analysis: Wolfgang Huber, Jan Schneider, Bettina Jungwirth, Sebastian Schmid.

Investigation: Wolfgang Huber, Jan Schneider, Tobias Lahmer, Claudius Küchle, Sebastian Schmid.

Resources: Roland M. Schmid.

Supervision: Wolfgang Huber, Roland M. Schmid.

Validation: Jan Schneider, Sebastian Schmid.

Visualization: Jan Schneider, Sebastian Schmid.

Writing – original draft: Wolfgang Huber, Sebastian Schmid.

Writing – review & editing: Jan Schneider, Tobias Lahmer, Claudius Küchle, Bettina Jungwirth, Roland M. Schmid.

Sebastian Schmid orcid: 0000-0002-1232-5350