Metrics Marked as Most Important

How SRTR determined which metric has the largest impact on patient survival after listing


Background

In response to feedback that the original 5-tier outcome assessment focused too much on posttransplant outcomes when pretransplant outcomes, e.g., transplant rate and waitlist mortality rate, may have a greater impact on overall patient survival, SRTR added 5-tier assessments for both survival on the waiting list (based on the standardized waitlist mortality rate ratio) and getting a transplant faster (based on the standardized transplant rate ratio). Methods for calculating the 5-tier assessments can be found here. An additional suggestion was to highlight the metric of most importance to survival after listing. In response to these suggestions, SRTR undertook an analysis to determine which of the three metrics to highlight. The updated version of the website includes the icon  above one of the three columns with tier assignments for heart, kidney, liver, and lung searches as shown in the figure.

Methods

SRTR estimated the association of the 5-tier evaluations for survival on the waiting list, getting a transplant faster, and 1-year organ survival at listing, with candidate mortality after listing. This analysis directly assesses the utility of each metric for informing patients of potential differences between programs in mortality after listing. The analysis also enables comparison of the relative importance of each metric on the scale in which the 5-tier evaluations are interpreted. For this example, we included candidates listed between July 12, 2011, and July 16, 2014. The tier rating for each metric at the time of listing was determined through archived program-specific reports (PSRs). Candidates were then followed prospectively for overall survival after listing, censoring follow-up on June 30, 2017. Cox proportional hazards models assessed the effect of a 1-tier increase in each metric on overall survival after listing. We adjusted for every factor in the SRTR risk-adjustment models for waitlist mortality in the January 2018 PSR release. Tier assignment (1-5) for each metric was included in each model to determine the relationship of the tier to survival after listing. The analysis was completed only for heart, kidney, liver, and lung candidates.

Results

The hazard ratios (HRs) associated with a 1-tier increase in each of the three metrics are shown in Table 1. Considering the row of results for kidney candidates, we see that a 1-tier increase in the survival on the waiting list  tier, i.e., changing from a tier-1 to a tier-2 program, or a tier-2 to a tier-3 program, etc., is associated with a 3% reduction in the hazard of death after listing (HR 0.97, 95% confidence interval 0.95-0.98); similarly, a 1-tier increase in the standardized getting a transplant faster ratio is associated with a 5% reduction in the hazard of death after listing (HR 0.95, 0.94-0.96). Finally, a 1-tier increase in the tier for first-year organ survival after transplant was not shown to affect survival after listing (HR 1.00, 0.99-1.01). From these results, we conclude that the getting a transplant faster tier has the greatest impact on a candidate’s hazard of death after listing and assign the  icon to that column for kidney program search results. Similarly, we conclude that the getting a transplant faster tier has the greatest impact for liver candidates (HR 0.90, 0.89-0.91) and heart candidates (HR 0.96, 0.93-0.99), while the tier for first-year posttransplant organ survival has the greatest impact for lung candidates (HR 0.95, 0.92-0.98).

Table 1. Effect of a 1-tier increase on risk of death after listing for kidney, liver, lung, and heart candidates.

 

Hazard ratio (95% confidence interval) for a 1-tier increase in survival on the waiting list tier, getting a transplant faster tier, or first-year posttransplant organ survival tier

Organ

Survival on the Waiting List Tier

Getting a Transplant Faster Tier

First-Year Posttransplant Organ Survival Tier

Kidney

0.97 (0.95-0.98)

0.95 (0.94-0.96)

1.00 (0.99-1.01)

Liver

0.98 (0.97-1.00)

0.90 (0.89-0.91)

0.97 (0.96-0.99)

Lung

0.97 (0.94-1.01)

1.02 (0.99-1.06)

0.95 (0.92-0.98)

Heart

1.00 (0.97-1.03)

0.96 (0.93-0.99)

1.00 (0.96-1.03)

 

The hazard ratios in Table 1 represent the effect of moving up 1 tier in performance. Table 2 translates these hazard ratios into the effect of moving 1 or more tiers. For example, a 4-tier difference in the getting a transplant faster tier for kidney programs, i.e., changing from a tier-1 to a tier-5 program, is associated with a 19% reduction in the hazard of death after listing.

Table 2. Effect of various differences in tier assignments on overall patient survival after listing.

 

 

Percent reduction in the hazard of death after listing for a 1-, 2-, 3-, or 4-tier difference in program tier assignment for the metric of most importance to overall survival following listing

Organ

Metric of most importance to overall patient survival following listing

1-tier difference

2-tier difference

3-tier difference

4-tier difference

Kidney

Getting a transplant faster tier

5%

10%

14%

19%

Liver

Getting a transplant faster tier

10%

19%

27%

34%

Lung

First-year posttransplant organ survival tier

5%

10%

14%

19%

Heart

Getting a transplant faster tier

4%

8%

12%

15%

 

Testing inclusion of this icon

Through a research project funded by the Agency for Healthcare Research and Quality, researchers at the Minneapolis Medical Research Foundation (Principle Investigators: Ajay Israni, MD, MS, and Cory Schaffhausen, PhD) conducted a randomized-controlled trial of various versions of the website with the goal of determining which version was most successful at guiding users to the column of greatest importance to overall survival after listing. Using a factorial study design, they conducted a randomized trial of website versions that included various combinations of the impact icon, the table of national rates, and the disclaimer that past performance does not necessarily reflect current or future outcomes. More than 190 participants were recruited per website version. They were randomly shown one version of the website and asked to choose from three hypothetical hospitals. The goal was to guide participants to select the program with the highest transplant rate (consistent with the current icon placement for heart, kidney, and liver programs); 47% percent chose the hospital with the highest transplant rate when the icon was present, compared with 39% when the icon was not present. Between the impact icon, the table of national rates, and the disclaimer, the impact icon was the only version that succeeded in guiding the most users to choose the program with the highest transplant rate. When the impact icon was not present, users tended to choose the program with the highest 1-year organ survival.

Conclusions

These analyses provide information on the relative utility of each metric to patient decision making, and the associations with candidate mortality after listing are arguably more important than eventual posttransplant survival. The public reporting should guide users to the metrics associated with candidate mortality after listing, especially the metrics with the strongest associations (i.e., the getting a transplant faster evaluation for kidney, liver, and heart transplantation).