Statistical methods—improvement and utilization outcomes
Prediction models for improvement and utilization outcomes use both ordinary least square regression and logistic regression in sequence to reduce the number of potential risk factors from more than 200 to a more manageable size. Recent prediction model development methods use a significance value of 0.0001 for inclusion in the prediction model. That is, for a risk factor to enter into the prediction equation, there needs to be less than 1 chance in 10,000 that the risk factor was included inappropriately. A series of repeated logistic regression equations are computed until all remaining risk factors meet this criterion.
The prediction equation for each improvement and utilization outcome is reviewed by a panel of experience home health clinical care experts. Additional materials reviewed include bivariate correlation matrices and previous prediction models for these outcome measures. Each risk factor in each model is checked for the appropriateness of including the risk factor as a predictor has a clinical basis and if the direction of the relationship (positive or negative) with the outcome has a clinical basis. Any prediction model where a risk factor is removed due to this clinical review is re-calculated and a second clinical review is performed.
Once the prediction model has been finalized, the model is validated using a different very large sample of episodes of care from the original population. Validation includes calculating correlations between the observed and predicted values for each episode of care for each of the outcomes. Odds ratios and confidence intervals for these odds ratios are also calculated. Finally, C-statistics for the models are calculated for each improvement and utilization outcome measure.
Significance testing—improvement and utilization outcomes
Historically, the median R2 value is approximately 0.12 and the median C-statistic value is approximately 0.73 for the prediction models of improvement and utilization outcomes reported on Home Health Compare. Given the nature of the phenomena being predicted, the prediction models have a substantive impact when applied to risk adjust these home health outcome quality measures.
Risk adjustment calculation
Risk-adjusted outcome and utilization measures are reported on Home Health Compare, along with national and state outcome measure averages. Risk adjustment is based on statistical prediction models estimated on a national sample of home health agency patients to predict individual patient outcomes based on patient health status and other attributes at admission to home health care. The method used to risk adjust home health agency outcome measures is as follows:
- The observed outcome rate for the agency is calculated for all eligible patients receiving care from the agency during the most recent 12 month period: Agencyobs= (# of patients achieving outcome)/(# of patients eligible for outcome)
- For each of the same patients, a predicted outcome probability is calculated based on the statistical risk model and the patient’s condition at admission to home health care.
- Predicted outcome probabilities are averaged across all of the patients served over a 12 month period, to yield a predicted outcome rate for the agency: Agencypred= (sum of predicted probability)/(# of patients eligible for outcome)
- National observed and predicted rates are calculated in the same manner for the same 12 month period, by aggregating across all patients served by any home health agency in the United States.
- The agency rate is risk adjusted by adding to the observed agency rate the difference between the national predicted rate and the agency predicted rate, using the following formula: Agencyra = Agencyobs + (National¬pred – Agencypred)
If applying the risk adjustment formula results in a number less than zero the risk-adjusted rate is set to zero. Similarly, if the result is greater than 100%, it is set to 100%.
Special notice to home health agencies
On the Outcome-Based Quality Improvement (CASPER) reports that home health agencies receive, the observed agency outcome rate is reported and the national reference rate is risk adjusted. This is done using the same method as for Home Health Compare, but the following formula is used: National¬ra = National¬obs + (Agencypred – National¬pred)