Asthma Population Management: Development and Validation of a Practical 3-Level Risk Stratification Scheme

Objective: To define and validate a practical risk stratification scheme based on administrative data for use in identifying patients at high, medium, and low risk of requiring emergency hospital care for asthma. Study Design: Retrospective cohort.

Patients and Methods: Predictors in 1999 were evaluated in relation to 2000 asthma emergency hospital care (any asthma hospitalization or emergency department visit) in a training set (n = 8789, 2000 emergency hospital care = 5.5%) and a testing set (n = 6104, 2000 emergency hospital care = 7.9%). Logistic regression was used to assign risk points in the training set, and positive and negative predictive values, sensitivities, and specificities were calculated in the training and testing sets.

Results: High risk was defined as asthma emergency hospital care in the previous year or use of >14 β-agonist canisters and oral corticosteroid use; medium risk was defined as no emergency hospital care but use of either >14 β-agonist canisters or oral corticosteroids; and low risk was defined as none of the above. For the high-risk groups in the training and testing sets, positive predictive values were 12.9% and 22.0%, sensitivities were 24.8% and 25.4%, specificities were 90.3% and 92.0%, and negative predictive values were 95.4% and 93.2%, respectively. The medium-risk groups identified another 32.6% of patients in the training set and 28.3% in the testing set requiring subsequent asthma emergency hospital care.

Conclusion: This simple risk stratification scheme is useful for identifying patients from administrative data who are at increased risk of experiencing emergency hospital care for asthma.

(Am J Manag Care. 2004;10:25-32)

The prevalence of asthma has increased substantially during the past several decades; it is now estimated that nearly 1 in 10 Americans have or will have asthma in their lifetime.¹ Not only is asthma increasingly common, but it causes substantial morbidity (hospitalizations, unscheduled physician and emergency department [ED] visits, absence from school and work, and reduced quality of life) and has very large associated direct and indirect costs.²

The prevalence and impact of asthma, combined with the availability of effective controller therapy, make it an appropriate disease for population management. Important processes in asthma population management include identification of asthmatic patients within a defined population, risk stratification, targeted intervention, and outcome measurement. The purpose of risk stratification is to define asthmatic patients who are most likely to experience morbidity and resource utilization and for whom targeted intervention would probably reduce these risks. For example, it is estimated that 80% of expenses for asthma are incurred by the 20% of patients with the most severe disease.²

Several studies^3-6 have investigated risk factors for asthma hospitalizations or ED visits using administrative data in large populations in a way that could potentially be translated into risk stratification schemes in managed care organizations. However, the results of these studies have frequently been too complicated to use for practical risk stratification applications. In addition, although they potentially identify high-risk patients, these studies do not define a medium-risk group, which is useful in population management to identify patients who are not at high risk but who need more targeted intervention than low-risk patients. The purpose of this study is to define and validate a relatively simple and practical risk stratification scheme for use in identifying high-, medium-, and low-risk asthmatic patients in a large defined population using administrative data.

METHODS

Participants

≥

This study was approved by the institutional review board of Southern California Kaiser-Permanente. Data for this study were derived from the Southern California Kaiser-Permanente Asthma Case Identification Database, based on linkage of computer data from a hospital discharge database, a diagnosis and procedures database, a membership database, and a prescription database. Patients are identified as having asthma if they meet1 of the following criteria:

1. Any discharge diagnosis (principal or other diagnosis) of asthma in the hospitalization database (International Classification of Diseases, Ninth Revision, code 493.xx).

β

2. Two or more asthma-related medication dispensings (excluding oral corticosteroids, inhaled ipratropium bromide, and inhaled combination albuterol sulfate/ipratropium bromide) in 1 year in the prescription database, including -agonists (excluding oral terbutaline sulfate), inhaled corticosteroids, other inhaled anti-inflammatory drugs, and oral leukotriene modifiers.

3. Any ED or regular clinic asthma-related visit in the diagnosis and procedures database.

This algorithm is based on evidence that identifying "physician-diagnosed asthma" is a valid way to identify asthmatic patents in epidemiologic studies.⁷ Physician diagnosed asthma is captured explicitly by criteria 1 and 3 and implicitly by criterion 2. Contamination of the population by patients with primarily chronic obstructive pulmonary disease (COPD) vs asthma is reduced by excluding patients without a diagnosis of asthma whose only respiratory medication is ipratropium or combination albuterol/ipratropium. In addition, the age group of this study (5-56 years) should reduce the prevalence of patients with COPD misclassified as having asthma (see next paragraph).

The training set consisted of all San Diego (Calif) asthmatic patients aged 5 to 56 years identified in the Asthma Case Identification Database who were continuously enrolled during 1999 and 2000 and had a Kaiser prescription benefit. The testing set consisted of all Fontana (Calif) asthmatic patients aged 5 to 56 years identified in the Asthma Case Identification Database who were continuously enrolled during 1999 and 2000. Only patients with a pharmacy benefit were included in the training set, but all patients were included in the testing set. The Kaiser pharmacy data used in the development of the risk stratification process would be of uncertain validity in patients who may obtain their prescriptions elsewhere. Although approximately 90% of patients have a pharmacy benefit, it was believed that the prediction rule would be best developed in a population in which all patients have a pharmacy benefit. In contrast, it was hoped that the resulting risk stratification scheme would be useful in the total population. Thus, the testing set included patients with and without a pharmacy benefit. The age group of 5 to 56 years was chosen for this study to conform to the National Committee for Quality Assurance Health Plan Employer Data and Information Set population.⁸ In addition, the proportion of patients in the 1999-2000 Asthma Case Identification Database cohort who were diagnosed as having coexistent COPD (International Classification of Diseases, Ninth Revision, codes 490.xx, 491.xx, 492.0, 492.8, 496, and 506.4) was 6.6% in the age group 5 to 56 years compared with 50.0% in patients older than 56 years.

Outcomes and Predictors

International Classification of Diseases, Ninth Revision

β

χ

The outcome being modeled in the training set and predicted in the training and testing sets was any 2000 asthma hospitalization (primary diagnosis of asthma, , code 493.xx) or ED visit for asthma. The available predictors were the following in 1999: (1) any asthma hospitalization or ED visit; (2) number of shortacting -agonist canisters dispensed; (3) number of oral corticosteroid dispensings; (4) number of different prescribers (as a measure of continuity of care); (5) number of routine visits for asthma; (6) number of inhaled anti-inflammatory (corticosteroids, cromolyn sodium, and nedocromil sodium) canisters dispensed; (7) ratio of the number of inhaled anti-inflammatory canisters to the number of total inhaler (anti-inflammatory plus short-acting ©¬-agonist) canisters dispensed, since higher ratios have been suggested as a quality measure of adequate controller therapy⁹ and lower ratios have been shown to be a risk factor for subsequent emergency hospital utilization^5,10,11; (8) sex; and (9) subsidized (by Medicaid or Medicare) vs commercial insurance. Univariate relationships between predictors and outcomes variables were assessed using² analysis.

Modeling

Using the training set, preliminary modeling was performed for continuous predictors by means of logistic regression using a forward stepwise selection algorithm to determine the most predictive cutoff point. Predictors in the preliminary models included any 1999 asthma hospitalization or ED visit (the strongest predictor in all models) and the candidate predictor cutoff points in the various categories. Preliminary models were performed in the total group and in children (aged 5-17 years) and adults (aged 18-56 years) separately.

The final models were constructed by means of logistic regression, using the binary predictors and the dichotomized continuous predictors, based on the results of the preliminary models. Since the ideal product of this risk stratification process was believed to be a high-risk group of approximately 10% of the population, a medium-risk group of approximately 30% of the population, and a low-risk group of approximately 60% of the population, potential risk factors with incidences >40% were excluded. All analyses were performed using statistical software (SAS Version 8.2 for Windows; SAS Institute Inc, Cary, NC).

Risk Stratification and Prediction Validation

Based on the contents and odds ratios of the final model, points were assigned to the final independent risk factors, and high-, medium-, and low-risk status was defined in the training set. The predictive accuracy of these risk groups was evaluated by determining the incidence of any 2000 asthma hospitalization or ED visit in the various risk groups in the training and testing sets. In addition, for the high-risk groups, sensitivity, specificity, positive predictive value (PPV), and negative predictive value were calculated in comparison with non-high-risk patients in the training and testing sets.

β

The predictive accuracy of the newly developed risk stratification scheme was compared with that of the scheme currently in use in Southern California Kaiser-Permanente. The current scheme was defined empirically, based on published data from Northern California Kaiser-Permanente.^4,5 It assigns 1 point for each of the following: asthma hospitalization or ED visit in 1999, use of >11 -agonist canisters in 1999, >2 oral corticosteroid dispensings in 1999, and >2 different prescribers in 1999. Low risk is defined as 0 points, medium risk as 1 point, and high risk as >1 point.

RESULTS

β

The training set (San Diego) consisted of 8789 patients and the testing set (Fontana) consisted of 6104 patients. Table 1 gives the distribution of the 1999 predictor variables and the 2000 outcome variables in the training and testing sets. The 1999 and 2000 asthma hospitalization and ED visit rates were higher in the testing set than in the training set. The testing set also had a higher proportion of patients with subsidized insurance, a lower proportion with use of >9 -agonist canisters, a lower proportion with no routine visits, and a higher proportion with no inhaled controllers (Table 1).

β

Univariate analyses suggested significantly increased 2000 asthma hospitalizations or ED visits in the training and testing sets in pediatric patients, patients with subsidized insurance, and patients with 1999 emergency hospital utilization; use of >14 -agonist canisters; oral corticosteroid use; more or less than 2 prescribers; and >2 routine visits (Table 1). Relationships between any 2000 asthma hospitalization or ED visit and inhaled controllers seemed to be different in the 2 populations. In the training set, >5 controllers and a ratio >0.5 were associated with a lower rate of emergency hospital care in 2000, whereas in the testing set, emergency hospital care in 2000 was higher in patents with >2 controllers and ratios >0.3 (Table 1).

Results of preliminary modeling to determine the most predictive cutoff points for continuous predictors are given in Table 2. Except for number of prescribers and a ratio <0.5, coefficients were similar for adults and children. There was no cutoff point for routine visits that was a significant predictor in the total sample or in children or adults separately, so this predictor was not considered for the final model. Owing to their high prevalence, <6 total inhaled controllers (82.8%) and a ratio <0.5 (69.1%) were not considered for the final model.

Remaining candidates for the final model included any 1999 asthma hospitalization or ED visit, any oral corticosteroid dispensing, >2 prescribers, subsidized insurance, and sex. Sex and >2 prescribers were not significantly related to the outcome in this final model. Independent predictors remaining in the final model are given in Table 3. Although subsidized insurance was a significant predictor (model 1 in Table 3), the performance of the model was not substantially different (c-statistic) without it (model 2 in Table 3). Thus, based on the principle of parsimony, model 2 (without subsidized insurance) was chosen.

&#946;

Based on the odds ratios in the final model, risk points were assigned as follows: >14 -agonist canisters used in 1999 = 1, any oral corticosteroids used in 1999 = 1, and any asthma hospitalization or ED visit in 1999 = 2. Low risk was defined as 0 points, medium risk as 1 point, and high risk as >1 point. The prevalence and predictive characteristics of this new risk stratification scheme compared with the old scheme in the training set are given in Table 4. In children and adults, the new scheme identifies a larger low-risk population and smaller high- and medium-risk populations. The outcomes in medium-risk patients are similar in the 2 schemes, but the PPV in the high-risk population is approximately 33% higher in the new scheme in children and adults. In the high-risk group, the sensitivity and negative predictive value are similar in the 2 schemes, but the new model provides a somewhat higher specificity in children and adults. The medium-risk stratum in the new scheme identifies a group of patients with an approximately 50% higher risk of subsequently requiring emergency hospital care for asthma compared with low-risk patients. This medium-risk group accounts for an additional 32.6% of the patents subsequently requiring such care. Together, the medium- and high-risk groups identify 57.4% of patients in the training set subsequently requiring emergency hospital care.

Comparison of the performance of the new and old risk stratification schemes in the testing set is given in Table 5. Again, the new scheme identifies a larger low-risk population and a smaller medium-risk population. The size of the child high-risk population is similar in the old and new schemes, whereas the adult high-risk group is smaller in the new scheme. The PPV in medium-risk children is higher in the new scheme, and the PPV in the high-risk population is approximately 30% higher in the new scheme, with the increased PPV in the new scheme being more marked in adults than in children. In the high-risk group, negative predictive values are similar in the 2 schemes, but the new model provides somewhat higher sensitivity in children and specificity in adults. The medium-risk stratum in the new scheme in the testing set identifies a group of patients with an approximately 33% higher risk of subsequently requiring emergency hospital care compared with low-risk patients. This medium-risk group accounts for an additional 28.3% of the patients subsequently requiring such care. Together, the medium and high-risk groups identify 53.7% of the patients in the testing set subsequently requiring emergency hospital care.

A comparison of the predictive performance of the new model for all patients in the training vs testing sets is shown in the Figure. The PPVs are higher in the testing set than in the training set. For all other variables, the values are similar, with the sensitivity and specificity being slightly higher in the testing set and the negative predictive value being slightly higher in the training set.

DISCUSSION

&#946;

An important aspect of population management is risk stratification. This study developed and validated a simple risk stratification scheme using administrative data for asthma population management that identifies a high-risk group of 10% to 12% of the asthmatic population that accounts for 25% of the patients who experience emergency hospital care for asthma the following year and who were 3 to 4 times more likely to need such care than the remainder of the population. In addition, this scheme identifies a medium-risk group representing approximately 30% of the population that accounts for another approximately 30% of the patients subsequently requiring an asthma hospitalization or ED visit. The scheme can be considered to be relatively simple because it is based on only 3 variables from the previous year-any asthma hospitalization or ED visit, any use of oral corticosteroids, and use of >14 canisters of -agonist. Although this scheme is based on odds ratios from logistic regression models and assignment of risk points, its usability and practicality are enhanced because it can be translated into simple definitions (Table 6).

The risk stratification scheme developed in this study performed better regarding outcome prediction in the training and testing sets and in children and adults than the scheme currently in use in Southern California Kaiser-Permanente. Moreover, the scheme was validated by showing that it functioned as well or better in the testing set as it did in the training set (Figure). The PPV was substantially higher in the testing set than in the training set. With a constant sensitivity and specificity, the PPV is directly related to the prevalence of the outcome in the population.¹² Since the sensitivities and specificities in the training and testing sets were similar, the observed differences in PPV can be attributed to the increased prevalence of the outcome in the testing set (Fontana) compared with the training set (San Diego).

&#946;

The results of the present study can be compared with findings from previous studies that developed prediction models from administrative data. Lieu et al⁴ developed and validated prediction models in 16,520 children from the Northern California Kaiser-Permanente medical care program. Final risk factors for subsequent emergency hospital care in their study included previous emergency hospital care, use of >6-agonist canisters every 6 months, or >2 prescribers in the previous 6 months. This relatively simple scheme created a high-risk group of 22%, with 53% sensitivity and 16% PPV. The hospitalization (1.8%) and ED visit (6.4%) rates were similar to those in our Fontana cohort. Compared with the performance of the present study's new risk stratification scheme in Fontana patients, the sensitivity in the Northern California scheme was doubled, but the size of the high-risk group was also doubled, and the PPV was approximately 38% higher in the present study's new scheme compared with the Northern California results. Their results were in children only, whereas our study included children and adults.

&#8805;

&#8805;

&#946;

Subsequently, Lieu and colleagues⁵ developed and evaluated the performance of prediction models for asthma hospitalizations or ED visits in 7141 adults from the Northern California Kaiser-Permanente medical care program. Again, compared with our new risk stratification scheme, their study identified a larger high-risk group (19%), with increased sensitivity (49%) but a somewhat lower PPV (10%). Their risk stratification scheme required a somewhat complicated decision tree process, starting with5 asthma medication prescriptions filled in the previous 6 months and including 2 oral corticosteroid prescriptions filled in the previous 12 months, anti-inflammatory--agonist ratios in the previous 12 months of <1.4, and at least 1 ED visit for asthma in the previous 6 months.

&#8805;

Two studies have used logistic regression models to develop risk stratification schemes for asthma hospitalizations. Grana et al³ evaluated 54,573 children and adults who were members of a large HMO. They identified a high-risk group of 4% of the population that accounted for 30% of patients subsequently hospitalized. However, their scheme required a complicated regression equation and many predictors: sex; Medicaid insurance; New York service area; COPD; ischemic heart disease; 5 severity levels, based on medication use; various levels of previous hospitalizations, ED visits, and primary care visits; 4 specialty visits; length of enrollment; and whether the patient was in the pediatric age group.

&#946;

Schatz et al⁶ previously evaluated children (n = 4197) and adults (n = 6904) separately from Southern California Kaiser-Permanente who were continuously enrolled during 1998 and 1999. That study identified high-risk groups of 11% to 13% of the population that accounted for 45% of the patients subsequently hospitalized. Independent risk factors in children were number of previous asthma hospitalizations, number of-agonist dispensings, number of total anti-inflammatory drug dispensings, and number of prescribers, all in the previous year. Independent risk factors in adults were number of hospitalizations and oral corticosteroid dispensings in the previous year and ZIP code-derived median household income. The major advantage of this scheme is its high sensitivity for asthma hospitalizations. Its practical disadvantages include (1) use of a complicated logistic regression equation, (2) use of continuous vs dichotomized risk factors, (3) use of ZIP code-derived median household income, and (4) separate models for children and adults.

Two potential predictors were not included in the current final new model because of their prevalence, but they could be important in asthma population management surveillance. The data in the training set suggest that <6 inhaled controller medicine canisters per year or a controller inhaler-total inhaler ratio <0.5 significantly increased the risk of subsequent emergency hospital utilization for asthma. These cutoff points could be useful for determining the adequacy of controller therapy from pharmacy data in asthmatic populations. However, the same relationships were not seen in the testing set. Although the amount of inhaled controller use could be a marker of adequate effective therapy, as it apparently was in the San Diego population, it could also be a marker of increased severity, as it seemed to have been in the Fontana population. Although it is not clear why the amount of inhaled controller use functioned differently in the 2 populations, the difference reinforces the decision not to include these variables in the final simplified risk stratification scheme.

Although this risk stratification scheme seems useful and compares favorably with previous similar efforts, it still identifies as low risk approximately 45% of patients subsequently requiring emergency hospital care for asthma. This is presumably because a variety of important risk factors cannot be easily identified from administrative data, such as lower pulmonary function,^13,14 race or ethnicity,^10,15-18 symptom severity,^13,19-21 and quality of life.²² We are hoping to develop methods of making the results of spirometry and interview-determined outcomes available electronically in our population in the future. This should allow identification of a higher proportion of the patients who are at risk of subsequent severe asthma exacerbations. Although this would presumably increase the cost of the risk stratification process, this increased cost would be justified by the increased yield in predicting (and hopefully preventing) expensive asthma hospital utilization. It is also important to point out that the performance of this risk stratification scheme could change as practice patterns change. We plan to regularly review the performance of this risk stratification scheme in our population over time.

In summary, this study produced a validated, practical, 3-level risk stratification scheme for use in asthma population management. Its major advantages compared with previous efforts are its simplicity, the manageable size of the high-risk group, its applicability to children and adults, and its PPVs. Its major disadvantage compared with previous efforts is its lower sensitivity, but the sensitivity exceeds that of previous schemes when the medium-risk group is included. We hope that others will find this risk stratification scheme useful for identifying patients in need of targeted intervention and that future outcome studies will show reduced morbidity in higher-risk patients who receive such targeted intervention.

From the Department of Allergy, Kaiser-Permanente Medical Center, San Diego (MS), the Department of Pharmacy Analytic Services, Kaiser-Permanente Medical Center, Los Angeles (RN), the Department of Clinical Services, Kaiser-Permanente Medical Center, Los Angeles (CHJ, RMR, AJ), and the Department of Research and Evaluation, Kaiser-Permanente Medical Center, Los Angeles (DP).

This study was funded by the Southern California Permanente Medical Group.

Corresponding author: Michael Schatz, MD, MS, Department of Allergy, Kaiser-Permanente Medical Center, 7060 Clairemont Mesa Blvd, San Diego, CA 92111. E-mail: michael.x.schatz@kp.org.

MMWR Surveill Summ.

1. Mannino DM, Homa DM, Akinbami LJ, Moorman JE, Gwynn C, Redd SC. Surveillance for asthma-United States, 1980-1999. 2002;51:1-13.

Am J Respir Crit Care Med.

2. Smith DH, Malone DC, Lawson KA, et al. A national estimate of the economic costs of asthma. 1997;156:787-792.

3. Grana J, Preston S, McDermott PD, Hanchak NA. The use of administrative data to risk-stratify asthmatic patients. Am J Med Qual. 1997;12:113-119.

Am J Respir Crit Care Med.

4. Lieu TA, Quesenberry CP, Sorel ME, Mendoza GR, Leong AB. Computer-based models to identify high-risk children with asthma. 1998;157:1173-1180.

5. Lieu TA, Capra AM, Quesenberry CP, Mendoza GR, Mazar M. Computer-based models to identify high-risk adults with asthma: is the glass half empty or half full? J Asthma. 1999;36:359-370.

Am J Manag Care.

6. Schatz M, Cook EF, Joshua A, Petitti D. Risk factors for asthma hospitalization in a managed care organization: development of a clinical prediction rule. 2003;9:538-547.

7. Toren K, Brisman J, Jarvholm B. Asthma and asthma-like symptoms in adults assessed by questionnaires: a literature review. Chest. 1993;104:600-608.

HEDIS 2002: Volume 2: Technical Specifications Information.

8. National Committee for Quality Assurance. Washington, DC: National Committee for Quality Assurance; 2001.

9. Frischer M, Heatlie H, Chapman S, Norwood J, Bashford J, Millson D. Should the corticosteroid to bronchodilator ratio be promoted as a quality prescribing marker? Public Health. 1999;113:247-250.

Chest.

10. Gottlieb DJ, Beiser AS, O'Connor GT. Poverty, race, and medication use are correlates of asthma hospitalization rates. 1995;108:28-35.

11. Osman LM, Friend JAR, Legge JS, Douglas JG. Requests for repeat medication prescriptions and frequency of acute episodes in asthma patients. J Asthma. 1999;36:449-457.

Buring: Epidemiology in Medicine.

12. Hennekins CH. Boston, Mass: Little Brown & Co Inc; 1987:327-347.

13. Cowie RL, Underwood MF, Revitt SG, Field SK. Predicting emergency department utilization in adults with asthma: a cohort study. J Asthma. 2001;38:179-184.

Am J Respir Crit Care Med.

14. Li D, German D, Lulla S, Thomas RG, Wilson SR. Prospective study of hospitalization for asthma: a preliminary risk factor model. 1995;151:647-655.

15. Mannino DM, Homa DM, Pertowski CA, et al. Surveillance for asthma¢®¨£United States, 1960-1995. MMWR CDC Surveill Summ. 1998;47:1-27.

J Allergy Clin Immunol.

16. Joseph CL, Havstad SL, Ownby DR, Johnson CC, Tiley BC. Racial differences in emergency department use persist despite allergist visits and prescriptions filled for anti-inflammatory medications. 1998;101:484-490.

17. Zoratti EM, Havstad S, Rodriguez J, et al. Health service use by African Americans and Caucasians with asthma in a managed care setting. Am J Respir Crit Care Med. 1998;158:371-377.

Am J Public Health.

18. Miller JE. The effects of race/ethnicity and income on early childhood asthma prevalence and health care use. 1999;90:428-430.

19. Tough SC, Hessel PA, Green FH, et al. Factors that influence emergency department visits for asthma. Can Respir J. 1999;6:429-435.

Heart Lung.

20. Janson-Bjerklie S, Ferketich S, Benner P, Becker G. Clinical markers of asthma severity and risk: importance of subjective as well as objective factors. 1992;21:265-272.

21. Wakefield M, Ruffin R, Campbell D, et al. A risk screening questionnaire for adult asthmatics to predict attendance at local hospital emergency departments. Chest. 1997;112:1527-1533.

Ann Allergy Asthma Immunol.

22. Eisner MD, Ackerson LM, Chi F, et al. Health-related quality of life and future health care utilization for asthma. 2002;89:46-55.

Download PDF: Asthma Population Management: Development and Validation of a Practical 3-Level Risk Stratification Scheme