Impact of Psychiatric Follow-Up Frequency on Outcomes and Waiting Times

ABSTRACT

Objectives: This study determined whether naturally occurring but significantly different outpatient follow-up frequencies are associated with clinical outcomes and service waiting times.

Study Design: Longitudinal retrospective study.

Methods: This study was conducted in an outpatient setting. Participants consisted of 340 patients with major depressive disorder who were randomly assigned to 4 psychiatrists and were followed at a variable frequency defined by the clinician. Patients were assessed at baseline and at every visit with structured interviews and self-reported questionnaires. These groups were also compared according to their baseline characteristics, treatment, and appointment frequencies. Little’s law was used to estimate the impact of modifying the appointment frequencies on the service waiting time.

Results: The demographic variables, prescriptions, and depression severity at intake of patients across the 4 groups were similar. The mean times between appointments of the 4 groups were significantly different (87.0, 46.9, 67.9, and 61.5 days, respectively; P < .001), but these differences in outpatient follow-up frequency were not associated with clinical outcomes (eg, mean last Quick Inventory of Depressive Symptomatology Self-Report score, 10.5, 10.0, 11.9, and 9.7; P = .25). However, different outpatient follow-up frequencies had an estimated impact on waiting times for access to care; using Little’s law, it was observed that the waiting list would be eliminated by reducing by 23.9% the follow-up frequencies of the 3 psychiatrists with the highest frequencies.

Conclusions: Although variations in appointment frequencies do not appear to have a major impact on clinical outcomes, they could be managed to achieve significant improvements in the accessibility of the clinic.

Am J Manag Care. 2024;30(2):e52-e58. https://doi.org/10.37765/ajmc.2024.89501

Takeaway Points

• Outpatient follow-up frequency in psychiatric care has not been well studied, but it has an important impact on access (ie, the total number of patients treated in a prespecified time window).
• There appeared to be no practical difference in clinical outcomes of patients with major depressive disorder who had naturally occurring but different follow-up frequencies, but different frequencies had a significant impact on the volume of patients treated.
• Additional studies are needed for evidence-based guidelines to help clinicians make decisions about appointment frequency and for investigating different aspects of follow-up outpatient visits.

Current clinical guidelines do not provide evidence-based recommendations concerning the frequency of outpatient visits in mental health services. For example, the clinical guidelines from the National Institute for Health and Care Excellence in the United Kingdom do not include any specific recommendations regarding the frequency of outpatient visits (except when monitoring patients with substance use disorder, withdrawal symptoms, and suicide risk).^1-7 Although guidelines from Kaiser Permanente and the Institute for Clinical Systems Improvement in the US provide some recommendations with respect to the frequency of in-person and/or virtual follow-up contacts for major depressive disorder (MDD), these are not supported by evidence.^8,9 This is in contrast with other domains of clinical practice in psychiatry, which are generally structured to limit subjective variability.

Grounding outpatient follow-up frequency decisions in empirical evidence is important due to a significant trade-off between high- and low-frequency appointments. On the one hand, high-frequency appointments limit access to medical resources and lead to unnecessary expenses if not clinically justified. On the other hand, low-frequency appointments can lead to poor clinical care and the deterioration of patients’ conditions.

Despite its apparent impact on access to treatment,¹⁰ recognition of depression,¹¹ cost,¹² and suicide risk,¹³ outpatient follow-up frequency has not been well studied in the psychiatry literature. There are limited and contradictory results with respect to the impact of counseling frequency on outcomes.^14-16 With respect to antidepressant monitoring, a meta-analysis of antidepressant trials found that visit frequency did not significantly influence the odds of response.¹⁷

Substantial disparities in follow-up frequency exist between primary care and psychiatric settings,^18-21 with patients with severe mental illness in primary care settings showing a higher consultation rate.²² Furthermore, the frequency of appointments was found to be less than the recommendation on the product label for a new course of antidepressant therapy,²³ and it was not found to increase after an October 2003 FDA advisory.²⁴ In addition, patients with depression or anxiety disorder show significant variation in their preferences with respect to treatment frequency.^25,26

The goal of this longitudinal retrospective study was to investigate the impact of naturally occurring variation in scheduling follow-up patient visits among psychiatrists working at an outpatient clinic treating patients with depressive disorders and the impact of this variation on clinical outcomes and waiting times for new patients to start receiving treatment at the clinic.

METHODS

Study Population

This study investigated all adult patients with diagnosed MDD who were followed at the Depressive Disorders Program (DDP) of the Douglas Mental Health University Institute (Montreal, Canada) between July 2006 and June 2015 by the 4 psychiatrists of the program. At the time of our study, these psychiatrists had 4 to 10 years of service at the clinic. As part of the activities of the clinical program, weekly team meetings discussing clinical management of patients and regular journal clubs to review up-to-date information on clinical management of depression were held. Nevertheless, the patient appointment frequencies were determined by each psychiatrist individually based on their personal clinical assessment.

All patients first met with an experienced nurse or psychologist who conducted a structured clinical assessment using validated questionnaires before meeting one of the treating psychiatrists. Diagnoses were made by means of Structured Clinical Interview for DSM-IV interviews²⁷ using Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) criteria²⁸ at baseline prior to acceptance in the DDP and to defining the treatment plan. Just before each follow-up visit, patients were subsequently asked to complete self-reported computerized measures as described below. All patients included in this study had at least the baseline assessment and 2 subsequent independent measures on follow-up.

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Declaration of Helsinki of 1975, as revised in 2008. All procedures involving human participants/patients were approved by the institutional review board at the Douglas Mental Health University Institute. Written informed consent was obtained from all patients.

Measures and Primary Outcomes

Four different groups of patients were defined corresponding to the 4 treating psychiatrists’ caseloads. The DDP randomly assigns new patients to 1 of the 4 treating psychiatrists, who will then treat the patient until discharge from the program. Data collected from each patient included information on treating psychiatrist, date of initial appointment, age at initial appointment, gender, and prescriptions until August 2015.

For a comprehensive examination of the patients, the following 4 self-reported test scores were used in this study: (1) Quick Inventory of Depressive Symptomatology Self-Report (QIDS-SR), which ranges from 0 to 27, with higher scores denoting higher depression severity²⁹; (2) Quality of Life in Depression Scale (QLDS), which ranges from 0 to 34, with higher scores denoting worse quality of life³⁰; (3) Scale for Suicide Ideation (SSI), which ranges from 0 to 38, with higher scores denoting higher levels of suicide ideation³¹; and (4) Frequency, Intensity, and Burden of Side Effects Rating (FIBSER), which ranges from 0 to 18, with higher scores denoting worse adverse effects.³² These test scores or variants of them are standard in most studies on major depression (eg, the STAR*D trial³³). We included information on the dates of the different tests and their scores up to 4 years after the initial appointment. These test dates defined the dates of the follow-up appointments. The range of dates for the scores of the included patients was from March 2007 to June 2017.

Finally, a responder was defined as a patient with an improvement of 50% or more on the last QIDS-SR score, and a remitter was defined as a patient with a last QIDS-SR score of less than or equal to 5.

Statistical Analyses

Categorical variables were described with frequency rates and percentages and compared with a χ² test. Continuous variables were described using mean and SD and compared with a 1-way analysis of variance. These analyses were performed with SciPy 1.5.2 and statsmodels 0.12.1 in Python. A Cox proportional hazards regression model was also used to assess whether the hazards functions were significantly different across psychiatrists for the total follow-up time. Patients were considered as right censored if they had an appointment in the past 365 days of data of their treating psychiatrist. In other words, this model was used to assess whether the total follow-up times differed across the psychiatrists while considering that the data of some patients were censored by the data collection. The R function cox.zph was also executed to test whether the hazard functions were proportional for each psychiatrist.³⁴ The survival analyses were performed with survival package (version 3.2-3) in R. The level of significance was set at P of less than .05 and 95% CIs were used.

Impact on Waiting Times

The impact of modifying the outpatient follow-up frequency on the waiting times experienced by the patients prior to the start of their treatment at the clinic was estimated by applying Little’s law,³⁵ a well-recognized method in operations management, to the queue of the clinic. This approach is appropriate for steady-state systems, which is the case here because no dramatic changes occurred over the study period in the arrival pattern of patients, follow-up times, and capacity of the clinic.

In this case, Little’s law was defined as L = λW, where L represented the average number of patients in the queue, λ represented the average arrival rate of patients to the queue, and W represented the average waiting time of a patient in the queue.

RESULTS

Baseline Characteristics

Of a total of 463 patients, 123 patients corresponded to single consultations with subsequent referral for follow-up with general practitioners or inappropriate references. Thus, they were excluded from this study. The remaining 340 patients were included in this study, resulting in 92, 120, 52, and 76 patients for psychiatrists A, B, C, and D, respectively.

As expected, given that patients were randomly distributed among psychiatrists, baseline clinical and demographic characteristics—including age, gender, and initial QIDS-SR, QLDS, SSI, and FIBSER scores—were well balanced across the 4 groups (Table 1). Antidepressants and other drugs, excluding “as needed” medication, that were prescribed to the patients in the first 30 days of follow-up are reported in Table 2. These characteristics were also well balanced across the 4 patient groups. In particular, the mean number of antidepressants per prescription were 1.3 (95% CI, 1.2-1.5), 1.4 (95% CI, 1.4-1.5), 1.5 (95% CI, 1.4-1.7), and 1.6 (95% CI, 1.5-1.7), respectively, and thus not significantly different across the 4 groups (P = .06).

Appointments

The appointments at the DDP are described in Table 3. A total of 4914 appointments had an electronic self-reported score. Psychiatrists A, B, C, and D provided 981, 2132, 809, and 992 appointments, respectively. The mean numbers of appointments per patient were significantly different between groups (F_3,336 = 10.07; P < .001) and were 10.7 (95% CI, 9.0-12.3), 17.8 (95% CI, 15.7-19.9), 15.6 (95% CI, 13.2-18.0), and 13.1 (95% CI, 11.0-15.1), respectively. When considering the period of observation for each patient, the mean numbers of appointments per patient per year were also significantly different between groups (F_3,336 = 12.32; P < .001) and were 6.0 (95% CI, 4.9-7.1), 9.8 (95% CI, 8.7-10.9), 6.8 (95% CI, 5.7-7.9), and 6.9 (95% CI, 6.2-7.6), respectively. The mean between-appointment intervals (in days) were also significantly different between groups (F_3,4570 = 75.32; P < .001) and were 87.0 (95% CI, 80.7-93.2), 46.9 (95% CI, 44.8-48.9), 67.9 (95% CI, 62.5-73.2), and 61.5 (95% CI, 57.0-65.9), respectively. Furthermore, as shown in eAppendix Table 1 (eAppendix available at ajmc.com), when the panel of psychiatrist B is excluded (as an outlier), the measures in Table 3 remain significant, except for the mean number of appointments per patient per year (P = .34).

For the total follow-up time per patient, we performed a likelihood ratio test based on a Cox proportional hazards model to assess whether the hazards functions were significantly different across psychiatrists. The difference was indeed significantly different (χ²₃ = 50.36; P < .001). In particular, as shown in eAppendix Table 2, the HRs for the different groups were 0.32 (95% CI, 0.21-0.47; P < .001) for group B, 0.46 (95% CI, 0.31-0.70; P < .001) for group C, and 1.02 (95% CI, 0.73-1.44; P = .89) for group D. In other words, the hazard functions of psychiatrists B and C were significantly different from the hazard function of psychiatrist A, and the hazard functions of psychiatrists A and D were not significantly different. The product-limit survival estimates for the follow-up duration are provided in the Figure. A test of proportionality³⁴ concluded that our data set does not allow us to reject the validity of our model (χ²₃ = 5.00; P = .17).

Outcomes

Patient clinical outcomes after follow-up are reported in Table 4. These outcomes were well balanced across the 4 groups for the number of responders (P = .62) and the number of remitters (P = .80). Balance was also present for the different tests with respect to the last scores (QIDS-SR, P = .25; QLDS, P = .22; SSI, P = .82; FIBSER, P = .34) and the differences between the first and last scores (QIDS-SR, P = .25; QLDS, P = .12; SSI, P = .72; FIBSER, P = .59). Overall, none of the 4 psychiatrists appears best across all measures. From a practical point of view, it is important to remember that for these self-reported measures, a difference in score of 1 or 2 is rather insignificant.

Impact on Waiting Times

According to Little’s law, the current average waiting time of a patient in the queue consisted of W_current = L / λ. By assuming that the average arrival rate of patients to the clinic queue, λ, was not affected when the psychiatrists modified their follow-up frequency, we found that the new average waiting time of a patient in the queue consisted of W_new = (L – x) / λ, where x denoted the reduction in the average number of patients in the queue due to the additional patients who can be followed by the psychiatrists after reducing their follow-up frequency. Hence, the reduction in the average waiting time of a patient in the queue equated W_current – W_new = x / λ.

By assuming that the patients’ outcomes and total follow-up times were not affected by modifications of the psychiatrists’ follow-up frequency, a decrease in the follow-up frequency directly resulted in an increase in the total number of patients followed within a cycle. For example, if a psychiatrist halved their follow-up frequency, they doubled the number of different patients who could be followed within a cycle. An increase (or decrease) in the number of different patients followed within a cycle also resulted in an equivalent decrease (or increase) in the average number of patients in the waiting list (ie, x was equal to the increase or decrease in the number of different patients followed within a cycle).

We estimated the number of different patients followed within a cycle by multiplying the mean time in weeks between appointments per patient (ie, the cycle) by the mean number of appointments per week. For each psychiatrist, the current mean numbers of patients seen in a cycle were 39.8, 64.6, 32.1, and 28.9, respectively. The impacts of modifying all psychiatrists’ frequencies to the shortest cycle (ie, 6.7 weeks) and to the longest cycle (ie, 12.4 weeks) on the mean waiting time in the queue are shown in the eAppendix Figure for different mean arrival rates to the queue; a gross estimate of the average arrival rate of the DDP was 1.2 patients per week. With a similar analysis, we identified that a relative reduction of 23.9% or more in the follow-up frequencies of psychiatrists B, C, and D would lead to an elimination of the waiting list.

DISCUSSION

To our knowledge, our paper constitutes the first study of the associations between outpatient follow-up frequency and the outcomes of patients with depression and the service waiting time. The need to better characterize parameters for optimal follow-up frequency has been highlighted in psychiatry^18,19,21,23 and in other areas of medicine.^36-41

Although the patient’s needs should remain the physician’s focus when determining the date of the next appointment, our work highlights the significance of appointment frequency with regard to the capacity to improve the overall accessibility of the services provided in the clinic. Psychiatric guidelines highlight the following factors in determining the next appointment time: demographic and psychosocial factors (eg, age,¹ family disruption,⁶ patient preference⁸), clinical factors (eg, symptoms,^1,2,6,8,9 episodes of depression,¹ impairment and/or distress,⁶ comorbid conditions,^1,3 risk of relapse¹), treatment-related factors (eg, new drugs,⁸ drug adverse effects and drug interactions,⁸ treatment adherence⁸), and safety-related factors (eg, effect of drugs on system,² drug toxicity,⁸ risk of harm to self and/or others,³ risk of suicide^1,4,8,9). The findings within this article provide a basis for weighing these factors pertaining to an individual patient against providing faster access to the patient population whose treatment start is delayed due to the clinic’s uptake capacity.

Because the frequency of appointments does not appear to be associated with either clinical outcomes or total follow-up times, it appears reasonable to conclude that reducing the frequency of appointments may reduce the waiting time at the DDP without affecting health outcomes or total follow-up times. Note that the outcomes are less affected by censoring than total follow-up times because a patient may be in remission multiple times within the maximum time period of 4 years, and the median follow-up duration was more than 2 years. Note also that a lower frequency of appointments does not appear to be associated with a longer follow-up duration because, for example, group A (with the lowest frequency of appointments) has a shorter follow-up duration than group B (with the highest frequency of appointments).

As shown in the eAppendix Figure, the follow-up frequency’s impact on the waiting time is significant; changing the psychiatrists’ average follow-up frequency to either the smallest or largest values results in a significant modification of the waiting time. In addition, a relative reduction of 23.9% or more in the follow-up frequencies of psychiatrists B, C, and D would lead to an elimination of the waiting list. It is interesting to note that the reduction in the average waiting time is linked to both the follow-up frequency and the number of appointments per week. Thus, among 2 psychiatrists, the impact of modifying follow-up frequency would be less for a psychiatrist who has fewer appointments per week.

Limitations

This study has limitations. First, only naturally occurring variations in frequency of appointments were analyzed. Thus, additional studies should test to what extent the variability investigated could be further expanded. Second, based on a sample of patients with MDD in a specific clinical outpatient program, our findings may not apply to different clinical populations and settings. Third, one psychiatrist left the DDP 1.5 years before the end of data collection. This issue was addressed by removing the data of the patients of the leaving psychiatrist after the psychiatrist’s departure date. Fourth, the psychiatrists did not start to use the electronic self-reported test scores at the same time. Thus, because of the 2 previous elements, the distribution of the observations’ lengths differs across each group. We tried to alleviate this issue by limiting the data collection to 4 years after the initial appointment because 4 years corresponds approximately to the time coverage of the psychiatrist who left. Fifth, the dates identified with the use of the electronic test scores may not capture all the appointments. In some cases, a patient may not be able to complete these tests before an appointment because of a technological issue or a late patient arrival. However, we believe that this occurred infrequently. Finally, the impact of follow-up frequency on waiting time is underestimated because this study does not capture all patients followed at the DDP; this value is, however, already quite large.

CONCLUSIONS

Outpatient follow-up frequency does not appear to be associated with clinical outcomes in patients with MDD, but it was shown to have an important impact on the management of volume of patients. These results support our call for additional research to identify the ideal outpatient follow-up frequency for patients with mental illnesses.

Acknowledgments

All authors would like to thank David Guan, who helped with the data collection needed for this study, and Marc Fredette, who provided support for the statistical analyses.

Author Affiliations: Department of Logistics and Operations Management, HEC Montréal (MC), Montreal, Quebec, Canada;Smith School of Business, Queen’s University (VV), Kingston, Ontario, Canada; Department of Psychiatry, Douglas Institute, McGill University (GT), Verdun, Quebec, Canada.

Source of Funding: None.

Author Disclosures: The authors report no relationship or financial interest with any entity that would pose a conflict of interest with the subject matter of this article.

Authorship Information: Concept and design (MC, VV); acquisition of data (MC, GT); analysis and interpretation of data (MC, GT); drafting of the manuscript (MC); critical revision of the manuscript for important intellectual content (MC, VV, GT); statistical analysis (MC); and supervision (VV, GT).

Address Correspondence to: Martin Cousineau, PhD, Department of Logistics and Operations Management, HEC Montréal, 3000 chemin de la Côte-Sainte-Catherine, Montreal, QC H3T 2A7, Canada. Email: martin.cousineau@hec.ca.

REFERENCES

1. Depression in Adults: Recognition and Management. National Institute for Health and Care Excellence clinical guideline 90; October 28, 2009. Accessed June 10, 2020. https://www.nice.org.uk/guidance/cg90

2. Alcohol-Use Disorders: Diagnosis, Assessment and Management of Harmful Drinking (High-Risk Drinking) and Alcohol Dependence. National Institute for Health and Care Excellence clinical guideline 115; February 23, 2011. Accessed June 10, 2020. https://www.nice.org.uk/guidance/cg115

3. Coexisting Severe Mental Illness (Psychosis) and Substance Misuse: Assessment and Management in Healthcare Settings. National Institute for Health and Care Excellence clinical guideline 120; March 23, 2011. Accessed June 10, 2020. https://www.nice.org.uk/guidance/cg120

4. Common Mental Health Problems: Identification and Pathways to Care. National Institute for Health and Care Excellence clinical guideline 123; May 25, 2011. Accessed June 10, 2020. https://www.nice.org.uk/guidance/cg123

5. Service User Experience in Adult Mental Health: Improving the Experience of Care for People Using Adult NHS Mental Health Services. National Institute for Health and Care Excellence clinical guideline 136; December 14, 2011. Accessed June 10, 2020. https://www.nice.org.uk/guidance/cg136

6. Psychosis and Schizophrenia in Adults: Prevention and Management. National Institute for Health and Care Excellence clinical guideline 178; February 12, 2014. Accessed June 10, 2020. https://www.nice.org.uk/guidance/cg178

7. Bipolar Disorder: Assessment and Management. National Institute for Health and Care Excellence clinical guideline 185; September 24, 2014. Updated December 2, 2023. Accessed June 10, 2020. https://www.nice.org.uk/guidance/cg185

8. Diagnosis and treatment of adult depression: clinician guide. Kaiser Permanente. August 2018. Accessed June 10, 2020. https://healthy.kaiserpermanente.org/content/dam/kporg/final/documents/community-providers/co/2021/depression-full-en-2021.pdf

9. Trangle M, Gursky J, Haight R, et al. Adult Depression in Primary Care. Institute for Clinical Systems Improvement. March 2016. Accessed June 10, 2020. https://www.icsi.org/wp-content/uploads/2021/11/Depr.pdf

10. Collins KA, Westra HA, Dozois DJ, Burns DD. Gaps in accessing treatment for anxiety and depression: challenges for the delivery of care. Clin Psychol Rev. 2004;24(5):583-616. doi:10.1016/j.cpr.2004.06.001

11. Menchetti M, Murri MB, Bertakis K, Bortolotti B, Berardi D. Recognition and treatment of depression in primary care: effect of patients’ presentation and frequency of consultation. J Psychosom Res. 2009;66(4):335-341. doi:10.1016/j.jpsychores.2008.10.008

12. Brown GD, Hunter WG, Hesson A, et al. Discussing out-of-pocket expenses during clinical appointments: an observational study of patient-psychiatrist interactions. Psychiatr Serv. 2017;68(6):610-617. doi:10.1176/appi.ps.201600275

13. Windfuhr K, While D, Kapur N, et al. Suicide risk linked with clinical consultation frequency, psychiatric diagnoses and psychotropic medication prescribing in a national study of primary-care patients. Psychol Med. 2016;46(16):3407-3417. doi:10.1017/S0033291716001823

14. Igarashi H, Kitamura T, Ohuchi K, Mitoma H. Consultation frequency and perceived consultation time in a Japanese psychiatric clinic: their relationship with patient consultation satisfaction and depression and anxiety. Psychiatry Clin Neurosci. 2008;62(2):129-134. doi:10.1111/j.1440-1819.2008.01745.x

15. Cuijpers P, Huibers M, Ebert DD, Koole SL, Andersson G. How much psychotherapy is needed to treat depression? a metaregression analysis. J Affect Disord. 2013;149(1-3):1-13. doi:10.1016/j.jad.2013.02.030

16. Chan SW, Adams M. Service use, drop-out rate and clinical outcomes: a comparison between high and low intensity treatments in an IAPT service. Behav Cogn Psychother. 2014;42(6):747-759. doi:10.1017/S1352465813000544

17. Rutherford BR, Cooper TM, Persaud A, Brown PJ, Sneed JR, Roose SP. Less is more in antidepressant clinical trials: a meta-analysis of the effect of visit frequency on treatment response and dropout. J Clin Psychiatry. 2013;74(7):703-715. doi:10.4088/JCP.12r08267

18. Simon GE, Von Korff M, Rutter CM, Peterson DA. Treatment process and outcomes for managed care patients receiving new antidepressant prescriptions from psychiatrists and primary care physicians. Arch Gen Psychiatry. 2001;58(4):395-401. doi:10.1001/archpsyc.58.4.395

19. Chen SY, Hansen RA, Farley JF, Gaynes BN, Morrissey JP, Maciejewski ML. Follow-up visits by provider specialty for patients with major depressive disorder initiating antidepressant treatment. Psychiatr Serv. 2010;61(1):1-5. doi:10.1176/appi.ps.61.1.81

20. Robinson RL, Long SR, Chang S, et al. Higher costs and therapeutic factors associated with adherence to NCQA HEDIS antidepressant medication management measures: analysis of administrative claims. J Manag Care Pharm. 2006;12(1):43-54. doi:10.18553/jmcp.2006.12.1.43

21. Jones LE, Turvey C, Carney-Doebbeling C. Inadequate follow-up care for depression and its impact on antidepressant treatment duration among veterans with and without diabetes mellitus in the Veterans Health Administration. Gen Hosp Psychiatry. 2006;28(6):465-474. doi:10.1016/j.genhosppsych.2006.08.002

22. Kontopantelis E, Olier I, Planner C, et al. Primary care consultation rates among people with and without severe mental illness: a UK cohort study using the Clinical Practice Research Datalink. BMJ Open. 2015;5(12):e008650. doi:10.1136/bmjopen-2015-008650

23. Stettin GD, Yao J, Verbrugge RR, Aubert RE. Frequency of follow-up care for adult and pediatric patients during initiation of antidepressant therapy. Am J Manag Care. 2006;12(8):453-461.

24. Morrato EH, Libby AM, Orton HD, et al. Frequency of provider contact after FDA advisory on risk of pediatric suicidality with SSRIs. Am J Psychiatry. 2008;165(1):42-50. doi:10.1176/appi.ajp.2007.07010205

25. Lokkerbol J, van Voorthuijsen JM, Geomini A, et al. A discrete-choice experiment to assess treatment modality preferences of patients with anxiety disorder. J Med Econ. 2019;22(2):169-177. doi:10.1080/13696998.2018.1555403

26. Lokkerbol J, Geomini A, van Voorthuijsen J, et al. A discrete-choice experiment to assess treatment modality preferences of patients with depression. J Med Econ. 2019;22(2):178-186. doi:10.1080/13696998.2018.1555404

27. First MB, Spitzer RL, Miriam G, Williams JBW. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition, (SCID-I/P). Biometrics Research, New York State Psychiatric Institute; 2002.

28. American Psychiatric Association. DSM-IV: Diagnostic and Statistical Manual of Mental Disorders. 4th ed. American Psychiatric Press Inc; 2000.

29. Rush AJ, Trivedi MH, Ibrahim HM, et al. The 16-item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry. 2003;54(5):573-583. doi:10.1016/S0006-3223(02)01866-8

30. McKenna SP, Doward LC, Kohlmann T, et al. International development of the Quality of Life in Depression Scale (QLDS). J Affect Disord. 2001;63(1-3):189-199. doi:10.1016/S0165-0327(00)00184-1

31. Beck AT, Kovacs M, Weissman A. Assessment of suicidal intention: the Scale for Suicide Ideation. J Consult Clin Psychol. 1979;47(2):343-352. doi:10.1037/0022-006X.47.2.343

32. Wisniewski SR, Rush AJ, Balasubramani GK, Trivedi MH, Nierenberg AA; STARD Investigators. Self-rated global measure of the frequency, intensity, and burden of side effects. J Psychiatr Pract. 2006;12(2):71-79. doi:10.1097/00131746-200603000-00002

33. Rush AJ, Fava M, Wisniewski SR, et al; STAR*D Investigators Group. Sequenced treatment alternatives to relieve depression (STAR*D): rationale and design. Control Clin Trials. 2004;25(1):119-142. doi:10.1016/S0197-2456(03)00112-0

34. Grambsch PM, Therneau TM. Proportional hazards tests and diagnostics based on weighted residuals. Biometrika. 1994;81(3):515-526. doi:10.1093/biomet/81.3.515

35. Little JDC. Little’s law as viewed on its 50th anniversary. Oper Res. 2011;59(3):536-549. doi:10.1287/opre.1110.0940

36. Teeny SM, York SC, Mesko JW, Rea RE. Long-term follow-up care recommendations after total hip and knee arthroplasty: results of the American Association of Hip and Knee Surgeons’ member survey. J Arthroplasty. 2003;18(8):954-962. doi:10.1016/j.arth.2003.09.001

37. Schatz M, Rodriguez E, Falkoff R, Zeiger RS. The relationship of frequency of follow-up visits to asthma outcomes in patients with moderate persistent asthma. J Asthma. 2003;40(1):49-53. doi:10.1081/JAS-120017206

38. Neuman HB, Weiss JM, Schrag D, et al. Patient demographic and tumor characteristics influencing oncologist follow-up frequency in older breast cancer survivors. Ann Surg Oncol. 2013;20(13):4128-4136. doi:10.1245/s10434-013-3170-8

39. Aurello P, Petrucciani N, Antolino L, Giulitti D, D’Angelo F, Ramacciato G. Follow-up after curative resection for gastric cancer: is it time to tailor it? World J Gastroenterol. 2017;23(19):3379-3387. doi:10.3748/wjg.v23.i19.3379

40. Hall SF, Owen T, Griffiths RJ, Brennan K. Does the frequency of routine follow-up after curative treatment for head-and-neck cancer affect survival? Curr Oncol. 2019;26(5):295-306. doi:10.3747/co.26.4949

41. Matsumoto Y, Mabuchi S, Isohashi F, Komura N, Ogawa K, Kimura T. Impact of a reduction in follow-up frequency on life expectancy in uterine cervical cancer patients. Int J Clin Oncol. 2020;25(6):1170-1177. doi:10.1007/s10147-020-01641-w