Abstract
Background
The Ask Suicide-Screening Questions (ASQ) tool is a simple suicide screening tool developed to screen patient suicide risk (SR). The purpose of this study was to verify the reliability and validity of the ASQ tool in hospitalized patients.
Methods
The internal consistency and test-retest reliability of the South Korean version of the ASQ tool were verified in 99 hospitalized patients admitted to a tertiary medical institution in Seoul. To verify the correlations and validity of each convergence with other scales, the Mini-International Neuropsychiatric Interview (MINI), Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder-7 (GAD-7), and Satisfaction with Life Scale (SWLS) were also conducted to determine convergent and discriminant validity. Then, the receiver operating characteristic (ROC) curve diagnosis values for suicide and depression levels with the highest correlations were analyzed.
Results
As a result, Cronbach’s alpha was 0.826, and when each item was removed sequentially, Cronbach’s alpha ranged from 0.736–0.840, showing stable internal consistency. Most of the corrected item-total correlation were over 0.500; however, a relatively low correlation was shown for the fourth and fifth questions, which had values of 0.429 and 0.410, respectively. The test-retest reliability was 0.830, and the MINI and PHQ-9 showed high values of 0.872 and 0.672, respectively. The area under the curve (AUC) according to the ASQ diagnosis value was also the highest for the MINI (0.936).
Graphical Abstract
With a suicide rate of 26.9 per 100,000 people, South Korea ranks first among Organization for Economic Co-operation and Development (OECD) countries. Suicide rates had decreased for a time but have continued to increase since 2018.1 In particular, the recent situation of the coronavirus disease 2019 (COVID-19) pandemic induced mental issues such as stress, anxiety, and depression along with financial burdens and a fear of infection, which may have resulted in increased suicidal tendencies.2
Mental illness plays a considerable role in suicide deaths, accounting for 90% of cases. Diagnoses of depression (15.0%), bipolar disorder (10.0%), schizophrenia (10%), and anxiety (11.0%) as well as drug and alcohol use problems3456 are related to suicide deaths.78
Suicide is a process that initiates from suicidal ideation, which leads to planning and behavior. Some people with suicidal ideation actually make suicide attempts.910 Furthermore, those with previous suicide attempts have an increased risk of recurrent suicidal ideation, which leads to an increased risk of suicidal behavior.11 Also, previous suicide attempts are the strongest predictor of future suicide attempts.1213 Therefore, as a preventative measure, early identification of patients with suicide risk (SR) is necessary. Furthermore, among various factors, social isolation experienced by people who are considering attempting suicide can be a strong predictor of SR.14 Increased social isolation in hospitalized patients and during the COVID-19 pandemic may increase SR; therefore, to address SR, proper intervention is required.
Healthcare providers use the Columbia Suicide Severity Rating Scale,15 Patient Safety Screener,16 and Revised Suicidal Behavior Questionnaire17 to identify SR. However, most of the tools developed for mental health examinations in outpatient and primary care clinics require a long time to perform, special education, and high cost and are mainly used only for depression screening.151618
Therefore, institutions with psychiatric wards recognize the necessity of a proven SR screening (SRS) tool to prevent suicide in suicidality risk patients. Among different tools, the Ask Suicide-Screening Questions (ASQ) tool was selected to be investigated in this study. ASQ is a good tool for screening patients at risk of suicide regardless of whether they are inpatients or patients,192021 and is particularly useful in emergency departments.22232425 The ASQ tool was developed by the National Institute of Mental Health (NIMH) in the United States to effectively screen for SR.
The ASQ tool is a test specifically developed for SRS in psychiatric patients and is comprised of five items.22 To confirm the efficacy of this tool, it was developed and validated for patients in general wards such as internal medicine/surgery, and the standard measure for suicide prediction was used for verification.262728 Therefore, this study also verified the reliability and validity of the ASQ SRS tool in psychiatric inpatients and determined appropriateness of the tool as a SR assessment for psychiatric inpatients in South Korea.
One hundred adult patients in their twenties who were admitted to the psychiatric department of a tertiary general hospital in Seoul from May 27, 2021, to December 31, 2021, were selected as participants for this study. Participants understood the study purpose and provided consent directly before participation. Those suffering from psychiatric Axis 2 diseases such as organic psychiatric disorders, intellectual disabilities, or personality disorders, neurological diseases, or significant internal/surgical diseases, and illiterate individuals were excluded from the subjects.
This toolkit was developed by the NIMH in the USA, and is comprised of five questions in the form of a self-report questionnaire. Questions one to four consist of ‘yes’ or ‘no’ responses, and if patients answer ‘yes,’ even to one question, it is regarded as a positive test. With a positive test, patients are required to answer question five, which is a susceptibility evaluation item. If this question is answered with ‘yes,’ which is regarded as positive susceptibility, the patient is considered to be at imminent risk.
The PHQ-9 is a self-report test designed to screen for and assess the severity of depression. It includes nine items that apply the diagnostic criteria for major depressive disorder from the Diagnostic and Statistical Manual of Mental Disorders (DSM), and the score ranges from 0 to 27. The Korean version of PHQ-9, which has been proven reliable and valid in a Korean standardization study, was used in this study.29
The GAD-7 anxiety assessment screens for generalized anxiety disorder and evaluates the severity of symptoms. This tool is a simple self-report test with seven questions on a four-point Likert scale.30 This tool screens for generalized anxiety disorder effectively in a short period of time. The Korean version of the GAD-7 was used to evaluate anxiety.31
The MINI is a simple and structured interview tool developed in USA and Europe in 1998 for the diagnosis of major psychiatric Axis 1 diseases in the DSM-IV and International Classification of Disease, 10th edition (ICD-10) and is used in multicenter clinical research or epidemiological studies. For domestic use, Yoo et al.34 developed a standardized South Korean version in 2006.
1) Cronbach’s alpha coefficient and item-total correlation were measured to verify the internal consistency and reliability of the ASQ tool.
2) The coexistence validity was determined to verify the similarity of the ASQ tool and previous SR assessments. For this purpose, the Pearson moment correlation coefficient was analyzed through correlation analysis of the ASQ tool total score, PHQ-9 depression assessment, SWLS, and MINI suicidality module.
3) To confirm the optimal reference score for the ASQ tool, receiver operating characteristic (ROC) curve analysis, which reveals diagnostic accuracy, was performed.
4) The sensitivity and specificity of the ASQ tool reference score were measured by confirming the sensitivity and specificity of each of the reference scores and comparing them with the MINI.
All data were analyzed using SPSS version 27 for Windows (IBM Corp., Armonk, NY, USA) and R 4.2.1 (R Development Core Team, Vienna, Austria; http://www.r-project.org).
A total of 99 subjects completed the questionnaire. There were 44 males (44.4%) and 55 females (55.6%), and the average age of the subjects was 48.21 ± 15.82 years. Accounting for the largest proportion, 48% of the patients were from internal medicine wards, and there were also inpatient (6.1%) and outpatient (12.1%) psychiatric patients (Table 1).
Cronbach’s alpha coefficient was 0.826 in this study. When each item was removed sequentially, stable internal consistency was observed, with values ranging from 0.736–0.840. Most of the corrected item-total correlations of the ASQ were over 0.500 (0.429–0.793); however, for the questions ‘Have you ever attempted suicide?’ (0.429) and ‘Are you thinking of suicide now?’ (0.441), the correlation was relatively low (Table 2).
Only 21 subjects responded when the retest was conducted after a week. The test-retest reliability was 0.929, and Guttman’s split-half reliability coefficient was 0.830 (Table 2).
For the suitability of item data for factor analysis, the Kaiser-Meyer-Olkin (KMO) sample fit test was performed. The KMO value was 0.760, and Bartlett’s sphericity test showed χ2 = 217.22 (P < 0.001), showing a statistically significant difference, indicating factorial validity (Table 3).
The results of the measured correlation with the PHQ-9 depression assessment, GAD-7 anxiety assessment, and SWLS for reference validity measurement are as follows: There was a significant positive correlation between the PHQ-9 depression assessment (r = 0.672, P < 0.001) and GAD-7 anxiety assessment (r = 0.519, P < 0.001). There was a significant negative correlation with the SWLS (r = −0.510, P < 0.001). The highest correlation was with the MINI (r = 0.872, P < 0.001). Finally, the correlation with the PHQ-10, which combines the tenth question from the PHQ that measures suicidal tendency showed the following correlation values: r = 0.668, P < 0.001 (Table 4).
As one factor, the factor standardized loading value of the ASQ tool was evaluated through EFA analysis, which showed high matrix values of 0.920, 0.730, and 0.87 for ASQ-1, ASQ-2, and ASQ-3, respectively. ASQ-4 and ASQ-5 had values of 0.480 and 0.460, respectively, showing component matrix value (Table 5, Fig. 1).
To determine the accuracy of the ASQ tool as a suicide screening tool, the optimal cut-off point was analyzed with Youden’s index (J) method utilizing the PHQ-9, PHQ-10, MINI scale, ROC curve analysis, sensitivity, and specificity. Thirty patients diagnosed with SR by the ASQ tool showed the highest MINI score with an AUC of 0.958 (95% confidence interval [CI], 0.910–1.007). Similarly, the PHQ-9 and PHQ-10 showed AUC values of 0.859 (95% CI, 0.776–0.943) and 0.855 (95% CI, 0.769–0.940) (Fig. 2), respectively. In the ASQ tool, ASQ-5 is a suicide susceptibility assessment that screens for current suicidal tendencies. In this study, six patients were diagnosed, with similar AUC values of the total ASQ questions (MINI, 0.936; 95% CI, 0.857–1.014; PHQ-9, 0.879; 95% CI, 0.788–0.971; PHQ-10, 0.882; 95% CI, 0.792–0.971). However, when compared with ASQ-5, which showed uniform values (MINI, 0.714; PHQ-9, 0.703; PHQ-10, 0.745), PHQ-10 in ASQ showed highest optimal cut point of 0.889 and other values included MINI, 0.632 and PHQ-9, 0.617 in the optimal cut-off point for the ROC curve (Table 6, Fig. 3).
The reliability and validity of the South Korean version of the ASQ tool were verified in this study. This tool has been used to screen for SR in pediatric and adult inpatients.2228 In particular, a rapid SRS tool is needed for patients admitted to psychiatric wards because they are at high risk from the early stage of admission. Therefore, this study was conducted on 100 patients admitted to a tertiary medical institution in Seoul to verify the reliability and validity of the ASQ tool, which consists of short items, in South Korea.
The internal consistency reliability showed a high value of 0.826 for the reliability of the ASQ tool. The internal consistency was high, when each item was removed, showing high reliability of the ASQ tool.3536 Furthermore, the correlation coefficient between each item and the total ASQ score also showed a statistically significant correlation. Remarkably, ASQ-4 and ASQ-5 showed relatively lower values of 0.429 and 0.447, respectively. In previous studies, ASQ-4, which evaluates past suicidal attempts (e.g., ‘Have you ever attempted suicide?’), is a time parameter that measures lifetime experiences.19 Therefore, when the subject has had any suicidal attempt experiences in their lives, all subsequent ASQ tests are regarded to be positive, regardless of changes in suicidal ideation or risk changes. Therefore, in the case of an ASQ-4-positive patient, the time point of the suicidal attempt should be determined, and this information must be considered according to the responses to ASQ-1–3 to clearly evaluate the current SR.
On the other hand, ASQ-5 evaluates the current SR by asking, ‘Are you thinking of suicide now?’. There were six patients with an ASQ-5 positive result in this study. Similar to ASQ-4, there was relatively low reliability; however, when compared with the total ASQ items, similar screening values were observed in the AUC and ROC curve optimal cut-off point as a suicide screening tool. Therefore, as the questions with low correlation values (ASQ-4 and ASQ-5) are regarded as single factors of the suicidal tendencies of the ASQ tool, and further interviews are required.
Among depression and suicide tools, the ASQ tool showed the highest significant correlation with the MINI, which measures suicidal tendencies. For the symptoms, depression showed a high correlation with suicide. In this study, the total ASQ items and ASQ-5 were separately analyzed, however, ASQ did not show significant difference. This demonstrates the validity and consistency of the ASQ tool as a suicide screening tool.
Similarly, in previous studies, the ASQ tool was considered to have high validity as an SRS tool even for individuals who are not psychiatric inpatients, as the four items from the ASQ tool evaluate the main components of SR.37 Furthermore, the ASQ tool has been researched as a useful tool that can predict adult psychology.192238 The results of this study, which revealed possible SR and psychology predictions, are congruent with those of previous studies.
Retesting was performed a week after the initial ASQ assessment to increase the reliability of this study. The test-retest reliability was high at 0.929, suggesting that ASQ suicidal-positive subjects do not change their suicidal ideations even with the passage of time. Therefore, the ASQ tool is consistent in assessing SR and can be managed as an SRS tool with verification through comparison with other suicide scales.39 In particular, with fewer questions than and similar cutoff values to other SRS tools, the ASQ tool is considered a useful SRS tool. Furthermore, the ASQ tool is useful in screening for suicide even in high-risk groups when used after the depression assessment through investigating the correlation with depression scales.
With these advantages, the American Academy of Child and Adolescent Psychiatry (AACAP) has created a clinical pathway for SRS in pediatric emergency rooms (ERs) and medical/surgical wards using the ASQ tool as an SRS tool,40 and according to a study by LeCloux, healthcare professionals in community and pediatric outpatient hospitals consider the ASQ tool easy to use with its short questions.41 Consequently, the ASQ tool is a valid, reliable, and effective SRS tool. In addition, it has the advantage of rapidly assessing suicide risk in psychiatry and especially in psychiatric emergencies through studies showing that it has high sensitivity, specificity, and NPV as a screening tool for suicide even with the 4 items of ASQ.22
The limitations of this study are as follows. First, it was not possible to measure variables of symptom changes according to each patient’s hospitalization period as the subjects were comprised of only hospitalized psychiatric patients. Second, the subjects were limited to hospitalized psychiatric patients in tertiary medical institutions, and further follow-up study is necessary to generalize the results of this study. Third, accurate measurement was limited as the responses could be denied or minimized due to the characteristics of psychiatric inpatients when they are not hospitalized.
Nevertheless, this study has significance. First, testing and retesting were conducted to confirm reliability, which has been rarely done in previous studies. Furthermore, the diagnostic criteria of the ASQ tool were analyzed as a screening tool through comparison with other suicide and depression tools. Second, accurate standardization was achieved through the reverse translation process performed by an English scholar. Third, unlike most of the previous studies, which were conducted on pediatric22 and adult patients17 in ERs,37 this study measured the validity and reliability of the tool in hospitalized patients who require SRS most significantly.
As the latest SRS tool, the ASQ tool is efficacious and has only short questions that take only 20 seconds, and studies on the ASQ tool have been conducted to verify its effectiveness since 2008. The results of these studies suggest the efficacy and validity of the ASQ tool in various subjects, and the results of this study ensured the reliability and validity of the ASQ tool in South Korea. However, a further suicide safety evaluation question: ‘What if the patient tests positive?’ is raised, and the current research is ongoing. Therefore, safety evaluation and intervention for additional SRS through the verification of the ASQ tool, which was confirmed through this study, should be further explored.
In conclusion, the reliability and validity of the ASQ tool were verified on inpatients in general hospitals. As a result, the ASQ tool showed high internal consistency and test-retest reliability, and its validity was verified through its high correlation with the MINI (suicide tendencies) and PHQ-9 (depression assessment). Furthermore, the high AUC value of the ASQ tool as an SRS tool and optimal cut-off point were identified through ROC curve analysis. Consequently, the ASQ tool is expected to be utilized as a simple and useful SRS tool in inpatients and patients in ERs. In addition, conducting other depressive tendency assessments along with the ASQ tool may contribute to screening and preventing suicide by increasing the stability of suicide screening.
References
1. Statistics Korea. Cause of Death. Daejeon, Korea: Statistics Korea;2020.
2. Sher L. The impact of the COVID-19 pandemic on suicide rates. QJM. 2020; 113(10):707–712. PMID: 32539153.
3. Cavanagh JT, Carson AJ, Sharpe M, Lawrie SM. Psychological autopsy studies of suicide: a systematic review. Psychol Med. 2003; 33(3):395–405. PMID: 12701661.
4. Lee JI, Lee MB, Liao SC, Chang CM, Sung SC, Chiang HC, et al. Prevalence of suicidal ideation and associated risk factors in the general population. J Formos Med Assoc. 2010; 109(2):138–147. PMID: 20206838.
5. Roth KB, Borges G, Medina-Mora ME, Orozco R, Ouéda C, Wilcox HC. Depressed mood and antisocial behavior problems as correlates for suicide-related behaviors in Mexico. J Psychiatr Res. 2011; 45(5):596–602. PMID: 21055767.
6. Tishler CL, Reiss NS. Inpatient suicide: preventing a common sentinel event. Gen Hosp Psychiatry. 2009; 31(2):103–109. PMID: 19269529.
7. Bentley KH, Franklin JC, Ribeiro JD, Kleiman EM, Fox KR, Nock MK. Anxiety and its disorders as risk factors for suicidal thoughts and behaviors: a meta-analytic review. Clin Psychol Rev. 2016; 43:30–46. PMID: 26688478.
8. Jae JM. Suicide and psychiatric disorder. J Korean Soc Biol Ther Psychiatry. 2004; 10(1):3–10.
9. Kumar G, Steer RA. Psychosocial correlates of suicidal ideation in adolescent psychiatric inpatients. Suicide Life Threat Behav. 1995; 25(3):339–346. PMID: 8553414.
10. Runeson BS, Beskow J, Waern M. The suicidal process in suicides among young people. Acta Psychiatr Scand. 1996; 93(1):35–42. PMID: 8919327.
11. Cooper J, Kapur N, Webb R, Lawlor M, Guthrie E, Mackway-Jones K, et al. Suicide after deliberate self-harm: a 4-year cohort study. Am J Psychiatry. 2005; 162(2):297–303. PMID: 15677594.
12. Walsh CG, Ribeiro JD, Franklin JC. Predicting risk of suicide attempts over time through machine learning. Clin Psychol Sci. 2017; 5(3):457–469.
13. Luby JL, Belden AC, Jackson JJ, Lessov-Schlaggar CN, Harms MP, Tillman R, et al. Early childhood depression and alterations in the trajectory of gray matter maturation in middle childhood and early adolescence. JAMA Psychiatry. 2016; 73(1):31–38. PMID: 26676835.
14. Davidson L, Linnoila M. Risk Factors for Youth Suicide. New York, NY, USA: Taylor & Francis;2013.
15. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011; 168(12):1266–1277. PMID: 22193671.
16. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001; 16(9):606–613. PMID: 11556941.
17. Boudreaux ED, Jaques ML, Brady KM, Matson A, Allen MH. The patient safety screener: validation of a brief suicide risk screener for emergency department settings. Arch Suicide Res. 2015; 19(2):151–160. PMID: 25826715.
18. Reynolds WM. Suicidal Ideation Questionnaire (SIQ). Odessa, FL, USA: Psychological Assessment Resources;1987.
19. Aguinaldo LD, Sullivant S, Lanzillo EC, Ross A, He JP, Bradley-Ewing A, et al. Validation of the ask suicide-screening questions (ASQ) with youth in outpatient specialty and primary care clinics. Gen Hosp Psychiatry. 2021; 68:52–58. PMID: 33310014.
20. Kemper AR, Hostutler CA, Beck K, Fontanella CA, Bridge JA. Depression and suicide-risk screening results in pediatric primary care. Pediatrics. 2021; 148(1):e2021049999. PMID: 34099503.
21. Inman DD, Matthews J, Butcher L, Swartz C, Meadows AL. Identifying the risk of suicide among adolescents admitted to a children’s hospital using the Ask Suicide-Screening Questions. J Child Adolesc Psychiatr Nurs. 2019; 32(2):68–72. PMID: 31025489.
22. Horowitz LM, Bridge JA, Teach SJ, Ballard E, Klima J, Rosenstein DL, et al. Ask Suicide-Screening Questions (ASQ): a brief instrument for the pediatric emergency department. Arch Pediatr Adolesc Med. 2012; 166(12):1170–1176. PMID: 23027429.
23. DeVylder JE, Ryan TC, Cwik M, Wilson ME, Jay S, Nestadt PS, et al. Assessment of selective and universal screening for suicide risk in a pediatric emergency department. JAMA Netw Open. 2019; 2(10):e1914070. PMID: 31651971.
24. Ballard ED, Cwik M, Van Eck K, Goldstein M, Alfes C, Wilson ME, et al. Identification of at-risk youth by suicide screening in a pediatric emergency department. Prev Sci. 2017; 18(2):174–182. PMID: 27678381.
25. Lanzillo EC, Horowitz LM, Wharff EA, Sheftall AH, Pao M, Bridge JA. The importance of screening preteens for suicide risk in the emergency department. Hosp Pediatr. 2019; 9(4):305–307. PMID: 30858170.
26. Mills PD, DeRosier JM, Ballot BA, Shepherd M, Bagian JP. Inpatient suicide and suicide attempts in Veterans Affairs hospitals. Jt Comm J Qual Patient Saf. 2008; 34(8):482–488. PMID: 18714751.
27. Combs H, Romm S. Psychiatric inpatient suicide: a literature review. Prim Psychiatry. 2007; 14(12):67–74.
28. Osman A, Bagge CL, Gutierrez PM, Konick LC, Kopper BA, Barrios FX. The Suicidal Behaviors Questionnaire-Revised (SBQ-R): validation with clinical and nonclinical samples. Assessment. 2001; 8(4):443–454. PMID: 11785588.
29. Park SJ, Choi HR, Choi JH, Kim KW, Hong JP. Reliability and validity of the Korean version of the Patient Health Questionnaire-9 (PHQ-9). Anxiety Mood. 2010; 6(2):119–124.
30. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006; 166(10):1092–1097. PMID: 16717171.
31. Lee SH, Shin C, Kim H, Jeon SW, Yoon HK, Ko YH, et al. Validation of the Korean version of the generalized anxiety disorder 7 self-rating scale. Asia-Pac Psychiatry. 2022; 14(1):e12421. PMID: 32893471.
32. Pavot W, Diener E. The satisfaction with life scale and the emerging construct of life satisfaction. J Posit Psychol. 2008; 3(2):137–152.
33. Lim YJ. Psychometric properties of the satisfaction with life scale among Korean police officers, university students, and adolescents. Korean J Psychol Gen. 2012; 31(3):877–896.
34. Yoo SW, Kim YS, Noh JS, Oh KS, Kim CH, Namkoong K, et al. Validity of Korean version of the mini-international neuropsychiatric interview. Anxiety Mood. 2006; 2(1):50–55.
35. Canan F, Ataoglu A, Nichols LA, Yildirim T, Ozturk O. Evaluation of psychometric properties of the internet addiction scale in a sample of Turkish high school students. Cyberpsychol Behav Soc Netw. 2010; 13(3):317–320. PMID: 20557252.
36. Khazaal Y, Billieux J, Thorens G, Khan R, Louati Y, Scarlatti E, et al. French validation of the internet addiction test. Cyberpsychol Behav. 2008; 11(6):703–706. PMID: 18954279.
37. Horowitz LM, Snyder DJ, Boudreaux ED, He JP, Harrington CJ, Cai J, et al. Validation of the ask suicide-screening questions for adult medical inpatients: a brief tool for all ages. Psychosomatics. 2020; 61(6):713–722. PMID: 32487323.
38. Pecache DR, Nancho RM, Cuisia-Cruz ES. 124. Validation of the Ask Suicide-Screening Questions (ASQ) as a suicide screening tool for adolescents in the outpatient population of a tertiary hospital. J Adolesc Health. 2022; 70(4):S66.
39. Thom R, Hogan C, Hazen E. Suicide risk screening in the hospital setting: a review of brief validated tools. Psychosomatics. 2020; 61(1):1–7. PMID: 31629482.
40. Chang BP, Tan TM. Suicide screening tools and their association with near-term adverse events in the ED. Am J Emerg Med. 2015; 33(11):1680–1683. PMID: 26346049.
41. Christensen LeCloux M, Aguinaldo LD, Lanzillo EC, Horowitz LM. Provider opinions of the acceptability of Ask Suicide-Screening Questions (ASQ) Tool and the ASQ Brief Suicide Safety Assessment (BSSA) for universal suicide risk screening in community healthcare: Potential barriers and necessary elements for future implementation. J Behav Health Serv Res. 2022; 49(3):346–363. PMID: 35266079.