Impact of Academic Detailing on Opioid Prescribing at the Veterans Health Administration

Abstract

Introduction

In 2014, the United States Veterans Health Administration (VHA) Pharmacy Benefits Management Academic Detailing Service (ADS) began national implementation of academic detailing (AD), a one-on-one educational outreach program delivered by specially trained clinical pharmacy specialists to address mental health disorders and pain management. Consequently, ADS adopted the mission of the VHA Opioid Safety Initiative (OSI) to align providers' opioid prescribing with evidence-based practice. This program evaluation assessed AD's impact on the monthly trends of high-dose opioid utilization and average morphine milligram equivalent (MME) between providers exposed (AD-exposed) and unexposed (AD-unexposed) to OSI-specific educational outreach.

Methods

A retrospective, repeated measures cohort study was performed to evaluate AD's impact on opioid utilization from October 2013-September 2016. Longitudinal data analyses using generalized estimating equations were performed to evaluate the rate of change in the monthly proportions of high-dose opioid users (defined as ≥ 100 MME) and average MME between AD-exposed and AD-unexposed providers.

Results

AD-exposed providers (11.3%; 1,813/15,993) had a 0.2% greater rate of reduction per month in the proportion of high-dose opioid users compared to AD-unexposed providers (95% CI: 0.01%-0.03%), which translates to a 41% and 35% reduction for the AD-exposed and AD-unexposed providers, respectively, adjusting for confounders. Additionally, AD-exposed providers had an average reduction of 0.071 MME per month greater than the reduction observed in AD-unexposed providers (95% CI: 0.035-0.108), which translates to a 38% and 33% reduction in MME for the AD-exposed and AD-unexposed providers, respectively, adjusting for confounders.

Conclusions

AD was associated with a reduction in monthly high-dose opioid utilization and average MME in Veterans.

Keywords

Academic detailing, Veterans, Opioids, Educational outreach, Morphine milligram equivalence

Introduction

Opioid overdose and mortality have become a serious national epidemic in the United States (U.S.). The annual death toll due to opioids was over 46,000 in 2018 [1] and exceeded that of motorized vehicles by 150% starting in 2014 [2]. This was paralleled by a 64% increase in opioid-related inpatient stays and a doubling of the rate of opioid-related emergency department visits from 2005 to 2014 in the general population [3]. In the U.S., mortality risk associated with drug overdose was twice as high in veterans compared to the general population (mortality rate = 1.96; 95% CI: 1.83, 2.08) [4], which was likely caused by increased opioid utilization between 2001 and 2007 (184% for methadone, 60% for synthetic and semisynthetic opioids, and 35% for non-synthetic opioids) [5].

In response, the United States Veterans Health Administration (VHA) implemented the Opioid Safety Initiative (OSI) to address and combat the rising opioid epidemic among Veterans [6]. The OSI is a nationwide system-level campaign implemented in October 2013 to promote safer opioid prescribing practices in order to prevent unwanted adverse events. As part of the OSI, a computerized clinical dashboard was created to provide stakeholders with a benchmark from which to evaluate the trends in opioid prescribing at the network-, facility-, provider-, and patient-levels. Facilities could then audit the data and provide feedback in order to improve opioid prescribing. Despite a national focus, there were concerns that variations in OSI implementation would negatively impact desired outcomes. Therefore, OSI partnered with the VHA Pharmacy Benefits Management (PBM) Academic Detailing Service (ADS) to improve the diffusion of OSI-specific key messages to providers across the VHA.

Academic detailing is a one-on-one educational outreach between clinicians designed to align clinical care with evidence-based practice [7,8]. Academic detailing is used to deliver balanced, non-commercial evidence-based information to providers. Previous studies have reported that academic detailing improved the judicious use of antibiotics [9], appropriate pain management [10], and adherence to antihypertensive guidelines [11]. In addition, academic detailing was associated with an increase in naloxone prescribing to Veterans at risk for opioid overdose [12,13] and reduction in unnecessary benzodiazepine use in Veterans who were elderly [14] and with posttraumatic stress disorder [15]. Clinical pharmacy specialists at the VHA were provided training in academic detailing to deliver OSI-specific educational outreach visits to providers who were prescribing opioids in order to align their prescribing with safer evidence-based practice. Academic detailers were trained to deliver information using communication techniques and strategies designed to influence behaviors (e.g., motivational interviewing-inspired techniques, audit and feedback tools) and were familiarized with the content via self-study, seminars, and face-to-face trainings.

Since the implementation of the OSI, opioid utilization has decreased across the VHA. High dose opioid users (defined as ≥ 100 morphine equivalent daily dose) decreased by 16% from October 2012 to September 2014, which translates to approximately 331 patients per month [6]. However, the incremental effects of academic detailing alongside OSI on the opioid prescribing practices at the VHA have not been studied. Based on our current knowledge, we hypothesized that providers exposed to OSI-specific educational outreach visits by an academic detailer would have a greater reduction in monthly high-dose opioid utilization and average morphine milligram equivalent (MME) compared to providers unexposed to academic detailing alongside OSI.

Objectives

The primary aim of this study was to evaluate the real-world impact of the VHA academic detailing program on the proportion of Veterans who are on high-dose opioids defined as an average of 100 MME or greater per monthly per provider. The secondary aim evaluated the impact of the academic detailing on the average MME per month per provider.

Methods

Design

This was a retrospective, repeated measures cohort study to evaluate academic detailing's impact on OSI measures from October 2013 to September 2016 (36 months). National implementation of academic detailing underwent a phased rollout starting in October 2013; therefore, VHA providers were exposed to OSI-specific educational outreach at various times. We treated exposure (receipt of AD) as time-varying, which meant that providers who were not exposed to OSI-specific academic detailing educational outreach at the beginning of the study may be exposed later. The primary outcome was the change in the monthly proportion of patients per provider who were on high-dose opioids defined as a monthly average 100 MME or greater during the study period. The secondary outcome was the change in the average monthly MME per patient per provider during the study period. MME was calculated using the methods determined by the CDC National Center for Injury Prevention and Control [16,17]. Only active VA opioid prescriptions were used in our calculations, which included codeine, butorphanol, fentanyl, hydrocodone, hydromorphone, levorphanol, meperidine, methadone, morphine, nalbuphine, oxycodone, oxymorphone, pentazocine, propoxyphene, and tapentadol. This study was reviewed and approved by the Edward J. Hines VHA Institutional Review Board in accordance with VHA Handbook 1058.05.

Intervention

VA academic detailers attended a 3- to 4-day interactive workshop targeting building relationships, using motivational language and communication techniques such as audit and feedback strategies, working on barriers and enablers to facilitate practice change, acquiring active and reflective listening techniques, applying feel-felt-found methods, and handling and dealing with objections to assist in effectively delivering evidence-based information to providers [18]. The information was disseminated primarily through one-on-one and small-group educational outreach sessions. Academic detailers met with the providers who have a panel of patients with the greatest opportunity for opioid reduction. The academic detailer focused on delivering targeted messages, embedded within the academic detailing Pain Management Opioid Safety provider handouts [19], and used audit and feedback strategies to help the provider identify patients for whom treatment plan adjustments may be warranted [20,21]. Academic detailers were also trained to provide customized content and barrier resolution strategies based on providers' individual needs and local resource availability.

Sample

The VHA is the largest integrated health care system in the U.S. with a network of 170 medical centers with 1,074 outpatient clinics serving over 8.9 million Veterans annually [22]. VHA health care covers veterans across all 50 U.S. states and her territories including Guam, the Philippine Islands, the Virgin Islands and Puerto Rico.

Providers at the VHA were included for analysis if they prescribed an opioid between October 2013 and September 2016. We used a closed cohort of providers who were actively working for the VHA and prescribing opioids from October 2013 to September 2016. Therefore, providers who left or entered the VHA service during this time frame were not included for analysis. This was done to reduce the bias associated with providing who were new or leaving the VHA and to stabilize the denominator. We only focused on opioid use for chronic non-cancer pain; therefore, we excluded patients from a provider panel if the opioids were used for cancer pain or if the patients were hospice care. Providers were considered exposed if they received one or more OSI-specific educational outreach by a trained academic detailer during the study period. Otherwise, they were categorized as unexposed.

Data source

Pharmacy claims and provider level data (age, gender, and tenure) came from the VHA Corporate Data Warehouse [23]. Academic detailing educational outreach visit data were captured using Salesforce.com® (San Francisco, CA), an online cloud-based platform designed for customer relationship management, data collection, and reporting. Academic detailers recorded their educational outreach interactions with providers including the date the visit occurred and topics discussed. These data were mapped to the provider- and patient-level data in the CDW to generate a monthly panel dataset for analyses.

Analysis

Baseline comparisons between AD-exposed and AD-unexposed providers were performed using independent t-test for continuous data and chi square test for discrete data. For the main endpoints, unadjusted comparisons on the changes from baseline were performed using difference-in-differences estimation. Additionally, longitudinal data analyses using generalized estimating equations (GEE) models with exchangeable correlation structures were constructed to account for repeated measures within each provider [24]. For the GEE model specification, a Gaussian family with an identity link was used [25]. Provider-level characteristics (age, gender, and months worked at the VA) in addition to baseline endpoints were controlled for in each model to adjust for imbalances across the groups.

In the GEE models, we included an interaction term between academic detailing exposure and time (months) to capture the difference-in-differences estimate across the study period. Difference-in-differences estimation provides us with the average change in the rate of the outcome between the study groups (AD-exposed and AD-unexposed providers) across the study period [26]. The outcomes for the primary and secondary aims were presented as the mean with 95% confidence interval (CI). Model fit was evaluated using the quasi-likelihood under the independence model criterion (QIC), which is an extension of the Akaike Information Criterion and used to compare different model specifications [27,28]. Statistical significance was defined as a two-tailed alpha of less than 5%. All analyses were performed using Stata SE version 15 (College Station, TX).

Results

A total of 1,813 (11.3%) providers were exposed to OSI-specific academic detailing educational outreach during the study period. AD-exposed providers were slightly older (51 versus 50 years, P < 0.001), had a larger proportion of females (49% versus 47%, P = 0.025), worked longer hours (0.96 versus 0.85 full time equivalents, P < 0.001), and had a shorter work history (33.6 versus 47.1 months, P < 0.001) compared to AD-unexposed providers (Table 1).

At baseline, the average proportions of patients on high-dose opioids for the AD-exposed and AD-unexposed providers were 2.05% and 1.14%, respectively; the average MME at baseline for the AD-exposed and AD-unexposed providers were 9.42 MME and 5.99 MME, respectively (Table 2).

For the primary aim, providers who were exposed to academic detailing had a larger reduction in the proportion of Veterans on high-dose opioids between month 36 and baseline compared to providers unexposed to academic detailing. In the unadjusted analysis, AD-exposed and AD-unexposed providers had a reduction of 40% and 39%, respectively (difference-in-differences = 0.40%; 95% CI: 0.20%, 0.60%, Table 2). In the GEE model, AD-exposed providers had a significantly greater rate of reduction in the proportion of Veterans on high-dose opioids compared to AD-unexposed providers controlling for baseline confounders (difference in the rate of high-dose opioid reduction per month was 0.02%; 95% CI: 0.01%, 0.03%; p = 0.001). In other words, when adjusting for confounders, AD-exposed providers had a significantly faster rate of reducing their proportion of Veterans on high-dose opioids than AD-unexposed providers over 36 months. This translates to a 41% reduction across the study period for the AD-exposed providers and a 35% reduction for the AD-unexposed providers after adjusting for baseline confounders (Figure 1). We verified our model specification using the QIC; the most appropriate model specification used a linear form with exchangeable correlation structure.

For the secondary aim, providers who were exposed to academic detailing had a larger reduction in the average MME between month 36 and baseline compared to providers unexposed to academic detailing. In the unadjusted analysis, AD-exposed and AD-unexposed providers had a reduction in average MME of 39% and 31%, respectively (Table 2). In the GEE model, AD-exposed provider has a significantly greater rate of reduction in the average monthly MME compared to AD-unexposed providers controlling for baseline confounders (difference in the rate of MME reduction per month was 0.071 MME; 95% CI: 0.035, 0.108; p < 0.001). In other words, when adjusting for confounders, AD-exposed providers had a significantly faster rate of reducing average MME than AD-unexposed providers over 36 months. This translates to a 38% reduction in MME across the study period for the AD-exposed providers and a 33% reduction for the AD-unexposed providers after adjusting for baseline confounders (Figure 2). Based on the QIC, the most appropriate model specification was a linear form with exchangeable correlation structure.

Discussion

Academic detailing plays a crucial role in the overall strategy to reduce opioid use and improve pain management. Overreliance on opioids for pain management has been due in large part to misinformation and inability to remain current on recent evidence-based literature. Despite the release of updated guidelines for chronic pain management by the CDC in 2016 [16] providers reported a lack of confidence as well as gaps in knowledge regarding the use of opioids in the management of chronic pain [29,30]. Academic detailing can close the knowledge gap by customizing the educational outreach to meet the provider's needs. Previous studies have reported that continuing education and training can improve providers' knowledge and confidence in chronic pain management with opioids [31-33]; however, focused, one-on-one educational outreach has not been studied in this area. This was the first study to demonstrate the reduction in opioid prescribing through educational outreach also known as academic detailing.

Using real world data, we reported that providers who were exposed to OSI-specific academic detailing educational outreach had a significantly greater rate of reduction in high-dose opioid users and average MME compared to providers who were unexposed. The VHA's response to the opioid epidemic included the implementation of OSI and ADS, which were designed to educate and align providers with evidence-based practices for pain management. The OSI has been reported to be associated with a 16% reduction in high-dose opioid users at the VHA after its implementation in October 2013 (in October 2012, there were 55,722 Veterans on opioids doses > 100 milliequivalents, and by September 2014, this number was reduced to 46,780 Veterans) [6]. With the combined effects of the OSI and academic detailing program, we expected academic detailing to enhance the reduction in high-dose opioid utilization of the OSI.

The OSI was implemented in October 2013 in response to the rising opioid epidemic. This was followed by the national implementation of the VHA PBM ADS in 2014 after successful completion of its pilot program. In March 2015, the Interim Under Secretary of Health, testified before Congress that academic detailing was a necessary program needed to address mental health and pain management issues [34]. Moreover, in 2017, the Secretary for the Department of Veterans Affairs, testified in his congressional hearing that academic detailing was a critical part of an overall strategy to address the opioid epidemic in the VA population [35]. As a result of these efforts the number of Veterans on opioids prescriptions decreased by 25% from 2012 to 2016 [36].

These results have important policy implications for the VHA and public health. Future and sustainable funding for academic detailing and OSI are necessary for the continual education of providers and patients. These programs provide additional benefits that go beyond the reduction in high-dose opioid users and average MME. Real-time surveillance tools (e.g., clinical dashboards) and metrics used to monitor opioid utilization are important for policymakers to make informed decisions about program expansion and sustainability [37]. In addition, these programs also focus on developing strategies to increase penetration into rural areas, which has been challenging. Academic detailing provides a means for providing educational outreach to providers in rural areas, which is a much-needed prevention strategy. Sustaining the improvements of academic detailing requires empirical support and positive return on investment. Previous cost-benefit analysis reported that academic detailing saves $2 for every one dollar spent [38]. However, it is unclear whether the results of the cost-benefit analysis are generalizable to the VHA or opioid utilization. Moreover, decision makers should consider the societal return on investments associated with reduced opioid-related abuse, crime, and deaths [39,40]. Future cost-benefits analysis will need to take into consideration the downstream consequences of opioid reduction in the Veteran population.

Additionally, there is concern that reducing or discontinuing patients who are chronic users of opioids may have unintended consequences. Oliva and colleagues investigated the impact of opioid discontinuation among Veterans and reported greater risk of overdose or suicide immediately after opioid discontinuation [41]. This risk increased the longer the Veterans were on an opioid before discontinuation. These findings suggest that a comprehensive treatment plan should be implemented alongside opioid discontinuation to safely reduce the risk attributable to both opioid use and discontinuation. Therefore, it is recommended that any opioid-related academic detailing is accompanied with education and counseling on how to individualize the Veteran's treatment plan and ensure safety of the Veteran.

There are several limitations associated with this study. First, we did not evaluate the implementation process of academic detailing at the VHA. Variability in academic detailing implementation due to differences in priorities, resources, or lack of stakeholder engagement may contribute to differences in a facility's performance. It was uncertain what the degree of implementation fidelity was for each facility; therefore, it was quite possible that the treatment effect of academic detailing on opioid utilization could vary. For example, some facilities may be fully implemented with support from local leadership and stakeholders, which would have a positive effect for academic detailing, whereas a facility with little support from local leadership and limited resources may not have enough penetration and little effect on opioid utilization. Midboe and colleagues evaluated academic detailers' and providers' perceptions about implementation of academic detailing at the VHA and identified that leadership support and buy-in were critical to promoting academic detailing; however, allocating enough time for academic detailing to be effective was a concern [42]. Second, we were unable to evaluate the dose-response relationship between multiple academic detailing interactions and opioid reduction. Reinforcement and repetition of academic detailing is an important factor for sustaining desirable behavioral changes. More frequent OSI-specific educational outreach and longer duration may impact the effect of academic detailing on opioid utilization. Previous studies have reported that academic detailing has a higher probability of changing provider's prescribing behavior with each additional educational outreach visit, but no additional impact was reported with increased duration of the visits beyond 30 minutes [38]. Moreover, it is unclear whether initial impact of academic detailing will decay if reinforcement or repetition is not performed with a follow-up visit. Future studies will need to evaluate the impact of repeat visits and educational outreach duration on opioid utilization. Third, we were unable to determine if reduction in opioid utilization would result in a substitution effect. In a report by Schnell and colleagues, states that had adopted a policy to use an abuse-deterrent version of oxycodone had an increase in heroin-related deaths, which may be attributable to a substitution effect [43]. It is unclear whether this effect also occurs among Veterans, but future investigations will need to evaluate this potential negative externality on the VHA's response to addressing the opioid epidemic. Finally, our study was performed in Veterans, which may not be generalizable to the U.S. population. However, the findings from our analyses may be useful for large, integrated healthcare systems struggling to effectively and safely reduce opioid prescribing.

Conclusions

Providers who were exposed to OSI-specific academic detailing educational outreach had a greater rate of reduction for high-dose opioid users and average MME over 36 months after initial implementation compared to providers who did not receive academic detailing. These results provide evidence that academic detailing, in collaboration with the broader VHA OSI, was effective at addressing the opioid epidemic in Veterans. However, future investigations should include assessments of potential unintended consequences of opioid discontinuation and implementation fidelity.

Acknowledgements

We would like to acknowledge the continual support from our Pharmacy Benefits Management leadership, Michael Valentino and Virginia Torrise. Their guidance, leadership, and encouragement have been instrumental in implementing essential programs that improve the lives of our Veterans. We would also like to acknowledge all the academic detailing clinical pharmacists who have worked tirelessly to reach out to providers and deliver vital information through their academic detailing interventions. We also want to extend our thanks to Priya Randeria-Noor who was an invaluable member of our team during our transition to a national program. The efforts and dedication of our supporters have contributed to the overall improvements to our health care system. Finally, to our Veterans who have sacrificed more than we can ever know at the altar of freedom, thank you.

References

1. Wilson N (2020) Drug and opioid-involved overdose deaths - United States, 2017-2018. MMWR Morb Mortal Wkly Rep 69: 290-297.
2. Rudd RA, Aleshire N, Zibbell JE, et al. (2016) Increases in drug and opioid overdose deaths - United States, 2000-2014. MMWR Morb Mortal Wkly Rep 64: 1378-1382.
3. Weiss AJ, Elixhauser A, Barrett ML, et al. (2017) Opioid-related inpatient stays and emergency department visits by state, 2009-2014: Statistical brief 219. In: Healthcare cost and utilization project (HCUP) statistical briefs. Agency for healthcare research and quality (US); 2006.
4. Bohnert ASB, Ilgen MA, Galea S, et al. (2011) Accidental poisoning mortality among patients in the Department of Veterans Affairs Health System. Med Care 49: 393-396.
5. Bohnert ASB, Ilgen MA, Trafton JA, et al. (2014) Trends and regional variation in opioid overdose mortality among veterans health administration patients, fiscal year 2001 to 2009. Clin J Pain 30: 605-612.
6. Lin LA, Bohnert ASB, Kerns RD, et al. (2017) Impact of the opioid safety initiative on opioid-related prescribing in veterans. Pain 158: 833-839.
7. Soumerai SB, Avorn J (1990) Principles of educational outreach ('academic detailing') to improve clinical decision making. JAMA 263: 549-556.
8. Avorn J, Soumerai SB (1983) Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based "detailing". N Engl J Med 308: 1457-1463.
9. Kisuule F, Wright S, Barreto J, et al. (2008) Improving antibiotic utilization among hospitalists: A pilot academic detailing project with a public health approach. J Hosp Med 3: 64-70.
10. Boothby LA, Wang LJ, Mayhew S, et al. (2003) Academic detailing of meperidine at a teaching hospital. Hosp Pharm 38: 30-35.
11. Siegel D, Lopez J, Meier J, et al. (2003) Academic detailing to improve antihypertensive prescribing patterns. Am J Hypertens 16: 508-511.
12. Bounthavong M, Harvey MA, Wells DL, et al. (2003) Trends in naloxone prescriptions prescribed after implementation of a national academic detailing service in the veterans health administration: A preliminary analysis. J Am Pharm Assoc 57: S68-S72.
13. Bounthavong M, Devine EB, Christopher MLD, et al. (2019) Implementation evaluation of academic detailing on naloxone prescribing trends at the United States Veterans Health Administration. Health Serv Res 54: 1055-1064.
14. Ragan AP, Aikens GB, Bounthavong M, et al. (2019) Academic detailing to reduce sedative-hypnotic prescribing in older veterans. J Pharm Pract.
15. Bounthavong M, Lau MK, Popish SJ, et al. (2020) Impact of academic detailing on benzodiazepine use among veterans with posttraumatic stress disorder. Subst Abuse 41: 101-109.
16. Dowell D, Haegerich TM, Chou R (2016) CDC guideline for prescribing opioids for chronic pain - United States, 2016. MMWR Recomm Rep 65: 1-49.
17. (2020) Alosa health.
18. Miller WR, Rollnick S (1991) Motivational interviewing: Preparing people to change addictive behavior. Guilford Press.
19. VA Academic Detailing Program Service. U.S. Department of Veterans Affairs (2014) Pain management opioid safety educational guide (2014).
20. Hysong SJ, Teal CR, Khan MJ, et al. (2012) Improving quality of care through improved audit and feedback. Implement Sci 7: 45.
21. Ivers N, Jamtvedt G, Flottorp S, et al. (2012) Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev.
22. Veterans Health Administration (2019) About VHA - Veterans health administration.
23. U.S. Department of Veterans Affairs (2014) 172VA10P2: VHA corporate data warehouse - VA. 79 FR 4377.
24. Liang KY, Zeger SL (1986) Longitudinal data analysis using generalized linear models. Biometrika 73: 13-22.
25. Hanley JA, Negassa A, Edwardes MD deB, et al. (2003) Statistical analysis of correlated data using generalized estimating equations: An orientation. Am J Epidemiol 157: 364-375.
26. French B, Heagerty PJ (2008) Analysis of longitudinal data to evaluate a policy change. Stat Med 27: 5005-5025.
27. Cui J (2007) QIC program and model selection in GEE analyses. Stata J 7: 209-220.
28. Pan W (2001) Akaike's information criterion in generalized estimating equations. Biometrics 57: 120-125.
29. Pearson AC, Eldrige JS, Moeschler SM, et al. (2016) Opioids for chronic pain: A knowledge assessment of nonpain specialty providers. J Pain Res 9: 129-135.
30. Pearson AC, Moman RN, Moeschler SM, et al. (2017) Provider confidence in opioid prescribing and chronic pain management: Results of the opioid therapy provider survey. J Pain Res 10: 1395-1400.
31. Jamison RN, Scanlan E, Matthews ML, et al. (2016) Attitudes of primary care practitioners in managing chronic pain patients prescribed opioids for pain: A prospective longitudinal controlled trial. Pain Med 17: 99-113.
32. Ruff AL, Alford DP, Butler R, et al. (2017) Training internal medicine residents to manage chronic pain and prescription opioid misuse. Subst Abuse 38: 200-204.
33. Zisblatt L, Hayes SM, Lazure P, et al. (2017) Safe and competent opioid prescribing education: Increasing dissemination with a train-the-trainer program. Subst Abuse 38: 168-176.
34. U.S. Congressional House Committee on Veterans' Affairs (2015) Statement of Dr. Carolyn Clancy interim under secretary for Health Veterans Health Administration Department of Veterans Affairs before the subcommittee on oversight and investigations committee on veterans affairs United States House of representatives.
35. U.S. Senate, Committee on Veterans' Affairs (2017) Senate hearing, 115^th congress - nomination of Hon. David J. Shulkin.
36. Gellad WF, Good CB, Shulkin DJ (2017) Addressing the opioid epidemic in the United States: Lessons from the department of veterans affairs. JAMA Intern Med 177: 611-612.
37. Lau MK, Bounthavong M, Kay CL, et al. (2019) Clinical dashboard development and use for academic detailing in the U.S. department of veterans affairs. J Am Pharm Assoc (2003) 59: S96-S103.e3.
38. Soumerai SB, Avorn J (1987) Predictors of physician prescribing change in an educational experiment to improve medication use. Med Care 25: 210-221.
39. McCollister K, Yang X, Sayed B, et al. (2017) Monetary conversion factors for economic evaluations of substance use disorders. J Subst Abuse Treat 81: 25-34.
40. Humphreys K, Wagner T, Gage M (2011) If substance use disorder treatment more than offsets its costs, why don't more medical centers want to provide it? A budget impact analysis in the veterans health administration. J Subst Abuse Treat 41: 243-251.
41. Oliva EM, Bowe T, Manhapra A, et al. (2020) Associations between stopping prescriptions for opioids, length of opioid treatment, and overdose or suicide deaths in US veterans: Observational evaluation. BMJ 368.
42. Midboe AM, Wu J, Erhardt T, et al. (2018) Academic detailing to improve opioid safety: Implementation lessons from a qualitative evaluation. Pain Med 19: S46-S53.
43. Schnell M, Currie J (2017) Addressing the opioid epidemic: Is there a role for physician education? National Bureau of Economic Research.