Evaluating the Implementation of Digital and In-person Diabetes Prevention Programs in a Large, Integrated Health System: Natural Experiment Study Design


Stephanie L Fitzpatrick, PhD1; Meghan Mayhew, MPH1; Chris L Catlin, BS1; Alison Firemark, MA1; Inga Gruß, PhD1; Denis B Nyongesa, MS1; Maureen O’Keeffe-Rosetti, MS1; Andreea M Rawlings, PhD, MS1; David H Smith, PhD, RPh1; Ning Smith, PhD1; Victor J Stevens, PhD1; William M Vollmer, PhD1; Stephen P. Fortmann, MD1

Perm J 2021;25:21.056


E-pub: 11/08/2021

Introduction: Implementation of a Diabetes Prevention Program (DPP) (in-person and digital) in health-care settings has been increasing. The purpose of this article is to describe the protocol of a mixed-methods, natural experiment study designed to evaluate the implementation of DPP in a large, integrated health system.

Methods: Kaiser Permanente Northwest patients who were 19 to 75 years with prediabetes (hemoglobin A1c or glycated hemoglobin, 5.7–6.4) and obesity (body mass index ³ 30 kg/m2) were invited, via the Kaiser Permanente Northwest patient portal, to participate in the digital (n = 4124) and in-person (n = 2669) DPP during 2016 through 2018. Primary (weight) and secondary (hemoglobin A1c or glycated hemoglobin level) outcome data will be obtained from electronic health records. A cost-effectiveness analysis as well as qualitative interviews with patients (enrolled and not enrolled in the DPP) and stakeholders will be conducted to examine further implementation, acceptability, and sustainability.

Conclusion: The mixed-methods, natural experiment design we will use to evaluate Kaiser Permanente Northwest’s implementation of the digital and in-person DPP builds on existing evidence related to the effectiveness of these two DPP delivery modes and will contribute new knowledge related to best practices for implementing and sustaining the DPP within large health systems over the long term.


The prevalence of prediabetes and obesity among US adults 40 years and older is significant, with more 30% having prediabetes and more 40% having obesity.1,2 Prediabetes and obesity increase the risk for diabetes, cardiovascular disease, and poor quality of life, and are responsible for substantial health-care costs.3 In fact, the estimated costs of managing diabetes in the US was $327 billion in 2017, a 26% increase since 2012.3 In response to the multilevel burden of prediabetes and obesity, there have been several efforts to prevent diabetes at the population level and reduce health-care costs,4,5 including national implementation and reimbursement of the successful Diabetes Prevention Program (DPP).

The DPP clinical trial demonstrated conclusively that a lifestyle intervention consisting of nutrition, physical activity, and behavioral counseling was more effective in reducing the incidence of type 2 diabetes and producing clinically significant weight loss than were metformin or placebo.6 These findings inspired numerous translational studies across the US, conducted in different settings (eg, primary care, community, churches). Meta-analyses of these studies found modest but clinically meaningful effects of the intervention in real-world settings on weight, hemoglobin A1c (HbA1c) and other cardiometabolic risk factors.7–10 The National Diabetes Prevention Program was created in 2010 to promote DPP dissemination across the US and currently recognizes both in-person and digital (remotely delivered, primarily internet-based) DPPs that meet their standards and operating procedures.11

Beginning in April 2018, the Centers for Medicare & Medicaid Services made a landmark decision to reimburse clinical and nonclinical settings for providing a DPP to Medicare beneficiaries (ie, Medicare DPP); this coverage is currently for in-person DPP only and not digital DPP.12,13 The Centers for Medicare & Medicaid Services’ decision to cover a DPP among older adults with prediabetes further catapulted efforts within health-care organizations to address the increasing number of individuals with diabetes receiving care in their facilities. The implementation and effectiveness of DPPs within large health systems—primarily the Veterans Health Administration—has been a focus of recent research.14–17 However, few studies have examined the sustainability of providing a DPP based on maintenance of the effect (ie, long-term weight change and HbA1c levels), health-care costs, participant experience, and organizational support. In addition, engaging individuals in a DPP and similar lifestyle change interventions remains a challenge, and identifying useful approaches is needed.18–21 Last, although the effectiveness of an in-person DPP is well-established, prior studies evaluating the effect of a digital DPP identified positive outcomes but had significant methodological limitations, such as a single arm pre-/posttest design and participant-reported outcomes.17,22–28

In 2017, Kaiser Permanente Northwest (KPNW), a large, integrated health system serving Oregon and southwest Washington, began piloting both digital and in-person versions of a DPP for its adult health plan members who were  19 to 75 years old with prediabetes (HbA1c, 5.7–6.4) and obesity [body mass index (BMI) ³ 30 kg/m2]. We describe a mixed-methods, natural experiment study design to evaluate this large health system initiative by assessing the effects of both a digital and in-person DPP on change in weight and HbA1c level, as well as sustainability based on cost-effectiveness and patient and health-care stakeholder perspectives. We present the approaches KPNW used to identify, recruit, and enroll patients with obesity at high risk for diabetes into a digital and in-person DPP. Furthermore, we describe the two DPP modalities and the mixed-methods approach used in this natural experiment.



KPNW provides comprehensive prepaid health care to more than 600,000 members in Oregon and southwest Washington. All patient contacts within the system and all services referred outside are recorded in a single, comprehensive electronic health record (EHR) (KP HealthConnect, based on Epic®). Approximately 30% of KPNW members have prediabetes, 12% have diabetes, and more than 40% are obese. From 2017 through 2020, KPNW piloted both digital and in-person versions of the DPP for adult health plan members who had both prediabetes and obesity. The decision to target health plan members with both prediabetes and obesity was based on recent predictive modeling using data from more than 77,000 health plan members, which established that an HbA1c level of 6.3 to 6.4 and a BMI ³ 30 were associated with more than a 15% probability of developing diabetes in 2 years.29 There were 2 distinct cohorts recruited by the KPNW health system for participation in the diabetes prevention programs. Cohort 1 included KPNW members who were invited by the health-care system to participate in the digital DPP, which was delivered from April 2017 through April 2018. Cohort 2 included KPNW members who were invited by the health-care system to participate in a group-based in-person DPP program across three waves of recruitment and intervention delivery from October 2017 through February 2020. Each cohort is described in further detail later.

Study Design

We use a mixed-methods natural experiment study design to assess clinical and cost outcomes among the two cohorts. Specifically, we will compare DPP enrollees to nonenrollees (digital DPP and in-person DPP cohorts will be analyzed separately) on changes in weight and HbA1c levels. Randomization was not feasible in this real-world implementation of the DPP; therefore, propensity score adjustment will be used to control for potential confounding. All data to be used in the analyses of clinical and cost outcomes are collected as part of standard health-care administration within the KPNW health-care system. Primary qualitative data are collected from a subset of DPP enrollees and nonenrollees in the 2 cohorts, as well as from health-care system providers and stakeholders. This study was approved by the KPNW Institutional Review Board (protocol no. STUDY00000693).

Eligibility and Interventions

Digital DPP (Cohort 1).

Cohort 1 individuals met the following inclusion criteria as noted in the EHR: 1) current KPNW member 65 to 75 years old, 2) BMI ³ 30, 3) HbA1c level between 5.7 and 6.4, and 4) no prior diagnosis of diabetes. In addition, patients had to use the KPNW electronic patient portal (approximately 80% of all KPNW members meet this criterion).

For Cohort 1, KPNW partnered with Omada Health to offer their program, a Centers for Disease Control and Prevention (CDC)–recognized11 translation of the DPP lifestyle intervention that is delivered in a digital format.30 The program consists of (virtual) small-group support, health coaching from Centers for Disease Control and Prevention–trained lifestyle coaches, a National DPP-approved curriculum,31 and electronic behavioral tracking tools. Participants are grouped with other individuals with similar demographics, geographic proximity, and BMI. Group participants communicate via a private social network and discussions are facilitated by a health coach. The health coach also communicates with participants through private messages or phone calls, and provides feedback on weight loss progress, and food and physical activity logs. Lessons from the DPP curriculum are posted each week, and participants can review these at their own pace. To facilitate self-monitoring, participants also receive a wireless weight scale and pedometer. The 12-month digital DPP program includes a 16-week intensive program and a 36-week maintenance program. Omada Health has several studies demonstrating the effectiveness of the program in producing clinically significant weight loss.22 The program was only offered once to KPNW members at no cost as part of this implementation pilot.

In-person DPP (Cohort 2).

Individuals eligible for in-person DPP included a wider age range of 19 to 75 years and a greater diabetes risk level (HbA1c, 6.0–6.4) to ensure those at highest risk for diabetes were being targeted given limited enrollment slots, but otherwise had the same criteria as cohort 1.

In Fall 2017, KPNW launched their Preventing Diabetes Program, a 12-month group-based, in-person DPP that closely followed the Centers for Disease Control and Prevention National DPP curriculum.11 The in-person DPP is overseen by the KPNW Health Engagement & Wellness Services, and group sessions are led by KPNW-registered dietitians. Participants attend sessions weekly for the first 6 months and then monthly for the next 6 months free of charge. Group sessions included approximately 20 participants each and were held at 2 KPNW clinic locations at a variety of times (daytime and evening) on weekdays and weekends so that participants could attend at their preferred time and place. Participants received hard copies of all curriculum materials, including logs to track weight, eating behaviors, and physical activity, and were encouraged to weigh themselves at home weekly. At the weekly meetings, participants recorded their weight, number of food records kept, and minutes of physical activity from the previous week, and shared these with the group facilitator.


Beginning in early 2017, KPNW staff identified eligible members based on their most recent BMI and HbA1c level documented in the EHR. Primary care providers were sent an email with a list of their eligible patients and were given 14 business days to opt out if they did not want their patients to be recruited for the DPP. The eligible members whose providers did not opt out were sent a secure email message within the KPNW patient portal inviting them to enroll in the DPP (Figures 1 and 2). The invitation message was signed as though it came from the patients’ primary care provider. The patient portal is used by 80% of KPNW members and is a common venue for patients to schedule appointments, refill prescriptions, get visit summaries and lab results, and communicate with care providers.

Figure 1. Digital Diabetes Prevention Program secure invitation email.

tpj21056f1 copy


Figure 2. In-person Diabetes Prevention Program secure invitation email.

tpj21056f2 copy


Figure 3 presents the timeline for cohort identification, recruitment, and delivery of the DPP at KPNW. A total of 4132 individuals were identified who met the criteria for cohort 1 and were sent secure messages in April 2017, inviting them to enroll in the digital DPP, which was the only program available at the time because KPNW’s in-person DPP did not launch until Fall 2017. Members were instructed to click on the unique web link embedded in the message to enroll in the digital DPP (Figure 1). Enrollment was tracked via a unique code, and Omada Health provided enrollment data back to KPNW. KPNW planned to offer a digital DPP once as part of the implementation pilot and, because of limited resources, allow only 500 enrollment slots.

Figure 3. Kaiser Permanente Northwest timeline for cohort identification, recruitment, and delivery of the Diabetes Prevention Program (DPP). EHR = electronic health record.

tpj21056f3 copy


Individuals included in cohort 2 were identified beginning in July 2017 and were recruited in 3 waves: first in August 2017, again in April 2018, and then in December 2018. There were 2669 invitation messages sent for in-person DPP. For each wave, invitation emails were sent in 2 to 3 batches during the recruitment period (Figure 2). Reminder emails were sent 14 days after the initial invitation. Enrollment slots were limited to 100 participants in each wave (300 total). In the invitation, members were instructed to call a KPNW health coach if interested in enrolling. Health coaches explained the program details and expectations, and assessed (via phone conversations) participant readiness. If determined to be ready, members signed up for their preferred class day and time.

Recruitment for the digital and in-person DPP did not overlap, as shown in Figure 3. To reduce potential contamination, cohort 1 patients enrolled in the digital DPP were not invited to sign-up for the in-person DPP after that program started. However, eligible patients in cohort 1 who did not enroll in the digital DPP were invited to participate in the in-person DPP.

Data Collection

Clinical and Cost Data.

The primary outcome is change in weight at 12 months for both the digital and in-person DPP cohorts. Secondary outcomes include change in HbA1c level at 12 months for both cohorts and, for the digital DPP cohort, change in weight and HbA1c level at 24 months. We will also conduct a cost-effectiveness analysis for each cohort at 12 months and, for the digital DPP cohort, at 24 months. Data used for these analyses come from the EHR and were collected during routine care. Clinical data entered in the EHR (eg, weight and HbA1c level) up to 12 months prior to DPP invitations/recruitment letters being sent out serve as baseline measures. Follow-up data come from the 14-month (for 12-month outcomes) or 26-month (for 24-month outcomes) period following the date the DPP invitation/recruitment letters were sent out; all available follow-up data will be used in analyses. Health-care use events documented in the EHR (ie, visits, pharmacy dispenses) for the 12-month baseline period described will be used to calculate costs per patient using standard costing algorithms32,33 and Medicare fee schedules, and will be compared to costs based on use events during the same 2 follow-up periods assessed for 12- and 24-month clinical outcomes. Clinical covariates (described further later) will also be obtained from the EHR. We obtained a waiver of informed consent for these population-based analyses.

Qualitative Data.

Interview guides were developed to ensure consistency in data collection during the semistructured interviews conducted with patients, health-care stakeholders, and providers.34–36 From September 2017 to July 2018, a subsample of DPP enrollees and nonenrollees from cohorts 1 and 2, were selected randomly and invited to participate in an hour-long, semistructured phone interview. Interview participants were offered a financial incentive for participation. The interview guide focused on patients’ perception of the methods used to invite them to participate in the DPP, patients’ reasons for participating, barriers and facilitators to enrollment and ongoing participation, patients’ perceived sense of usefulness of program components, and recommendations for improving and sustaining the program. In addition, from June 2018 to August 2018, health-care system leaders and providers were invited to participate in hour-long semistructured phone interviews to understand barriers and facilitators to sustaining the DPP within the health-care system. Verbal consent was obtained from all interview participants prior to initiation, and all interviews were audio-recorded with participants’ permission and were transcribed professionally.

Data Analysis

Power Considerations for Primary Outcome Analysis.

Power calculations were conducted for pairwise comparisons of each DPP modality (digital and in-person) vs usual care with the following assumptions: 1) Bonferroni-adjusted type 1 error rate of 0.05/2 » 0.025, 2) the observed weight loss or HbA1c level change for DPP enrollees would be the same for both DPP modalities, 3) there would be 2000 KPNW members eligible for each DPP cohort, and 4) DPP enrollment in each cohort would vary from as low as 5% of eligible KPNW members (n = 100) to as high as 20% (n = 400). This range of enrollment is consistent with previous experience offering such programs using similar recruitment methods in smaller, select populations of patients with prediabetes. The observed weight/HbA1c level change is thus a weighted average of the assumed true DPP effect in DPP enrollees and the assumed change for those in usual care.

For weight, we used a true DPP effect of 9.85 lbs13,30 and a usual care effect of 0.93 lb6 as an average of reported effects from the literature. For the standard deviation of weight change, we assumed 1.2 lbs.30 As indicated in Table 1, we have excellent power to detect significant differences even with enrollment rates as low as 5%. For the HbA1c level, we used a true DPP effect of a reduction of 0.24% as an average of reported results30,37 and 0.0% for patients receiving usual care.37 For the standard deviation of HbA1c level change, we assumed 0.07%.30 As indicated in Table 1, with these assumptions we again have excellent power if enrollment rates are at least 10%.

Table 1. Power for comparison of each Diabetes Prevention Program modality vs usual carea

Enrollmentb Observed weight lossc Observed HbA1c decreased
UC DPP Power UC DPP Power
5% 0.93 1.38 > 0.99 0.00 0.01 0.989
10% 0.93 1.82 > 0.99 0.00 0.02 > 0.99
15% 0.93 2.27 > 0.99 0.00 0.04 > 0.99
20% 0.93 2.71 > 0.99 0.00 0.05 > 0.99

aAssumes 2-sided with Bonferroni-adjusted type 1 error of 0.05/2 and n = 2000 eligible per cohort.

bProportion of DPP participants who enroll in the program (i.e., 5% = 100 DPP enrollees per cohort).

cAssumes weight loss of 9.85 lbs for DPP enrollees.

dAssumes HbA1c decrease of 0.24% for DPP enrollees.

DPP = Diabetes Prevention Program; HbA1c: hemoglobin A1c or glycated hemoglobin; UC = usual care.

Analysis of Clinical Outcomes.

We plan to model 12- and 24-month weight trajectories using a linear mixed-effects model. Based on visual inspection of a scatterplot of weights, we will consider using a piecewise linear spline function with a knot at 6 or 7 months given the “checkmark” phenomenon often seen in behavioral weight loss studies.38,39 Random effects for the intercept and slope(s) will be included in the model to allow for person-specific trends in weight trajectories, and we will determine the correlation structure for the random effects based on best model fit in terms of Akaike information criterion and Bayesian information criterion values. We will evaluate as covariates the following a priori-chosen variables: age (modeled continuously), race/ethnicity (non-Hispanic White and non-Whites, which includes Blacks, Hispanics, and other), gender, minutes of exercise per week (0 minutes, 10–140 minutes, ≥ 150 minutes), Charlson comorbidity index score (0, 1, 2, 3, or more),40,41 baseline tobacco use (current, former, never), census tract-level proportion with college education or higher, census tract-level median household income, baseline weight and HbA1c level, and metformin use (treated as time-varying as always yes after the first dispense of the medication).

To control further for potential confounding, models will be adjusted for propensity score. We will estimate propensity scores for enrolling in the DPP (digital and in-person) by fitting a logistic regression model, with enrollment status as the outcome and all covariates listed earlier as predictors. To test the robustness of our primary findings, we will conduct propensity score matching as a sensitivity analysis. In addition to the covariates and propensity score, we will also include a 2-way interaction term between time and DPP enrollment status (digital and in-person will be modeled separately) to compare the time trend of weight trajectories between enrollees and nonenrollees. Furthermore, we will include 2-way interactions for time by gender and time by weight at enrollment. Model assumptions will be checked by examining residuals and predicted values cross-sectionally and over time. We will estimate marginal means (95% confidence intervals), model-estimated weights averaged over all covariates in the model (ie, all covariates are held constant at their means) at months 1 through 12. The primary contrasts of interest will be the differences in estimated weight change from time of recruitment between enrollees and nonenrollees at 12 months. Similar analyses will be conducted to compare digital DPP enrollees and nonenrollees on 24-month weight trajectories. HbA1c trajectories for 12 months (both cohorts) and 24 months (digital DPP) will be modeled similarly to weight using a linear mixed-effects model. We plan to include the same covariates as the weight model, but exclude the interactions for time by gender and time by weight at enrollment.

Analysis of Cost Outcomes.

We will conduct an economic evaluation over the 12-month follow-up period for both the digital and in-person DPP cohorts as well as over the 24-month period for the digital DPP cohort from the perspective of the health plan, following best practices,42 and guided by previous economic analyses of DPP interventions.43–48 We will compare the health-care cost functions between DPP enrollees and nonenrollees for each cohort, adjusting for baseline differences using a propensity score similar to that used in the clinical evaluation, but including adjustments for baseline costs. Cost data will include medical care and the cost of intervention delivery. Medical care use will be enumerated using the EHR and will include pharmacy, primary and specialty care office visits, inpatient stays, and laboratory tests. Health-care events (eg, visits, medication) will be costed using existing algorithms.33 We will report differences in medical care costs between DPP enrollees and nonenrollees. We will consider methods appropriate for cost data (eg, right skewness and potentially censored follow-up time), including 2-part models,49 bootstrapping,50,51 and proportional means regression.52 Last, we will assess the cost-effectiveness of the DPP using net benefit regression methods,53,54 and we will estimate the probability of the DPP being cost-effective at various levels of willingness-to-pay per unit of improvement in clinical outcomes.42

Analysis of Qualitative Data.

A coding dictionary based on review of transcript content and interview questions with patients invited to each DPP format as well as health system stakeholders will be used by 2 trained coders to establish interrater reliability. Any differences in coding will be resolved through discussion, and coders will meet regularly to discuss and refine coding processes. A qualitative database will be compiled, coded, and analyzed using a qualitative software program (NVivo 12). After all transcripts are coded within the software program, we will apply text retrieval and grouping functions on specific codes and combinations of codes for a particular topic, and will summarize the issues, agreements, and disagreements in the content for each item (eg, barriers to participation compared across both DPP modalities). This type of theme-focused analysis provides qualitative data that can be integrated with our quantitative findings and improve our breadth of understanding of the DPP program, its implementation options, and effectiveness.


This study will address 2 of the most pervasive and potentially costly health conditions in the US: obesity and prediabetes. The mixed-methods natural experiment design we will use to evaluate KPNW’s implementation of a digital and in-person DPP builds on existing evidence related to the effectiveness of these two DPP delivery modes and will contribute new knowledge related to best practices for implementing and sustaining a DPP within large health systems over the long term. Specifically, many translational studies have documented the success of an in-person DPP outside the research setting; however, evidence related to digital DPP effectiveness remains limited as a result of varied design limitations in studies conducted to date17,22–27 and low methodological quality.28 The digital DPP shows promise in facilitating access to diabetes prevention services for individuals with physical or geographic barriers,55 and evidence supporting its clinical benefit could facilitate expanded coverage of this delivery mode for Medicare and Medicaid beneficiaries. We propose a rigorous study design to evaluate the digital DPP, using a longitudinal, usual care comparison cohort and objectively assessed measures to determine longer term (12- and 24-month) clinical and cost outcomes while applying propensity score adjustment.

In addition, the sustainability of both in-person and digital programs is less certain, particularly in the context of a large health system hoping to offer a DPP to potentially tens of thousands of patients.21,56 Further vetting of the Centers for Medicare & Medicaid Services  payment model for DPP by health-care systems is needed.12,55,57,58 Our cost-effectiveness analysis will determine the impact of a digital and in-person DPP on health-care costs and the return on investment for health-care systems, which could have significant implications for the expansion and sustainability of a DPP.

There are some limitations to this study design. Specifically, enrollment slots for both the digital and in-person DPP are capped at 500 and 300 slots, respectively, because of limited resources, which makes it more challenging to assess program reach. The limited number of slots may have implications for generalizability in terms of enrollee characteristics and study outcomes. In addition, the health-care system restricting recruitment to those with a web-based patient portal account may also limit the generalizability of our findings. Although 80% of KPNW members use our patient portal, based on prior studies, patients who underuse the patient portal or other EHR-based tools tend to be from underserved populations with a high burden of diabetes.59–62 Furthermore, each DPP modality will be offered at separate times and to cohorts of individuals with slightly different eligibility criteria (ie, ranges for age and HbA1c levels at baseline for inclusion). This prevents the opportunity to do a head-to-head comparison of digital vs in-person on enrollment based on patient preference.


The mixed-methods natural experiment design we will use to evaluate KPNW’s implementation of a digital and in-person DPP will build on existing evidence related to DPP effectiveness across the 2 delivery modes on change in weight and HbA1c level at 12 months, and for digital DPP at 24 months. In addition, the cost-effectiveness analysis will determine the impact of a digital and in-person DPP on return on investment for health-care systems and sustainability of the program. Findings from our evaluation will therefore inform best practices for implementing and sustaining a DPP within large health-care systems.

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Financial Support

This study is supported by the National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases (project no. 1R01DK115237).

Author Affiliations

1Kaiser Permanente Center for Health Research in Portland, OR

Corresponding Author

Stephanie L Fitzpatrick, PhD (Stephanie.L.Fitzpatrick@kpchr.org)

Author Contributions

Stephanie L Fitzpatrick, PhD, Chris L Catlin, BS, Alison Firemark, MA, David H Smith, PhD, RPh, Ning Smith, PhD, Victor J Stevens, PhD, William M Vollmer, PhD, and Stephen P Fortmann, MD, have contributed to the development of the study. Stephanie L Fitzpatrick, PhD, and Stephen P Fortmann, MD, conceived of the study and overall study design in consultation with Victor J Stevens, PhD, and William M Vollmer, PhD. Ning Smith, PhD, Andreea M Rawlings, PhD, MS, and Denis B Nyongesa, MS, developed and contributed to writing the statistical analysis plan. Alison Firemark, MA, and Inga Gruß, PhD, developed and contributed to writing the qualitative analysis plan; and David H Smith, PhD, and Maureen O’Keeffe-Rosetti, MS, developed and contributed to writing the cost analysis plan. Stephanie L Fitzpatrick, PhD, Meghan Mayhew, MPH, and Andreea M Rawlings, PhD, MS, developed the manuscript draft. All authors contributed to the editing of the manuscript and have read and approved the final version.


The views presented here are solely the responsibility of the authors.


1. Gerstein HC, Santaguida P, Raina P, et al. Annual incidence and relative risk of diabetes in people with various categories of dysglycemia: A systematic overview and meta-analysis of prospective studies. Diabetes Res Clin Pract 2007 Dec;78(3):305–12. DOI: https://doi.org/10.1016/j.diabres.2007.05.004

2. Centers for Disease Control and Prevention. National diabetes statistics report, 2017. Atlanta, GA: Centers for Disease Control and Prevention, U.S. Dept of Health and Human Services; 2017. https://dev.diabetes.org/sites/default/ files/2019-06/cdc-statistics-report-2017.pdf.

3. American Diabetes Association. Economic costs of diabetes in the U.S. in 2017. Diabetes Care 2018;41(5):917–28. DOI: https://doi.org/10.2337/dci18-0007, PubMed PMID: 29567642, PubMed Central PMCID: PMCPMC5911784.

4. Ackermann RT, Kenrik Duru O, Albu JB, et al. Evaluating diabetes health policies using natural experiments: the natural experiments for translation in diabetes study. Am J Prev Med 2015;48(6):747–54. DOI: https://doi.org/10.1016/j.amepre.2014.12.010, PubMed PMID: 25998925, PubMed Central PMCID: PMCPMC5210173.

5. Ali MK, Wharam F, Kenrik Duru O, et al. Advancing health policy and program research in diabetes: Findings from the Natural Experiments for Translation in Diabetes (NEXT-D) Network. Curr Diab Rep 2018;18(12):146. DOI: https://doi.org/10.1007/s11892-018-1112-3, PubMed PMID: 30456479, PubMed Central PMCID: PMCPMC6640642.

6. Knowler WC, Barrett-Connor E, Fowler SE, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 2002 Feb;346(6):393–403. DOI: https://doi.org/10.1056/NEJMoa012512

7. Whittemore R. A systematic review of the translational research on the Diabetes Prevention Program. Transl Behav Med 2011;1(3):480–91. DOI: https://doi.org/10.1007/s13142-011-0062-y, PubMed PMID: 24073067, PubMed Central PMCID: PMCPMC3717627.>

8. Ali MK, Echouffo-Tcheugui J, Williamson DF. How effective were lifestyle interventions in real-world settings that were modeled on the Diabetes Prevention Program? Health Aff (Millwood) 2012 Jan;31(1):67–75. DOI: https://doi.org/10.1377/hlthaff.2011.1009

9. Mudaliar U, Zabetian A, Goodman M, et al. Cardiometabolic risk factor changes observed in diabetes prevention programs in US settings: A systematic review and meta-analysis. PLoS Med 2016;13(7):e1002095. DOI: https://doi.org/10.1371/journal.pmed.1002095, PubMed PMID: 27459705, PubMed Central PMCID: PMCPMC4961455.

10. Dunkley AJ, Bodicoat DH, Greaves CJ, et al. Diabetes prevention in the real world: Effectiveness of pragmatic lifestyle interventions for the prevention of type 2 diabetes and of the impact of adherence to guideline recommendations: A systematic review and meta-analysis. Diabetes Care 2014 Apr;37(4):922–33. DOI: https://doi.org/10.2337/dc13-2195

11. Prevention CfDCa. CDC Diabetes Prevention Recognition Program standards and operating procedures. Atlanta, GA: Centers for Disease Control and Prevention; 2018.

12. Centers for Medicare & Medicaid Services. Proposal rules, pp 46413–8. July 15, 2016. Federal Register, vol. 81, no. 136. www.federalregister.gov/documents/2016/07/15/2016-16097/medicare-program-revisions-to-payment-policies-under-the-physician-fee-schedule-and-other-revisions.

13. Hinnant L, Razi S, Lewis R, et al. Evaluation of the Health Care Innovation Awards: Community resource planning, prevention, and monitoring annual report 2015: Awardee-level findings: YMCA of the USA. 2015. Accessed September 22, 2021. https://innovation.cms.gov/Files/reports/hcia-ymcadpp-evalrpt.pdf.

14. Brunisholz KD, Joy EA, Hashibe M, et al. Stepping back to move forward: Evaluating the effectiveness of a diabetes prevention program within a large integrated healthcare delivery system. J Healthc Qual 2017 Sep/Oct;39(5):278–93. DOI: https://doi.org/10.1097/JHQ.0000000000000103

15. Brunisholz KD, Kim J, Savitz LA, et al. A formative evaluation of a diabetes prevention program using the RE-AIM framework in a learning health care system, Utah, 2013–2015. Prev Chronic Dis 2017;14:E58. DOI: https://doi.org/10.5888/pcd14.160556, PubMed PMID: 28727546, PubMed Central PMCID: PMCPMC5524524.

16. Moin T, Damschroder LJ, AuYoung M, et al. Diabetes prevention program translation in the Veterans Health Administration. Am J Prev Med 2017;53(1):70–7. DOI: https://doi.org/10.1016/j.amepre.2016.11.009, PubMed PMID: 28094135, PubMed Central PMCID: PMCPMC6699500.

17. Moin T, Damschroder LJ, AuYoung M, et al. Results from a trial of an online diabetes prevention program intervention. Am J Prev Med 2018;55(5):583–91. DOI: https://doi.org/10.1016/j.amepre.2018.06.028, PubMed PMID: 30262149, PubMed Central PMCID: PMCPMC6699502.

18. Chambers EC, Rehm CD, Correra J, et al. Factors in placement and enrollment of primary care patients in YMCA’s Diabetes Prevention Program, Bronx, New York, 2010–2015. Prev Chronic Dis 2017;14:E28. DOI: https://doi.org/10.5888/pcd14.160486, PubMed PMID: 28358669, PubMed Central PMCID: PMCPMC5386615.

19. Venkataramani M, Pollack CE, Yeh HC, Maruthur NM. Prevalence and correlates of diabetes prevention program referral and participation. Am J Prev Med 2019 Mar;56(3):452–7. DOI: https://doi.org/10.1016/j.amepre.2018.10.005

20. Ali MK, McKeever Bullard K, Imperatore G, et al. Reach and use of diabetes prevention services in the United States, 2016–2017. JAMA Netw Open 2019;2(5):e193160. DOI: https://doi.org/10.1001/jamanetworkopen.2019.3160, PubMed PMID: 31074808, PubMed Central PMCID: PMCPMC6512285.

21. Ackermann RT, O’Brien MJ. Evidence and challenges for translation and population impact of the Diabetes Prevention Program. Curr Diab Rep 2020 Feb;20(3):9. DOI: https://doi.org/10.1007/s11892-020-1293-4

22. Sepah SC, Jiang L, Ellis RJ, McDermott K, Peters AL. Engagement and outcomes in a digital diabetes prevention program: 3-Year update. BMJ Open Diabetes Res Care 2017;5(1):e000422. DOI: https://doi.org/10.1136/bmjdrc-2017-000422, PubMed PMID: 28948027, PubMed Central PMCID: PMCPMC5595194.

23. Sepah SC, Jiang L, Peters AL. Long-term outcomes of a web-based diabetes prevention program: 2-Year results of a single-arm longitudinal study. J Med Internet Res 2015;17(4):e92. DOI: https://doi.org/10.2196/jmir.4052, PubMed PMID: 25863515, PubMed Central PMCID: PMCPMC4409647.

24. Kim SE, Castro Sweet CM, Cho E, Tsai J, Cousineau MR. Evaluation of a digital diabetes prevention program adapted for low-income patients, 2016–2018. Prev Chronic Dis 2019;16:E155. DOI: https://doi.org/10.5888/pcd16.190156, PubMed PMID: 31775010, PubMed Central PMCID: PMCPMC6896833.

25. Castro Sweet CM, Chiguluri V, Gumpina R, et al. Outcomes of a digital health program with human coaching for diabetes risk reduction in a Medicare population. J Aging Health 2018;30(5):692–710. DOI: https://doi.org/10.1177/0898264316688791, PubMed PMID: 28553807, PubMed Central PMCID: PMCPMC5944079.

26. Chen F, Su W, Becker SH, et al. Clinical and economic impact of a digital, remotely-delivered intensive behavioral counseling program on Medicare beneficiaries at risk for diabetes and cardiovascular disease. PLoS One 2016;11(10):e0163627. DOI: https://doi.org/10.1371/journal.pone.0163627, PubMed PMID: 27706216, PubMed Central PMCID: PMCPMC5051965.

27. Lee PG, Damschroder LJ, Holleman R, Moin T, Richardson CR. Older adults and diabetes prevention programs in the Veterans Health Administration. Diabetes Care 2018;41(12):2644–7. DOI: https://doi.org/10.2337/dc18-1141, PubMed PMID: 30377187, PubMed Central PMCID: PMCPMC6245214.

28. Joiner KL, Nam S, Whittemore R. Lifestyle interventions based on the diabetes prevention program delivered via eHealth: A systematic review and meta-analysis. Prev Med 2017;100:194–207. DOI: https://doi.org/10.1016/j.ypmed.2017.04.033, PubMed PMID: 28456513, PubMed Central PMCID: PMCPMC5699208.

29. Glauber H, Vollmer WM, Nichols GA. A simple model for predicting two-year risk of diabetes development in individuals with prediabetes. Perm J 2018;22:17–050. DOI: https://doi.org/10.7812/TPP/17-050, PubMed PMID: 29309270, PubMed Central PMCID: PMCPMC5760055.

30. Sepah SC, Jiang L, Peters AL. Translating the Diabetes Prevention Program into an online social network: Validation against CDC standards. Diabetes Educ 2014 Jul;40(4):435–43. DOI: https://doi.org/10.1177/0145721714531339

31. Centers for Disease Control and Prevention. National Diabetes Statistics Report, 2017 [updated 10/03/2016; cited 11/09/2016]. Available from: http://www.cdc.gov/diabetes/prevention/lifestyle-program/index.html.

32. Hornbrook M, Goodman MJ. Adjusting health benefit contributions to reflect risks. In: Risk-based contributions to private health insurance. Hornbrook M, editor. Greenwich, CT: JAI Press: 1991; p. 41.

33. O’Keeffe-Rosetti MC, Hornbrook MC, Fishman PA, et al. A standardized relative resource cost model for medical care: Application to cancer control programs. J Natl Cancer Inst Monogr 2013;2013(46):106–16. DOI: https://doi.org/10.1093/jncimonographs/lgt002, PubMed PMID: 23962514, PubMed Central PMCID: PMCPMC3748000.

34. Patton MQ. Qualitative research & evaluation methods. 3rd ed. Thousand Oaks, CA: Sage Publications; 2002.

35. Bernard R, Ryan GW. Analyzing qualitative data: Systematic approaches. Thousand Oaks, CA: Sage Publications; 2009.

36. Silverman D. Doing qualitative research: A practical handbook. Thousand Oaks, CA: Sage Publications; 2010.

37. Ackermann RT, Finch EA, Brizendine E, Zhou H, Marrero DG. Translating the Diabetes Prevention Program into the community: The DEPLOY pilot study. Am J Prev Med 2008;35(4):357–63. DOI: https://doi.org/10.1016/j.amepre.2008.06.035, PubMed PMID: 18779029, PubMed Central PMCID: PMCPmc2610485.

38. Knowler WC, Barrett-Connor E, Fowler SE, et al. Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin. N Engl J Med 2002 Feb;346(6):393–403. DOI: https://doi.org/10.1056/NEJMoa012512, PubMed

39. Look ARG. Eight-year weight losses with an intensive lifestyle intervention: The Look AHEAD Study. Obesity (Silver Spring, Md) 2014;22(1):5–13. DOI: https://doi.org/10.1002/oby.20662

40. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992 Jun;45(6):613–9. DOI: https://doi.org/10.1016/0895-4356(92)90133-8

41. Charlson M, Szatrowski TP, Peterson J, Gold J. Validation of a combined comorbidity index. J Clin Epidemiol 1994 Nov;47(11):1245–51. DOI: https://doi.org/10.1016/0895-4356(94)90129-5

42. Sanders GD, Neumann PJ, Basu A, et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: Second panel on cost-effectiveness in health and medicine. JAMA 2016 Sep;316(10):1093–103. DOI: https://doi.org/10.1001/jama.2016.12195

43. Diabetes Prevention Program Research Group. Within-trial cost-effectiveness of lifestyle intervention or metformin for the primary prevention of type 2 diabetes. Diabetes Care 2003;26(9):2518–23. DOI: XXXXXXX, PubMed PMID: 12941712, PubMed Central PMCID: PMCPmc1360736.

44. Diabetes Prevention Program Research Group. The 10-year cost-effectiveness of lifestyle intervention or metformin for diabetes prevention: An intent-to-treat analysis of the DPP/DPPOS. Diabetes Care 2012;35(4):723–30. DOI: https://doi.org/10.2337/dc11-1468, PubMed PMID: 22442395, PubMed Central PMCID: PMCPmc3308273.

45. Herman WH. The cost-effectiveness of diabetes prevention: Results from the Diabetes Prevention Program and the Diabetes Prevention Program Outcomes Study. Clin Diabetes Endocrinol 2015 Sep;1(1):9. DOI: https://doi.org/10.1186/s40842-015-0009-1

46. Hoerger TJ, Hicks KA, Sorensen SW, et al. Cost-effectiveness of screening for pre-diabetes among overweight and obese U.S. adults. Diabetes Care 2007 Nov;30(11):2874–9. DOI: https://doi.org/10.2337/dc07-0885

47. Krukowski RA, Pope RA, Love S, et al. Examination of costs for a lay health educator-delivered translation of the Diabetes Prevention Program in senior centers. Prev Med 2013 Oct;57(4):400–2. DOI: https://doi.org/10.1016/j.ypmed.2013.06.027

48. Zhou X, Siegel KR, Ng BP, et al. Cost-effectiveness of diabetes prevention interventions targeting high-risk individuals and whole populations: A systematic review. Diabetes Care 2020 Jul;43(7):1593–616. DOI: https://doi.org/10.2337/dci20-0018

49. Diehr P, Yanez D, Ash A, Hornbrook M, Lin DY. Methods for analyzing health care utilization and costs. Annu Rev Public Health 1999 20(1):125–44. DOI: https://doi.org/10.1146/annurev.publhealth.20.1.125

50. Briggs A, Gray A. The distribution of health care costs and their statistical analysis for economic evaluation. J Health Serv Res Policy 1998 Oct;3(4):233–45. DOI: https://doi.org/10.1177/135581969800300410

51. Briggs A, Nixon R, Dixon S, Thompson S. Parametric modelling of cost data: some simulation evidence. Health Econ 2005 Apr;14(4):421–8. DOI: https://doi.org/10.1002/hec.941

52. Başer O, Gardiner JC, Bradley CJ, Yüce H, Given C. Longitudinal analysis of censored medical cost data. Health Econ 2006 May;15(5):513–25. DOI: https://doi.org/10.1002/hec.1087

53. Hoch JS, Briggs AH, Willan AR. Something old, something new, something borrowed, something blue: A framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002 Jul;11(5):415–30. DOI: https://doi.org/10.1002/hec.678

54. Hoch JS. Improving efficiency and value in palliative care with net benefit regression: An introduction to a simple method for cost-effectiveness analysis with person-level data. J Pain Symptom Manage 2009 Jul;38(1):54–61. DOI: https://doi.org/10.1016/j.jpainsymman.2009.04.010

55. Ritchie ND, Sauder KA, Gritz RM. Medicare Diabetes Prevention Program: Where are the suppliers? Am J Manage Care 2020 Jun;26(6):e198–201. DOI: https://doi.org/10.37765/ajmc.2020.43496

56. Tice JA CR, Shore KK, Seidner M, Ollendorf DA, Weissberg J, Pearson SD. Diabetes prevention programs: Effectiveness and value. 2016.

57. Parsons AS, Raman V, Starr B, Zezza M, Rehm CD. Medicare underpayment for Diabetes Prevention Program: Implications for DPP suppliers. Am J Manage Care 2018 Oct;24(10):475–8. DOI: XXXXXXXXXX

58. Ritchie ND, Gritz RM. New Medicare diabetes prevention coverage may limit beneficiary access and widen health disparities. Med Care 2018 Nov;56(11):908–11. DOI: https://doi.org/10.1097/MLR.0000000000000981

59. Jain V, Al Rifai M, Lee MT, et al. Racial and geographic disparities in Internet use in the U.S. among patients with hypertension or diabetes: Implications for telehealth in the era of COVID-19. Diabetes Care 2021;44(1):e15–7. DOI: https://doi.org/10.2337/dc20-2016, PubMed PMID: 33139408, PubMed Central PMCID: PMCPMC7876593.

60. Nouri SK, EH, Lyles CR, Karliner L. Addressing equity in telemedicine for chronic disease management during the COVID-19 pandemic. N Engl J Med Catal 2020 ▪▪▪. DOI: 10.1056.

61. Graetz I, Huang J, Brand RJ, Hsu J, Yamin CK, Reed ME. Bridging the digital divide: mobile access to personal health records among patients with diabetes. Am J Manage Care 2018;24(1):43–8. DOI: XXXXX, PubMed PMID: 29350505, PubMed Central PMCID: PMCPMC6382280.

62. Graetz I, Huang J, Muelly ER, Fireman B, Hsu J, Reed ME. Association of mobile patient portal access with diabetes medication adherence and glycemic levels among adults with diabetes. JAMA Netw Open 2020;3(2):e1921429. DOI: https://doi.org/10.1001/jamanetworkopen.2019.21429, PubMed PMID: 32074289, PubMed Central PMCID: PMCPMC7646995.

Keywords: diabetes, health services research, implementation, obesity, prevention


Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.


2 million page views of TPJ articles in PubMed from a broad international readership.


Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.




ISSN 1552-5775 Copyright © 2021 thepermanentejournal.org

All Rights Reserved