Variation in Positivity Rates of Computed Tomography Pulmonary Angiograms for the Evaluation of Acute Pulmonary Embolism Among Emergency Department Physicians



 
The Permanente Journal

Kori Higashiya, MD1; James Ford, MD2; Hyo-Chun Yoon, MD, PhD3

Perm J 2022;26:21.019 • E-pub: 04/05/2022 • https://doi.org/10.7812/TPP/21.019

Volume 26, Issue 1

Corresponding Author
Kori Higashiya, MD
Kori.higashiya@wsu.edu

Author Affiliations
1John A. Burns School of Medicine, University of Hawaii, Honolulu, HI, USA

2Hawaii Permanente Medical Group, Emergency Medicine, Honolulu, HI, USA

3Hawaii Permanente Medical Group, Diagnostic Imaging, Honolulu, HI, USA

Author Contributions:
Kori Higashiya, MD, participated in the acquisition of data, analysis and interpretation of the data, drafting, review, and submission of the final manuscript. Hyo-Chun Yoon, MD, PhD, participated in the study concept and design, acquired and analyzed the data, and assisted with the drafting and review of the final manuscript. James Ford, MD, participated in the study concept and design, acquired the data, and assisted with the review of the final manuscript.

Disclosures
Conflicts of Interest: None declared
Funding: None declared

Copyright Information
© 2022 The Permanente Federation. All rights reserved.

Abstract

Computed tomography pulmonary angiography (CTPA) is an imaging study for which there is substantial evidence for its overuse in the evaluation of acute pulmonary embolism (PE). Prior literature has reported low positive PE rates, but the variability in positive rates among the ordering physicians has not been as well studied. The purpose of this study was to evaluate variation in ordering and positive rates among physicians in an emergency department (ED) within an integrated health care system.

This study was based in a single ED that is part of a geographically isolated integrated health care system. We reviewed the patient records for all patients who underwent a CTPA for the evaluation for acute PE in the ED between January 1, 2018, and December 31, 2019. For each CTPA examination, we recorded the ordering ED physician, serum d-dimer value (mcg/mL), if any, and the results of the CTPA.

Review of CTPAs over the 2-year period revealed 1380 CTPAs ordered by 23 ED physicians with a range of 25–141 studies per physician (mean of 60 + 31 CTPAs). The overall positive rate for PE was 6.9%. Individual ED physician positivity rates showed wide variability ranging from 0% to 18.4% (mean positive rate 7.6 + 4.4%). The results of this study confirm the need for greater adherence to existing guidelines using clinical decision rules and d-dimer testing when appropriate among all ED physicians but especially those who order a greater number of studies and have low rates for positive PE.

Introduction

Computed tomography pulmonary angiography (CTPA) is an imaging study in evaluating acute pulmonary embolism (PE). Especially in the US, however, there is considerable evidence for its overuse. In a recently published 4-Level Pulmonary Embolism Clinical Probability Score (4PEPS) study,1 the authors used the existing data from 3 prospective studies on PE to evaluate the use of a 4-level clinical pretest probability score to potentially decrease imaging. The positive PE rates in the 2 European cohorts were 28% and 18%,2,3 while the positive rate in the US cohort was 5.9%.4 Three retrospective studies published in a variety of settings in the US reported a CTPA positive rate of acute PE of 1.8% in 716 CTPA studies performed in an urban community hospital,5 7.4% in 2031 CTPA studies performed in a large hospital system,6 and 5.4% in 295 CTPA studies performed at a university hospital.7 While the overall positive PE rate is consistently low in all studies from the US, the variability in positive rates among the ordering physicians has not been as well studied. In our hospital, the overwhelming majority of CTPAs is ordered by physicians in the emergency department (ED).

The purpose of this study was to evaluate variation in ordering of CTPAs and the positive PE rates among physicians in a single ED within an integrated health care system, as well as to see how the application of a clinical decision rule might affect these results.

Methods

STUDY DESIGN

The Institutional Review Board approved this study.

STUDY SETTING AND POPULATION

This study was based in a single ED within a geographically isolated integrated health care system that serves approximately 250,000 members. The ED sees 48,000 patients per year, and all patient encounters are included in a comprehensive electronic medical record. We reviewed the patient records for all patients who underwent a CTPA for the evaluation for acute PE in the emergency department between January 1, 2018, and December 31, 2019. Based on internal data from our patient population,810 our medical group recommended the use of a higher d-dimer threshold (1 mcg/mL FEU) for patients in whom a CTPA study is requested.

Study Protocol: All CTPA studies were performed on one of two 64-slice CT scanners (GE Medical Systems, Milwaukee, WI) with image reconstructions in the axial, coronal, and sagittal planes. All d-dimer tests were performed using the STA-Liatest (Diagnostica Stago, Parsippany, NJ). For each CTPA examination, we recorded the ordering ED physician, serum d-dimer value (mcg/mL), if any, and the results of the CTPA. Results were recorded as either positive, negative, or indeterminate. Indeterminate was used if the report of the CTPA specifically mentioned that patient or technical factors precluded accurate assessment of segmental or more distal branches or if the report questioned if there was a possible embolism. For these indeterminate studies, we assessed what further imaging studies were performed, if any, during the same patient encounter or hospitalization. For positive PE studies, we recorded if the thrombus was limited to the segmental or subsegmental branches (peripheral PE) or in the lobar or more proximal pulmonary arteries (central PE). We excluded any CTPA ordered by an ED physician who only ordered studies from the ED during only one of the 2 years of this study.

To determine if greater adherence to accepted clinical criteria such as the Wells score and appropriate utilization of the serum d-dimer could be used to discriminate between ED physicians with lower PE positive rates from those with higher PE positive rates, we ranked the remaining ED physicians by their positive PE rates. We estimated that we would need to review approximately 120–170 charts to detect a 15% difference in adherence rates between groups of physicians with .05 significance and .80 power. Therefore, we combined the 4 physicians with the highest positive PE rates and who ordered a minimum of 30 PCTAs into one group (High group) and the 4 physicians with the lowest positive PE rates and who ordered a minimum of 30 PCTAs into a comparator group (Low group). We then reviewed the electronic medical record of each ED encounter leading to a CTPA for the patients who had been seen by a physician of the High group. One ED physician in the High group had ordered more than 50 CTPA studies during the study period, and we only reviewed 50 randomly selected patients for that physician in order to minimize any bias from having too many patients from any one ED physician. We also reviewed the electronic medical record of only 50 randomly selected ED encounters leading to a CTPA for each of the 4 ED physicians in the LOW group, except for one physician who ordered only 45 studies during the study period. All 45 of these encounters were included in the review.

For each patient encounter that was reviewed, we calculated the Wells score. We used the following algorithm to determine whether PE was as likely or more likely than any alternative diagnosis: If the patient’s chief complaint on ED physician record was shortness of breath or dyspnea, then we assumed PE was the most likely diagnosis unless 1) the patient had a history of congestive heart failure and chest x-ray was suggestive of edema, 2) the patient had signs and symptoms of a respiratory infection and an abnormal chest x-ray, or 3) the patient had a history of asthma or chronic obstructive pulmonary disease and clinical symptoms of an asthma or chronic obstructive pulmonary disease exacerbation. If the patient’s chief complaint was chest pain, then we assumed PE was the most likely diagnosis unless the patient had a history of coronary artery disease, prior myocardial infarction, or cardiomyopathy.

However, if the chest pain was further described as substernal, crushing, or radiating to the back or left arm, PE was not assumed to be the most likely diagnosis. For a chief complaint of unilateral leg pain or swelling, PE was assumed the most likely diagnosis unless there was a specific finding in the reported history to suggest a more likely alternative diagnosis. We assumed that the patient did not have signs or symptoms of deep venous thrombosis (DVT) unless the ED physician report specifically mentioned lower extremity edema concerning for DVT. Tachycardia was presumed present if the first recorded heart rate was equal to or greater than 100 beats per minute or the ED physician reported tachycardia. In addition to the standard definition of immobilization (3 consecutive days of bed rest or major surgery within 4 weeks), we also considered this positive if the ED physician noted a recent transpacific flight in their report because this was a frequently mentioned factor. No previous history of DVT or PE was presumed unless it was mentioned in the ED physician report or it was already coded into the patient’s problem list at the time of the ED encounter. Hemoptysis was also considered absent unless specifically mentioned in the ED physician report. Finally, active malignancy was considered absent unless specifically mentioned in the ED physician report or the patient had a diagnosis of malignancy for which they were receiving treatment or in palliative care at the time of the ED encounter based on the electronic medical record. The patients were then stratified into low (Wells criteria score 0–4), moderate (4.5–6), or high (> 6) probability, as was performed in the recent prospective Canadian trial.11

We then compared the distribution of the Wells scores between the High and Low groups. We compared the utilization of d-dimer tests between these 2 groups of physicians. Finally, we compared appropriate utilization between the groups, where appropriate utilization was defined as a CTPA that was ordered for a patient with a low or moderate Wells score and d-dimer ≥ 1.0 ug/mL or high Wells score irrespective of a d-dimer value, if any.

Results

TOTAL CTPA EXAMS AND POSITIVITY RATES

Between January 1, 2018, and December 31, 2019, 1507 CT pulmonary angiograms were performed. Of them, 127 CTPA studies ordered by 9 physicians were omitted because these physicians worked during only one of the 2 years of this study. The final data set included 1380 CT pulmonary angiograms ordered by 23 ED physicians. Out of all of them, 95 were positive for a PE, 1219 were negative, and 66 were indeterminate. Review of the medical records of the patients with indeterminate findings demonstrated that none were treated for acute PE. Nine of the 66 patients had a US evaluation for DVT that was negative. One patient with a questionable filling defect in a single segmental artery had a follow-up CTPA, which was negative. The remaining 56 patients had no further imaging evaluation for PE. Over this 2-year period, the overall positive rate for PE was 6.9%.

Table 1 lists the number of CTPAs ordered by ED physicians, positive PE rate, number of serum d-dimers ordered, those > 1 ug/mL, and the number of CTPAs performed without a corresponding serum d-dimer. The median number of CT pulmonary angiogram exams ordered by individual ED physicians was 49 (mean 60 ± 31 CTPAs with a range of 25–141, interquartile range 38). The median positivity rate among ED physicians was 7.7% (range of 0%–18.4%, interquartile range 7.0%).

PhysicianTotal CTPAsPositive rateD-dimer orderedD-dimer >1No D-dimer ordered
1 40 12.5% 32 22 8
2 63 3.12% 35 23 28
3 31 12.9% 14 11 17
4 100 5.0% 62 50 38
5 121 2.5% 75 44 46
6 37 10.8% 25 25 12
7 26 3.8% 3 0 23
8 102 6.9% 46 23 56
9 75 13.3% 40 37 35
10 62 11.3% 27 21 35
11 36 11.1% 22 20 14
12 47 4.3% 20 19 27
13 37 8.1% 16 14 21
14 70 4.3% 52 44 18
15 75 4.0% 18 14 57
16 45 0.0% 20 18 25
17 76 9.2% 54 46 22
18 49 18.4% 29 18 20
19 39 7.7% 16 10 23
20 49 6.1% 28 20 21
21 34 8.8% 18 14 16
22 141 3.6% 87 49 54
23 25 8.0% 24 23 1

Table 1: Individual ED physician data

The 4 physicians who ordered a minimum of 30 CTPA studies over the 2-year study period and had the highest positive CTPA rates (physicians 1, 3, 9, and 18 in Table 1, referred to as the High group) and had individual CTPA positive rates that ranged from 12.5% to 18.4%. These 4 physicians ordered a total of 195 studies with a range from 31 to 75 CTPAs. The 4 physicians who had the lowest positive CTPA rates (physicians 2, 5, 16, and 27 in Table 1, referred to as the Low group) had individual CTPA positive rates that ranged from 0.0% to 3.6%. These physicians ordered a total of 310 studies with a range from 45 to 141 studies.

Among all the reviewed ED encounters for both the High and Low physician groups, the Wells score was mentioned only twice, and no specific value was given in either record. Both mentions of the Wells score were by the same physician in the Low group. In one encounter, the ED physician noted “low risk by Wells.” We imputed a Wells score of 1.5 for this 74-year-old woman who did not have acute PE on her CTPA. In the other encounter, a “high risk by Wells” was mentioned. We imputed a Wells score of 6 for this 86-year-old man, who did not have acute PE on his CTPA.

The calculated Wells scores are presented in Table 2 for the representative sample of randomly selected patients, which was limited to 50 patients for each physician to minimize the bias associated with the different number of studies ordered by each physician. There was no significant difference in the distribution of the Wells scores between High and Low groups of physicians, χ2(2, 370, 4.84, p = 0.09). There was no significant difference between the groups for the distribution of patients with or without a d-dimer test, χ2 (1, 370, 0.49, p = 0.48).

Measurement MethodPhysicians with high positive PE ratePhysicians with low positive PE rate
Wells score    
Low (0–4) 116 (68.2%) 143 (73.3%)
Moderate (4.5–6) 49 (28.8%) 41 (21.0%)
High (> 6) 5 (2.9%) 11 (5.6%)
D-dimer    
Not drawn 67 (39.4%) 83 (42.5%)
< 1 mcg/mL 26 (15.2%) 44 (22.5%)
≥ 1 mcg/mL 77 (45.2%) 68 (34.8%)

Table 2: Wells score and d-dimer for Low vs High groups

Among the patients who underwent CTPA ordered by the High group physicians with the highest positive PE rates, there were 103 patients who had a d-dimer level drawn. Twenty-six of these patients had a serum d-dimer < 1.0 mcg/mL and did not have an estimated Wells score > 6. These 26 patients would be considered to have inappropriately undergone CTPA. The remaining 77 patients would be considered to have appropriately undergone CT. Among the 4 ED physicians with the lowest positive PE rates, there were 112 patients who had a d-dimer level drawn, and 44 of them had a serum d-dimer < 1.0 mcg/mL and did not have an estimated Wells score > 6. These patients would be considered to have inappropriately undergone CTPA, which shows that a lower percentage of patients in the High group (25.2%) inappropriately underwent CTPA compared to the Low group (39.2%), χ2(1, 215, 4.82, p = .03). There were no patients in the High group who had an imputed Wells score > 6 who had a d-dimer value less than 1. There was only one patient in the Low group with an imputed Wells score of 7.5, who would be considered to have appropriately undergone CTPA despite a d-dimer value 0.75 mcg/mL.

There were 3 patients who underwent CTPA ordered by the HIGH group who had estimated Wells score ≤ 6 but had d-dimer values greater than their age-adjusted threshold, but less than the 1 mcg/mL threshold recommended by our medical group. All 3 CTPA studies were negative for PE. There were 2 patients who underwent CTPA ordered by the Low group who had estimated Wells score ≤ 6 but had d-dimer values greater than their age-adjusted threshold but less than the 1 mcg/mL threshold recommended by our medical group. Both CTPA studies were negative for PE. If we include these patients as having appropriately undergone CTPA, there is still a significant difference in the percentage of appropriate patients who underwent CTPA between these 2 groups of physicians (Low 62.5% vs. High 77.6%), χ2(1, 215, 5.85, p = .02).

Discussion

This study confirms that there is substantial variability between ED physicians in the number of CTPAs ordered, as well as the positivity rates of PE in their patients. The difference between the ED physician who ordered the greatest number of CTPA studies and the ED physician who ordered the least number of studies was 5.6 times. The median positivity rate of 7.7%, with a range of 0% to 18.4% in these 23 ED physicians, is similar to that reported by Salehi et al, whose study examined 77 ED physicians who had a median positivity rate of 9.1% with a range of 0% to 33%.12 In that study, there was up to 22-fold difference in the number of CTPAs ordered per ED encounter between physicians.3

The use of a clinical decision rule, such as Wells criteria, was rarely recorded in the ED encounter within the electronic medical record. Therefore, in order to compare the patients seen by the 2 groups of ED physicians, we retrospectively scored the patients based on the ED encounter reports. However, the distribution of the imputed Wells scores was not significantly different between the High and Low physician groups. There have been mixed results reported in the literature on the mandated use of a clinical decision rule in ordering CTPAs. In a study by Geeting et al, inputting a modified Wells score as part of the ordering process did not significantly increase the positivity rate from 6.9% to 7.5%.13 These authors found that while the percentage of appropriately ordered CTPAs increased after ED physicians were required to input the Wells score, the overall utilization of CTPA and positive PE rates did not significantly change. They attribute the observed increased appropriate utilization to the increased use of the subjective component of the Wells score by the ordering physicians. In another study at a quaternary academic center, the implementation of the Wells criteria into the ordering process increased the overall positive PE rate from 9.2% to 12.6%, but there was large variability among the 25 physicians included in the study with a preintervention range of 2.6% to 20.5% and a postintervention range of 0% to 38%. There was only an increase in the positive rates of PE among the patients of 3 physicians.14

There was no significant difference between the High and Low physician groups in the rates at which they ordered d-dimer levels on their patients with possible PE. However, there was a significantly greater proportion of patients in the LOW physician group who underwent CTPA despite a d-dimer < 1.0 mcg/mL and an estimated Wells score ≤ 6 compared to those patients seen by physicians in the High group. The reason for ordering a CTPA despite a d-dimer < 1 mcg/mL was usually not mentioned in the reviewed reports. There were 2 physicians in the Low group who occasionally noted an “elevated dimer” in their ED reports on patients with d-dimer values <1.0 mcg/mL. However, this was not seen in all their reports. For the other 2 physicians in the Low group and all 4 physicians in the High group, there were no specific mentions of why a CTPA was ordered despite a d-dimer < 1 mcg/mL. Our study would suggest that while it is important to stress that all ED physicians should use clinical decision rules supplemented by d-dimer values when appropriate, those with the lowest positive PE rates may benefit the most from education on the importance of both in determining appropriate utilization of CTPA.

Our acute PE positivity rate of 6.9% is consistent with other reported US studies.57 Positivity rates in the US have decreased since the 1990s and appear to be an ongoing issue for most centers in the US. As noted by Prologo et al, positivity rates for acute PE in the US decreased from 27.1% in 1997–1998 to 5.7% in 2002–2003.15 A key concern resulting from the low prevalence of acute PE in patients undergoing CTPA is that this can result in more false positives than true positives. Cronin and Kelly demonstrated that if the prevalence of acute pulmonary embolism in a study population is between 5% and 10%, the PPV of CTPA would only be 48.9–66.7%.16 Since our PE positive rate is 6.9%, some 50% of the reported positive PE studies may in fact be falsely positive. In a study by Hutchinson et al, the authors reported that out of 174 CTPA examinations (performed on 64-slice CT scanners, similar to those used in the current study) that were initially reported as positive for PE, 45 (25.9%) cases were subsequently reported as negative when reviewed by expert chest radiologists. Approximately 84.4% (38 out of 45) of the discordant cases were caused by pulmonary embolisms in segmental or subsegmental arteries.17 Given that 57.9% of the positive PEs were reported in the segmental and subsegmental arteries in this study, there is concern that a number of these studies may have been falsely positive and resulted in the unnecessary treatment of patients.

Limitations

In this retrospective review, it is difficult to determine the exact reasoning each physician used for ordering a CTPA. Our integrated health care system does not mandate the use of a clinical decision rule such as the Wells or revised Geneva score prior to ordering a CTPA. Therefore, we used the best available information in the electronic medical record to estimate each patient’s Wells score. We did not capture the total number of encounters seen by each ED physician during the 2-year study period, the number of hours worked, or the different shifts during the day. Therefore, some of the difference in the total number of CTPAs ordered by individual physicians could be attributed to difference in the overall number of patient encounters. Finally, we did not have a method to reliably determine how often a d-dimer < 1 mcg/mL was sufficient for an ED physician not to order a CTPA study. While the data on the d-dimer values from all ED encounters can be extracted from the medical record, each ED physician note would have to be reviewed to determine if the patient was being evaluated for acute PE and if the d-dimer value was the reason that a CTPA was not ordered.

Conclusion

The results of this study confirm the need for utilization of existing clinical decision rules and adherence to appropriate d-dimer thresholds among all ED physicians, but especially for those who order a greater number of studies and have low rates for positive PE.

References
1.     Roy PM, Friou E, Germeau B, et al. Derivation and validation of a 4-level clinical pretest probability score for suspected pulmonary embolism to safely decrease imaging testing. JAMA Cardiol 2021;6(6):669–677. DOI: https://doi.org/10.1001/jamacardio.2021.0064
2.     Roy PM, Meyer G, Vielle B, et al., Appropriateness of diagnostic management and outcomes of suspected pulmonary embolism. Ann Intern Med 2006;144(3):157–164. DOI: https://doi.org/10.7326/0003-4819-144-3-200602070-00003
3.     Roy PM, Durieux P, Gillaizeau F, et al. A computerized handheld decision-support system to improve pulmonary embolism diagnosis: a randomized trial. Ann Intern Med 2009;151(10):677–686. DOI: https://doi.org/10.7326/0003-4819-151-10-200911170-00003
4.     Kline JA, Courtney DM, Kabrhel C, et al. Prospective multicenter evaluation of the Pulmonary Embolism Rule-out Criteria. J Thromb Haemost 2008;6(5):772–780. DOI: https://doi.org/10.1111/j.1538-7836.2008.02944.x
5.     Yin F, Wilson T, Fave AD, et al. Inappropriate use of d-dimer assay and pulmonary CT angiography in the evaluation of suspected acute pulmonary embolism. Am J Med Qual 2012;27(1):74–79. DOI: https://doi.org/10.1177/1062860611407907
6.     Alhassan S, Sayf AA, Arsene C, Krayem H. Suboptimal implementation of diagnostic algorithms and overuse of computed tomography – pulmonary angiography in patients with suspected pulmonary embolism. Ann Thorac Med 2016;11(4):254–260. DOI: https://doi.org/10.4103/1817-1737.191875
7.     Osman M, Subedi SK, Ahmed A, et al. Computed tomography pulmonary angiography is overused to diagnose pulmonary embolism in the emergency department of academic community hospital. J Commun Hosp Intern Med Perspect 2018;8(1):6–10. DOI: https://doi.org/10.1080/20009666.2018.1428024
8.     Abcarian PW, Sweet JD, Watabe JT, Yoon HC. The role of a quantitative d-dimer assay in determining the need for pulmonary CT angiography in the evaluation of acute pulmonary embolism. Am J Roentgenology 2004;182(6):1377–1381. DOI: https://doi.org/10.2214/ajr.182.6.1821377
9.     Hirai LK, Takahashi JM, Yoon HC. A prospective evaluation of a quantitative d-dimer assay in the evaluation of acute pulmonary embolism. J Vasc Interv Radiol 2007;18(8):970–974. DOI: https://doi.org/10.1016/j.jvir.2007.04.020
10.   Char S, Yoon HC. Improving appropriate use of pulmonary computed tomography angiography by increasing the serum d-dimer threshold and assessing clinical probability. Perm J 2014 fall;18(4):10–15. DOI: https://doi.org/10.7812/TPP/14-007
11.   Kearon C, de Wit K, Parpia S, et al. Diagnosis of pulmonary embolism with d-dimer adjusted to clinical probability. N Engl J Med 2019;381(22):2125–2134. DOI: https://doi.org/10.1056/NEJMoa1909159
12.   Salehi L, Phalpher P, Ossip M, Meaney C, Valani R, Mercuri M. Variability in practice patterns among emergency physicians in the evaluation of patients with a suspected diagnosis of pulmonary embolism. Emerg Radiol 2020;27(2):127–134. DOI: https://doi.org/10.1007/s10140-019-01740-w
13.   Geeting GK, Beck M, Bruno MA, et al. Mandatory assignment of modified Wells score before CT angiography for pulmonary embolism fails to improve utilization or percentage of positive cases. Am J Roentgenol 2016;207(2):442–449. DOI: https://doi.org/10.2214/AJR.15.15394
14.   Prevedello LM, Raja AS, Ip IK, Sodickson A, Khorasani R. Does clinical decision support reduce unwarranted variation in yield of CT pulmonary angiogram? Am J Med 2013;126(11):975–981. DOI: https://doi.org/10.1016/j.amjmed.2013.04.018
15.   Prologo JD, Gilkeson RC, Diaz M, Asaad J. CT pulmonary angiography: A comparative analysis of the utilization patterns in emergency department and hospitalized patients between 1998 and 2003. AJR Am J Roentgenol 2004;183(4):1093–1096. DOI: https://doi.org/10.2214/ajr.183.4.1831093
16.   Cronin P, Kelly AM. Influence of population prevalences on numbers of false positives: An overlooked entity. Acad Radiol 2011;18(9):1087–1093. DOI: https://doi.org/10.1016/j.acra.2011.04.011
17.   Hutchinson BD, Navin P, Marom EM, Truong MT, Bruzzi JF. Overdiagnosis of pulmonary embolism by pulmonary CT angiography. AJR Am J Roentgenol 2015;205(2):271–277.

ETOC

Click here to join the eTOC list or text ETOC to 22828. You will receive an email notice with the Table of Contents of The Permanente Journal.

CIRCULATION

2 million page views of TPJ articles in PubMed from a broad international readership.

Indexing

Indexed in MEDLINE, PubMed Central, EMBASE, EBSCO Academic Search Complete, and CrossRef.


                                             

 

 

ISSN 1552-5775 Copyright © 2021 thepermanentejournal.org

All Rights Reserved