Submitted: 04 September 2019 | Approved: 10 September 2019 | Published: 11 September 2019
How to cite this article: Abolfotouh MA, Al-Anazi H, Hassan S. The validity of progesterone level on hCG injection day in the prediction of IVF/ICSI cycles’ outcome. Clin J Obstet Gynaecol. 2019; 2: 095-100.
Copyright: © 2019 Abolfotouh MA, et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Keywords: Pregnancy rate; Premature progesterone rise; Prediction: Validity; Ovarian hyper stimulation; Pregnancy outcome
Abbreviations: hCG: human Chorionic Gonadotropin; PE: Premature Progesterone Elevation; P4: Progesterone level on hCG day; E2: Estrogen level, PMS: Percentage Mean Score; COH: Hyperstimulation; ROC: Receiver Operating Characteristic Curve; AUC: Area Under the Curve; ARR: Absolute Risk Reduction, RRR: Relative Risk Reduction; HER: Electronic Health Records; KAIMRC: King Abdullah International Medical Research Center; IRB: Institutional Review Board
Background: Previous studies highlighted the negative effect of premature progesterone elevation (PE) during IVF cycles on the cycle outcomes. The aim of this study was to assess the validity of progesterone level on hCG day (P4) in the prediction of IVF/ICSI cycles’ outcome.
Methods: In a retrospective cohort study, all fresh cycles of 256 patients who underwent IVF or ICSI cycles in 2017 at reproductive endocrinology & infertility unit/ Obg/Gyn department at King Abdulaziz Medical city, Riyadh, Saudi Arabia, were followed up. They were started on gonadotropin medications for ovarian hyperstimulation, followed by serial transvaginal U/S and serum estrogen levels each visit. Patients having 2 or more 18mm follicles were triggered by hCG 10,000 IU and ovum pickup was done 34-36 hrs after. Data were collected on patients’ characteristics [age, BMI infertility type], cycles’ characteristics [number of follicles and endometrium thickness on hCG day, P4 and estrogen levels], rates of pregnancy and pregnancy outcomes. Receiver operating characteristic curve was applied to determine the cut-off of P4 that corresponds with a negative pregnancy test. Logistic regression analysis was used and significance was considered at p - value of ≤0.05.
Results: Pregnancy rate in the study sample was 36.7%. The mean P4 level in cycles with negative pregnancy tests was significantly higher than the mean in cycles with positive tests (p = 0.018). After adjusting for confounders, significant negative association between P4 and pregnancy rate was evident (p < 0.03). The optimum trade-off of P4 for prediction of a negative pregnancy test was 1.5nmol/L. This cut-off level had a 59% sensitivity, 51% specificity and 68% positive predictive value and 10% & 15% absolute and relative risk reductions respectively. Cycles with mean P4 of ≥1.5nmol/L were significantly associated with primary infertility (p = 0.011), lower mean BMI (p = 0.009) higher mean estrogen level (p < 0.001), lower live birth rate (p = 0.048), higher abortion rate (p = 0.039), and higher ovarian hyperstimulation rate (p = 0.027).
Conclusion: Premature elevation of progesterone level on the hCG day in IVF/ICSI cycles may have adversely impacted the pregnancy rate and pregnancy outcome. The cutoff point of 1.5nmol/L for this P4 was not valid in predicting pregnancy outcomes.
In the normal menstrual cycle, progesterone level is rising after luteinizing hormone (LH) surge and ovulation in the luteal phase. In in vitro fertilization (IVF) cycles, progesterone level sometimes rise in the follicular phase on the day of human chorionic gonadotropin (hCG) administration. This premature progesterone rise (PE) was called premature luteinization referred to premature LH surge. With the introduction of GnRH analogues in IVF cycles, the progesterone level (P4) was decreased and premature LH surge was inhibited. This is because suppressing of granulosa cell steroidogenic activity. Despite the wide use of GnRH analogous in ART nowadays, the rise of progesterone level is still observed in some cycles. The effect of that on the endometrium and cycle outcome is still controversial and area for discussion [1,2].
Although the frequency of PE varies, incidences as high as 35% of stimulated cycles in women treated with GnRH agonists [3,4] and 38% of cycles in women treated with GnRH antagonists [5,6], have been reported. However, in a large retrospective analysis of over 4000 cycles, the incidence of PE above 1.5ng/ml was estimated to be 8.4% in agonist and antagonist cycles [7,8].
The relationship between PE and pregnancy rate has been analyzed by using different thresholds of serum progesterone on the day of hCG. The thresholds were varied and found to be between 0.4ng/ml and 3ng/ml . For example, in the analysis of a large series by Bosch, et al. , the optimal progesterone threshold over which a detrimental effect on IVF outcome might be observed has been estimated at 1.5ng/ml. Many studies reported a negative effect of PE during IVF cycles on the cycle outcomes including endometrial receptivity [10-12]. Lower pregnancy rate and higher pregnancy loss with PE in IVF cycles had been observed in studies conducted by Silverberg, et al. , although the mechanism of this remains controversial [4,13]. Then, several authors investigated the effect of PE during the IVF cycles with GnRH analogues, some of these reported negative impact on the live birth and pregnancy rate, while others didn’t find any association [14-18]. The aim of this study was to assess the validity of progesterone level on hCG day (P4) in the prediction of IVF/ICSI cycles’ outcome in a Saudi setting.
Our study was undertaken at King Abdul-Aziz Medical City of the National Guard- health affairs, department of obstetrics and gynecology, reproductive endocrinology and infertility unit, in Riyadh, Saudi Arabia. In a retrospective cohort study, all fresh cycles of patients who underwent IVF or ICSI cycles in 2017 with GnRH agonist or antagonist medications, were followed up. Frozen cycles were excluded. After reviewing all patients’ files for infertility treatment at that period, 302 patients were eligible to be included in our study, 46 patients were cancelled for different reasons (21 patients had failed fertilization, 8 showed poor response, one developed ovarian hyperstimulation, 3 with no oocytes retrieved, one had an endometrial polyp, 6 had immature oocytes, and 6 had arrested embryos). Only 256 patients were followed up. Patients were started on antagonist [n = 241], short [n = 47] or long [n = 14] agonist protocols, according to their infertility evaluation. They were started on gonadotrophin medications for ovarian hyperstimulation. The starting dose was based on patient’s age, AFC, BMI and previous response. The stimulation cycle was followed by serial transvaginal U/S and serum estrogen level (E2), each visit. When patient a had 2 or more 18mm follicles, she was triggered by hCG 10,000 IU and ovum pickup was done 34-36 hrs after. We documented all patients’ characteristics at the starting of the cycles. Number of follicles and endometrium thickness on hCG triggering day were also documented. We checked Progesterone level on the day of triggering (P4) as well as the estrogen level (E2).
Pregnancy rate was calculated. Among patients with positive pregnancy test, we calculated the rates of term live birth, preterm birth, abortion and chemical pregnancy. Patient’s data and cycles’ details for all patients were reviewed using patients’ files and the EHR (Best-Care).
All values of the progesterone on the hCG day of IVF/ICSI cycles were cross classified according to their pregnancy test result (negative or positive), and by various cut-off points along with the range of progesterone levels above which subjects may be considered having a negative pregnancy test. From these tabulations, the sensitivity, specificity and positive predictive value were computed for progesterone level at each cut-off point.
The sensitivity of progesterone level diagnosis for the pregnancy test result “gold standard” was determined by calculating how frequent the correct progesterone level diagnosis was made in each pregnancy test diagnosis. The specificity of progesterone diagnosis was determined by calculating how frequently the progesterone diagnosis was not made when the corresponding pregnancy test diagnosis was positive. Positive predictability indicated how frequently the progesterone diagnosis correctly reflected the negative pregnancy test. Negative predictability indicated how frequently the progesterone diagnosis was not made when the corresponding pregnancy test diagnosis was positive. Also, the level of agreement between these two methods was determined at each cut-off point by the calculation of kappa coefficient (k). Absolute risk reduction (ARR) and relative risk reduction (RRR) were calculated.
The Receiver Operating Characteristic (ROC) curve of a diagnostic test is a graph of the pairs of sensitivity and 1 minus specificity that correspond to each possible cut-off for the diagnostic test result. This curve was used to determine the threshold value of P4 that corresponds to the negative pregnancy test result. Analyses were performed using SPSS (version 23). Both descriptive and analytical statistics were applied. Rates of pregnancy, abortion, live births were estimated. Chi-square test and Fisher exact were used to investigate the association between the levels of progesterone and outcome parameters. Logistic regression analysis was used to investigate if the change in progesterone level is a predictor of outcome. Significance was considered at a p - value of ≤0.05. The study was approved by the IRB of the Ministry of National Guard-Health Affairs (Ref. # RC17/335/R). This study was conducted in accordance with the Declaration of Helsinki.
Of a total of 302 cycles, 46 cycles (15.2%) were cancelled. The pregnancy rate in the followed up cycles (n = 256) was 36.7%. There was a significant negative association between progesterone level at HCG day and the pregnancy outcome, with significantly higher mean progesterone level among those with a negative pregnancy test than its counterpart among those with a positive test (2.26 ± 1.81 vs 1.84 ± 1.01, t = 2.37, p = 0.018, Table 1).
Applying the ROC curve for progesterone levels on hCG day and the pregnancy test results shows that the optimum trade-off level for progesterone was 1.5nmol/L, figure 1. This cut-off level had a sensitivity of 59% and a specificity of 51% for prediction of a negative pregnancy test. The computed positive predictive value was 68%. The area under the curve (AUC) was 0.55. This reflects the low validity of this cut-off in the prediction of pregnancy test results. At this cut-off level, the absolute risk reduction was 10% and the relative risk reduction was 15%, i.e.; pregnancy failure would be reduced by 15% when P4 level is < 1.5nmol/L. At this cut-off, the level of agreement between the P4 level and the pregnancy test results, as calculated by kappa coefficient, was not significant (k = 0.10, p = 0.077), Table 1.
Table 2 shows the comparison between cycles with <1.5nmol/L and ≥1.5nmol/L mean progesterone level on the hCG day (P4), in terms of; patients characteristics, cycle characteristics, cycle outcome and complications. Cycles whose mean level of progesterone at hCG day is ≥1.5nmol/L showed significantly higher proportion of primary infertility (54.1% vs 39.2%, χ2 = 6.54, p = 0.011), lower mean BMI (28.13 ± 5.6 vs 29.76 ± 4.9, t = 2.62, p = 0.009) and higher mean estrogen level (9602.09 ± 6952 vs 5518.36 ± 4359, t = 6.22, p <0.001). They showed also less favorable outcome in terms of term lower proportion of live birth (47.8% versus 68.1%, χ2 = 3.92, p = 0.048), higher proportion of abortion (24.4% versus 8.2%, χ2 = 4.25, p = 0.039), and higher proportion of ovarian hyperstimulation (8.1% vs 2.3%, χ2 = 7.22, p = 0.027). Endometrial polyp was seen only in patients with low progesterone level. The incidence of premature progesterone elevation (PE) was 55.4% in all stimulation protocols. It was 58.5% (141/241) in antagonist protocol group, and 50.8% (31/61) in agonist protocol group (χ2 = 1.17, p = 0.28).
After adjusting for BMI, type of infertility and estrogen level at hCG day, the significant negative association between P4 and pregnancy outcome was retained, with a p-value of <0.03 (Table 3).
Pregnancy rate in our study was 36.7%. This figure was comparable to figures of 30.6%  and 34%  in previous studies. A progesterone rise during the late follicular phase (P4) has been considered a negative predictive factor for clinical outcome in both GnRH agonist [4,21] and antagonist protocols [7,22]. In our study, there was a significant negative association between P4 level on hCG day and the pregnancy outcome, with significantly higher mean P4 level among those with negative pregnancy test than its counterpart among those with positive test. This finding was in agreement with what was reported by Mascarenhas, et al.  and Ashmita, et al. , Higher P4 level in our study was also associated with lower rate of term live birth and higher rates of abortion and ovarian hyperstimulation. Huang, et al. , in a study of 2566 patients, reported that PE negatively correlated with live birth in fresh embryo transfer cycles. Data from large previous retrospective  and prospective  studies supported the notion that pregnancy rates are inversely related to P4 levels, especially when a threshold of 1.5ng/ml is adopted.
The relationship between progesterone elevation (PE) and pregnancy rate has been analyzed by using different thresholds of serum progesterone on the day of hCG. The thresholds were varied and found to be between 0.4ng/ml and 3ng/ml . The analysis of a large series by Bosch, et al. , the optimal progesterone threshold over which a detrimental effect on IVF outcome might be observed has been estimated at 1.5ng/ml. In our study, applying the ROC curve for P4 and the pregnancy test results showed that the optimum trade-off level for progesterone was 1.5nmol/L. This cut-off level had a sensitivity of 59% and a specificity of 51% for prediction of a pregnancy test result. The computed positive predictive value was 68%. Thus, at this cut-off level, the progesterone test correctly diagnosed 59% of negative pregnancy test results, missed 41% of these negative pregnancy tests, but misclassified 49% of positive pregnancy results as negative results (false positives). At this cut-off level, absolute risk reduction was 10% and relative risk reduction was 15%, i.e.; pregnancy failure would be reduced by 15% when the level of P4 is less than 1.5nmol/L. The level of agreement between the P4 and the pregnancy test results, as calculated by kappa coefficient, was not significant (k = 0.10, p = 0.077). Meanwhile, there was no significant difference between the group with P4 <1.5nmol/L and those with ≥1.5nmol/L in pregnancy rate. These findings may reflect that the low validity of the cut-off of 1.5nmol/L for P4 in the prediction of pregnancy outcome. Even, using the cut-off of ≥1.2nmol/L, recommended by others , would result in a very low (28%) sensitivity, 69% specificity and 61% positive predictive value.
The pathogenesis of PE in controlled ovarian hyperstimulation (COH) cycles is still poorly understood. The incidence of PER in our study was 55.5% (142/256), based on a cut-off level of 1.5nmol/L for P4. This figure is high if compared with figures of 13.19% , 13.02%  and 38.3% . However, comparison is difficult when the definition of PE is different in various studies. Among factors associated with PE are the type of protocol, the type and total dose of gonadotropin given, E2 levels on the day of trigger and the number of intermediate follicles recruited . Bosch, et al.  concluded that estrogen values on the day of hCG trigger were associated with increased progesterone levels (P <0.0001). In our study, higher mean level of P4 of ≥1.5nmol/L showed significantly higher mean estrogen level (E2). This was in agreement with the findings of Ashmita, et al.  who reported a higher incidence of PE in the cycles with E2 ≥2500 IU. The incidence of elevated progesterone concentrations was higher in rFSH‑treated patients than in HMG‑treated patients , and in cases with large doses of gonadotropins given [19,24]. However, in our study, there was no significant association between the type of protocol and P4 level. The mean number of oocytes retrieved was significantly higher in patients with higher P4 levels , but our study showed no such association. In our study, significant associations were shown between P4 level and only 3 variables; type of infertility and BMI and E2 level, however, none of these variables had a significant association with the pregnancy rate. Adjusting for these variables, P4 level was the only significant predictor of pregnancy outcome.
Premature progesterone level on the day of hCG in IVF/ICSI cycles might have a role in predicting pregnancy outcome. However, the cut-off of ≥1.5 nmol/L for progesterone on the hCG day is not a valid threshold in the prediction of pregnancy outcome. Further studies are necessary to confirm these findings.
Ethics approval and consent to participate
This study was approved by the institutional review board (IRB) of the Ministry of National Guard-Health Affairs, Riyadh, Saudi Arabia [Ref. # RC17/335/R]. The need for informed consent was waived by the IRB, as all these retrospective data were retrieved from records without the disclosure of identifiers.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
HA, SH and MAA contributed to concept development, manuscript preparation and final writing, HA and SH contributed to concept development, research proposal writing and data collection, MAA and HA conducted data analysis and interpretation, and manuscript drafting. All authors read and approved the final manuscript.
This study was initiated and supported by King Abdullah International Medical Research Center (KAIMRC), King Saud bin-Abdulaziz University for Health Sciences, Ministry of National Guard –Health Affairs, Saudi Arabia. The final draft of the manuscript was English language edited by Macmillan Science Communication.
- Hugues JN, Massé-Laroche E, Reboul-Marty J, Boîko O, Meynant C, et al. Impact of endogenous luteinizing hormone serum levels on progesterone elevation on the day of human chorionic gonadotropin administration. Fertil Steril. 2011; 96: 600-604. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21880277
- Del Castillo JL, Bousamra M, De La Fuente L, Ruiz-Balda JA, Palomo M. The impact of serum progesterone levels on the results of in vitro fertilization treatments: a literature review. JBRA Assist Reprod. 2015; 19: 141-147. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27203093
- Edelstein MC, Seltman HJ, Cox BJ, Robinson SM, Shaw RA, et al. Progesterone levels on the day of human chorionic gonadotropin administration in cycles with gonadotropin-releasing hormone agonist suppression are not predictive of pregnancy outcome. Fertil Steril. 1990; 54: 853-857. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/2121554
- Silverberg KM, Burns WN, Olive Dl, Riehl RM, Schenken RS. Serum progesterone levels predict success of in vitro fertilization/embryo transfer in patients stimulated with leuprolide acetate and human menopausal gonadotropins. J Clin Endocrinol Metab. 1991; 73: 797-803. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/1909704
- Bosch E, Valencia I, Escudero E, Crespo J, Simón C, et al. Premature luteinization during gonadotropin-releasing hormone antagonist cycles and its relationship with in vitro fertilization outcome. Fertil Steril. 2003; 80: 1444-1449. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/14667881
- Ubaldi F, Albano C, Peukert M, Riethmüller-Winzen H, Camus M, et al. Endocrinology: subtle progesterone rise after the administration of the gonadotrophin-releasing hormone antagonist Cetrorelix in intracytoplasmic sperm injection cycles. Hum Reprod. 1996; 11: 1405-1407. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/8671476
- Bosch E, Labarta E, Crespo J, Simon C, Remohi J, et al. Circulating progesterone levels and ongoing pregnancy rates in controlled ovarian stimulation cycles for in vitro fertilization: analysis of over 4000 cycles. Human reproduction. 2010; 25: 2092-2100. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20539042
- Al-Azemi M, Kyrou D, Kolibianakis EM, Humaidan P, Van Vaerenbergh I, et al. Elevated progesterone during ovarian stimulation for IVF. Reprod Biomed Online. 2012; 24: 381-388. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22377153
- Venetis CA, Kolibianakis EM, Bosdou JK, Tarlatzis BC. Progesterone elevation and probability of pregnancy after IVF: a systematic review and meta-analysis of over 60 000 cycles. Hum Reprod Update. 2013; 19: 433-457. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23827986
- Labarta E, Martínez-Conejero JA, Alamá P, Horcajadas JA, Pellicer A, et al. Endometrial receptivity is affected in women with high circulating progesterone levels at the end of the follicular phase: a functional genomics analysis. Hum Reprod. 2011; 26: 1813-1825. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21540246
- Li R, Qiao J, Wang L, Li L, Zhen X, et al. MicroRNA array and microarray evaluation of endometrial receptivity in patients with high serum progesterone levels on the day of hCG administration. Reprod Biol Endocrinol. 2011; 9: 29. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/21375772
- Haouzi D, Bissonnette L, Gala A, Assou S, Entezami F, et al. Endometrial receptivity profile in patients with premature progesterone elevation on the day of HCG administration. BioMed Res Int. 2014; 2014. PubMed: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4022194/
- Schoolcraft W, Sinton E, Schlenker T, Huynh D, Hamilton F, et al. Lower pregnancy rate with premature luteinization during pituitary suppression with leuprolide acetate. Fertil Steril. 1991; 55: 563-566. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/1900481
- Xu B, Li Z, Zhang H, Jin L, Li Y, et al. Serum progesterone level effects on the outcome of in vitro fertilization in patients with different ovarian response: an analysis of more than 10,000 cycles. Fertil Steril. 2012; 97: 1321-1327. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22494924
- Papanikolaou EG, Pados G, Grimbizis G, Bili E, Kyriazi L, et al. GnRH-agonist versus GnRH-antagonist IVF cycles: is the reproductive outcome affected by the incidence of progesterone elevation on the day of HCG triggering? A randomized prospective study. Hum Reprod. 2012; 27: 1822-1828. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22422777
- Ochsenkühn R, Arzberger A, von Schönfeldt V, Gallwas J, Rogenhofer N, et al. Subtle progesterone rise on the day of human chorionic gonadotropin administration is associated with lower live birth rates in women undergoing assisted reproductive technology: a retrospective study with 2,555 fresh embryo transfers. Fertil Steril. 2012; 98: 347-354. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22633265
- Huang CC, Lien YR, Chen HF, Chen MJ, Shieh CJ, et al. The duration of pre-ovulatory serum progesterone elevation before hCG administration affects the outcome of IVF/ICSI cycles. Hum Reprod. 2012; 27: 2036-2045. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22561057
- Huang PC, Chen MJ, Guu HF, Yi YC, Ho JY, et al. Effect of premature serum progesterone rise on embryo transfer outcomes and the role of blastocyst culture and transfer in assisted reproductive technology cycles with premature progesterone rise. Taiwan J Obstet Gynecol. 2015; 54: 641-646. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26700978
- Ashmita J, Vikas S, Swati G. The impact of progesterone level on day of hCG injection in IVF Cycles on clinical pregnancy rate. J Hum Reprod Sci. 2017; 10: 265-270. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29430153
- Mascarenhas M, Kamath MS, Chandy A, Kunjummen AT. Progesterone/estradiol ratio as a predictor in the ART cycles with premature progesterone elevation on the day of hCG trigger. J Reprod Infertil. 2015; 16: 155-161. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26913234
- Elnashar AM. Progesterone rise on the day of HCG administration (premature luteinization) in IVF: an overdue update. J Assist Reprod Genet. 2010; 27: 149-155. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/20177771
- Papanikolaou EG, Kolibianakis EM, Pozzobon C, Tank P, Tournaye H, et al. Progesterone rise on the day of human chorionic gonadotropin administration impairs pregnancy outcome in day 3 single-embryo transfer, while has no effect on day 5 single blastocyst transfer. Fertil Steril. 2009; 91: 949-952. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17555751
- Andersen AN, Devroey P, Arce JC. Clinical outcome following stimulation with highly purified hMG or recombinant FSH in patients undergoing IVF: a randomized assessor-blind controlled trial. Human Reproduction. 2006; 21: 3217-3227. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16873892
- Kiliçdag EB, Haydardedeoglu B, Cok T, Hacivelioglu SO, Bagis T. Premature progesterone elevation impairs implantation and live birth rates in GnRH-agonist IVF/ICSI cycles. Arch Gynecol Obstet. 2010; 281: 747-752. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19862542
- Kyrou D, Al-Azemi M, Papanikolaou EG, Donoso P, Tziomalos K, et al. The relationship of premature progesterone rise with serum estradiol levels and number of follicles in GnRH antagonist/recombinant FSH-stimulated cycles. Eur J Obstet Gynecol Reprod Biol. 2012; 162: 165-168. PubMed: https://www.ncbi.nlm.nih.gov/pubmed/22425288