Jump to comment:

• Two issues with this paper, one of which is a significant calculational error
  Ed J Gracely
  Published on: 11 August 2023

• Published on: 11 August 2023

  Two issues with this paper, one of which is a significant calculational error

  • Ed J Gracely, Faculty Drexel University, Philadelphia, PA (USA)

  The systematic review by Paultre et al. supports the use of turmeric or curcumin extract for knee osteoarthritis pain.

  They did not perform a formal meta-analysis but summarize the results of individual studies by calculating effect sizes based on the data in the original papers. Unfortunately there are two problems with these, one major and the other more modest.

  The major issue is with the last study reported in table 3, Srivastava (2016). Paultre et al. report very large effect sizes for this study, such as 8.6, 9.5, and even 11 for a visual analogue scale. These effect sizes are the usual "d" value, that is the mean difference divided by the standard deviation. Effect sizes of such high magnitudes should raise a red flag that something is wrong, as they are rarely attained in clinical studies.

  The authors' impressive effect sizes for Srivastava are errors due to using a standard error of the mean (SE) as if it were a standard deviation (SD). Srivastava et al. define the statistic used in the statistical methods: "The results are presented as mean ± SE." The values shown are also impossibly small to be standard deviations, which is what caught my attention. Both at 60 days and 120 days, the "standard deviations" shown for a 10-point VAS scale are around 0.1. This suggests a range of responses of about 0.5, which is not plausible.

  The SEM is the SD divided by the square root of the sample size and represents...

  Show More

  The systematic review by Paultre et al. supports the use of turmeric or curcumin extract for knee osteoarthritis pain.

  They did not perform a formal meta-analysis but summarize the results of individual studies by calculating effect sizes based on the data in the original papers. Unfortunately there are two problems with these, one major and the other more modest.

  The major issue is with the last study reported in table 3, Srivastava (2016). Paultre et al. report very large effect sizes for this study, such as 8.6, 9.5, and even 11 for a visual analogue scale. These effect sizes are the usual "d" value, that is the mean difference divided by the standard deviation. Effect sizes of such high magnitudes should raise a red flag that something is wrong, as they are rarely attained in clinical studies.

  The authors' impressive effect sizes for Srivastava are errors due to using a standard error of the mean (SE) as if it were a standard deviation (SD). Srivastava et al. define the statistic used in the statistical methods: "The results are presented as mean ± SE." The values shown are also impossibly small to be standard deviations, which is what caught my attention. Both at 60 days and 120 days, the "standard deviations" shown for a 10-point VAS scale are around 0.1. This suggests a range of responses of about 0.5, which is not plausible.

  The SEM is the SD divided by the square root of the sample size and represents the accuracy of the sample mean, not the distribution of individual subject values. Effect sizes incorrectly calculated using the SEM will thus be inflated by a factor of the square root of the sample size. Correcting for this leaves several still large effects (near 1) but other effect sizes drop to below 0.5. None are over 2.

  The modest issue is with the first study in the table, Panda. I don't see any actual errors, but the standard deviations on the VAS are smaller than is typical for a 100 point scale. I believe this is because an inclusion criterion for that study was, "VAS score during the most painful knee movement between 40-70mm." Restricting to a max of 70 for "most painful" is a significant restriction and results in rather small standard deviations for the VAS.

  On a 100-point scale a mean difference of 10 is not a large effect clinically, but Paultre et al. show differences of 5 and 7 as "large" for the Panda study because of the small standard deviations. Not until day 60 is there a mean difference much bigger than 10 on the VAS.

  I would be interested to see if the authors would modify their conclusions after addressing these issues.

  ====
  References (both are from the reviewed paper)

  Srivastava S, Saksena AK, Khattri S, et al. Curcuma longa extract reduces inflammatory and oxidative stress biomarkers in osteoarthritis of knee: a four-month, double-blind, randomized, placebo-controlled trial. Inflammopharmacology 2016;24:377–88

  Panda SK, Nirvanashetty S, Parachur VA, et al. A randomized, double blind, placebo controlled, parallel-group study to evaluate the safety and efficacy of Curene® versus placebo in reducing symptoms of knee oa. Biomed Res Int 2018;2018:1–8.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top