Statistics from Altmetric.com
I read with interest the manuscript “Randomized controlled trial of vertebroplasty versus kyphoplasty in the treatment of vertebral compression fractures” published in the Journal of NeuroInterventional Surgery.1 Although the authors make some valid scientific points in the article, the study has significant shortcomings that should be recognized.
One of the critical issues with this paper is the lack of inclusion of the statistical power calculations necessary to prove or disprove the null hypothesis. The authors use pain and function (Roland–Morris Disability Questionnaire) as primary endpoints for comparing vertebroplasty (VP) and kyphoplasty (KP). The formula described here is typical for calculating sample size in each group. If the desired power is 80% and the level of statistical significance is 1.96, we can use the value of 1–2.5 as a statistical difference threshold in pain after treatment according to the largest VP versus KP manuscript published to date along with other standard references of measurement.2–4 If the standard method of calculation for each group is used to detect a standard difference in pain there would need to be a minimum of 512 patients in each group—that is, 1024 patients in total. The KAVIAR study calculated that a total of 1234 patients would be necessary to determine an 8.7% difference in their primary endpoint—subsequent radiographic fractures. This patient requirement alone indicates that the number of patients (59 enrolled in one group and 56 in the other in the study of Evans et al) was 909 patients short of the number of patients required to determine a difference in their primary endpoint of pain difference.
The number of patients necessary to conclude that there is no difference in VP and KP for pain and disability is less than the number needed to compare the difference in additional fractures, but that number also has not been used in their study.
Pain and function are reasonable primary endpoints but other endpoints that characterize a difference between VP and KP are also important but were not included in this evaluation, including the level of fracture reduction, the difference in additional or adjacent fractures and differences in cement extravasation rates. When randomized controlled studies (RCTs) such as this are performed, a Bayesian adaptive design with interim analyses conducted by an independent data coordinating center can be used to determine when an adequate patient number has been reached.
This statistical shortcoming combined with the definitive statement that this study “provides strong evidence that VP and KP are equally effective” goes far beyond the conclusions that can be reached with this study design. It is a bold statement for a study that has no power calculations and only 115 patients, with over 25% of them unavailable for data collection at 1 year. The authors have not demonstrated that their study has sufficient power to make this distinction, much less to justify bold statements about their study providing strong evidence.
The safety claims are also an important concern with this manuscript. Safety is mentioned 10 times in the manuscript and it is stated that to compare the safety of these two procedures is a primary goal. As stated in the abstract the authors present the results of a “RCT evaluating safety and efficacy of vertebroplasty versus kyphoplasty in treating vertebral body compression fractures”. The difficulty with this approach is recognized by the authors in the ‘Limitations’ section where they write “Our study is limited by a modest sample size” and “given this limitation, it is difficult to compare outcomes of rare complications such as new-onset fractures”. Not only is it difficult to compare, it is not possible, given the underpowered state of the study, and their declaration that additional fractures as rare is simply not true and is at odds with the vast majority of data that report an additional fracture rate of around 20%.5
Comparison of safety between the two procedures would require a study similar to recently published RCTs with approximately 300 patients each, which compared KP with non-surgical management (FREE) and KP with Kiva (KAST).6 ,7 Given the low rate of complications seen with both procedures, detecting a difference between VP and KP would require far more patients. That fact did not prevent the authors from stating in their conclusion that “This study confirms the findings of two previous RCTs that vertebroplasty and kyphoplasty are equally safe and effective in reducing pain and disability related to vertebral compression fractures”. This conclusion of safety is not warranted by the statistical power of the study.
They also dismiss other important points of comparison such as the level of fracture reduction, the difference in additional fracture rate and differences in cement extravasation. As we know from the recently published data from the FREE trial, the greater the amount of reduction, the greater the improvement in pain, function, and quality of life. The best RCT data to date show that fracture reduction is indeed important but is not addressed by this small RCT. The differences in additional fracture rate would have taken over 1000 additional patients, as mentioned previously, and the lack of comparison of cement extravasation rates completely discounts the importance of the most common cause of a procedural complication.
In summary, the manuscript, although an RCT, is underpowered, focuses on only two limited factors of comparison, overstates the strength of their evidence and tries to equate VP with KP without comparing the elements most likely to indicate a difference.
Correction notice This article has been corrected since it published Online First. The competing interests statement has been amended.
Twitter Follow Doug Beall at @dougbeall
Competing interests Medtronic – advisory board, consultant, research funding; Amendia – consultant; Spineology – consultant, stock owner; Dfine – consultant; Osseon – consultant; Lilly – advisory board, consultant; Xten – consultant; Smith & Nephew – consultant; Ascendx Spine – consultant, research funding; Vertiflex – consultant; Synthes – consultant; Depuy – consultant; Johnson & Johnson – consultant; Orthovita – consultant, research funding; Vitacare – consultant, research funding; Ortho Kinematics – consultant, research funding; Alphatech Spine – consultant, research funding; Dfine – consultant, research funding; Advanced Technologies and Regenerative Medicine – research funding; Algea-Globus – consultant; Benvenue – consultant, research funding, medical director, stock owner; Bone Support – consultant, research funding; Convatec – advisory board, consultant; Integral Spine Solutions – advisory board, consultant; Spinal Ventures – consultant; Medical Metrics – consultant, research funding; Zyga – consultant, research funding; Liventa – consultant, research funding; Vexim – consultant, board member; Mesoblast – research funding.
Provenance and peer review Not commissioned; internally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.