Recent device failures

Adoption of new technologies is not without risk. While initial experience may demonstrate benefit, further experience or longitudinal measures may detect concept, design or manufacturing flaws that were not immediately evident. The most prominent of such devices is the ASR XL Acetabular System (DePuy, Johnson & Johnson, Warsaw, Indiana, USA), which was approved for use by the FDA through 510(k) clearance (described below) and introduced into the US market in 2005. This device has gained considerable negative media attention6 ,7 with numerous websites recruiting clients for plaintiff attorneys and over one million unique web pages produced after a Google search using the keywords ‘Depuy ASR hip recall’. The ASR featured a metal-on-metal acetabular cup design that was borrowed from a second device, the ASR Hip Resurfacing System, and fitted onto a predicate hip implant. Depuy applied for 510(k) clearance and the new device was deemed substantially equivalent to the prior hip implant without rigorous safety and effectiveness testing. Between 2005 and 2010, approximately 100 000 ASR Acetabular systems were implanted. By 2008 the FDA had received about 300 complaints regarding the device, most arising from patients who had had to undergo early revision surgery.6 Recent studies have demonstrated an increased rate of implant dysfunction with need for revision surgery that far exceeds that of other hip replacement devices.8 ,9 In fact, results presented at the British Hip Society meeting in 2011 indicated a failure rate nearing 50% at 6 years, which is three times the rate of other devices (approximately 15% at 5 years).8 Furthermore, elevated levels of blood chromium and cobalt were identified as a side effect of dysfunctional joints. Based on these data, a voluntary recall of the ASR devices was enacted in August 2010 after an estimated 100 000 ASR devices had been implanted (one-third in the USA) and 6 months after the company warned physicians of a high early failure rate.6 ,7 Examination of the dysfunctional implants after removal identified flaws inherent to the design.10 It is possible that more rigorous safety testing prior to market release, or close post-market clinical follow-up, would have detected irregularities and prevented (or halted) the implantation of ASR devices.

A more familiar neurointerventional device recently drawing considerable negative attention is the Wingspan Stent System (Stryker, Kalamazoo, Michigan, USA), a stent designed for use with the Gateway PTA Balloon Catheter in the treatment of intracranial atherosclerotic disease. The stent was approved under a FDA HDE in 2005, based on a safety study conducted in 45 patients at 12 sites in Asia and Europe, for the treatment of intracranial atherosclerotic disease refractory to medical therapy in intracranial vessels with stenosis of ≥50%. Early retrospective analyses of outcomes performed by independent centers indicated both safety and efficacy with the Wingspan,11 and many clinicians involved in stroke care were optimistic about how the system would fare in a randomized controlled trial of stroke prevention. The Stenting and Aggressive Medical Management for Preventing Recurrent stroke in Intracranial Stenosis trial (SAMMPRIS), the first randomized trial comparing best medical therapies to angioplasty and stenting, began enrolling its first patients in October 2008. However, enrollment for SAMMPRIS was halted prematurely in a report in September 2011 owing to a 30-day stroke rate of 14.7% in the angioplasty and stenting arm compared with 5.8% in the medical management arm.12 These results have led a consumer advocacy group to seek the repeal of the Wingspan HDE and to criticize the FDA for the original approval.13–15 However, these efforts are not without controversy as the patient population evaluated in SAMMPRIS was in some respects different from the population indicated on the patient HDE (who would only comprise a subset of the patients evaluated in SAMMPRIS), and the 1-year stroke rate of 20.2% was still perceived as a dramatic improvement over the 24.9% stroke rate demonstrated in the WASID16 study for the HDE-approved population. Thus, while portrayed as ‘dangerous’ by groups such as Public Citizen, outcomes with Wingspan in SAMMPRIS were no different from those observed in the same patient cohort treated with conventional medical therapy. The true advance in the SAMMPRIS trial was an observation that was independent of the actual device in that aggressive medical management resulted in a primary event rate that was half the rate (12.2% over 1 year) expected on the basis of the WASID study (24.9%). So while no one debates that aggressive medical management is superior to angioplasty and stenting in the SAMMPRIS study population, this unexpected finding in no way indicates a breakdown of the regulatory process but merely reflects a tremendous advance in the medical management of the disease process.

A final example of a neurointerventional device not performing as anticipated is the Cerecyte coil (Micrus Endovascular, San Jose, California, USA), a specific type of detachable bioactive coil designed for the endovascular embolization of intracranial aneurysms. The Cerecyte coil contains a polyglycolic acid element within the wind of the coil, in contrast to traditional coils which are composed of bare platinum. The Cerecyte coil was approved for use in the USA via the 510(k) process in 2004. Early non-randomized studies suggested better results than bare platinum coils,17–21 leading the device manufacturer to charge a premium for these coils as they were deemed superior to traditional coils. However, the Cerecyte Coil Trial, a company-sponsored randomized controlled trial comparing Cerecyte coils with bare platinum coils, demonstrated no benefit for Cerecyte over traditional coils.22–24 Although it is unlikely that patients were physically harmed due to the use of this technology, the amount of money spent on premiums for what was eventually determined to be an equivalent product is substantial. This problem is not isolated to the Cerecyte coil; other devices such as the Matrix coil (Stryker)25 were similarly charged at a premium for years, only to reveal no difference in primary outcomes in later definitive trials. This scenario represents an additional point of contention: unvalidated increases in financial expenditures following a market release without rigorous testing and post-market follow-up.

Pathways to obtaining FDA approval for use

Postmarket device reporting

All devices approved for market have mandatory manufacturer and facility reporting requirements. Most notably, manufacturers and the facility must report all device-related deaths, serious injuries and adverse events secondary to device failure or adverse events in which the device may have contributed. These include 30-day reports in which manufacturers have 30 days from time of event to report device-related deaths, serious injuries or malfunctions to the FDA (available at http://www.fda.gov/downloads/Safety/MedWatch/HowToReport/DownloadForms/UCM082728.pdf); 5-day reports which require manufacturers to report serious public health concerns stemming from device use to the FDA within 5 days of becoming aware of the concern; baseline reports which are for first-time adverse events; supplemental reports; and annual certifications. However, substantial criticism exists with this reporting process as no formal system is in place to ensure capture of all events. For the most part, the reporting process is dependent upon physicians reporting any events back to the company. It is probably fair to say that, currently, such reporting is sporadic at best.

The FDA may order manufacturers of certain Class II and Class III devices to establish tracking systems in which each individual device may be tracked to the patient in which it was used. This provision allows the FDA and manufacturer to locate and expeditiously remove those devices from the market that have been identified in postmarket reporting as potentially dangerous or defective (facilitating device recalls) or to notify treated patients of a potential health concern associated with device use. Generally, devices subject to such tracking provisions are those that are intended for implantation in the body for >1 year, those that may cause significant harm or death should the device malfunction, or those that are intended for use outside the treatment facility and are life-saving or life-sustaining.

The FDA may also order holders of a PMA or HDE to perform a 522 postmarket surveillance (522PMS) study to help assure continued safety and effectiveness after the device has been released on the market for a period of up to 36 months. The FDA has authority to require a 522PMS on any Class II or Class III device that meets one of the three tracking criteria (listed above) and/or is expected to have significant use in a pediatric population. The 522PMS is highly specific to the given device and may range from animal studies to randomized controlled trials. Frequently, a required 522PMS will involve active or enhanced surveillance studies where the incidence, distribution and trends of adverse events are actively or passively recorded and reported.

Growing concerns about the current FDA approval processes

There is a growing discussion in both the medical literature and in public commentary regarding potential faults in the FDA approval process for medical devices. Most notably, the 510(k) and HDE processes have sparked considerable controversy due to the Depuy ASR and Wingspan System, as well as other devices, which were approved through ‘fast track’ routes lacking scientific evidence confirming device safety and effectiveness.

Safety concerns have been raised over the FDA's 510(k) clearance process whereby devices demonstrated to be ‘substantially equivalent’ to previously approved devices, and therefore thought to share similar safety and benefit profiles, are approved for marketing without clinical trials. These concerns led the Institute of Medicine (IOM) to recommend eliminating the 510(k) process altogether in their July 2011 FDA-commissioned report.27 Within this report the IOM argues that the 35-year-old 510(k) process cannot ensure device safety or effectiveness because it lacks any means to do so; it can merely determine equivalence to a predicate device. The IOM therefore argues that the 510(k) process should be disbanded and a new forward-thinking process developed. Furthermore, the report argues for enhanced post-marketing surveillance monitoring of devices, a feature that 510(k) approval currently lacks. Unfortunately, the report does not deliver new blueprints for overhauling the system; it merely identifies that the 35-year-old 510(k) process is antiquated and no longer appropriate. The report does list a number of attributes that would be ideal for any new FDA approval system for Class II devices including evidence-based, fair, clear, self-improving, risk-based, and others.28 This report has been met with praise29 but also criticism, particularly from the medical device industry which argued that changes to regulation would slow technological innovation, cost jobs and harm patients.30 ,31

An additional concern of this process revolves around the financial incentive for manufacturers to develop new devices via the 510(k) clearance process with only minor improvements. As stated by Curfman and Redberg in a commentary published in the New England Journal of Medicine: “Since regulatory approval hinges on claims of similarity to previously approved devices, the process may encourage the development of devices that provide only small improvements at higher cost than their predecessors. The trade-offs between incremental improvement and the additional costs and technical complexity of the required procedure are poorly understood and seldom investigated rigorously.”29 As seen with the Cerecyte coils, the manufacturer was able to charge a premium due to purported superiority over traditional coils without any prospective evidence confirming superiority. When a prospective trial was performed, equivalence was confirmed but superiority was not, indicating that patients had been charged an increased cost for years for a device that was no more effective than its cheaper alternatives.

The HDE approval process may not adequately require substantial proof of efficacy. Nevertheless, the majority of these concerns are addressed at individual devices, not at the fundamental principles of the HDE approval process. Concerns have arisen due to the recent SAMMPRIS results regarding the Wingspan System13–15 as well as devices for other treatments such as deep brain stimulation in obsessive-compulsive disorder.32

Critics note the differing level of requirement for the introduction of drugs versus medical devices. Ironically, the FDA is being criticized at both ends of the spectrum while remaining substantially under-funded for the difficult tasks that it must oversee. On the one hand, there are concerns over the seemingly slow pace of introduction of new devices and drugs to the US medical market. Simultaneously, the FDA faces criticism for allowing drugs and devices to enter the market prematurely. Finally, some critics argue that the FDA and the Centers for Medicare and Medicaid Services (CMS), who are responsible for reimbursement for devices, play a critical role in the speed at which enrollment occurs in important randomized controlled trials based upon reimbursement patterns, and whether or not devices are reimbursed outside the context of a clinical trial. One example of this phenomenon is new acute ischemic stroke devices, which some argue are enrolling patients for randomized trials very slowly due to the fact that these devices are being reimbursed prematurely by CMS without Class 1 evidence supporting their use.33 ,34 In addition, poor coordination between the FDA and CMS with regard to physician reimbursement for FDA-approved devices (eg, foreign body retrievers with a stroke indication) further contributes to this issue. Finally, an inefficient FDA approval process is resulting in an increasing number of device manufacturers outsourcing their randomized trials to Europe or other countries in an effort to expedite accrual and trial completion. This fact may be a further indication that flaws inherent to the current FDA regulatory processes may be beginning to undermine the ability of the USA to remain competitive in the medical device industry.

The role of experience in device effectiveness and safety

An important issue that has so far been left out of the FDA clearance debate is the role of operator experience in determining device safety and effectiveness. Most new medical technologies have a learning curve wherein clinicians receive initial training once the device is released for use and then subsequently improve with experience. Logically, practitioners using new devices are more likely to cause patient harm when first learning how to use the device than after proficiency has been obtained. The ‘learning curve’ effect plays a significant role in those devices that require new advanced skill sets and, through its effect on patient outcomes, may be a substantial contributor to the early results of mandated safety and effectiveness trials for devices. Recognition of this learning curve effect by the FDA has led to important FDA-manufacturer agreements regarding training for some new devices such as the Pipeline Embolization Device, wherein clinicians must undergo course training and then be supervised by a proctor for a designated number of cases before being able to use the device independently.

An excellent example of the learning curve effect comes from the Carotid Revascularization Endarterectomy versus Stenting Trial (CREST), a randomized trial comparing carotid endarterectomy to carotid angioplasty and stenting with the primary outcome of stroke, myocardial infarction or death.35 Although carotid angioplasty and stenting was first described in 1994, significant improvements in device technology, such as distal embolic protection devices, were not widely available until the beginning of the 21st century. The fundamental goal of CREST was to compare a new and exciting technology—angioplasty and stenting (with which most clinicians having only limited experience)—with a tried and tested technique—carotid endarterectomy, a commonly-performed procedure introduced in the 1950s. CREST began enrolling patients in 2000 and finished in 2008, with 50% of total enrollment reached in 2006. The final results of the trial demonstrated statistical equivalence of stenting with endarterectomy for the primary outcome, with a higher risk of stroke in the stenting group and higher risk of myocardial infarction in the surgical group. Interestingly, the risk of major stroke or death in the stenting group was 2.5% over the period 2000 to 2005 (n=361) and <1% from 2006 to 2008 (n=770).36 The substantial reduction in serious complications during this time period for the endovascular treatment arm is most likely secondary to improvements in technique from gained operator experience at the treatment centers, but may also be partly due to changes in enrollment criteria during the study period. The learning curve effect is well illustrated in this example because of the longevity of the trial, which provided ample time for operators to develop proficiency with the devices and techniques. Had the trial been halted before 2005, the results from stenting would have appeared worse than endarterectomy because operator experience was poor and complications were high. However, we now know that carotid stenting is a safe and effective option for patients with carotid stenosis because the trial allowed ample time for operators to become proficient and for associated technologies to be developed and widely implemented, with the lower complication rate towards the end of the trial nullifying the higher rate of poor outcomes at the beginning.

Expanding the concept of a learning curve effect to the approval process makes the situation even more complicated. Assuming this process occurs universally for most new technology, early trials evaluating the effectiveness of a new device are likely to overestimate complications and underestimate effectiveness because clinicians have limited experience with the device and are more prone to error. Consequently, early studies are likely to show no difference in outcomes (or potentially worse) compared with standard of care therapies. Studies performed years after approval of a device, after clinicians have gained experience with the technology and acquired proficiency, are more likely to report lower complication rates and a better safety profile. Extrapolating this argument further, one can predict how rigorous early testing of new technology, such as in a PMA, has a bias towards device rejection. Conversely, continued post-marketing monitoring of device safety and effectiveness in the years that follow approval is likely to show improved results as time progresses. Therefore, while early tests are crucial in detecting devices that are unsafe, post-marketing monitoring may be just as important in capturing the true risks and benefits of new technology.

There certainly exists a subset of newly-approved devices with inherent flaws that will continue to show inferior results regardless of advancements in proficiency (eg, the Depuy ASR). However, there are probably devices approved by the FDA with mediocre initial results that could show improvements in safety and effectiveness with time, and eventually become a standard of care therapy with profound benefits to patients with a particular condition.

Optimizing the FDA approval process

Recent and major device failures suggest a rationale for change within the FDA device approval framework. There has been surprisingly little argument from neurointerventional physicians regarding the PMA process and the need for rigorous testing demonstrating safety and effectiveness of new high-risk Class III devices. Furthermore, although the HDE process has limitations, the rarity of the diseases for which the devices are designed to treat makes obtaining effectiveness data impractical. In addition, it seems impractical to mandate that new devices with only small iterations upon previously-approved medical devices (ie, 510(k) approved devices) show robust effectiveness and safety data prior to approval. However, it is difficult to support the notion that merely being able to argue that a new device has ‘substantial equivalence’ to a predicate is an acceptable surrogate to actually having demonstrated it through clinical studies.

A potential solution to resolving the problems with the 510(k) and HDE processes does not necessarily lie in a complete overhaul of the system but, instead, lies in the realm of post-marketing monitoring and reporting. Mandatory post-marketing reporting of outcome, safety and complication data on new 510(k) or HDE devices by clinicians using newly-approved devices would provide an additional screening process by which 510(k) devices actually demonstrate equivalence and by which HDE devices actually demonstrate effectiveness. Essentially, this solution would make 522PMS mandatory for all newly-approved Class II or III devices, with most devices requiring active surveillance studies for recording and reporting of all adverse outcomes. As an example, it could be mandated that the first 1000 devices used (or implanted) after FDA approval are monitored closely for early and long-term outcomes. This would allow physicians to treat patients with new technology that they deem to be of benefit and allow manufacturers to continue to profit from their research and development by selling devices, while simultaneously providing a validation process for new technologies that can weed out those that are causing harm. This would not only allow patients to be treated with cutting edge technology but would continue to support technological innovation. If stringent post-marketing monitoring was performed for the Depuy ASR, it is certainly possible that an unacceptably high revision rate would have been detected much earlier and the device could have been removed from the market. Additionally, with mandated ongoing data accrual, it is not unreasonable to expect that the overall field would actually benefit as subsequent iterative advances would be based on valuable newly collected data rather than on anecdotal and marketing projections.

Limitations of mandated post-marketing monitoring for HDE and 510(k) devices largely appear to be related to the additional costs of data collection and data review. Unless strictly regulated, data provided by physicians or industry to the FDA would likely also contain an inherent bias. Strict guidelines for accurate, honest and clear reporting would be an essential element of any post-marketing amendment to the approval process. An adjudication procedure in which unbiased external experts review and evaluate clinical and outcome data in the post-marketing period may be necessary to ensure the quality of the post-marketing monitoring process.

The authors of this commentary are sensitive to the numerous burdens facing healthcare providers and medical device manufacturers. The Affordable Care Act produced legislative changes to healthcare greater than many US-based doctors have experienced in their professional lifetime. As part of the funding for the Affordable Care Act, device manufacturers have had a 2.3% tax imposed on the sale of their products.37 Physicians have increased demands on their time with diminishing reimbursements. Mandating post-marketing monitoring has the potential to be perceived as an unfunded mandate. We propose it in the absence of an alternative to potential changes of a more draconian nature as could occur by a sensitive or reactive FDA. Neurointerventionists, like other medical specialists with practices closely tied to the availability, safety and effectiveness of cutting edge medical devices, should be involved in designing and refining such processes to ensure that the proposed post-marketing monitoring remains efficient and effective in capturing credible information.