FDA’s Quality Metrics Initiative – From Compliance to Performance

February 17, 2015

By Mark Land, VP Operations and Regulatory Affairs at Boiron

The 2012 Food Drug Administration Safety and Innovation Act (FDASIA) gave FDA new tools to encourage high quality manufacturing of pharmaceuticals and enhanced the agency’s ability to respond to, prevent and mitigate the risk of drug shortages. Section 705 of the Act specifically requires FDA to implement a risk-based inspection regimen, while section 706 allows FDA to request, and firms to provide, records in advance of or in lieu of inspections1. Early in 2013, FDA reached out to industry and the public for input into the use of quality metrics as a means to assist in the evaluation of product manufacturing quality aimed at predicting and preventing drug shortage issues and as a tool for use in the risk based inspectional model2. The metrics program is an extrapolation of the aims of FDASIA Title VII attempting to refocus industry from a compliance only mentality towards a quality performance management approach.   The Quality Metrics initiative has addressed obstacles and gained consensus among stakeholders. There are quality and business implications for AAHP members.

Product quality should be the inherent goal of medical products companies. Historically, the standard of quality has been compliance. Highly publicized recalls, product bans, and drug shortages have forced companies and the FDA to rethink quality. Janet Woodcock of CDER has said that industry must move from compliance to performance in pursuit of quality3. The Quality Metrics Initiative is essential, allowing FDA to make risk based decisions, ranging from inspection scheduling to assessing the potential for drug shortages because of product quality issues. The FDA wants objective measures of product quality, site operations, and site systems performance. FDA’s new Office of Pharmaceutical Quality in CDER will likely support these efforts by collecting performance metrics and conducting trend analysis to identify both compliant operations and outliers4. FDA has invested heavily in public comment and consensus building. Trade association proposals and FDA’s considerations were the subject of written public comment and in person meetings.

In a Feb. 12, 2013 Federal Register notice, FDA solicited information, from manufacturers and industry trade associations on the variety of industry standards and quality assurance approaches related to Quality Metrics5. The FDA received more than 150 responses including more than 600 potential metrics6. The responses indicate that pharmaceutical and biotech companies regularly monitor error rates, for example, to identify non-conformances, discrepancies, and deviations and many more. Industry trade groups including; ISPE, BIO, PhRMA, PDA and the GphA provided feedback. Industry and FDA arrived at consensus objectives for the program and consensus stakeholder metrics during a 2 day meeting in May of 20147. FDA continues to review findings from outreach efforts and the agency hopes to draft guidance soon aimed at outlining a set of metrics indicative of a company’s ability to produce high quality products consistently  Final guidance will follow a one year comment and revision period. Enforcement will begin one year after publication of the final guidance.

Development of meaningful metrics presents a number of challenges. The old adage that you get what you measure may be true and FDA has worked hard with stakeholders to address the risks of unintended consequences. Metrics are considered leading or lagging indicators of product quality. Leading metrics tend to predict future quality while lagging indicators report on historic quality failures. FDA seeks metrics that are meaningful, allowing insight into a manufacturers systems and consistent with the continuous improvement aims of ICH Q108. Metrics may also be viewed as high risk for unintended consequences or deceptive reporting. Balancing metrics are included to avoid risks of that nature. For example, if the lot rejection rate presented motivation for excessive rework, the batch “right first time rate” is capable of balancing that consequence9.

FDA has published a list of Metrics of Potential Interest10

Lot Acceptance Rate = 1 – (the number of lots rejected by the establishment in a year divided by the number of lots attempted by the same establishment in the same year).

Right First Time Rate = 1 – (the number of lots with at least one deviation by the establishment in a year divided by the number of lots attempted by the same establishment in the same year).

Product Quality Complaint Rate – the number of complaints received by the manufacturer of the product concerning any actual or potential failure of a unit of drug product to meet any of its specifications, divided by the total number of lots released by the manufacturer of the product in the same year.

Invalidated Out-of-Specification (OOS) Rate – the number of OOS test results invalidated by the establishment, or contracted establishment in a year divided by the total number of tests performed by the establishment in the same year.

Annual Product Review (APR) on Time Rate – the number of APRS generated within 30 days of annual due date at the establishment divided by the number of products produced at the establishment.

Management Engagement – the most senior manager that signed each annual product review, reported as one of the following: (1) none specified, (2) line quality manager, (3) site operations manager, or (4) corporate executive manager.

– Performance Question 1 – A “yes” or “no” value of whether the establishment calculated a process capability or performance index for each critical quality attribute as part of that product’s APR.

– Performance Question 2 – A “yes” or “no” value of whether the establishment has policy requiring a CAPA at some lower process capability or performance index.

– Performance Question 3 – If “yes” to question 2 – what is the process capability or performance index that triggers a CAPA. If “no” to question 2 – please do not respond.

Corrective and Preventative Action (CAPA) Rate – the number of corrective or preventative actions that were initiated due to an APR, divided by the total number of APRs generated.

One of the more challenging areas for both industry and FDA involve metrics aimed at evaluating a site’s “quality culture”. Management engagement and performance are examples of Quality Culture measures. Metrics can be further stratified as product or site level. Product level metrics are often a requirements of Post Marketing reporting for NDA drug products. For OTC’s or generics this information may only be available at the site level. FDA is mindful of legal and practical limitations of its ability to collect information across innovator, generic and OTC product classes. FDA’s metrics program is likely to evolve over time but will play a vital role in inspection scheduling and establishing a site quality grade.

Industry is encouraged to become aware of the details of the Quality Metrics Initiatives program and think deeply about the implications for your manufacturing sites. Information responsive to the metrics program may reside in multiple systems within the organization or may not exist at all. Consider services provided by contract manufacturers and external laboratories as well. Developing mechanisms to collect, report and compare quality performance is an obvious opportunity to comply. Embracing the managerial promise of the compliance to performance incentive of the metrics program is a way to fast track implementation and deepen your organization’s overall resilience and effectiveness. Focusing attention on the effectiveness of your quality and manufacturing systems performance in addition to complying with regulations allows the manufacturer to pinpoint challenges and implement durable fixes. FDA plans to summarize metrics information for each site into a Quality Score. The Score has implications beyond inspection frequency.

FDA will issue overall results for industry, and also provide each manufacturer with a confidential score that then can be compared to industry averages. This will allow the individual to see their specific score and also how it fits overall results11. A company would know their individual score, and their decile ranking within the various ranges, but your next door neighbor would not know your score12. Retail buyers could than ask a company for its score and its decile ranking. It would then be up the company to release their score or not. Readers are encouraged to develop a collaborative discussion within quality and business departments to realize compliance, managerial and business development opportunities embedded within this program.

  1. Public Law 112–144 Food and Drug Administration Safety and Innovation Act http://www.fda.gov/RegulatoryInformation/Legislation/FederalFoodDrugandCosmeticActFDCAct/SignificantAmendmentstotheFDCAct/FDASIA/ucm20027187.htm
  2. Comments available at the Federal eRulemaking Portal: www.regulations.gov, Docket No. FDA-2013-N-0124.
  3. Ellen Leinfuss, Life Science Leader April 2, 2014The FDA’s Focus on Metrics, Performance and Quality http://www.lifescienceleader.com/doc/the-fda-s-focus-on-metrics-performance-and-quality-0001
  4. Wechsler, J. FDA Seeks Metrics to Define Drug Quality Manufacturing standards are considered key to preventing drug recalls and shortages. Pharmaceutical technology Oct. 2, 2013 http://www.pharmtech.com/fda-seeks-metrics-define-drug-quality-1?rel=canonical.
  5. Federal Register, Vol. 78, No. 29, p 9928-9929, Feb. 12, 2013
  6. Wesdyk, R. USFDA CDER/OSP Quality Metrics Update Stakeholder Feedback, Goals, and Gaps May, 1 2014 http://www.brookings.edu/~/media/events/2014/5/measuring%20pharma%20quality/quality_metrics_final_for_posting.pdf
  7. Mullins Theresa. United States Food and Drug Administration CDER’s Quality Metrics Initiative Program Update http://www.pqri.org/workshops/EVP2014/Mullin.pdf
  8. Parenteral Drug Association Points to Consider: Pharmaceutical Quality Metrics 2013 http://www.pda.org/docs/default-source/website-document-library/general/pda-pharmaceutical-quality-metrics
  9. Brookings Institute Measuring Pharmaceutical Quality through Manufacturing Metrics and Risk-Based Assessment May 1 & 2, 2014 http://www.brookings.edu/~/media/events/2014/5/measuring%20pharma%20quality/quality_metrics_final_for_posting.pdf
  10. Mullins Theresa. United States Food and Drug Administration CDER’s Quality Metrics Initiative Program Update http://www.pqri.org/workshops/EVP2014/Mullin.pdf
  11. Wechsler, J. FDA Readies Quality Metrics Measures Pharmaceutical Technology Oct 06, 2014 http://www.pharmtech.com/fda-readies-quality-metrics-measures
  12. Nielsen Hobbs, M. FDA GMP Quality Metrics Will Be Industry’s ‘SAT Score’ The Tan Sheet November 3 2014. https://www.pharmamedtechbi.com/publications/the-tan-sheet/22/44/fda-gmp-quality-metrics-will-be-industrys-sat-score

 

Posted in Articles, Compliance, Regulatory