APPLYING CRITICAL THINKING TO AUDIT TRAIL REVIEW

There continues to be uncertainty regarding the expectations of and approach to audit trail review within the industry.

The April / May 2015 edition of ISPE Pharmaceutical Engineering included the article “A Risk Based Approach to Audit trail Review” by Randy Perez, Chris Reid, and Sion Wyn, which summarised the ISPE GAMP position relating to audit trail review.

Key points of the article included:

• Audit trails have a specific purpose in providing traceability of GxP record creation, modification, and deletion
• Audit trails are only one tool in the armoury for ensuring data integrity
• Audit trails provide most value when reviewed in process i.e., in conjunction with the GxP record they relate to
• Audit trail reviews should be based on a documented risk assessment, with greatest focus on records that have the highest impact to patient safety and product quality

A risk-based approach is essential to ensuring maximum value from the audit trail review and for focussing key resources where needed. For less critical GxP records, audit trail review is a useful tool in support of quality investigations.

Regulations and Regulatory guidance support this risk-based approach:

• "Routine data review should include a documented audit trail review where this is determined by a risk assessment." MHRA GXP Data Integrity Guidance and Definitions; Revision 1: March 2018

• "If the review frequency for the data is specified in CGMP regulations, adhere to that frequency for the audit trail review."

• "If the review frequency for the data is not specified in CGMP regulations, you should determine the review frequency for the audit trail using knowledge of your processes and risk assessment tools. The risk assessment should include evaluation of data criticality, control mechanisms, and impact on product quality" FDA Data Integrity and Compliance with Drug CGMP Q&A Guidance for Industry Dec 2018

• "The frequency, roles and responsibilities of audit trail review should be based on a risk assessment according to the GMP/GDP relevant value of the data recorded in the computerised system." PICS Good Practice for Data Management and Integrity in Regulated GMP/GDP Environments Jul 2021

• "Based upon a documented and justified risk assessment – the frequency, roles and responsibilities, and the approach used to review the various types of meaningful metadata, such as audit trails. " WHO Technical Report Series 996 Annex 5 Guidance on good data and record management practices, 2016

A risk-based approach should employ critical thinking in determining whether the audit trail should be reviewed in process or as a tool supporting investigations. The risk-based approach should consider:

Record Criticality: The impact on patient safety in the event that the record is incorrect or falsified. Where there is no direct impact on patient safety or product quality, audit trails are most useful in supporting quality investigations. For those records that have a direct impact of patient safety or product quality, an in-process review of the audit trail may be most beneficial.

Probability: The probability of a GxP record being inadvertently or consciously changed is dependent of a number of factors. Such factors include:

• Role based security that ensures only authorised people can change a record
• Date entry limit check that ensures that data can only be entered with permissible limits
• File formats that do not permit a change
• Segregation of duties that ensure that the same person cannot change a method, configuration or control parameters as the person operating the process.
• Segregation of duties that ensure the person operating the process and recording the data is not the same person reviewing the final data
• Knowing that data reviews are taking place to ensure that the correct methods or control parameters are used
• System controls that ensure that data is locked once reviewed and approved

Detection: The obvious form of detection is review of the audit trail; however, the audit trail review is not the only mechanism for detecting changes. Data reviews will also determine:

• When records are created
• Data that is out of limits
• Use of approved methods / recipes e.g., when contained in the final results
• Second person checking of critical data

It should also be noted, that changes to data are more often due to human error rather than falsification of data. As such, it is not practical to investigate every data change. Critical thinking should be employed in conjunction with changes, e.g., a change to a record sometime after the initial data entry may (but not always) be more indicative of falsification.

Consider what is in the audit trail that can tell you there is an indication of suspicious behaviour, not just individual data changes. Periodic review of audit trails will seldom provide valuable information about specific changes. For data changes to be in an audit trail, the change must have been made by an authorised person and must be within permitted limits to be accepted. Where periodic reviews are conducted, they should focus on behavioural trends e.g., batch recipe changed before the start of a batch and reverted at the end of a batch, or alarm limits were modified during a batch.

For those of you that are ISPE members, we would recommend revisiting the Pharmaceutical Engineering article from March / April 2015 as it remains valid today.