What exactly is an Effectiveness Review (ER)?
Think of it as the final “Evaluate” phase of our IMP. It’s a deep dive, not just a check-the-box exercise, to confirm a corrective action truly fixed a problem at its source.
An effective ER is essential for validating that a solution genuinely addresses the underlying root cause(s) of an issue, moving beyond mere surface-level symptoms. It serves to ensure the fix is durable and sustainable, preventing recurrence of the same or similar problems. Furthermore, the review confirms that the solution did not introduce any new, unintended consequences or operational risks. Ultimately, the ER demonstrates verifiable performance improvement, solidifying our commitment as a learning organization.
When is an ER required?
The IMP uses a risk-based approach to focus our assurance efforts where they matter most. The need for an ER depends on the issue’s severity: It is Required for all corrective actions addressing high-risk issues, and Recommended (strongly encouraged) for medium-risk issues. This isn’t an immediate sign-off. ERs are typically performed 6 to 12 months after the corrective action is implemented. This essential “run-time” allows new processes to become fully institutionalized, giving us a clear picture of their long-term effectiveness.
How is an ER actually done?
To guarantee an unbiased assessment, an ER is conducted by a chartered team of objective individuals who were not involved in implementing the original fix. They use a variety of methods to get a complete picture, including: Reviewing key documents like the original Root Cause Analysis and updated procedures; Observing work in the field to see if new processes are being followed correctly in a real-world setting; Analyzing performance metrics to provide tangible proof of improvement, such as a downward trend in adverse events; and Interviewing personnel to confirm staff understanding, buy-in, and compliance with the changes.
What happens after the review?
The team compiles its findings into a formal report, concluding whether the fix was effective, partially effective, or ineffective. If a corrective action is found to be only partially effective or ineffective, new or revised corrective actions must be developed. This ensures we don’t leave a problem half-solved and truly close the loop on the issue.
Any examples of recent Effectiveness Reviews at LBNL?
A: Absolutely. Over the past five years, teams across the Lab have conducted more than 30 Effectiveness Reviews, applying them to a wide range of situations. These reviews fall into two main categories:
Strengthening Lab-wide Systems: ERs are used to validate major improvements in our foundational programs. For example, reviews have been performed to confirm that corrective actions successfully addressed gaps in our Chemical Management Program, Personnel Security Program, and key financial processes like invoice certification. This ensures our core operational systems are robust and working as intended.
Ensuring We Learn from Incidents: After serious incidents occur, ERs are a critical final step to confirm that the changes we made are effective. Reviews have been completed for corrective actions following events such as the telehandler incident, a serious hand injury, and a chemical container burst. These reviews make sure that our actions were effective and that safety lessons are truly embedded in our work practices to prevent recurrence.
These examples show how versatile and vital the ER process is. It’s a tool we use not only to respond to incidents but also to proactively ensure the health and integrity of our most important programs.
Can I get some help to perform an ER?
ICAO’s Issues Management Program has put together some templates and tools in a Getting Started with an Effectiveness Review toolkit. We also offer a training session on a need basis if any Effectiveness Review is to be conducted. If you have any questions, please feel free to reach out to LBNL’s Issues Management Program Manager, Manjot Grewal, at Issues-Management@lbl.gov.
