news

Quality Assurance (QA) helps the Lab plan and execute work in a reliable, repeatable and sustainable manner and is a key component of stewardship. While successfully meeting our mission and key objectives is critical, the manner with which we achieve that is just as important.

Suspect/Counterfeit Items (S/CIs) are a subset of nonconforming items. Nonconforming items are items that we receive and inspect or maintain that do not meet the Lab’s specifications or requirements and/or perform as intended. 

Suspect items that don’t conform to specifications and documentation, appearance, performance, material, or other characteristics of the items may have been misrepresented by the supplier (e.g., distributor, manufacturer, subcontractor).

Counterfeit items are copied or substituted without legal right or authority that have been misrepresented by the supplier (e.g., distributor, manufacturer, subcontractor).

Do you have an example of the consequences resulting from a suspect or counterfeit item being used? How did it happen? What would have prevented it?

Using S/CIs can result in all sorts of adverse situations including death, injury, exposure, incidents/events, loss of integrity, inoperability, etc.

One example of an S/CI case involves the Bayview Project where site preparation was being made to accommodate the future construction of the Integrated Genomics Building (IGB) and BioEpic. An inspection was performed of items that were going to be used in a water distribution system that included water to be used for the Fire Protection Program. It was found that large amounts of pipe that, according to the Lab’s specifications were to be new and coated with epoxy to prevent or minimize corrosion once in the ground, looked used and a number of areas of the epoxy were flaked off and/or rusted. Had this piping been used, it would have compromised the lifecycle of the water distribution system, and if a fire would’ve occurred at a future date once the IGB was constructed, it’s very possible that the piping would’ve corroded, water output wouldn’t be available or significantly minimized and the fire department wouldn’t have been able to put the fire out.

The subcontractor segregated or tagged the piping to prevent inadvertent use of the items and replaced the piping. The new piping was inspected by the Fire Marshall and approved for installation. 

S/CI’s can be prevented or significantly minimized. Things you can do to protect yourself include:

1. If you have a critical item or if an item has a critical attribute that it must meet, clearly articulate/spell out and document the specifications that it needs to meet in formal procurement documentation such as the Request for Proposal (RFP), contract, Statement of Work (SOW), specification. Verbally communicating these specifications doesn’t obligate the supplier to meet them. Formally communicating your User specifications via procurement documentation is the only way you’ll have legal recourse if an item or service doesn’t meet your specifications or requirements.
   1. Pointing to the supplier’s or manufacturer’s advertised specifications is not the same as identifying your specific User specifications and requirements the item must meet.  Just because they say their product can do xyz and perform a certain way, doesn’t mean that it can or will. 
2. Upon receipt and before the item is put into service, inspect or test it to validate that it meets specifications, and document the inspection, test and results. This helps to ensure that nonconforming items, including S/CIs, are not inadvertently installed or put into service.

Are there any general things to look for? Who do you tell if you find something S/CI?

1. Always check the item(s) against the procurement documents that have the specifications and requirements that the item needs to meet. General things to look for include:
   1. Low quality packaging used to ship items;
   2. Low quality materials used to make items;
   3. “New” items show use or wear;
   4. Irregular markings or markings have been scraped off;
   5. Testing laboratory markings are incorrect for that particular model or part number;
   6. Serial/model numbers don’t match the manufacturer’s numbers;
   7. Safety seals are broken; and
   8. Supplier quality documentation (e.g., Certificates of Conformance, Certificates of Calibration or Analysis, Inspection or Test results) validating that the item meets User specifications and other requirements in procurement documents contain typographical or grammatical errors, inconsistencies in font size or type, missing information, etc. 
2. Tell your supervisor or work lead, and let the Institutional QA office know at QA@lbl.gov, and we can make the determination as to whether or not something is suspect or counterfeit. Our office is obligated to report S/CIs to our BSO counterparts, the Contracting Officer and the Inspector General. 
3. Hold on to any potential S/CI and do NOT return it to the supplier or discard the item until the Institutional QA Office has said it is ok to do so. The item may need to be turned over to the Institutional QA Office as evidence.

Do you foresee an uptick in the amount of S/CI items coming in from foreign countries due to increased tariffs?

The S/CI industry is a multi-trillion dollar business annually, making it the largest criminal enterprise worldwide surpassing the combined value of illegal drug and human trafficking industries.

Yes; I foresee an uptick in S/CIs due to tariffs. It’s all about “supply and demand”. Tariffs raise the prices of authentic products making it more tempting to purchase cheaper, suspect and/or counterfeit alternatives. 

Counterfeiters often operate outside of legal trade channels, allowing them to bypass tariffs and sell goods at extremely low prices. The increased demand for cheaper goods, including counterfeit goods, makes the counterfeiting business profitable. As consumers look for cheaper goods to avoid the high prices of authentic products, the demand for counterfeit items can increase. 

Tariffs, coupled with the global (as opposed to local) supply chain, make it more likely that S/CIs will enter into the supply chain. This may impact the ability of items to meet User specifications and perform as intended. 

Now, more than ever, it will be important to look at the price of an item before you purchase it (if it’s too good to be true, it probably is) and inspect or test them prior to installation and operation to ensure that they meet User specifications and other requirements, and perform as intended. This will help to prevent inadvertent use of non-pedigreed or substandard items in Lab systems and subsystems.

Where do I find guidance on S/CI for anything QA related?

Additional guidance and resources can be located in the Lab’s QA Program Description (QAPD) and on the Lab’s S/CI website.

About Melanie Gravois: Melanie has worked in the Institutional Contractor Assurance Office (ICAO), formerly the Institutional Assurance & Integrity Office (OIAI). She was hired to help build the Institutional Assurance office as we currently know it today and to design, develop and roll out the Issues Management Program. After that was accomplished, she became the Institutional Quality Assurance Program (QAP) Manager and has been serving in this role for the last 15 years. 

Prior to coming to the Lab, Melanie spent 7.5 years at a DOE nuclear facility, the Waste Isolation Pilot Plant (WIPP) and Central Characterization Project (CCP) headquarters, in New Mexico as the Institutional Site QA Officer. There, she oversaw the QA activities associated with identifying, shipping and disposing of radiological waste generated at Federal nuclear facilities from the Cold War era.

At the Lab, she manages the QAP to ensure work is performed in a reliable, repeatable, and sustainable manner. She improves efficiency by preventing rework through establishing policies, procedures, and training, partnering with stakeholders to solve problems and ensuring consistent QA application. For Melanie, QA allows all of us to “work smarter, NOT harder”, and is a fundamental approach that allows us to focus on those things that are most important.

As an LBNL employee, how do I know I’m performing work within the requirements of the Prime Contract C31?

Lab employees have various understandings of what the Prime Contract is and how it applies to them. The Prime Contract 31 (Prime Contract or Contract) is the Contract between the DOE and UC which establishes the requirements that LBNL must meet. 

 Just like Lab employees must perform their work within applicable federal and state regulations, employees must also perform their work, so the Lab is in compliance with the Prime Contract requirements.  However, that doesn’t mean that each employee needs to memorize the Prime Contract in order to ensure compliance with each of the Prime Contract requirements.  So how does LBNL ensure that the Prime Contract requirements are met?  There are many methods used to ensure compliance with the Prime Contract requirements, such as the institutional assessment process. However, assurance that we meet the Prime Contract requirements begins before a new or revised requirement is implemented into the Prime Contract.  

ICAO’s Contract Assurance program begins working with division leads, assurance leads, and subject matter experts from relevant divisions before a new or revised requirement is implemented into the Prime Contract to ensure that a complete impact analysis is completed to adequately assess the impact to the Lab, which may include, but is not limited to the cost, resource requirements, and the time required to implement the requirement.  Once the requirement is implemented into the Contract an implementation analysis is performed to determine the appropriate implementation actions that must be performed in order to ensure compliance with the requirement by the Lab.

 The implementation efforts may include actions such as creating or revising training programs, new or upgraded IT systems to accommodate actions required by the requirements, new or revised institutional documentation, or hiring additional employees or contractors to perform the work required by the requirement.  This is not a complete list of implementation actions that may have to be completed but there are some common actions.  These actions are how we ensure we are meeting the Prime Contract requirements and why each employee is not responsible for memorizing the Contract requirements.  Prime Contract requirements flow down to our institutional documentation such as LBNL policies and procedures.  By ensuring that our policies and procedures are in accordance with the Prime Contract and by providing adequate training to employees on these policies and procedures, employees can ensure they are following the Prime Contract requirements by completing work in accordance with the LBNL policies and procedures. 

What does assurance mean to you? Assurance is a vital concept in quality assurance and contractor assurance. In simple terms, assurance is about making sure that a product, service, or system meets certain standards, requirements, or regulations. It’s a systematic process that involves planning, monitoring, and evaluation to ensure that everything is working as it should be. The goal is to provide confidence that the product or service is reliable, safe, and of high quality. In the context of Berkeley Lab, we could say that we are working to provide confidence that all of the work we do at the Lab is performed in a safe, compliant, and high-quality fashion.

The benefits of assurance are numerous. It helps us improve quality, increase confidence, and reduce risk. It also ensures compliance with relevant regulations and standards, which can help avoid costly mistakes or reputational damage. By prioritizing assurance, we demonstrate our commitment to quality and excellence, which can have a positive impact on our reputation and the success of our programs.

-RLV

You can now easily access these lessons learned by following links in the Project Management Lessons Learned Guide and document repository  to both the Institutional Lessons Learned and Best Practices (LLBP) Database and DOE OPEXshare. DOE OPEXshare has also recently expanded its inventory of project management lessons learned from other DOE labs.

Have your own valuable operating experience to share? Contact lessonslearned@lbl.gov for assistance with writing or entering new lessons learned or best practices into the LLBP Database or DOE OPEXshare.

If you plan to revise an existing policy, please remember to contact the Requirements Management Program Manager first to obtain the latest Word version of the policy. All revisions should be made using either Track Changes mode in Word or Suggesting mode if the document is converted to a Google Doc. To initiate the revision process, please submit your request via the Policy Change Request Form. Keep in mind that most revision requests also require additional forms such as the Significance Rating Determination and Approvals. For a detailed overview of the procedure, please refer to the Guidance for Updating the RPM. To request a copy of a policy or if you need any assistance, please reach out to requirementsmgmt@lbl.gov.

A Root Cause Analysis is a relatively loosely used term in industry, but here at LBNL, we only call an analysis a Root Cause Analysis if it meets a certain level of rigor, primarily based on the method by which it is accomplished. 

Root causes are the underlying or basic cause of an issue that can reasonably be identified and management has the control to fix, and when fixed, will preclude recurrence or significantly reduce the likelihood of recurrence of the same or similar issue. Root causes are typically discovered through deeper analysis and are one level beyond apparent causes.

The key distinction between RCA and ACA is that an ACA does not require the use of formal analytical methodologies—such as barrier analysis, change analysis, or error precursor analysis—as outlined in LBNL’s Issues Management Manual (PUB-5519). These formal methodologies are consistent with those used in Department of Energy (DOE) causal investigations. An ACA is a more basic analytical approach used to identify the evident or likely reasons behind an incident or finding.

Because ACAs are less methodologically rigorous, they require fewer resources and less time for analysis and fact-finding compared to RCAs. Just like many programs at the Lab, LBNL applies a risk-graded approach and only requires RCAs for high-risk issues and ACAs for medium-risk issues. There are a few other process requirements for the RCA process, which are defined in detail in PUB-5519. 

Risk and issue are terms that are frequently used interchangeably.  While they are interrelated and share some common characteristics – they are not the same thing.

 A risk refers to a potential problem or event that may occur in the future, which could have an impact on an organization. Risks are uncertain and may or may not materialize. They are often identified in advance, allowing for proactive measures to be taken to mitigate them. For example, we may identify a risk of supply chain disruptions due to natural disasters or economic changes.

 An issue is something that has already occurred. Issues are actual problems that require attention and resolution. Unlike risks, issues are certain and have a direct impact on an organization. For instance, a software team may encounter an issue with a bug that is causing system crashes, requiring immediate fixing. Understanding the difference between risks and issues is crucial for effective risk management and issue resolution, enabling proactive and reactive measures to be taken to minimize their impact.

Institutional, or enterprise, risk management, is a critical function that provides the framework to provide the University of California, Berkeley Lab Senior Management, and DOE with an integrated view of the significant risks facing the Laboratory and to help assure that these risks are managed effectively.

Of course, we all manage risks every day as part of our day-to-day work. This more tactical risk management is critical to Lab success. Institutional risk management allows us to develop a holistic, portfolio view of significant risks to our achieving the Lab’s important objectives.

Institutional risk management at the Lab is performed in accordance with the Risk Management Description for Institutional Risks.  As we work to refine and improve our approach to risk management, that program document is being revised, and a newly formed Institutional Risk Management Committee will be appointed to oversee and manage enterprise risks.

More to come soon.

-Robbie Leftwich-Vann

The CATS Database is the official LBNL Issues and Corrective Action tracking system. The database enables LBNL employees to document, track, and formally close Issues and their associated corrective actions. The database is the central repository for issues management information, which includes retrieval and reporting capabilities to gauge the implementation and effectiveness of corrective actions and to monitor and trend adverse conditions. 

The CATS Database has three core functions: 

• Issues and Corrective Action Management
  The database supports the documentation workflow of the issues management process. The workflow includes entry, review and approval, tracking and closure of issues, and associated corrective action(s) based on risk severity levels.

• Records Management / Data Warehouse
  The database supports electronic documentation and retrieval of issue, corrective action and objective evidence data. This includes the capability to upload multiple documents and file types and URLs to demonstrate issue resolution/corrective action implementation.

• Ongoing Performance Analysis
  The database supports trending and analysis of issues, with various search and reporting capabilities. This aids in monitoring, analyzing, and identifying recurring issues/trends and areas of improvement for quality, efficiency and reliability. 

Who can enter Issues and Corrective Actions in CATS?

Anyone with a Berkeley Lab Identity password (LDAP) can access CATS, and enter Issues and Corrective Actions in it.

The following are requirements for CATS Database documentation, tracking and monitoring: 

• Issues and risks pertaining to injury, damage, loss, noncompliance, safety, or operational deficiencies (Refer to section 1.0 of the Issues Management program manual) and associated corrective actions, regardless of risk level, are entered into the CATS Database. The entry of observations or recommendations in the database is at management’s discretion. Personnel performance (human resources) issues and associated corrective actions are not entered in the CATS Database. 
• Documenting immediately corrected issues/fixed upon identification (“on-the-spot”) in the CATS Database is recommended to demonstrate assurance of issues management. However, the decision to document these issues in the database is at management’s discretion based on risk severity and administrative burden. 
• An issue should be entered into the CATS database as soon as it is characterized and compensatory and/or corrective actions are identified. For high and medium risk issues, issues should be entered in the CATS Database following the CAP development/corrective action development process to enable a more comprehensive and collaborative approach to managing and resolving the issue.

Exceptions:

Immediately corrected issues, service requests, such as Information Technology (IT) help desk tickets and Facilities general, preventive, and corrective maintenance work requests are not entered in the CATS Database. Similarly, ethical/integrity, health issues, employee concerns, traffic incidents, security breaches, and ergonomics evaluation issues are not required to be entered into the CATS Database.

Training & help:

Anyone needing help with learning more about CATS can reach out at Issues-Management@lbl.gov

The Institutional Contractor Assurance Office (ICAO) has developed the Lab’s first Institutional Approved Supplier List (ASL), which identifies items, services and suppliers that have been evaluated by the Institutional Quality Assurance (QA) office; It is designed to help end users plan procurements and buyers to process purchase requisitions (PRs) more efficiently by leveraging prior QA Reviews; and identifying items, services and suppliers that need a QA Review.  A pilot of the automated Institutional ASL will take place during FY25 1st Quarter to identify improvement opportunities.

The Institutional ASL includes a number of process efficiencies as well as provides a “One-Stop-Shop” for data warehousing, analytics and reporting. Key improvements:

• Ensure all data is consistently entered;
• Allow faster processing of PRs by:
  • Reducing the number of emails between the buyers, ICAO, end users and the EHS electrical inspection group through automatic notifications;
  • Allowing the End User Risk Assessment Questionnaire to be tailored based on how the end user responds to questions; and
  • Leveraging prior QA reviews so that future purchases of items/services from the same supplier for the same end use won’t need to be reviewed again; and 
  • Ensuring current data is reflected on the ASL in real time.

Additionally, the Institutional ASL and will be published in real-time and available to Lab community 24/7 on the QA Website and Quick Links.