James (Jim) Dent, LSSBB, DTMx2 In most of my past companies I worked for, QMS changes were treated the same as CAPAs whether they were Corrective or Preventive. The final step before closure was verifying the effectiveness of the change - which usually took a while because we defined effectiveness as also including sustainability of the change; therefore a change may be monitored and periodically audited for up to 6 months before closing. The monitoring and auditing were documented and were kept as part of the CA or PA records.

Thanks Jim. Six months seems reasonable, as for longer periods, other KPI's plus Mgmt review will carry forward the watch on effectiveness. Any other views on this?

Usually, we tracked effectiveness through our CAPA or management review. Depending on the changes, effectiveness can varied; some changes only required couple of months some changes required longer. If the effectiveness depending on customer using and if you have a large customer base, then it will take less time to get enough statistical data to confirm your changes.

Ee Bin Liew Hi,

if you split change control into QMS (process and administrative) change, and (product and process) engineering change.. then effectiveness is being handled differently. All of this needs to be documented.

QMS (process and administrative) change is normally associated with the effectiveness of its training of that document change. something in that document changes, we as users need to understand them and depending on what is actually changed the effectiveness check could be immediate and objective (everyone uses this form from this point onwards etc.) or it could be long drawn (e.g. procedure of the way of doing something has changed, hence we need to buddy up or have a monitoring period or spot audits for verification after some time etc.)

(product and process) engineering change would mean longer term effectiveness checks such as monitoring for complaints (sounds a bit negative but really, no news is good news!), customer satisfaction, service calls, consumption of consumables etc. or in process engineering changes, that would have a direct (and more immediate) impact to the cycle time, defect rates, or in general process control (which is more long drawn), where you have statistics to help you.

true that it's very convenient for management review to be the platform for effectiveness check (since everyone that needs to be present is present - in theory) but truly the measure depends on the change itself (as explained above and the MR just looks at all these data as a snapshot and really should not be taken as an effectiveness check measure, in my opinion.

Cheers,
Ee Bin

Ee Bin - thanks for that insight, which makes perfect sense. Looking more tactically now, do you leave a CC file open until agreed effectiveness checks are in or do you open e.g. a separate CAPA to track this? Or to put it another way, how do you differentiate via KPI's between the different steps? Risk is that CC''s could have a very long cycle time. Thoughts? Mike M

Ee Bin Liew Hi Mike - what is the real impact of a long-drawn CC if the position is defensible? What regulators want are defensible scenarios, right? One thing about sending such stuff over the wall to CAPAs or to just close it and 'monitor' it in some other way .. runs the risk of losing traceability during the transfer. Another thing is that if the CCs are indeed too long, they will either be caught in any change review board or MR, or everyone says .. "oh, that one, we have the justifications for it'.

the key here is to minimise time extensions - that's always one of the least defensible things in an inspection. We can make a "mistake" once (wrong estimation of the duration for effectiveness check) but not all the time. So make it a team effort to get the timing as close to reality as possible.

What regulators want (and what the company needs) - if put in simple words - is for people to be on top of things and stay motivated to get them done properly

hope it helps, feel free to shoot more questions and we'll respond.

Cheers,
Ee Bin

Hi. Id recommend the following:
1. Code your changes by type of change eg. Engineering, material, document, test method.
2. Identify on the documentation for each change whether an effectiveness check is required.
3. Record the success criteria for the change to be considered effective.
Close the change when the work is complete
1. At either a change review or management review list the changes that required egfectiveness checks and have been closed in the past year. List the success criteria next to each change and record the actual performance next to this.
2. Flag up the ones that have not met the success criteria and either reopen the change, or open a new change or capa.
Cheers
Pete

Rajani Kumar Sindavalam, PMP® Based on my personal work experience, I feel that you are getting confused between the ‘Change Control’ and a ‘CAPA’. Change control management talks about how best to handle changes to the approved configuration (Considering all the right cross functions, Doing a proper impact assessment, Identifying all the necessary work, execution – which includes updates to DMR documents, verification, validation etc., implementation of the changes to production and closing the change control). CAPAs follow their own workflow. They usually follow the DMAIC (Define-Measure-Analyze- Improve-Control). When you identify that there are systemic issues (or) Procedural issues, we do create Change controls to address the root causes identified as part of CAPA. But, the effectiveness check must happen as part of CAPA workflow. You can very well close the change package once the intended work for that package is complete. In general, change controls don’t really involve effectiveness checks.