The curse of the med-tech QMS…
…and where should we be heading?
The signature headache
For anyone who’s ever worked in medical device development, the numbers 13485 will probably provoke a reaction of some sort, whether it’s anxiety from thinking about preparing for an audit or just dismay at the recollection of all the hours wasted on collecting approval signatures on documents that were then updated a week later.
So why does the ISO 13485 standard make us jump through all the hoops? Who decided that we need to sign all our documents so many times?
It might come as a surprise to lot of people in med-tech to hear that the standard doesn’t actually mention signatures once. Even the Medical Device Regulation (MDR) in Europe only obliges the manufacturer to sign a couple of things: the CE certificate of conformity (Annex IV) and the formal application and contract with the notified body for assessment (Annex VII).
Q: So, how did we get to where we are now?
A: Legacy.
But before I explain what I mean, a little detour.
Documents and signatures
The key words for document management are ‘review’ and ‘approval’. The 13485 standard expects that any document that needs to be generated during the execution of a quality management system (QMS) will be subjected to review and approval (§4.2.4). Implicit in that expectation is being able to be confident that the document hasn’t been altered after each of the steps.
It is also expected that the persons responsible for generating, reviewing and approving documents are sufficiently independent (§5.5.1).
Sidenote: There’s a distinction between ‘documents’ and ‘records’ in the standard, but I’m using the word document loosely. For the time being, just think of what a layperson would mean when they say document, but I’ll get back to this.
So, how do you indelibly record that the people involved in the various document stages are identifiable, independent and that the steps (authoring, reviewing, approving) were done in the right order?
Simple, right? You ask them to sign and date the paperwork (and hopefully you keep specimen signatures for reference).
You probably even make their life easy by having a nice little table on the front page of the document which is pre-filled with their name and title, so all they have to do is quickly scribble with their pen.
Unfortunately, the act of physically signing a document has taken on greater importance than the authors of the standard ever intended. We should be asking people to apply their intellectual power to the process of assessing the quality of their colleagues’ work, but instead we’re usually asking them to focus on remembering to write the date in a specific format…
“Is it 11–06–2020 or 2020–06–11? I can never remember…”
The idea behind the ISO 13485 standard is to help an organisation to “demonstrate its ability to provide medical devices and related services that consistently meet customer and applicable regulatory requirements”.
But the customer doesn’t care about the document signature. In a lean process, it’s muda (“futility, uselessness, wastefulness”). The customer cares about well-made products that are safe and effective — and that’s the real intent behind the standard, I contend.
Solving the wrong problem?
At this point, I’m waiting for the vendors of electronic QMS software to jump in and say, “Hey, we’ve solved that headache, all you need to do is migrate to our tool, and the document review and approval process is completely digital”.
Well yes, except the problem is not the concept of signing documents that needs to change, it’s the concept of documents that needs to change.
What is a document?
Back to the topic of legacy. What do we mean by a document? Well it’s a bunch of words and maybe pictures, that hopefully conveys meaning to the reader, right? We’ve all seen loads of documents: a product requirement specification, a risk analysis document, a CAPA report, a bill of materials etc. Move along, nothing to see here…
Except a document is actually only an extract, a fragment, a snippet of something much bigger. I’ll try to explain.
The network effect
Consider a risk analysis. For most places I’ve ever worked, the risk analysis has taken on a standard form: there’s causes, hazardous situations, harms, probability and severity estimates and hopefully some mitigations, all wrapped up in a Word table, an Excel sheet, or wherever. But what happens when you look a bit more closely at the contents of those columns of data?
If you’re lucky, the mitigations are linked to specific product requirements, probably with ID numbers or the like. But what about the other columns?
Realistically, the ‘Cause’ probably falls into one of three categories: a) product failure b) process failure or c) user/environment failure.
Examples:
a) “Resistor R23 becomes short-circuit”, “Software unit ‘get_last_result’ returns an invalid number”, “The aluminium mounting bracket snaps”
b) “The mains voltage tester fails next calibration”, “Assembly technician uses Loctite 243 instead of 242”
c) “The user forgets to toggle the battery backup switch”, “The device is dropped”
So, wouldn’t it be nice if these causes were somehow linked to the items involved, i.e. the part number for resistor R23, the serial number of the mains voltage tester, the user interface specification for the battery backup switch.
Similarly, the ‘Harm’ column is a list of all the ways an operator or patient could suffer. Shouldn’t the harms make reference to the various user populations identified in the intended use and/or usability file?
Connections
When you apply this same thinking to other documents, you discover the same thing:
Everything is connected
You notice that the requirements traceability matrix contains requirement statements, references to the component(s) that are implementing the requirements, pointers to a verification protocol and report…
…the verification report lists the serial number of the item that was tested, the name of the employee who executed the test and the multimeter she used…
…somewhere there’s a training record for the employee to show that she read the SOP for conducting verification tests, and somewhere else there’s a calibration certificate for the multimeter…
…the serial number of the DUT is stored in a device history record, which points to a specific revision of the bill of materials…
…the b.o.m. is just a list of part numbers, pointing to other part numbers, and maybe it references datasheets and suppliers…
…for each supplier, somewhere in the QMS paperwork is a supplier quality agreement and a supplier evaluation, maybe an audit report too…
Everything is connected
Rewind — what is a document?
We’ve been brought up to believe that the (approved) document is the truth. That it contains the meaning that we are usually searching for. If we want to know something, we tend to ask ourselves, “in which document will I find the answer?”.
The reality is that a document is only ever an extract, a fragment, a snippet of something bigger.
What we should be asking ourselves is, “Where do I want to start my journey in the massive network of connected information, and what do I hope to find out?”
Bill of materials? —
Start with a top-level assembly part number and (recursively) explore the complete tree of parts connected via the relationship ‘is composed of’.
Risk analysis? —
Find a component, explore every failure mode, follow the cascade of dependent events until you reach an external user interface failure. Track from this failure to a user harm. Repeat for all components.
Find a manufacturing process step, explore every failure mode, follow the resultant cascade as above…
Find a user scenario, look at all the possible use errors, determine the component / subsystem / product affected, follow the cascade…
And don’t forget to note the probability of each step, and accumulate the possible mitigations along the way (which turn out to be product requirements).
And so on…
I once started to draw a network showing all the useful relationships between the various entities in the QMS, but I got as far as this and gave up.
The knowledge network
Documents aren’t the truth.
And this is not just because documents can end up contradicting each other — maybe someone updated a requirement spec document but forgot to update the risk analysis, or maybe the spec sheet for a given part lists an approved supplier that is different from the supplier named in the bill of materials.
Even with the most diligent colleagues, documents can’t be the truth. They are just an extract, a fragment, a snippet of something bigger.
So, is the ‘knowledge network’ the truth?
Well, no. Because it is dependent on humans to be created, it is inevitably flawed.
But the knowledge network is hopefully our most accurate repository of information about reality.
The value of the network
The fantastic value of using a ‘knowledge network’ comes from what it can tell you that you didn’t already know you needed to look for.
For example, an engineer wants to change a bracket in a particular sub-assembly from aluminium 6063 to 6061. She makes sure to find and update the CAD drawings for the relevant parts in the sub-assembly. Hopefully, she remembers to update the b.o.m. If she’s any good, she also thinks to check whether this bracket is used in any other sub-assemblies, in case her change could impact those.
But what about the assembly work instructions? The related process risk analysis? Is this the only component bought from a particular supplier, and might it be wise to choose an alternate supplier who has better prices? Is that supplier approved for critical parts? Was there a biological safety evaluation conducted for this component and does it need repeating?
Where previously we had people filling the role of ‘document author’, we should have people taking on the role of ‘knowledge network editor’.
The ‘knowledge network editor’ should focus on making the right decisions based on evaluating the impact of a possible change, rather than focussing on the document formatting, version numbering, signatures etc.
And the reviewer’s job is now to provide a second pair of eyes to confirm that the intended effect of the change and all side-effects have been considered sufficiently. Finally, the approver is agreeing that this is a change that would align with the intent of the standard: keeping the customer satisfied.
Thinking about ISO 13485 from scratch
Assuming that a tool existed that was able to provide users with the ability to create and navigate a knowledge network, the implementation and quality of its UX/UI would be critical to success.
All changes would naturally take place at the most granular level: updating the wording of a single requirement, changing the number of drops of adhesive used in an assembly process, adding a new approved supplier to a specific component and so on.
In effect, every entity in the network is now a ‘document’, subject to its own review and approval process, and there are thousands of them, maybe tens or even hundreds of thousands for the biggest projects. And these changes (edit, review, approval) would all need to be frictionless. No muda.
Furthermore, once the knowledge network is established, anyone who needs to utilise the information contained would always have access to the latest data, instead of referring to a Word document that could have become obsolete a day after being created.
To me, this is where QMS should be heading, not just working towards technology-assisted versions of the old paper-based systems.
Just in case you were thinking that this was all a marketing piece for convincing you to buy a new piece of software, I can assure you that it’s not — I have yet to find the perfect knowledge network management tool (please let me know if you find it).
UPDATE: Since writing this, I’ve found a software platform tool that is capable of providing the foundation on top of which I have been able to build a fantastic knowledge network QMS. It may not yet be the perfect knowledge network management tool, but it’s as close as I have ever experienced 🧘♂️
If you’re interested in being an early adopter, drop me a line, and I will help implement it in your company.
The future of med-tech quality management
I long for the day when comments like “We need to roll Document ABC from version 3.0 to version 4.0” (followed by wearied sighs all round) are no longer heard in med-tech company offices the world over.
Instead, we’ll focus on discussing whether that idea we have for changing x to y is really such a great idea after all. And then if we all agree that it is, the change can be made to the knowledge network in the time it takes to click your fingers.
Until that day, I think it’s worth occasionally taking a step back and thinking about how we could do better and do more with the tools currently available, whilst still adhering to the intent of the standard.
