Using EFQM RADAR to improve the quality of auditing

I think most people would agree that the quality of internal auditing (of any discipline) isn’t always what it should be. Typically reports are run off in a rush, with a poor level of detail, calling into question how rigorous the audit process actually was. And as for overall value …

There are a few techniques that people can use to increase the chances that the audit process will have the rigour and depth of scrutiny necessary to generate decent management information, turtle diagrams, for instance. Lately, though, I’ve been looking at how we can borrow part of the EFQM assessment approach to improve our internal audits. In an earlier post I highlighted a few EFQM weaknesses, but I think there is an opportunity to take one of its most simple and easy to apply strengths, and apply it to internal auditing – and it is equally applicable to quality, environmental and health & safety audits

For those of you who don’t know, the EFQM assessment process uses a numerical scoring system. Its not an exact science, of course, but it does help assessors rank and rate comparative data between companies and processes, and maintain a consistent approach. At the moment that scoring system is based on the “RADAR” Scorecard (see below)

radar.matrix

The EFQM RADAR assessment approach uses a type of balanced scorecard to encourage the assessor (or auditor for that matter) to ask a series of key questions, namely;

R. To what extent are results used to set targets for process performance?

A. To what extent is a clear approach (procedures for example) defined and understood?

D. To what extent is the approach deployed (i.e. does everyone follow the approach or is deployment patchy)?

A. To what extent is the process assessed (i.e. measurement that asks the question “is it working the way it should”)?

R. To what extent is the process reviewed? (i.e. a review of whether the overall approach is still relevant and suitable)?

My view is that many internal audit systems could benefit from using this type of framework. It would improve the depth and consistency of process scrutiny no end. Internal audit report forms and checklists could be constructed to force auditors to follow the discipline. A similar scoring system could even be used to try to generate comparative data, audit to audit

Its a different approach, right enough, but worth thinking about maybe? Drop me a line if you’re interested in working with me to integrate this approach

This entry was posted in Auditing, EFQM and tagged , , , , , . Bookmark the permalink.

11 Responses to Using EFQM RADAR to improve the quality of auditing

  1. Rob says:

    The big problem with this is that you need to ensure that you incorporate the compliance aspect of auditing against the governing standard too. I see it as an interesting addition to the traditional internal audit but it won’t replace it.

  2. Shaun says:

    Ah well, this is it in a nutshell, Rob. There are compliance audits and there are effectiveness audits and the need to use the right tool for the right job is as important as ever. You are quite right that the added value that could be achieved by an approach like this will be greater in effectiveness audit situations. I do find however that, even though every company will have a need for a mix of compliance and effectiveness audits, they either use one or the other (usually compliance). I don’t want to run compliance auditing down. It will always have its place, but when we’re auditing at process or system level we need to ask more questions

    Thanks for commenting. Its been a while. Thought you’d been abducted by aliens, or met an unfortunate Toyota-related end

  3. Rob says:

    Nope, not inducted by aliens, just mad busy at work, factor in my training for an ironman triathlon, seeing the kids, and a fairly feeble attempt to maintain a moderate level of alcohol consumption, has made commenting on blogs a lower priority!

    I tried this sort of approach to auditing a few years ago and the compliance aspect was a major concern during 3rd party audits. It also served to confuse the team of internal auditors too: it was just much to take in and try to do during the audit; unless you were super-experienced at audits. I think if you conducted one of these audits say, once per quarter, and perhaps used ISO 9004 too, you may get some benefit?

    Saying that, I would be interested in experimenting with this approach across process or value-streams.

  4. shaun says:

    I can see how that would be a problem

    I have just been invited to speak at the 2010 IRCA Conference in Yokohama on the subject of compliance versus effectiveness audits. This made me start thinking about what sort of themes I may cover, and this one sprung to mind

    “ISO 9004” and “EFQM” are by far the most popular search terms used by new visitors who find my blog, so there is undoubted interest either in a more modern approach, an excellent approach or whatever. The big worry is that people start seeing compliance auditing as some sort of outdated approach when clearly it is not. Take a look at RADAR – even that demonstrates that scrutiny works on two levels – assessment and then review. That basically translates into “compliance checking” (or basic QC) and Review for effectiveness. That fits in with the conclusion you have already reached in that the effectiveness review is a complimentary high level “time out” that asks the question “is this still the right thing to do”

    So I completely agree with you. We now have potentially not one but two tools in our kit bag, which puts us in the happy position of having a choice over our approach as we plan each audit

    The whole thrust of what I am trying to do here with this Capable People mallarky is to encourage a thought-out and intelligent approach to management in general. Most of the time I feel like John the Baptist, and cultivate feelings of mild despair. It seems too many people can only cope with one point of view or approach at a time, being more happy to switch lock-stock from one approach to another, than to sit with the luxury (in my opinion) of having a choice of two or more approaches. It comes as no surprise to me how easily some people can justify a myopic stance, as the old adage goes “If the only tool you have in your bag is a hammer, don’t be surprised if all your problems look like nails”

    Thanks for finding the time to comment

  5. Andy Nichols says:

    Interesting idea, Shaun. But then, I’ve come to expect that from you!

    I like the idea. Furthermore, I don’t see why the compliance issue can’t be incorporated as another ‘leg’ to the radar (or spider’s web chart, as I’ve used). Appeasing the 3rd Party auditors isn’t always simply from reports as Rob suggests, surely it’s also in the approach to the audit – for example using my ‘football’ diagram to plan a specific audit shows how much of the compliance aspects are considered.

  6. shaun says:

    Yes, Rob is a friend of mine. He runs the Learn Sigma Blog and he is quite right.

    I’ve thought about this effectiveness versus compliance question against the ISO 9001 standard and even considered a matrix that identifies, clause by clause, which requirements lean towards an effectiveness bias (e.g most of sections 5 and 8), and which lean towards a compliance approach (e.g. most of section 7). Then I realised just what sort of awful direction this could go in and retreated from the idea at 100 miles per hour.

    My mission is to try to get people to think and apply an intelligent approach and simply replacing one form of tick list with another just could have been something I’d struggle to forgive myself for

    Thanks for commenting Andy . Please call again

    PS. I have tried to respond to your email, but it keeps bouncing back. I think you may have put an incorrect character in the email address you added to the contact form

  7. Travis says:

    I’m sure we all know, if the Internal Audit is carried out in the above fashion (first paragraph) then, it should be questioned (under Audit) if it really achieved the results it was set out to do.

    Rob, I’m sure you’re fine with your Ironman training, but if you want to bounce some ideas, I have some experience.

  8. admin says:

    Thanks for the comment Travis. If you really want to get the low down on Rob’s Triathlon training and find out what toothpaste he uses, you can follow RobThompson on Twitter

  9. Ashok Deobhakta says:

    This is with reference to your comments on the CQI Linked in group.
    I like this idea. However, it will be necessary to train the internal auditors. It will be a good idea to first analyze the current audit reports and show how improvements can be made to adopt the RADAR approach. It will be interesting to develop a training module for this purpose.
    Further, such an exercise will help in identifying areas where the quality system is weak, or silent, in specifying the requirements.

  10. Shaun says:

    Thanks for taking the time to read and comment, Ashok, I appreciate it.

    Obviously training will be a key issue, it requires a different approach and additional abilities. However, analysing past audit reports (assuming the RADAR approach was not used) in my experience doesn’t help much with regard to taking things forward. It is usually hard to judge, when looking at the conformity statements contained in typical audit reports, the actual process, for good or bad, that was followed, just what was written. The key strength is that it forces a balanced view of effectiveness versus conformity, which is often lacking in the tick box approach most people fall into

  11. Ashok Deobhakta says:

    I agree. What I was contemplating was that the current reports was to build the case for adopting new RADAR approach. The Training Program could include demonstration of the new approach (by actual audit)and generating report to illustrate improvements made. Subsequent Training Program, say after one or two audits, could be organized to harmonize internal auditors.

Leave a Reply

Your email address will not be published. Required fields are marked *