08 Jun 2011 Lies, damn lies and statistics – interpreting accuracy claims for scanning
I once heard of an investment that was guaranteed to double my money. What wasn’t guaranteed however, was how long it would take to double!
Caveat Emptor
You have to be very careful how you interpret claims made by vendors and claims made for the accuracy of a scanning based document management solution process are no exception. If you want to include scanning of invoices as a component of an AP automation program you need to know that you are likely to in excess of 80% 1st time match. But what does 80% accuracy mean?
If your vendor is promising that all of your invoices will match to an 80% accuracy – that means that none will be matched. 20% of the data has not been recognised and manual intervention will be required 100% of the time. What you probably want is that 80% of your invoices are matched 100% accurately. There’s a big difference!
OCR accuracy
OCR software has been pretty sophisticated you some time and many solution providers could credibly claim OCR accuracy percentages in the high 90s. In other words, the software could recognise 98-99% of the characters – what it won’t be able to do is interpret them.
OCR is dumb. When reading an invoice, it doesn’t understand what a P.O. number is. It won’t know if the invoice is from a valid supplier and it doesn’t have idea what to look for to find a VAT number. A claim of 99% OCR accuracy is unhelpful.
A scanning solution in an AP automation context requires some intelligence. It needs to know what it’s looking for and have the ability to find it. It needs to be able to compare what it thinks it finds with P.O. records and supplier records in the ERP system and make intelligent judgements about what is and is not a good match.
A clean scan of a decent invoice and accurate OCR is a good start but it’s the intelligent steps that follow that allows matching at line level to the level of accuracy that will support full AP automation.
