How I Built an AI to Detect License Plate Number Registration (ANPR)

This is an AI Automatic Number Plate Reader (ANPR) that I installed in my car!

It uses AI to detect a license plate, convert it to text, and then check if that plate is registered by looking up the Victorian Vehicles Register (VVR).

Now, you’d expect technology like this to be worth thousands of dollars or more specifically, $78,636 because that’s how much it cost on average for Victoria Police to fit Automatic Number Plate Recognition into their cars.

But my solution only ended up costing a few hundred dollars, so let me show you how I built it.

(Watch the full video above...or keep reading...or both...it's up to you)

Hardware

We start with a USB3 Camera (Omnivision OG02B10) with a 23mm lens which provides a really clear vision of license plates that are as far as 10m away.

That camera is then wired through the engine bay, passenger firewall, and into the glove compartment which connects to an Asus Tinker Edge T (ATET) Single Board Computer which performs AI plate detection, and vehicle registration checks (Asus sent me a free one last year).

From that, we have an HDMI cable that connects to a portable monitor.

And a 12v battery pack to power the computer.

As for internet connection, the ATET is tethering from my phone.

Software

I trained a Deep Learning model with Tensorflow using over 300 images of Victorian license plates via Transfer Learning.

And then wrote an app in Python to detect license plates, and check that plate against the Victorian Vehicles Register.

Put this all together and for about $764, you have an Automatic License Plate Reader in your car. So, let’s see how well this thing works by starting with a few basic unit tests.

Unit Tests

I stood 10 metres away and held up some photoshopped license plates that I printed out on A4 paper.

This main window here is the Asus Tinker Edge T which is hooked up to the portable monitor.

The right side shows my Pixel 4A recording next to the car.

Just look at how much more zoomed-in the 23mm lens is compared to a phone camera!

Unit Test 1: A55

The model is detecting the plate, but it’s failing to convert it to text - I think it might be too small.

Unit Test 2: FLICK

It has detected the plate again, and this time converted to text with 83% confidence.

Unit Test 3: PEN15

The model detects the plate but reads it as PEH15.

With thorough testing done, it’s time to hit the road.

Parking Lot Test

With both cars stationary, it works perfectly.

You can see the owner has paid its rego until 25-Sep-2021. The VVR returns the make, model, colour, VIN, and engine number which look correct.

The app then summarises that detail into this Dataframe here that shows the rego number, if rego has been paid, and if there are any sanctions or fines on the rego. If everything is okay, it’ll return a circle, and an x in red if there's something wrong.

Highway Driving Test

At low speeds and stopping at intersections, it performed quite well. Once it locked onto a plate it took between 5-10 seconds to convert the text and return results from the VVR.

In daylight where plates had a white background, and blue text that follow the number-letter-letter-number-letter-letter pattern, the plate reader worked beautifully.

Issue #1 - O vs Q vs Zero

On several occasions, my app struggled to tell the difference between an O, Q, and Zero.

I used the OpenALPR library to convert the image to text and as amazing as that library is, I just couldn’t figure out a way around this. Let me know if you have any ideas!

Issue #2 - Black-Coloured Plates

Black-coloured plates struggled which is why I think my unit tests didn’t go so well but I know the solution is to do a bit of preprocessing on the image.
OpenALPR just couldn't convert it to text.

Issue #3 - Plastic Housing

License plates with plastic housing had too much glare.

Issue #4 - Low Light

The app didn’t work well in low light when the sun was going down.

The plate either became too dark or my headlights added too much glare.

You can see it just looks like a hot blurry mess.

And this is pretty much why Victoria Police have way better technology in their cars than what I’ve put together!

Summary: Building My ANPR System

My ANPR system produced amazing results in these situations:

• when my car and the target car were both stationary (e.g., car parks, and stopping at intersections),
• at low speeds with plenty of daylight, and
• with license plates that follow the #AA#AA pattern.

However, my ANPR system struggled during the following scenarios:

• differentiating between O, Q, and Zero (Fix: Unsure)
• black-coloured plates (Fix: OpenCV image pre-processing)
• clear plastic housing on the plate (Fix: Add a lens filter to reduce glare)
• dusk / lack of daylight (Fix: Add infrared lights)

Therefore, my advice is - if you want to bypass my ANPR system, buy black-coloured plates, put a clear plastic housing around it, and drive only after the sun goes down :P