First we need to perform object detection to find the license plates on moving vehicles. For this we can use ultralytics Yolo.

Then, we need to track the object (license plate), using sort.

Finally, after detecting and tracking the license plate, we need to read the license plate. For this we can use the python library EasyOCR:

In the end, we want to create a stream of data that basically has two items:

• A timestamp

• The license plate number

Training Yolo object detection

We need a dataset of license plate images to train Yolo. We could take a bunch of pictures of license plates, or we could use a publicly available image dataset:

Open Images Dataset v7

So, we can download many images of license plates. To do this, we need a script:

If we are using our own images of license plates, we would need to annotate the dataset.

CVAT.ai

1. Create a new user account (openvino)

2. Start a new project (matricula_tester)

3. Add a label (alpr)

4. Create a new task

To set up your Raspberry Pi with YOLO (You Only Look Once) and the Pi IR-wide camera to capture and read vehicle license plate numbers, follow these steps:

Install Python dependencies:

You'll need OpenCV, NumPy, and other Python packages for image processing.

Install YOLOv5

You can use YOLOv5, which is easier to set up on the Raspberry Pi.

Installing Hailo

Set pcie to gen2/gen3(gen3 is faster than gen2):

Add following text to /boot/firmware/config.txt

NOTE

If you want to use gen2, please comment dtparam=pciex1_gen=3

Install hailo-all and reboot:

Open the terminal on the Raspberry Pi 5 and enter the following command to install the Hailo software

Check Software and Hardware:

Open terminal on the Raspberry Pi5, and input command as follows to check if hailo-all have been installed.

Open terminal on the Raspberry Pi5, and input command as follows to check if hailo-8L have been connected.

Validate installation of TAPPAS

Set Up the Pi IR-wide Camera

• Enable the camera on the Raspberry Pi: Run sudo raspi-config, go to Interfacing Options, and enable the camera. Then reboot the Pi.

• Test the camera: You can test whether the camera works using raspistill or raspivid.

• Install picamera2: (Optional if you are using picamera library)

  Copy code

  sudo apt install python3-libcamera python3-kms++ python3-picamera2

3. Prepare YOLO for License Plate Detection

• Download a pretrained model: YOLOv5 has several pretrained models. Use one of them (like yolov5s.pt) for testing.

  Copy code

  python3 detect.py --weights yolov5s.pt --source 0 # 0 for Pi camera

• Train or fine-tune for License Plate Detection: To accurately detect license plates, you might want to fine-tune YOLO on a dataset of license plates or use a pretrained model for license plate detection.

  You can download pre-trained models from sources like OpenALPR or create a custom dataset with labeled images of license plates.

4. Connect YOLO with License Plate Recognition

You can integrate YOLO with a license plate recognition software like OpenALPR:

• Install OpenALPR:

  Copy code

  sudo apt-get install openalpr openalpr-daemon openalpr-utils

• Run OpenALPR: Once you have the image captured using the Pi camera, run OpenALPR on the image to extract the license plate:

  Copy code

  alpr -c us your_image.jpg

5. Create the Pipeline

• Capture images using the Pi IR-wide camera.

• Use YOLO to detect the license plates in the image.

• Pass the detected license plate region to OpenALPR for text recognition.

Example Python Script:

Copy code

import cv2 from picamera2 import Picamera2 import torch # Load YOLO model model = torch.hub.load('ultralytics/yolov5', 'custom', path='best.pt') # Initialize the Pi camera picam2 = Picamera2() picam2.start() # Capture frame frame = picam2.capture_array() # Perform YOLO detection results = model(frame) # Process results to find license plates for result in results.xyxy[0]: x1, y1, x2, y2, conf, cls = result # Crop the license plate region license_plate = frame[int(y1):int(y2), int(x1):int(x2)] # Save or process the license plate with OpenALPR cv2.imwrite('license_plate.jpg', license_plate) os.system("alpr -c us license_plate.jpg")

This script captures a frame, runs YOLO to detect license plates, and uses OpenALPR to recognize the license plate number. You can fine-tune it based on your needs.

Let me know if you need help with any specific part of this setup!