In the field of computer vision and image processing, OpenCV (Open Source Computer Vision Library) is a powerful tool that enables a wide range of applications, from facial recognition to object detection. One interesting application of OpenCV is detecting and processing currency bills, which can be valuable in various fields including retail, banking, and automated vending systems. In this article, we will explore how to detect and analyze currency bills using OpenCV, breaking down the process into understandable steps and providing practical code example.

Steps to perform detection of bills using OpenCV

1. Morphological Operations
2. Removing Background
3. Edge and Contour Detection
4. Identifying Corners
5. Transforming the perspective

Code Implementation to perform detection of bills using OpenCV

Here, is the step by step implementation to perform detection of bills using OpenCV

Step 1: Morphological Operations

Morphology is a broad range of image processing procedures that manipulate images depending on shape [1]. Morphological operations add a structural element to an input image, resulting in an output image of the same size. A morphological operation calculates the value of each pixel in the output image by comparing it to its neighbors in the input image.

We can execute manipulations with the morphologyEx() function. In Morphology, the “close” operation is equivalent to Erosion, which is followed by Dilation.

import cv2 
import numpy as np 

# Morphology 
kernel = np.ones((5, 5), dtype = np.uint8) 
original_image = cv2.imread('Data/test.png')
image = cv2.morphologyEx(original_image, cv2.MORPH_CLOSE, kernel = kernel) 

# resize your image according to your test image
image = cv2.resize(image, (int(750 * 0.7), int(1000 * 0.7)))
cv2.imshow("Document Scanner", image) 
cv2.waitKey(0)
cv2.destroyAllWindows() 

Output:

Image after performing morphology operation

We’ll start with a blank page because the contents will get in the way while we work on the edges, and we don’t want to risk erasing them.

Step 2: Removing Background

The parts of the images that do not depict our subjects must also be eliminated. Similar to cropping a photograph, we will just focus on preserving the necessary section of the image. Use of the GrabCut library is possible. GrabCut removes any elements outside the boundary after receiving the input image and its bounds. To use GrabCut to detect the backdrop, we may provide users the option of manually selecting the document’s boundary. For the time being, GrabCut will be able to automatically determine the foreground by removing 70 pixels from each corner of the image and using them as the background.

1. The “mask” variable initializes the mask used by the GrabCut algorithm. The mask should be the same height and width as the image, with only one channel (grayscale).
2. “bgdModel”, “fgdModel” these arrays are used internally by the GrabCut algorithm.
3. The “rect” variable here represents the boundary that we are ready to split. You may find that some of the background goes inside the line, which is acceptable. Our goal is that no part of the object extends beyond the limit. Adjust the coordinates (x, y, width, height) to better suit your image.
4. cv2.grabCut() initializes the GrabCut algorithm with the given rectangle and iterates five times.
5. “mask2” converts the mask to binary form, where pixels marked as probable background (2) or sure background (0) are set to 0, and all others are set to 1.
6. “image” line multiplies the original image by the mask to remove the background.

# Create an initial mask
mask = np.zeros(image.shape[:2], np.uint8)

# Create temporary arrays used by the algorithm
bgdModel = np.zeros((1, 65), np.float64)
fgdModel = np.zeros((1, 65), np.float64)

# Define the rectangle around the object
rect = (50, 50, image.shape[1] - 70, image.shape[0] - 70)

# Apply GrabCut algorithm with the rectangle
cv2.grabCut(image, mask, rect, bgdModel, fgdModel, 10, cv2.GC_INIT_WITH_RECT)

mask2 = np.where((mask == 2) | (mask == 0), 0, 1).astype('uint8')

# Multiply the original image with the new mask to remove the background
image = image * mask2[:, :, np.newaxis]

Output:

Image after performing grabcut algorithm

Although it was not precise, it can approximate the image by removing the background.

Step 3: Edges and Contour Detection

We currently have a blank document of the same size as the original document. On the similar note, we shall do edge detection. We’ll use the Canny function for that. We also use Gaussian Blur to reduce document noise.

1. “image” This should be a binary image (e.g., the result of edge detection using Canny or thresholding).
2. “cv2.RETR_EXTERNAL” This flag retrieves only the outermost contours, ignoring any nested contours.
3. “cv2.CHAIN_APPROX_SIMPLE” This flag compresses horizontal, vertical, and diagonal segments, leaving just the termination points. For example, a rectangle contour could be represented using only four points.

The second return value is the hierarchy of the contours, which is not needed in this particular use case, so it’s discarded.

Finally, we dilate the image in the last line. Following this, we may proceed with the contour detection. We will just record the largest contour and apply it to a new blank document.

# Convert to grayscale and blur
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (11, 11), 0)

# Edge detection
canny = cv2.Canny(gray, 0, 200)
dilated = cv2.dilate(canny, cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5)))
# Find contours
contours, _ = cv2.findContours(image, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

Output:

Edge Detection

Step 4: Identifying Corners

We’ll align the paper with the four corners that have already been marked. Using the “Douglas-Peucker” method with the approxPolyDp() function.

# Loop over the contours to find the document
for c in contours:
    # Approximate the contour
    epsilon = 0.02 * cv2.arcLength(c, True)
    corners = cv2.approxPolyDP(c, epsilon, True)
    # If our approximated contour has four points, we assume it's the document
    if len(corners) == 4:
        break

# Draw the contours and corners on the image
con = np.zeros_like(image) 
cv2.drawContours(con, [c], -1, (0, 255, 255), 3)
cv2.drawContours(con, [corners], -1, (0, 255, 0), 10)

# Sort corners and label them
corners = sorted(np.concatenate(corners).tolist())
for index, corner in enumerate(corners):
    character = chr(65 + index)
    cv2.putText(con, character, tuple(corner), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 0, 0), 1, cv2.LINE_AA) 

Output:

Step 5: Standardizing Orientation of the four points

To ensure that the quadrilateral’s corners are accurately identified, we begin with a function that sorts the provided points. This function takes four points and organizes them in the following order: top-left, top-right, bottom-right, bottom-left. This ordering is required for consistent and accurate perspective conversions.

def order_points(pts):
    rect = np.zeros((4, 2), dtype='float32')
    pts = np.array(pts)
    s = pts.sum(axis=1)
    rect[0] = pts[np.argmin(s)]
    rect[2] = pts[np.argmax(s)]
    diff = np.diff(pts, axis=1)
    rect[1] = pts[np.argmin(diff)]
    rect[3] = pts[np.argmax(diff)]
    return rect.astype('int').tolist()

Step 6: Calculating the Dimensions

With the corners sorted, the next step is to compute the quadrilateral’s maximum width and height. The width is obtained by measuring the distances between the top and bottom corners of each side. Similarly, the height is derived by measuring the lengths between the left and right corners on both ends. The maximum width and height are then calculated to verify that the modified rectangle can accommodate the full quadrilateral.

# Order the corners
ordered_corners = order_points(corners)

(tl, tr, br, bl) = ordered_corners

# Finding the maximum width
widthA = np.sqrt(((br[0] - bl[0]) ** 2) + ((br[1] - bl[1]) ** 2))
widthB = np.sqrt(((tr[0] - tl[0]) ** 2) + ((tr[1] - tl[1]) ** 2))
maxWidth = max(int(widthA), int(widthB))

# Finding the maximum height
heightA = np.sqrt(((tr[0] - br[0]) ** 2) + ((tr[1] - br[1]) ** 2))
heightB = np.sqrt(((tl[0] - bl[0]) ** 2) + ((tl[1] - bl[1]) ** 2))
maxHeight = max(int(heightA), int(heightB))

Step 7: Defining Destination Coordinates

Finally, we define the coordinates of the destination rectangle.These coordinates reflect the rectangle’s corners, beginning at the top left corner and moving clockwise. The maxWidth and maxHeight specify the dimensions of this rectangle.

# Final destination coordinates
destination_corners = np.array([
    [0, 0],
    [maxWidth - 1, 0],
    [maxWidth - 1, maxHeight - 1],
    [0, maxHeight - 1]], dtype="float32")

Step 8: Perspective Transformtion

The coordinates of the source photos must now match the destination locations we discovered before. After completing this stage, the image will look to have been captured from above at a perfectly regular perspective.It is now obvious that the image, which was captured at an angle, has been perfectly caught using a 0-degree transform.

# Perspective transform
M = cv2.getPerspectiveTransform(np.array(ordered_corners, dtype="float32"), destination_corners)
warped = cv2.warpPerspective(original_image, M, (maxWidth, maxHeight)) 

Output:

Scanned image

These codes can be tested on multiple photographs in various orientations, and you can do the same. It has performed admirably in all of them. It would also result in an inaccurate perspective transformation as well. Deep Learning algorithms are used in popular document scanning applications because their results are more detailed and reliable.

Conclusion

Detecting and processing currency bills using OpenCV involves a series of well-defined steps including morphological operations, background removal, edge and contour detection, corner identification, and perspective transformation. By following these steps, we can accurately isolate and analyze currency bills from various images. Although traditional computer vision techniques like these can achieve impressive results, integrating deep learning models can enhance the accuracy and reliability of bill detection in more complex and varied scenarios. This approach provides a solid foundation for applications in retail, banking, and automated systems, demonstrating the power and versatility of OpenCV in practical, real-world tasks