Abstract
In this paper, we designed a fast and low-cost mechatronic system for recognition of eight current Persian banknotes in circulation. Firstly, we proposed a mechanical solution for avoiding extra processing time caused by detecting the place of banknote and paper angle correction in an input image. We also defined new parameters for feature extraction, including colour features (RGBR values), size features (LWR) and texture features (CRLVR value). Then, we used a Multi-Layer Perceptron (MLP) neural network in the recognition phase to reduce the necessary processing time. In this research, we collected a perfect database of Persian banknote images (about 4000 double-sided prevalent images). We reached about 99.06% accuracy (average for each side) in final banknote recognition by testing 800 different worn, torn and new banknotes which were not part of the initial learning phase. This accuracy could increase to 99.62% in double-sided decision mode. Finally, we designed an ATmega32 microcontroller-based hardware with 16MHz clock frequency for implementation of our proposed system which can recognize sample banknotes at about 480ms and 560ms for single-sided detection and double-sided detection respectively, after image scanning.
Keywords
1. Introduction
There is a rapid growth in the use of vending machines in developing countries. A vending machine is made up from different mechanical and electrical parts, but one of the main parts of any vending machine is the mechatronic system for banknote recognition which is also used in the other different automatic banknote systems, like the note sorter and note receptor in ATMs.
There are two main challenges in developing the mechatronic systems for banknote recognition systems, including cost and fast response time, which are essential factors for commercial purposes. In a real practical system with heavy processing, it is necessary to use high speed processors and complex hardware, which means increasing the total cost.
In this research, we have a perfect database of Persian real and ordinary banknote images for the learning stage.
We collected them from several print versions in different years because different versions have small differences in the colours used, so this makes a more reliable database.
In this paper, we designed a fast and low-cost mechatronic recognition system for Persian banknotes. In our proposed system, we designed a special mechatronic solution to reduce heavy pre-processing when detecting the place of a banknote image in relation to an original image; therefore, there is no need for paper angle correction. Fortunately, the colours, sizes and textures of common Persian banknotes also vary and have different images on the front and back (figure 1). Therefore, we also used colour and simple texture features for recognition of Persian banknotes by a Multi-Layer Perceptron (MLP) neural network to reach fast speeds in recognition and verification.
Image samples of eight different Persian banknotes
In the next sub-section, we briefly review some research literature relevant to the recognition of banknotes. In section 2, we present the proposed mechatronic system which includes mechanical parts, proper feature extraction and recognition algorithms. To evaluate the performance of the proposed system, in section 3 we present some results of implementation on a wide variety of common Persian banknotes (more than 4000 samples) which are often used in bank circulation. We also propose and implement low-cost implementation electronic hardware for the system used. The mechanical part of the proposed system has only just been designed. The simulation results were also verified through the implemented electronic hardware. Finally, the paper closes with a concluding section.
1.1. Brief Review
So far, features such as size, colour and texture have been used in banknote recognition researches. Takeda and Omatu (1995) proposed a technique for Japanese and US banknote recognition. They used Fourier power spectra and time series for their inputs to the network. They could reduce the input scale of the Artificial Neural Networks (ANN) without reducing system performance [1].
Frosini et al. (1996) also proposed an MLP network for recognition and verification techniques in a real banknote machine. Their mechanism is based on optoelectronic devices which produce a signal associated with the light refracted by the banknotes. They used additional algorithms in their system to classify banknotes of different countries [2].
Aoba et al. (2003) proposed the use of two types of ANNs, including a three-layered perceptron and a Radial Basis Function (RBF) network, for euro banknote recognition. They use a three-layered perceptron for classification and several RBF networks for validation. They announced that the RBF network has the ability to reject invalid banknotes because RBF networks always estimate the probability distribution of the sample data. They also propose using infra-red (IR) and visible images as input data to the system since euro banknotes have quite significant features in IR images [3].
Thai banknote recognition has been discussed by Takeda et al. (2003). The slice values, which are the digitized characteristics of banknote by the mask set, are extracted from each banknote image [4]. These slice values are the summation of non-masked pixel values of each banknote. Then, they used ANN to execute the learning and recognition process. Their system shows some unreliability because of the output fluctuation by the mask set and threshold values.
A method based on principal component analysis (PCA) for increased reliability of banknote recognition is used by Ahmadi et al. (2004), who use six different US dollar banknote types [5]. Their data was acquired through an advanced line sensor. They classified test samples with a Linear Vector Quantization (LVQ) network. They evaluated the reliability of the system for 1200 test samples. They assert that their method has up to 95% reliability when the number of PCA components, as well as the number of LVQ codebook vectors, are taken into proper account. In order to compare the results of classification, they also applied hidden Markov models (HMMs) as an alternative classifier.
Lee et al. (2004) proposed a point extraction and recognition algorithm for euro banknotes. For distinctive point extraction, they used a coordinate data extraction method from specific parts of a banknote representing the same colour. To recognize banknotes, they train several ANNs. One of them is used to detect inserting side direction and the other four are used for detecting banknote value. Their algorithm is designed to minimize recognition time by using a minimal amount of recognition data. With this method, they had some errors, except for 100 and 200 euro banknotes [6].
Combined output with single ANN proposed for recognizing Turkish lira and Cyprus pound banknotes has been proposed by Khashman and Sekeroglu (2005). They used a combination of both image processing and neural networks to provide sufficient and robust solutions. A back propagation (BP) learning algorithm is implemented using an input layer with 100 neurons, one hidden layer with 30 neurons and one combined output layer with nine neurons. The flexibility, usability and reliability of this intelligent banknote identification system are shown through the results and a comparison is drawn between using separate neural networks and using a combined neural network for each currency [7].
Kagehiro et al. (2006) used a hierarchical method for high-speed classification of US banknotes, with 99% accuracy. A number of discrete points are selected from the overall image and the average of the pixel at each point and its adjacent pixels is taken as the observed value for each point [8]. They used about 32,850 samples from 12 kinds of US banknote. The banknote is classified by measuring the distance between the template vectors and the feature vectors from the observation points for use in classification. High-speed processing is realized by using low-dimensionality vectors; therefore, the computational costs decrease.
Hinwood et al. (2006), aiming to assist blind Australians [9], proposed ‘Money Talker’, which takes advantage of the different patterns and colours on Australian banknotes and recognizes them with an electronic device. They showed the light reflection and transmission properties for colour feature recognition. Different coloured lights were used for detecting distinct ranges of values, depending on the colour of the note by corresponding sensors. Their system should be accurate, easy to use, inexpensive, quick and portable at the same time. They reported the execution time is about three seconds for single note recognition. The total accuracy of their device is 99.4% for the whole dataset.
Gunaratna et al. (2008) proposed a system for Sri Lankan banknote recognition, exploiting several changes in image quality. Hence, a special linear transformation function is adapted to wipe out noise patterns from backgrounds without affecting the notes' characteristic images and allowing images of interest to reappear [10]. Their proposed system, “SLCRec”, offers a solution focusing on minimizing false rejection of notes. Their proposed transformation maps the original greyscale range into a smaller range of zero to 125. Edge detection after the transformation provided better robustness for noise and fair representation of edges for new and old damaged notes. A three-layer back propagation ANN used was for edge detection. The implemented system, comprising compressed greyscale range and the three layers BP-ANN, produced suitable results for Sri Lankan currency notes.
Hassanpour and Farahabadi (2009) proposed a method based on a hidden Markov model (HMM) for banknote recognition. The texture features of banknotes are modelled as a random process [11]. The proposed algorithm is used for separation of paper currency from different countries. In their method only intact examples of paper currencies from each denomination are used for training the system. Their algorithm is used as a similarity measure for the classification. They tested their method on more than 100 denominations from different countries and they asserted that it was able to recognize 95% of data correctly.
Debnath et al. (2010) proposed a currency recognition system using an Ensemble Neural Network (ENN). The individual ANNs in an ENN are trained via Negative Correlation Learning (NCL). The objective of using NCL is to train individuals on different parts or portions of the input patterns in an ensemble. By using an image histogram, the frequency of different colours in a paper currency is calculated and compared with the one in the reference paper currency. The image of each type of note is converted into greyscale and compressed in the desired range. Each pixel of the compressed image is given as an input to the network [12]. They assert that their system can be learned with one valid sample of each banknote type.
However, each country uses its own banknotes which are different in size, colour and texture. This means that a banknote recognition system should be designed especially for each country, which can help to reduce the total cost of the system. We also focused on our country's banknotes and proposed our system.
2. Proposed Mechatronic System
In this research, we collected a comprehensive database with the widest range of banknote images, which is necessary for robust learning and evaluation. Our input images of200×450 pixel resolution were taken by two A6 600DPI fast flat scanners. Then we merged an original input image with a 180 degree rotated one (Figure 6).
We suggest using an intelligent mechatronic system in the entrance of our proposed mechanism, which eliminates the need for an angle correction algorithm. This means that using our proposed mechanism also saves pre-processing time, resulting in reducing overall processing time. As another advantage, it was not sensitive to the entrance side of banknotes and this was accomplished with a simple and quick technique.
In our proposed recognition method, we also used the ratio of length and width instead of using only the size of banknote image. This reduced sensitivity to scaling and damaged angle. We also considered complete banknote images which reduced sensitivity to special parts in the picture such as numbers, small detail in pictures, handwriting and small laceration.
2.1. Mechanical Part
Our mechanical system was designed with regard to the crooked entrance angle potential of cash. This means that we did not need heavy processing for image angle correction that results in saved processing time. In this method, by preparing an intelligent gate in the cash entrance and using a sensor near it, cash moving can be controlled and then the initial angle of entry is corrected. After sensing the banknote, the entrance motor feeds for a short time without the entrance shutter opening; this action corrects the paper entrance angle to 90 degrees. When the cash angle is corrected and all the input sensors are detected, the note shutter opens and the cash enters into the mechanism for scanning immediately (Figure 2). We also used two scanners for double-sided scanning. The scanners can scan paper with two modes, ultra violet or visible light.
Proposed cash entrance and transporter mechanism
We apply a black background for scanning which helps to remove background easily. The system scans both sides of the paper simultaneously, saving processing time. Sometimes, it is necessary to balance a crooked image after image scanning by obtaining the slope of the image with random external edge points; if the slope is more than a predefined threshold then the crooked input image should be balanced (Figure 3).
Simple extraction and separation of the main banknote image from the input image
2.2. Recognition Algorithm
Our final goal is to reach a low-cost and fast mechanism which is suitable for commercial products. Therefore, we must use a simple algorithm for reducing processing time and cost. We select a fast algorithm which can be implemented in a low-cost microcontroller-based hardware, enhancing commercial product possibility. Our algorithm, which includes feature extraction and recognition, must avoid heavy calculation and it should use little time-consuming computation such as multiplication and division.
2.2.1. Feature Extraction
In the proposed feature extraction, we only use three types of feature: size, colour and texture.
2.2.1.1 Colour Features
Many countries use colours to distinguish the denominations of the banknotes easily. Thus, colour is a feature that characterizes the denomination of banknotes; besides, the banknotes keep their colour despite being mistreated, because the colour is distributed throughout the surface of the banknotes. By adding colour information, the characterization of the banknotes becomes more robust and the classification becomes more accurate. Therefore, colour and texture data are suitable features that contribute to classifying banknotes. In fact, people usually recognize the banknotes firstly by their colour and secondly by their picture pattern [13].
To classify the banknotes by their colour, it must be selected an adequate colour space model to characterize the colours. Since we are interested in classifying the banknotes' colours, the RGB model is the most adequate for this purpose [15]. In the RGB model, each colour appears with its primary spectral components of red, green and blue components.
The colour of every pixel is a linear combination of the three-dimensional base vectors red, green and blue. The colour of a pixel can be written as:
Where C(x, y) is the colour vector of pixel (x, y), x = 1, …, N and y = 1, …, M where N and M are the numbers of image rows and columns, respectively. The scalars R, G and B are the vector components that correspond to Red, Green and Blue, respectively.
As it has been explained before in this sub-section, each denomination class is characterized by a dominant colour. We name it as Base colour, C T , which is obtained by summation of colour vectors of all image pixels.
This vector is placed within the unit colour cube near the location of the most repeated colour vector. The magnitude of this vector is very large; however, what is relevant is the location of the vector within the cube, not its magnitude. So, the colour vectors are normalized as follows:
Thus, the vector NC T characterizes the colour feature. The scalars NR T , NG T and NB T are the average of each colour in the image, but these parameters are very diverse in worn and new banknotes. We decided base colour by comparing these values (Figure 4).
Base colour extraction
In our proposed method, we have three extracted features: Red to Green Ratio (RGR), Blue to Red Ratio (BRR) and Green to Blue Ratio (GBR). These parameters have less sensitivity to worn and handwritten features. We nominate it the ‘RGBR method’ and it can be defined as follows:
Table 1 shows these parameters which are used as classifier input for Persian banknotes. It is clear that there are sufficient differences in these features necessary for classification.
RGBR values for different Persian banknotes
2.2.1.2 Size and Texture Features
Length and width of image are two parameters which show similarity in some banknotes in the different countries. However, in our country (Iran), there is little variation (Table 2). This can help us to improve recognition. In order to improve separation ability, we used Length-to-Width Ratio (LWR). LWR has less sensitive to image angle rotation (Figure 5).
Length to width ratio (LWR)
Merging original input image with 180 degree-rotated image
We reduced the necessary time for entrance side direction, considering our proposed method which merged the original and the 180 degree-rotated image (Figure 6). At first, it seems that we may have lost many details, but our investigations show that overall performance does not change.
Sizes of different Persian banknotes and corresponding LWR
In the next step, a greyscale adjusting algorithm, like histogram equalization, is applied to improve the image contrast and details in banknote images (Figure 7). The computational cost of this algorithm is low, but it improves overall performance.
Greyscale image adjusting
The original scanned images are 200×450 pixels, but for the next step we divided them into 25×25 pixel blocks by applying Moving Average (MA) filtering on the input image; therefore, the obtained image is 8× 17 (figure8).
Image resized to 8×17 pixels
Important features can be obtained from this reduced image, which include variation in columns' average brightness and variation in rows' average brightness. These parameters are important but, in some, images have little similarity. Therefore, we proposed a new parameter, Column-to-Row Light Variation Ratio (CRLVR) which shows a suitable discrimination between different banknotes (Figure 9). As shown in Figure 10, our proposed texture features are symmetrical, which means that collecting half of the data is sufficient.
CRLVR for sample Persian banknotes
Some samples of worn and torn banknotes
2.2.2. Classifier Design
Computational cost is a very important parameter for practical and commercial designs. Among different types of classifiers used in banknote recognition, several kinds of ANNs, such as RBF, MLP and LVQ are popular [1–10], among which MLP-based classifiers are most attractive. Although MLP-based classifiers sometimes suffer from heavy computation in the learning phase, which is off-line processing, they need low computational cost in the recognition phase since their computations are limited to summations, multiplications and sigmoid-based function calculations which can be implemented by a simple lookup-table technique.
Therefore, we used a three-layer MLP-based classifier with29 neurons in the input layer and nine neurons in the output layer. In the input layer, one neuron is used for LWR, three neurons for base colours, three neurons for RGBR values, four neurons for row averages, nine neurons for column averages and finally nine neurons for CRLVR values(some values are symmetrical such as row and column averages and CRLVR values). Nine neurons in the output layer are used to classify eight types of Persian banknotes and the remaining one neuron is used for unknown input images.
In order to select the best MLP-based classifier, we tested different numbers of neurons in the hidden layer [14]. Finally, we used 15 neurons in the hidden layer which resulted in the best performance. We also apply sigmoid function for the hidden layer and hard limited function for the output layer. Our practical recognition system used a double-sided image of the paper simultaneously with two separated ANNs. In this way, the overall processing time can be reduced using two implementations of parallel hardware. Again, we can have two modes for our proposed system, one-sided or double-sided detection, where double-sided detection is more accurate (Table 3). The double-sided detection algorithm reached a higher recognition rate without increasing computational cost and execution time.
Again, the recognition rate of the single-sided detection algorithm is lower than that of the double-sided detection algorithm, but the computational cost and execution time is also lower. Using single-sided or double-sided detection algorithms depends on the relative importance of recognition rate and computational cost. The difference of recognition for each side depends on the kind of image in them, but if the banknote has large or many lacerations it may be misrecognized.
3. Implementation Result
3.1. Banknote Database
In this research, we prepare a perfect database of Persian real and ordinary banknote images for the learning phase as well as for system evaluation. There are worn, torn (Figure 10), fit and unfit images in our database which contains 4000 double-sided images (500 double-sided images of each type). We try to collect them from several print versions of different years because different versions have some differences in the colours used. Using this database guarantees reliable learning, which is necessary for practical applications. Our images are acquired from two A6 600DPI fast flat scanner sand their resolution is 200×450 pixels.
Recognition accuracy of the proposed system for Persian banknotes (D-Side: Double-sided, S-Side: Single-sided, F: Front, and B: Back banknote images)
3.2. System Accuracy
Simulation and evaluating the system performance is done in Matlab R2010a [15]. We used 3200 images (400 images for each of eight different banknote types) in the learning phase and 800 double-sided images (100 other double-sided images for each of eight different banknote types) in the evaluation phase that were different from those in the learning phase. On average, we reach about 99.06% accuracy in the evaluation phase for each side and this can increase when using the doubled-sided results in the decision criteria. The accuracies are shown in Table 3 for all Persian banknote types. Our electronic implementation confirmed the simulated results.
3.3. Proposed Implementation Hardware
We used two low-cost high speed A6 600DPI fast flat scanners of the model CP06 (Prober & OEM), the scanning time of which is less than one second. We also applied our proposed algorithm in two separate ATmega32 microcontroller-based hardware systems with 16MHz external clock source. We saved our final ANN's weights (about 700 weights, biases and threshold values) into ATmega32 EEPROM and then we retrieved them once after ATmega32 reset from EEPROM into ATmegat32 SRAM. This will help us with the next upgrade as we will not have to change flash codes for future improvements. We connected the scanners to microcontrollers withFT232 which is a single channel USB 2.0 high Speed (480Mb/s) to UART/FIFO IC. Our system used each scanner for one-sided recognition.
For single-sided recognition, our simulation shows that we need about 648000 summation operations and 24000 multiplication operations plus some references to a lookup table. In relation to the 16MHz clock source, Table 4 shows the necessary processing times in two cases, single-sided and double-sided recognition. The second of these modes needs a little more time because some calculations are done in parallel in two separate modules and finally one of the two modules (the master module) decides the recognition result. Our results also show that the processing speed is sufficient for real time applications.
Average processing time
4. Conclusion
Simplicity, robustness, operational speed and total cost of a banknote recognition system are some of essential parameters in a banknote-based system designing. In this paper, we designed a mechatronic system for Persian banknote recognition. We consider a mechanical entrance gate for image angle correction useful for reducing pre-processing time. We merged an original input image with a 180 degree-rotated one for removing the side detection process. We also defined some simple and efficient features in the feature extraction step, including RGBR value based on colour information and CRLVR value based on texture information. Finally, we used a MLP classifier for recognition of the input banknote image.
We used 3200 ordinary images (400 double-sided images of each type) for the learning phase and 800 other images (100 double-sided images of each type) for system evaluation. We can recognize the Persian banknotes with 99.06% overall average accuracy for each side, which can be increased using the double-sided mode.
This small margin of error occurs when there is some damage to the input image, as in Figure 11, which is very rare in current banknote circulation.
Very damaged banknote image
We also designed a simple and low-cost ATmega32 microcontroller-based hardware which has the facility for connecting to the scanners to implement our recognition algorithm. Our simulation shows that the overall processing time of the proposed hardware is 480 msec and 560 msec, for single-sided and double-sided modes respectively which are sufficient in most automatic banknote-based real-time applications.
Moreover, we believe the proposed method can be simply used for other banknotes in different countries.