Investigation on an industrial-feasible approach for measurement and assessment of large-sized micro-structured surfaces based on grayscale matching

Abstract

Multi-scale micro-structured functional surfaces can enhance the interfacial effect on a component by the microtopological structure, likely through realizing or enhancing its lubrication, drag reduction, noise reducing, sealing, and other specific surface functions. However, the realization of these functions requires high-precision machining quality and the occurrence of any defects on the structured surfaces will possibly influence actual performances of the component. The distribution density, morphology, geometry, configuration feature, and position of those microstructures play a decisive role in affecting the functional performance of the component. It imposes a challenge for rapid and accurate measurement of micro-structured surface particularly at a reasonable large size in industrial-scale production. This article presents an innovative industrial-feasible measurement method for micro-structured surfaces using normalized grayscale matching combined with pyramid search strategy, discriminating-step matching strategy, and sensitive area analysis. The method renders the high matching accuracy and rapid processing capability, which also provides the technical solution for industrial-scale micromanufacturing of large-sized micro-structured surfaces. Well-designed experimental trials are carried out to further evaluate and validate the measurement accuracy and efficiency.

Introduction

Micro-structured functional surfaces typically refer to various kinds of micro-featured topological surface configurations, such as microgrooves/riblets, microdimples, and convex hull arrays, which improve and change some functional features of mechanical components and products. The micro-structured functional surfaces can distribute in multi-scale with feature sizes generally at 1–100 µm scale. ¹ The surface can significantly enhance the functional performance of mechanical components or products, such as reduction of the noise, surface friction, energy consumption, enhancement of the surface sealing, lubrication, and service life. This is achieved through changing the tribological interface and/or thermal effect of the surfaces at components or products as illustrated in Figure 1. There are a number of methods for manufacturing micro-structured surfaces. For instance, micromilling combined with mass replication processes such as injection molding or rolling and microlaser processing are the most common ones. Weng et al. ² used the magnetic-aided roller to process microstructure and Mastud et al. ³ developed the reverse micro-electrical discharge machining method to process microstructure. Zhou and Cheng ⁴ suggest the quality of the machined surface topography and features is highly dependent on the machining dynamics and machining strategies employed and material properties. As the surface–interface effect of micro-structured surfaces is a complex nonlinear physical phenomenon, the intrinsic relationship between its complex geometric shape and precision and other underlying mechanisms in the machining process are not fully understood scientifically. The defects in the process of manufacturing micro-structured surfaces are still unavoidable, although different manufacturing methods have their own respective pros and cons. Therefore, the industrial-feasible measurement and assessment of micro-structured surfaces are essential and much needed particularly for their industrial-scale manufacturing and applications.

Figure 1.

Typical engineering applications of micro-structured functional surfaces.

The measurement approaches fall in contact and non-contact methods generally. The dynamic testing range of contact measurements is large but tedious in operations. For instance, Mu et al. ⁵ used tactile sensors to measure the machined freeform surfaces; Hu et al. ⁶ adopted the coordinate measuring machine to measure a known micro-structured surface. But the surfaces are susceptible to be damaged, and the features and spatial positions are limited, as the probe’s diameter needs to be smaller than the features and measurable, and measurement efficiency being low for hundreds of features. Furthermore, because the scale of the microstructure is normally at 1–100 µm scale or even smaller, it imposes stringent requirement for contact measurement particularly in terms of the size and precision of the probe. In many application cases, the microfeatures with the three-dimensional shape with geometric mutation are so complex, which is very challenging to use traditional methods to measure the surfaces swiftly as required in the production. ⁷

Therefore, the non-contact methods are more suitable for measuring large-sized micro-structured surfaces, such as using confocal microscopy, laser measurement, and white light interference. However, almost all non-contact methods need precise and expensive devices that are not widely available for the industry. In the last two decades or so, image matching has been developed rapidly with the development of image processing and being applied in many fields. For instance, terminal guided cruise missile uses scene matching to determine the exact position. In stereo vision, in order to obtain a depth map of the images, it is essential to find the conjugate pair in the scene of the same point in two images, which also falls in image matching researching; Furthermore, face recognition based on hidden Markov model, ⁸ object recognition, and tracking are also widely used image matching technology.

In order to guarantee the quality of the micro-structured surfaces, it is critical to develop an accurate, efficient, and swift approach to the surfaces measurement and its industrial-scale implementation. The advantage of using image matching technology is similar to non-contact measurement, that is, having much higher efficiency and no damage to micro-structured surfaces in measurement. Furthermore, image matching technology normally applies inexpensive digital camera devices, but other non-contact method is not the case. However, the image matching still has some technological challenges particularly for the measurement of micro-structured functional surfaces. Therefore, the research presented in this article aims to develop an improved normalized grayscale matching method combined with pyramid search strategy, discriminating-step matching strategy, and sensitive area analysis, so as to obtain an efficient and accurate industrial-feasible approach for measuring the micro-structured surfaces at the industrial scale. A series of experimental tests have been carried out to evaluate and validate the measurement accuracy, efficiency, and industrial feasibility of using the improved measurement approach. The results and corresponding discussion are also presented in light of the experimental testing.

The surface measurement approach and analysis based on grayscale matching

Grayscale matching algorithm calculates a relative value that is the similarity of images between a template and the measurement image by some similarity measuring function, to evaluate the similarity degree of images. For instance, Parker and Hou ⁹ use high-contrast grayscale images in diffuse images of topographical defects to facilitate effective defect detection, which provides a foundation for characterizing the severity of defects presented in a given region.

After grayscale processing, assuming that the size of the measurement image is M × N, the template’s size is A × B, M > A, N > B. M and A stand for the number of pixels in the height, and N and B stand for the number of pixels in the width. During the matching process, the template will translate on the measurement image pixel by pixel. On some position, the overlap between template and measurement image named testing sub-graph, denoted by S(i,j), and (i,j) stands for the corresponding coordinate of template’s top left corner on measurement image. ¹⁰

The methods of grayscale matching are various, such as absolute balance search (referred to as ABS), sum of squared differences (referred to as SSD), and normalized cross correlation (referred to as NCC). ABS is the earliest algorithm proposed based on grayscale matching, ¹¹ but the thresholds in ABS are different in different situation and it is susceptible to noise interference. Due to the decrease of pixels selected, the accuracy of SSD is poor and it is susceptible to noise interference, too. ¹² This article adopts the deformation of NCC and its definition formulation is as follows

$$\begin{array}{c}\mathrm{R}(\mathrm{i},\mathrm{j})=\hfill \\ \frac{\sum _{\mathrm{m}=1}^{\mathrm{M}}\sum _{\mathrm{n}=1}^{\mathrm{N}}[{\mathrm{S}}^{\mathrm{i},\mathrm{j}}(\mathrm{m},\mathrm{n})-\overline{{\mathrm{S}}^{\mathrm{i},\mathrm{j}}}]\times [\mathrm{T}(\mathrm{m},\mathrm{n})-\overline{\mathrm{T}}]}{\sqrt{\sum _{\mathrm{m}=1}^{\mathrm{M}}\sum _{\mathrm{n}=1}^{\mathrm{N}}{[{\mathrm{S}}^{\mathrm{i},\mathrm{j}}(\mathrm{m},\mathrm{n})-\overline{{\mathrm{S}}^{\mathrm{i},\mathrm{j}}}]}^2}\times \sqrt{\sum _{\mathrm{m}=1}^{\mathrm{M}}\sum _{\mathrm{n}=1}^{\mathrm{N}}{[\mathrm{T}(\mathrm{m},\mathrm{n})-\overline{\mathrm{T}}]}^2}}\hfill \end{array}$$

where $\mathrm{R}(\mathrm{i},\mathrm{j})$ is the relative matching degree of grayscale; ${\mathrm{S}}^{\mathrm{i},\mathrm{j}}(\mathrm{m},\mathrm{n})$ and $\overline{{\mathrm{S}}^{\mathrm{i},\mathrm{j}}}$ are the grayscale of each point on measurement image and the average grayscale of each point on the measurement image; and $\mathrm{T}(\mathrm{m},\mathrm{n})$ and $\overline{\mathrm{T}}$ are the grayscale of each point on template and the average grayscale of each point on template.

NCC’s accuracy and resistance to white noise present well under the condition of small grayscale variation and geometric distortion. Therefore, it is widely used in grayscale matching of images. However, the main problem is that its matching speed is not so rapid, and its matching problem is susceptible to some tiny interference (such as the beam angle of environment).

Traditional NCC is that uses an image cut out from measurement image to match with a template, whose size is the same as the template. And the matching process begins from the top left and then scans from left to right, from top to bottom per-pixel along, so its efficiency and speed are slow. In order to solve this problem, we adopt pyramid search strategy and discriminating-step progressive search method to improve NCC in allusion to its defects, making it a feasible, fast, and efficient measurement method.

The pyramid search strategy reduces the number of matching process through reducing image’s size and distinguished ability according to the predetermined threshold. When the grayscale between the template and the reduced image matches to a higher value (compared to a preset threshold), then the template returns to the original image to do some exact matching to get the position where the template in the measurement image.

Traditional NCC’s matching step is 1, and it only goes forward a pixel after each matching. This leads to its low matching efficiency. Since the microstructures are lined by a certain distance, when the template is far to the true position of the measuring unit, the difference between two pictures is such big and their correlation is low so that their grayscale matching values are low, too. Then, when the template is near the true position of the measuring unit, the grayscale matching values increase with the increase in correlation degree to a peak, as is shown in Figure 2.

Figure 2.

The matching values scatter diagram of NCC algorithm and improved NCC algorithm: (a) the scatter diagram of NCC algorithm and (b) the scatter diagram of discriminating-step algorithm.

From Figure 2, it is found that there are so many matching values near the peak value. This means that there will be a lot of repeated values, leading to much repetition near the true position appearing in the process of image’s matching. This not only reduces the accuracy but also adds much calculation. When the matching value is low, the template is much different to the measurement image, almost belonging to useless process so that can be skip during measurement.

Therefore, according to the correlation characters between template and measurement image, we set several thresholds to avoid some repeated matching. When the matching value is small, it stands for the template is far away from the measuring units and we can adopt greater progressive step; while the value is big, we should use a smaller step to approach the peak. After a successful matching, since there will be many high matching values near the measuring unit, we adopt a big step to skip this area to avoid repeated matching. As is shown in Figure 1, there is the matching scatter diagram of NCC adopted discriminating step (the scatters are bolded).

From the scatter diagrams, it is found that discriminating-step method can reduce much calculation. When the matching value is greater than the threshold (the value is 0.55 in the diagrams), there are a few matching values around the match point and only one value near the peak generally, that is, the grayscale matching value of this matching position. Table 1 lists the tabulated step settings.

Table 1.

Tabulated step settings.

Values	Step (X direction)
<0.4	0.07a
0.4–0.5	0.05a
0.5–0.55	0.03a
>0.55	1 pixel
After a matching	0.4a

a: the width of a microunit.

Pyramid search strategy lessens the image by the preset threshold to reduce its distinguished ability, that is, to reduce the pixels to reduce the matching times. When the matching value is higher, return to the original image and matching exactly to get the position where the template is on the measurement image.

When pyramid search strategy is used to lessen the image, the programs will turn the grayscale of several adjacent pixels into one pixel by weighted average, essentially. This may cause positional deviation or multiple positions. While it returns to the original image, it will lead to a decreasing accuracy.

When the template matches by steps progressively, there may be several matching values close to the maximum values near the true position. And we set a greater step to increase the matching speed, which may lead to jumping over the best matching point.

So in order to solve these problems, we introduce a kind of sensitive area methods. When the grayscale matching value is greater than the threshold, delimit a wider area that contains where the testing microunit is, as is shown in Figure 3. The external blue wireframe is the range of sensitive area, and the internal red one is the position of true microunit.

Figure 3.

(a) The measurement (testing) image, (b) template, and (c) the measuring equipment.

The measurement routine will conduct matching exactly to get a maximum value that is regarded as the true matching value of the microstructures in this position. Then, the measurement routine will return to the position of the true matching value on the pyramid layer, conducting a greater lateral leap to make the template far away from the matching position, and then enter the next matching area rapidly.

With regard to the complex image recognition issue, the ant colony algorithm or artificial neural network algorithm can be applied to resolve it. But the pyramid search algorithm and discriminating-step algorithm have a unique advantage on coping the issue, since the micro-structured surface is a kind of repetitive onefold structure, which enables a simplified image recognition through automated computational comparison.

Measurement experiments, results, and discussion

In these experiments, the measurement image and the template are shown in Figure 3. We used different methods to compare the traditional method with the improved methods (NCC matching strategy, pyramid matching strategy, discriminating-step matching strategy, and rapid matching strategy that uses pyramid strategy and discriminating-step strategy at the same time). In the measurement experiments, the zoom ratios of pyramid matching are 0.5 and 0.25, and the one of rapid matching is 0.5.

In order to reflect the difference between the testing microstructures and template intuitively, the matching degree will be classified in the measurement routine through the different preset thresholds, and the matching images in different grades will be distinguished by different colors. The matching degrees can be divided into following five grades: matching values from 0.55 to 0.65 are identified by red wireframes, matching values from 0.65 to 0.70 are identified by pink wireframes, matching values from 0.70 to 0.75 are identified by yellow wireframes, matching values from 0.75 to 0.90 are identified by green wireframes, and matching values over 0.90 are identified by blue wireframes. As shown in Figure 4, the processing quality can be made out intuitively. Then, the measurement routine will return the matching positions and grades into two-dimensional color blocks diagram to eliminate the interference from low matching values in some position, as is shown in Figure 5.

Figure 4.

Repeatability comparison diagrams of different matching strategies: (a) NCC strategy, (b) pyramid strategy, (c) discriminating-step machining strategy, and (d) rapid matching strategy.

Figure 5.

Accuracy comparison diagrams with different matching strategies: (a) NCC strategy, (b) pyramid strategy, (c) discriminating-step machining strategy, and (d) rapid matching strategy.

In terms of repeatability, it is found that the polyisomerism is serious if the measurement method is without discriminating-step strategy from Figure 4.

In terms of accuracy, NCC strategy is real and accurate. Therefore, from Figure 4, it is found that the accuracy of these improved algorithms changes little on the basis of NCC and the defects can be measured by them. The accuracy of the measurement method that adopts discriminating-step matching strategy will decrease a little, but it does not affect the defects measurement (all algorithms found where the defects are). Therefore, this rapid matching algorithm can be adopted in micro-structured surfaces’ measurement.

The running environment of these comparison experiments is Core i5, dual-core (2.40 GHz, 2.40 GHz) Intel processor, 4 GB for memory, and the software platform is MATLAB R2012a. We still used the picture shown in Figure 2, whose size is 768 × 576 pixels, to do the comparison experiments. The time comparison of the final result is listed in Table 2. The number in the parenthesis is the zoom ratio of the pyramid strategy.

Table 2.

Matching time table of different methods.

Matching algorithms	NCC algorithm	Pyramid algorithm (0.5)	Pyramid algorithm (0.25)	Discriminating-step algorithm	Rapid matching algorithm (0.5)
Time (s)	215.1	63.4	18.3	3.8	2.9

NCC: normalized cross correlation.

It is obvious that discriminating-step algorithm can shorten the time required significantly. And the pyramid searching algorithm can also shorten the time though the time shortened is determined by the zoom ratio. If the zoom ratio is 0.5, the image’s size will become 0.25 times than the original size and matching times will reduce 75%. However, excessive lessening will lead to image distortion and then cause a decreasing matching accuracy. Therefore, the zoom ratio should not be too small. Although there is volatility in the matching time, the improved rapid matching algorithm still owns greater advantage than other algorithms in speed.

Conclusion

The accuracy of NCC algorithm is quite high in allusion to the repeated structure, so it can be a useful measurement method for micro-structured surfaces as long as its drawback of slow matching can be overcome. Adopting pyramid searching and discriminating-step algorithm can increase the matching speed substantially, but the measurement accuracy is still a serious problem as restricted by applying this improved algorithm. In order to resolve this issue, sensitive area analysis is further integrated into the improved algorithms development. To get a satisfactory result, the size of the sensitive area must be determined by the actual situation, such as the size of the microunit surface and the threshold preset, and the corresponding setting optimization.

On the other hand, in order to ensure the accuracy of measurement results, the step of discriminating-step algorithm and the zoom ratio of pyramid searching algorithm must be set cautiously. The step should be set according to the precision requirement of processing. It will affect the matching speed if the step is too small, otherwise it will affect the accuracy. As for the zoom ratio of pyramid searching algorithm, if it is set big it will increase the measurement accuracy, otherwise it will reduce the accuracy while increasing the matching speed. Therefore, the zoom ratio must be maintained as a balance between the matching accuracy and matching speed according to the practical measurement requirements. If high measurement precision is needed for the component with micro-structured surfaces, the larger zoom ratio is recommended, and if high matching speed is needed, the smaller ratio recommended.

The measurement routine for the micro-structured surfaces can be further improved by adopting the in-process measurement. It can immediately measure the components in-process with a higher speed as long as there are a set of high-resolution image acquisition device and a complete data transmission system in accessing the measurement routine. Setting with a threshold according to the processing interval, the image acquisition device will sample an image at intervals under the computer control. By transferring the image data to the measurement routine through the data transmission system, the measurement routine can likely integrate the in-process measurement with the machining process for the component with micro-structured surfaces, which is essential for future industrial-scale micromanufacturing. ¹³