Sage Journals: Discover world-class research

Abstract

As to the problems of local stereo matching methods, such as matching window selection difficulty, fuzzy disparity edges and low accuracy in weak texture regions, this paper proposes an efficient stereo matching algorithm to improve the stereo matching accuracy in these regions. First of all, we segment the stereo images and calculate the adaptive support window according to the area of each segmentation region. Second, the matching cost is computed based on the feature fusion of color and gradient, and then the initial disparity can be achieved. Finally, the ultimate matching disparity can be obtained through a series post-processing, including consistency checking, mismatch correcting, disparity refinement and so on. Test results of Middlebury Stereo Datasets show that the proposed algorithm is effective with high matching precision, and especially can tackle well with the weak texture and slope surfaces regions.

Keywords

Stereo matching image segmentation disparity adaptive support window multi-feature fusion

Introduction

Stereo matching is one of the most active research areas in computer vision, and it is widely used in three-dimensional surface modeling, 3D object recognition, vision navigation, aviation and space exploration, industrial monitoring, image-based rendering and so on.¹ The Stereo matching problem is formulated with the process of computing a disparity map through a pair of stereo images given before, and the effect of three-dimensional reconstruction is directly affected by matching results. Domestic and foreign scholars have done a lot of researches on the stereo matching in recent years. Scharstein et al. summarized and evaluated some representative stereo matching algorithms in Scharstein and Szeliski,² and divided them into local and global algorithms. In global stereo matching algorithm, disparity can be estimated by minimized a global optimization function through establishing data item and smooth item. Graph cut algorithms,³ belief propagation^4,5 and dynamic programming⁶ are some kinds of global algorithms with good effect. Global algorithms can handle well with occluded and weak texture regions; however, complex parameters setting, tedious calculation and optimization for global energy function are time-consuming, which makes it not suitable for real-time platform application. On the contrary, in local methods, disparity in complex texture regions can be recovered quickly and efficiently, but it is difficult to select the size and shape of the support window adaptively; especially, mismatch maybe appear in the weak texture and disparity discontinuous regions, and thus match accuracy decreases in the final result.

In local methods, the emphasis is on the selection of match support window. Matching algorithm-based window is often trying to enlarge the support window size to enhance the distinguish ability in weak texture regions. However, it would induce excessive smooth disparity and “foreground hypertrophy” effect in discontinuous areas. As to this problem, many representative local matching algorithms come up, and they not only consider the gray level of image, but also take color into account. All these local methods are divided into two categories in Zhang et al.⁷ One category is mainly selecting the best window in the pre-given windows as the support window,^8,9 or choosing the size and shape for support window point by point adaptively.^10–12 When computing the matching cost, the match accuracy can be improved by changing the window size adaptively. With this kind of algorithms, the disparity precision in difficult matching regions can be improved, and the foreground hypertrophy phenomenon can be inhibited as well, but because of the fixed shape of the window and the lack of flexibility, matching error rate is still high and disparity edge is not distinct. The other kind of method is based on the idea of pixel weighted in the support window with established size and shape. Yoon et al. proposed a locally adaptive support-weight approach with highly match precision in Yoon and Kweon,¹³ in which the segmentation weights adapts according to the color and geometric relationship between pixels within the specified window and pixels to be matched, but it is computational complexity and high operation cost, it could not reflect the efficiency of local algorithm. Federico et al. improved the weighted function by color segmentation in Tombari et al.,¹⁴ which improved the match precision, but further increased the operation cost. A highly precision disparity map can be obtained with the method of the combination with adaptive weights, non-parametric and disparity refinement in Gu et al.,¹⁵ but the combination produces a higher operation cost. An adaptive weighted method with highly precision disparity map is proposed in the literatures,^16,17 the pixel in the match window is assigned weight adaptive based on the difference in local region, but the method increases the match window size and operation cost. In Yu and Rong,¹⁸ the pixels in match window are weighed by multiple mode, but weight calculation algorithm is much complex. Segmentation algorithm based on color proposed in Ni et al.¹⁹ exhibits superior performance in weak texture regions for stereo matching, but it depends on the stability of segmentation algorithm and the accuracy of the initial disparity, so it is not suitable for too large weak texture regions.

As to the problems mentioned in local methods which are difficult to select support window and the precision and speed are contradictory, a new stereo matching algorithm based on color segmentation in the CIELAB color space for adaptive window and multi-feature fusion is proposed in this paper, which combines the advantages of color segmentation and local window. Experiment result shows that highly match precision can be gained by the proposed algorithm in regions with discontinuities disparity and weak texture, especially there is better effect in large regions with weak texture, slope plane and disparity edge.

Basic assumption and process

As concluded above, there are some problems in local stereo matching.²⁰ First of all, it is difficult to select the matching support window, for small window size will induce mismatch in weak texture regions, but large window size will lower the match efficiency. Second, non-weighted window improves the efficiency and has noise immunity, but it will cause over smooth in large window edge and sensitive to occluded regions, and then can give rise to mismatch easily.

So we assume that disparity of pixels in the same segment region is approximately the same or smooth; in this premise, we propose an local stereo matching algorithm using adaptive support window and multi-feature fusion based on segmentation, ASWMFF for short. First of all, the stereo images are segmented by Mean-shift method in CIELAB color space, the area of each segment region is counted, and the size of each matching window is calculated according to the area. The key point is to select these pixels inside the window to compute the match cost which with the matching points belong to the same segmentation region. Meanwhile, the matching cost is calculated by using the multi-feature linear weighting fusion method, which includes color and gradient features. Then the initial disparity maps of the stereo image pairs, respectively, can be obtained by using sum of absolute differences (SAD) to measure similarity. Finally, the ultimate refined disparity map can be gained through a series of experiments by the optimization of the initial disparity map, including mismatch check, classification and disparity refinement, in which reliable and unreliable disparity can be gained by horizontal disparity consistency check and winner-take-all(WTA) method, and then classify the unreliable disparity as mismatch or occlusion, and optimize the unreliable disparity by the neighborhood of reliable disparity.

Adaptive window selection based on color segmentation

Color image segmentation based on CIELAB space

A more suitable segment image can be got for human visual system in CIELAB color space, for which is a kind of apperceive uniform color space, the luminance and color in this space are independent with each other, and the Euclidean distance of different colors in visual has equivalent chromatic perception in different direction and different place. The proposed method first of all converts the RGB image into CIELAB color space and then segments the converted image by Mean-Shift; finally, calculate the size of match window according to the segmentation. The segmentation results as shown in Figure 1.

Figure 1.

Left image and segmentation result. (a) Left image and (b) result of segmentation.

Adaptive selection of matching window

According to the color information, we can segment the image into several regions. If the segment region is too large, though pixels inside the window only with close color can be segmented into the same area, the disparity inside the window may change greatly when there is slope plane or curve surface. So over-segmentation is often used to avoid this situation, but it is usually with high requirement of parameter setting. However, in the premise that disparity of pixels in the same segment area is approximately the same or smooth, this paper proposes an adaptive window selection method based on segmentation, which can overcome the limitation of too large segmentation, and even if it is so, the ratio calculation method can get proper matching windows.

The first step of calculating adaptive window is to get area of every segment region, i.e. compute the total pixel number of every segment region, and then determine the size of the initial rectangular window centered at the pixel to be matched using equation (1). Next, we choose some pixels in the initial rectangular window to form the adaptive matching support window, which belongs to the same segmentation region with the pixel to be matched, and the matching window may be the region of any shape and size rather than a fixed region. Detailed process is as follows.

Adaptive initial rectangular window selection.

i is segment label for image to be matched,

s (i)

is area of the segment region i, and

L (i)

is the size of scaling adaptive rectangular window in segment patch i. The size of rectangular match window for segment patch i is

L (i) \times L (i)

, and the equation for scaling adaptive rectangular window is as follows

k (i) = {[s (i) / b 1] s (i) > b 1 b 2 others

(1)

L (i) = {k (i) kisodd k (i) + 1 kiseven

(2)

Based on a large number of experiments, we empirically set b1 in equation (1) as 100 in this paper, but k is set as 1 or 0 by scaling calculation when the size of segment patch is smaller than 100, and then it is point match rather than window match. In this situation, it is easy to induce mismatch in weak texture regions for the area of segment region is too small, and so we define a fixed match window b2. Through a large number of experiments, we empirically set b2 as 5 as the fixed match window for small area. Equation (2) is a rectangular window with

L (i)

is odd.

Selection for adaptive support match window

The initial matching window is selected according to the segment region without considering segmentation boundary limit. The pixels inside the window may cross more than one segment region and exist in discontinuous or occluded region as well. The accuracy of the match result will be disturbed if they all take part in calculating match cost, and it will cause boundaries of disparity fuzzy and over smooth and mismatch. In order to get disparity map with clear boundaries, many researchers describe the relationship between pixels inside the rectangular window and the pixels to be matched using weight method, although highly match accuracy got, the calculation of weight value is very complex with low efficiency. To improve the efficiency and meanwhile ensure the accuracy, we propose a simple adaptive matching support window selection method by re-selecting the initial rectangle match window of pixels to be matched. We just select the pixels belonging to the same segment region with the pixel to be matched as the actual matching region, and then a new adaptive deformable matching region is formed, the shape of which is changeable with different pixels to be matched and segment results, and so a reliable matching support window can be provided for the latter match cost calculation. The comparison results of five optional adaptive variable matching windows are shown in Figure 2. White rectangular region in Figure 2(a) is the initial rectangle window, and the blue region in Figure 2(b) is the ultimate selection of adaptive matching support window region.

Figure 2.

Adaptive support matching window sketch map. (a) Initial rectangle windows and (b) adaptive support matching window.

Matching cost computation of multi-feature fusion and disparity optimization

Matching cost of multi-feature fusion

Matching cost based on segmentation of CIELAB color space

CIELAB color space is a perceptual uniform color space, in which luminance information and chrominance information are independent, which is visual to the human eye with visual characteristics, so the chroma and luminance information of the three-channel in LAB space is selected to calculate match cost in this paper.

It is very important to select the appropriate similarity measure function for the disparity accuracy of stereo matching, and SAD is a simple and efficient similarity measure function used in local matching process commonly. This paper calculates stereo matching cost as well by using SAD. Chroma segmentation matching cost is calculated by equation (3). I_L and I_R denote the reference image and the target image. Suppose p is an arbitrary pixel point in I_L, q is the pixel in I_R, i.e. $p \in I_{L}$ , $q \in I_{R}$ , p and q are assumed to be corresponding matching points, and the disparity between them is d. The match cost can be calculated as follows

C_{Lab} (p, q, d) = \sum_{p_{i} \in W_{p}} min (e_{1} (p_{i}, q_{i}, d), T_{r})

(3)

where C is the match cost of region W_P, W_P is the adaptive matching support window region for pixel p in the reference image obtained by the proposed method, T_r denotes a fixed pre-set cut-off threshold, which is based on M-estimator that can reduce the impact of the outliers and increase the robustness of the algorithm.

e_{1} (p_{i}, q_{i}, d)

is the matching cost of the pixels p and q when disparity is d, which represents the distance of two pixels in the CIELAB color space, defined as follows

e_{1} (p_{i}, q_{i}, d) = | L_{p_{i}} - L_{q_{i}} | + | A_{p_{i}} - A_{q_{i}} | + | B_{p_{i}} - B_{q_{i}} |

(4)

where

L_{p_{i}}

A_{p_{i}}

, and

B_{p_{i}}

represent LAB three-channel component of p_i point, respectively, and

L_{q_{i}}

A_{q_{i}}

, and

B_{q_{i}}

represent LAB three-channel component of q_i point, respectively.

Combined matching cost with introducing gradient magnitude

The premise of combined matching cost based on chrominance segmentation is that disparity of pixels in the same segment region is approximately the same or smooth; in the actual case, a sudden change in the color is often the edge of depth map, i.e. the regions of discontinuous disparity, and therefore the matching accuracy can be improved in regions of weak texture and disparity discontinuous regions based on chrominance segmentation, but it will cause identical disparity or disparity smooth in the same segment region if only dependent on chrominance segmentation. It will cause blurred edges and other mismatch, i.e. when adjacent region is with similar color but different depth.

The problem of mismatching and depth edge blur can be avoided as far as possible by the proposed method, i.e. using a scaled-down adaptive matching window algorithm, but mismatching cannot be completely avoided, thanks to the image gradient features, which can describe the feature of edges and slope surface regions more accurate.²¹ Therefore, in order to further improve the matching accuracy in disparity discontinuous regions, we introduce the image gradient information into the matching objective function, combining with Lab three-channel component information to form the weighted-combination matching cost calculation as shown in equation (8), thus further improving the matching accuracy of edges and weak texture regions.

The gradient magnitude characteristic of any point (x, y) in the image is calculated as follows

{G_{x} (x, y) = H (x + 1, y) - H (x - 1, y) G_{y} (x, y) = H (x, y + 1) - H (x, y - 1) \nabla (x, y) = \sqrt{G_{x} (x, y)^{2} + G_{y} (x, y)^{2}}

(5)

where

G_{x} (x, y)

is the horizontal gradient,

G_{y} (x, y)

is the vertical gradient,

H (x, y)

is the pixel intensity, and

\nabla (x, y)

is the gradient magnitude of the pixel point

(x, y)

p and q are assumed to be corresponding matching points and the disparity between them is d. The match cost can be calculated as follows

C_{\nabla} (p, q, d) = \sum_{p_{i} \in W_{p}} e_{2} (p_{i}, q_{i}, d)

(6)

e_{2} (p_{i}, q_{i}, d) = | \nabla_{p_{i}} - \nabla_{q_{i}} |

(7)

where

C_{\nabla}

is the match cost of region W_P,

\nabla_{p_{i}}

is the gradient magnitude of pixels point p_i in the reference image,

\nabla_{q_{i}}

is the gradient magnitude of pixels point q_i in the target image, e₂ is the matching cost of pixel point p_i and pixel point q_i when the disparity between them is d.

We calculate the matching cost by using the linear weighted method of the two features, and in this paper, an improved C model combined with $C_{Lab}$ and $C_{\nabla}$ , called C_W, is proposed as follow

C_{W} (p, q, d) = \sum_{p_{i} \in W_{p}} min (\cos t (p_{i}, q_{i}, d), T_{r})

(8)

\cos t (p_{i}, q_{i}, d) = ω 1 \times C_{Lab} (p_{i}, q_{i}, d) + ω 2 \times C_{\nabla} (p_{i}, q_{i}, d)

(9)

where

ω 1

and

ω 2

are the two weight coefficient of matching cost fusion,

ω 1

is the gradient feature weighted coefficient, and

ω 2

is the color feature weighted coefficient,

ω 1 + ω 2 = 1

The initial disparity maps of the reference left image and target right image can be obtained by the method described above.

Disparity optimization

The ultimate dense and high accuracy disparity map can be obtained after a series of optimization for the initial disparity maps. First of all, we check the initial left and right disparity map by disparity consistency check and winner-take-all method, then classify them into sets of reliable and unreliable disparity, and then classify the unreliable disparity²⁰ into three kinds, such as mismatch, occlusion, and both mismatch and occlusion. Optimize each kind unreliable disparity according to the method in Haizhong et al.²⁰

Experiments and results

The proposed algorithm is realized in MATLAB. In order to verify its efficiency, we test and evaluate the ASWMFF algorithm in the Middlebury dataset, which is released on Middlebury website (http://vision.middlebury.edu/stereo/) recognized by the current academic community. During the experiment, we set $b 1 = 100$ and $b 2 = 5$ for match window selection; we set $ω 1 = 0.4$ and $ω 2 = 0.6$ for matching cost computation, the weighted coefficient setting is taken into account that the noise immunity of color feature is stronger than that of texture features, and we empirically set them for solving the problem of weak texture region. The evaluation algorithm error threshold is 1.

Weighted method of data fusion is a simple and effective method, and the weighted coefficient determines the effect of fusion. Of course, the matching result will be affected by different values of $ω 1$ and $ω 2$ _. The gradient feature is more effective when the color difference of complicated textures is not obviously. Therefore, $ω 1$ should be set larger than $ω 2$ in order to enhance the role of the gradient feature in multi-feature fusion. Through the above analysis, we believe that the dynamic weight method is more in line with a variety of different situations stereo datasets. In order to achieve the feature weight conform with the realistic feature distribution on different stereo datasets, and thus further improve the adaptability of stereo matching algorithm, the authors plan to research the dynamic weights setting method in the next future.

The disparity map result obtained by the proposed algorithm is shown in Figure 3. As can be seen from the figure, the proposed algorithm is simple and effective, and precision dense disparity can be obtained in the majority difficulty problems of stereo matching, such as weak texture, occlusion and discontinuous regions, even in large weak texture regions and slope surface.

Figure 3.

Experimental results on Middlebury dataset. Left-to-right: Tsukuba, Venus, Teddy, Cones, respectively. Top-to-bottom: color images, ground truth, results of our method, and error maps with threshold equal to 1.0.

Disparity map for Teddy image is obtained by four different methods is shown in Figure 4. The algorithm proposed in the literature² is a classic graph cut matching algorithm. In Figure 4(a) and (d), we can see that our method can overcome the fuzzy mismatch at disparity edges, and significantly improve the match accuracy and obtain dense disparity in the blue oval regions, cabinet edges and other regional disparity discontinuities in the side of the cabinet and other large slope weak texture regions; the algorithm proposed in literature¹⁴ is one of the best local algorithm, and the method proposed in the literature²² is one of the latest outstanding local algorithm. As seen in Figure 4(b), (c) and (d), our algorithm has a certain advantage in the blue oval regions, cabinets and other disparity discontinuities regions.

Figure 4.

Processing results for Teddy using four methods: (a) results of Scharstein and Szeliski,² (b) results of Tombari et al.,¹⁴ (c) results of Jiao et al.²⁷ and (d) results of our method.

To evaluate the performance of the proposed algorithm, we upload our results to the Middlebury website (http://vision.middlebury.edu/stereo/) and the error threshold is 1 pixel. The evaluation result feedback from the website is partly listed in table 1, sorted by the comprehensive ranking feedback from the website, and evaluated on the basis of the three quantitative algorithm offered by the website. In which, nonocc is the mismatch of non-occlusion regions, all is the mismatch of whole image, disc is the mismatch of disparity discontinuous regions and our method is the proposed algorithm.

Table 1.

Quantitative evaluation of the proposed method compared with other methods on Middlebury.

Error Threshold = 1 Algorithm	Tsukuba			Venus			Teddy			Cones			Average percent of bad pixels
Error Threshold = 1 Algorithm	Nonocc All Disc			Nonocc All		Disc	Nonocc All		Disc	Nonocc All		Disc	Average percent of bad pixels
AdaptingBP²²	1.11	1.37	5.79	0.10	0.21	1.44	4.22	7.06	11.8	2.48	7.92	7.32	4.23
CoopRegion²³	0.87	1.16	4.61	0.11	0.21	1.54	5.16	8.31	13.0	2.79	7.18	8.01	4.41
CCRADAR²⁴	1.15	1.42	6.23	0.15	0.27	1.89	5.39	10.6	14.7	2.01	7.37	5.88	4.75
SegAggr²⁵	1.99	2.39	8.59	0.12	0.21	1.68	2.19	3.73	7.02	2.16	6.52	6.37	3.58
GC + LSL²⁶	2.43	2.73	10.6	0.25	0.36	2.89	2.02	3.77	6.99	2.77	7.37	8.05	4.19
AdaptDispCalib¹⁵	1.19	1.42	6.15	0.23	0.34	2.50	7.80	13.6	17.3	3.62	9.33	9.72	6.10
Our method	1.43	1.68	7.73	0.24	0.36	3.23	7.52	12.8	20.9	3.57	8.91	10.1	6.54
SegmentSupport¹⁴	1.25	1.62	6.68	0.25	0.64	2.59	8.43	14.2	18.2	3.77	9.87	9.77	6.44
OverSegmBP²⁷	1.69	1.97	8.47	0.51	0.68	4.69	6.74	11.9	15.8	3.19	8.81	8.89	6.11
CostAggr + occ²⁸	1.38	1.96	7.14	0.44	1.13	4.87	6.80	11.9	17.3	3.60	8.57	9.36	6.20
AdapWeight²⁹	1.38	1.85	6.90	0.71	1.19	6.13	7.88	13.3	18.6	3.97	9.79	8.26	6.67
SemiGlob³⁰	3.26	3.96	12.8	1.00	1.57	11.3	6.02	12.2	16.3	3.06	9.75	8.90	7.50
HistoAggr³¹	2.47	2.71	11.1	0.74	0.97	3.28	8.31	13.8	21.0	3.86	9.47	10.4	7.33
GC + occ³²	1.19	2.01	6.24	1.64	2.19	6.75	11.2	17.4	19.8	5.36	12.4	13.0	8.20
TreeDP³³	1.99	2.84	9.96	1.41	2.10	7.74	15.9	23.9	27.1	10.0	18.3	18.9	11.7
H-Cut³⁴	2.85	4.86	14.4	1.73	3.14	20.2	10.7	19.5	25.8	5.46	15.6	15.7	11.7
DP²	4.12	5.04	12.0	10.1	11.0	21.0	14.0	21.6	20.6	10.5	19.1	21.1	14.2
SSD + MF²	5.23	7.07	24.1	3.74	5.16	11.9	16.5	24.8	32.9	10.6	19.8	26.3	15.7

In Table 1, algorithms of Coop Region, Segment Support, Cottager + occ, AdaptDispCalib, AdapWeight, SegAggr, AdaptLocalSeg and so on are the latest outstanding local matching algorithms, and algorithms of AdaptingBP and OverSegmBP are some of the most outstanding global matching algorithms. Seeing mismatch quantitative data feedback from the website in Table 1, the proposed method is slightly worse than the most outstanding algorithms such as CoopRegion and AdaptingBP, but much better than GC + occ, TreeDP, SemiGlob and other global algorithms, and in the AdapWeight, SSD + MF and other local algorithms, the matching accuracy is superior to most of algorithms, and neck by neck with the current mainstream adaptive window algorithms such as SegmentSupport and AdaptLocalSeg, and meanwhile, the simple and effective adaptive window selection method proposed in the paper can improve the computational efficiency and applicability.

Conclusions

With adaptive window selection based on color segmentation and multi-feature fusion, we proposed a stereo matching algorithm. First of all, the stereo images are segmented by Mean-shift method in CIELAB color space. Pixel disparity is same or smooth in the same segmented region. The initial matching window is acquired by each segmented area, in which the final matching region is obtained by choosing pixels in the same segmented region with the pixel to be matched. And then the matching cost is calculated by the multi-feature fusion method, and the method overcomes the fuzzy disparity boundaries and enhances the accuracy of matching in weak texture regions. Finally, we get high precision dense disparity map by reliable classification and disparity optimization. The experiment results show that our algorithm is simple and effective. The feedback data from Middlebury website shows that the proposed algorithm exceeds most of the current outstanding matching algorithms, and illustrates the effectiveness of the proposed algorithm in weak texture and occluded regions, as well as the disparity discontinuous regions. However, mismatch ratio of disc is higher compared with the other two indexes in the three quantitative evaluation indexes. The future work will focus on further improving the matching accuracy of the disparity discontinuous regions and research the dynamic weights setting method, and getting better dense disparity.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The research is supported by the grant from the Principal Research Foundation of Xi’an Technological University (grant no. XAGDXJJ14024).

References

Zitnick CLand Kang

. Stereo for image-based rendering using image over-segmentation. Int J Comput Vision 2007; 75: 49–65.

Scharstein

Szeliski

. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vision 2002; 47: 7–42.

Felzenszwalb

Zabih

. Dynamic programming and graph algorithms in computer vision. Pattern Anal Mach Intell 2011; 33: 721–740.

Yang

Wang

Yang

. Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. Pattern Anal Mach Intell 2009; 31: 492–504.

Klaus A, Sormann M and Karner K. Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: Proceedings of the 18th international conference on pattern recognition, ICPR, IEEE, Hong Kong, China, 20–24 August 2006, pp.15–18.

Deng Y and Lin X. A fast line segment based dense stereo algorithm using tree dynamic programming. In: Proceedings of 9th European conference on computer vision, ECCV, Graz, Austria, 7–13 May 2006, Berlin Heidelberg: Springer, pp.201–212.

Zhang

Lafruit

. Cross-based local stereo matching using orthogonal integral images. Circuit Syst Video Technol 2009; 19: 1073–1079.

Zhou

Wen

Wang

. Fast stereo matching using adaptive window. Chinese J Comput 2006; 29: 473–479.

Fusiello A, Roberto V and Trucco E. Efficient stereo with multiple windowing. In: Proceedings of IEEE Computer Society conference on computer vision and pattern recognition, CVPR, IEEE, Portland, Oregon, USA, 17–19 June 1997, pp.858–858.

10.

Veksler

. Stereo correspondence with compact windows via minimum ratio cycle. Pattern Anal Mach Intell 2002; 24: 1654–1660.

11.

Veksler O. Fast variable window for stereo correspondence using integral images. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (vol. 1), CVPR, IEEE, Madison, WI, 2003, pp.I-556–I-561.

12.

Xu Y, Wang D, Feng T, et al. Stereo computation using radial adaptive windows. In: Proceedings of 16th international conference on pattern recognition, ICPR, IEEE, Quebec, Canada, 11–15 August 2002, pp.595–598.

13.

Yoon KJ and Kweon IS. Locally adaptive support-weight approach for visual correspondence search. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, CVPR, IEEE, San Diego, CA, USA, 2005, pp.924–931.

14.

Tombari

Mattoccia

Di Stefano

. Advances in image and video technology, Berlin Heidelberg: Springer, 2007, pp. 427–438.

15.

Liu

. Local stereo matching with adaptive support-weight, rank transform and disparity calibration. Pattern Recog Lett 2008; 29: 1230–1235.

16.

De-Maeztu

Villanuea

Cabeza

. Stereo matching using gradient similarity and locally adaptive support-weight. Pattern Recog Lett 2011; 32: 1643–1651.

17.

Hosni

Bleyer

Gelautz

. Secrets of adaptive support weight techniques for local stereo matching. Comput Vision Image Understand 2013; 117: 620–632.

18.

Rong

. Multi-pattern stereo matching with intensity-weighted correlation. Int J Advance Comput Technol 2012; 4: 466–473.

19.

Weiji

Guili

. Fast stereo matching based on color segmentation and adaptive window. Chinese J Scientific Instrument 2011; 1: 194–200.

20.

Chen

Zhang

. A new adaptive stereo matching method based on local edge features. Optical Technique 2013; 06: 510–516.

21.

Liu

Zhang

. Stereo matching algorithm of adaptive window based on gradient. Comput Modern 2012; 1: 67–69.

22.

Klaus A, Sormann M and Karner K. Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: Proceedings of 18th international conference on pattern recognition, ICPR, IEEE, Hong Kong, China, 20–24 August 2006, pp.15–18.

23.

Wang ZF and Zheng ZG. A region based stereo matching algorithm using cooperative optimization, In: Proceedings of IEEE conference on computer vision and pattern recognition, CVPR, IEEE, Anchorage, AL, USA, 24–26 June 2008, pp.1–8.

24.

Jiao

Wang

. Local Stereo Matching with Improved Matching Cost and Disparity Refinement. IEEE Transactions on Multimedia 2014; 21(4): 16–27.

25.

Muninder V, Soumik U and Krishna A. Robust segment-based stereo using cost aggregation. In: Proceedings of international conference on British Machine Vision Conference, BMVC, IEEE, Nottingham, UK, 1–5 September 2014.

26.

Taniai T, Matsushita Y and Naemura T. Graph cut based continuous stereo matching using locally shared labels. In: Proceedings of IEEE conference on computer vision and pattern recognition, CVPR, IEEE, Columbus, OH, 2014, pp.1613–1620.

27.

Zitnick

Kang

. Stereo for image-based rendering using image over-segmentation. Int J Comput Vision 2007; 75: 49–65.

28.

Min

Sohn

. Cost aggregation and occlusion handling with WLS in stereo matching. IEEE Transact Image Process 2008; 17: 1431–1442.

29.

Yoon

Kweon

. Adaptive support-weight approach for correspondence search. IEEE Transact Pattern Analysis Mach Intell 2006; 28: 650–656.

30.

Hirschmuller H. Accurate and efficient stereo processing by semi-global matching and mutual information. In: Proceedings of IEEE Computer Society conference on computer vision and pattern recognition (vol. 2), CVPR, IEEE, San Diego, CA, 2005, pp.807–814.

31.

Min D, Lu J and Do MN. A revisit to cost aggregation in stereo matching: How far can we reduce its computational redundancy? In: Proceedings of 13th international conference on computer vision, ICCV, IEEE, Barcelona, Spain, 2011, pp.1567–1574.

32.

Kolmogorov V and Zabih R. Computing visual correspondence with occlusions using graph cuts In: Proceedings of 8th international conference on computer vision (Vol.2), ICCV, IEEE, Vancouver, Canada, 2001, pp.508–515.

33.

Veksler O. Stereo correspondence by dynamic programming on a tree. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition (Vol.2), CVPR, IEEE, San Diego, CA, 20–26 June 2005, pp.384–390.

34.

Miyazaki D, Matsushita Y and Ikeuchi K. Interactive shadow removal from a single image using hierarchical graph cut. In: Proceedings of the 9th Asian conference on computer vision, ACCV, Springer Berlin Heidelberg, Xi’an, China, 23–27 September 2009, pp.234–245.

Segment-based adaptive window and multi-feature fusion for stereo matching

Abstract

Keywords

Introduction

Basic assumption and process

Adaptive window selection based on color segmentation

Color image segmentation based on CIELAB space

Adaptive selection of matching window

Matching cost computation of multi-feature fusion and disparity optimization

Matching cost of multi-feature fusion

Matching cost based on segmentation of CIELAB color space

Combined matching cost with introducing gradient magnitude

Disparity optimization

Experiments and results

Conclusions

Footnotes

Declaration of conflicting interests

Funding

References