Intelligent Carom Billiards Assistive System for automatic solution path generation and actual path prediction with principal component analysis-based ball motion detection

Abstract

This article proposes an Intelligent Carom Billiards Assistive System for a novice at Carom Billiards. Since a novice player is very difficult to find the solution paths of the cue ball, the ICBAS automatically calculates the solution paths for given ball configuration which is detected by a monocular vision and displays them in a guidance monitor. For rapid and systematically calculating the solution paths, the five and half system that is the most representative method in the field of billiard is implemented. In addition, the paths of the cue ball changed by the direction of the cue stick are predicted in real time and projected onto the pool table bed. By comparing the solution paths of the cue ball with the projected paths, a player can hit the cue ball easily and successfully. For automatically providing the solution paths and the predicted paths during billiard game, the ball movement is detected by the principal component analysis under linear motion constraints robustly against image noise, occlusion, and so on. Whenever all of the balls stop, both paths are newly calculated and provided for a player. For verifying the feasibility of the proposed Intelligent Carom Billiards Assistive System, the experiments are conducted with the real pool table.

Keywords

Carom billiards five and half system monocular camera ball path generation principal component analysis

Introduction

According to the report of the Ministry of Culture, Sports and Tourism of the Republic of Korea in 2014, 62.2% of leisure activities are relaxation, but only 8.6% are sports activities. That is because most sports require an understanding of complex rules and a certain level of athletic ability. In addition, in the case of intense sports, the risk of injury can be felt. Unlike other sports, cue sports (or billiard sports) are rarely injured and are popular sports for all ages. However, there are still barriers to entry because it takes a lot of time to learn some complicated principles and billiards skills.

There are three major games in cue sports: Carom billiards, Pool (Pocket billiards), and Snooker. The difference between Carom Billiards and Pool and Snooker is that Carom Billiards has no pockets on the pool table whereas Pool and Snooker have pockets. There are many different types of games in Pool and Snooker, but the basic rule is for a player to hit the cue ball to put the object ball into the pocket. Because the rule is relatively simple, many assistive systems have been developed for Pool and Snooker.^1

–9 Some study was conducted to provide a player with a simple top view of the pool table using intelligent glasses for self-training.¹ However, most of them focus on the cue ball’s path prediction according to the cue stick’s direction and provide the predicted path for a player. The predicted path is obtained by considering collisions with not only cushions but also object balls. In some research studies, physics-based cue ball’s path prediction was studied for a billiards robot, not for a human being. The multiple neural network compensator was developed for a billiard robot to master the cue ball’s trajectory with multiple cushion rebounds,² and the fuzzy neural network was proposed to predict the cue ball’s final position after stroking and collision.³ In another research,⁴ the four frame long short-term memory (LSTM) model was proposed for training the predictive dynamics model of billiards with sequential visual inputs. Thanks to the physics-based approach, the cue ball’s trajectory and final position can be obtained more accurately, but it requires higher computational complexity due to neural network training and inference. In previous research studies,^5,6 a camera was attached to the player’s head and body for ball and cue stick detection. During a billiards match, it may cause inconvenience to the player’s movement. In other research studies, a band and a marker were used for the cue stick detection.^7,8 In this case, some additional settings are required in advance. Despite these inconveniences, the previous assistive systems seem to be very helpful for a player to find the cue ball path that can put the object ball into the pocket.

Unlike Pool and Snooker, the rule of the Carom billiards is more difficult and complicated. The rule of the three-cushion billiards among games within Carom billiards is that the cue ball hit by a player must hit two object balls and hit the second object ball after contacting at least three cushions. For a novice of the three-cushion billiards, it is challenging to find the solution path of the cue ball satisfying the three-cushion rule. In this article, we propose an Intelligent Carom Billiards Assistive System (ICBAS) for helping a novice player. The proposed ICBAS provides not only the solution paths of the cue ball satisfying the three-cushion rule but also the predicted paths of the cue ball that are changed in real time by the direction of the cue stick. To do so, the ICBAS first detects three balls and a cue stick using a monocular camera with no additional settings required in previous research studies. For robust detection to illumination change, the red–green–blue (RGB) image obtained from the camera is converted into the hue–saturation–value (HSV) image. By applying image preprocessing techniques, such as Gaussian filtering and binary thresholding, and contour extraction technique to the HSV image sequentially, the contour image can be obtained. For detecting three balls, the circle Hough transform (CHT)¹⁰ is applied to the contour image. Finally, each ball color is determined considering the light reflection problem on the balls. Recently, many researchers use convolutional neural networks for edge or contour detection. In the research,¹¹ the fully convolutional encoder–decoder network is trained end-to-end to effectively highlight object contours with suppressing other edges. In the research,¹² the convolutional oriented boundaries are proposed by considering not only contour strength but also orientation for more accurate results. In the research,¹³ the architecture-fusion network is trained end-to-end to effectively reduce the misrecognition of both internal and background edges inside objects without degradation of object detection precision. Deep neural networks (DNNs)-based methods are efficient and useful for object contour detection from various images, but training and inference of DNNs require powerful and expensive systems including graphics processing units (GPUs). Thus, the deep learning system with GPUs seems over-spec to apply to the proposed ICBAS as ball detection on the pool table bed is relatively simple.

With the configuration of the detected three balls, the solution paths of the cue ball can be systematically calculated by using the five and half system¹⁴ that is the most representative method in the field of three-cushion billiards. The ICBAS displays the obtained solution paths in a guidance monitor with the tip-point for the desired spin of the cue ball and the overlap width between the cue ball and the first object ball. A novice player can refer to the solution paths in the monitor during billiard game. However, a player may be difficult to map the solution paths to the actual pool table. Thus, the ICBAS also predicts the cue ball paths changing in real time according to the direction of the cue stick and projects the paths on the actual pool table using a beam projector. That is a kind of augmented reality system. Consequently, a player can successfully play a billiard game conveniently comparing the solution paths and the predicted paths.

Next, it is very important to appropriately provide the solution paths and the predicted paths for a player without interfering with the player’s game. In previous research, it was difficult for the billiard game to proceed smoothly because a player had to trigger using a keyboard, a mouse, and so on manually, to calculate the solution paths every time.¹⁵ To solve this problem, the ICBAS automatically recalculates and provides the solution paths and the predicted paths whenever a player hits the cue ball and all of the balls stop. In this case, the ICBAS must detect the movement of the balls robustly against false detection caused by image noise, occlusion, and so on. Thus, for detecting the motion of the balls after the player hits the cue ball, the principal component analysis (PCA)^16,17 is used under linear motion constraints since the normal motion of the balls is linear. Of course, the motion of the cue ball may not be linear because of spin, but its motion is eventually linear after the spin disappears. Using the PCA with linear motion constraints, the ICBAS can exactly detect the motion of the balls robustly against the abnormal change of the ball position caused by image noise, occlusion, and so on.

The remaining part of this article is organized as follows. The second section describes the concept of the ICBAS and the flowchart of the proposed algorithm. In the third section, the image processing techniques for detecting three balls and a cue stick are described. In addition, the table bed is detected and the homography transformation from the image coordinates to the object coordinates is obtained from the four corner points of the table bed. In the fourth section, for automatic generation of the solution paths and the predicted paths of the cue ball, the detection method of the ball motion is introduced. In this case, the PCA is used for detecting whether all of the balls stop or not. Next, the solution paths of the cue ball are obtained by using the five and half system in the fifth section. In the sixth section, for verifying the feasibility of the proposed ICBAS, experiments are conducted with an actual pool table. Finally, concluding remarks are described in the seventh section.

Intelligent Carom Billiards Assistive System

The basic concept of the proposed ICBAS is shown in Figure 1. The ICBAS provides both the solution paths and the predicted paths for a player to easily make a successful billiard shot. The solution paths are calculated by using the five and half system that is the most representative technique in the field of billiards to systematically find the paths satisfying the three-cushion rule. The resultant solution paths are displayed together with the tip-point for the required spin of the cue ball and the overlap width between the cue ball and the first object ball in a guidance monitor. For player’s convenience, the guidance monitor is continuously controlled to head for a player during billiard game. For automatic generation of the solution paths, the three ball positions are detected by a monocular camera. Next, the actual paths of the cue ball are predicted and projected by a beam projector onto the pool table bed. According to the change of the cue stick direction, the predicted paths are repeatedly recalculated and reprojected as shown in Figure 1. In this case, a player changes the direction of the cue stick to find the predicted path that is coincided with the solution path in the guidance monitor and then just hits the cue ball along the predicted path. In this way, the proposed ICBAS helps a player enjoy his or her own game even though he or she is a beginner of Carom Billiards.

Figure 1.

Basic concept of ICBAS which provides not only the solution paths of the cue ball in the guidance monitor but also the predicted paths on the pool table for a player during billiard game. ICBAS: Intelligent Carom Billiards Assistive System.

In Figure 2, the algorithm of the ICBAS is depicted as the flowchart in detail. The image obtained from the camera is first calibrated by using the GML C++ Calibration Toolbox.¹⁸ Next, the calibrated image is preprocessed for noise reduction and contour detection. From the detected contour image, the pool table bed and the three balls are detected. Using the obtained corner points of the table bed, the homography transformation matrix is calculated from the image coordinates to the object coordinates. Using the homography transformation matrix, the three ball positions in the object coordinates can be calculated from the points detected in the image coordinates. Next, the ICBAS detects the motion of the balls by applying the PCA to the trajectory points of the balls represented in the object coordinates. If all of the balls stop, the ICBAS automatically finds the solution paths of the cue ball and displays them in the guidance monitor. In addition, the cue ball path according to the direction of the cue stick is predicted and projected onto the table bed. If at least one of the balls begins to move, the ICBAS just receives the next image from the camera and continuously checks the ball motion until all of the balls stop again. In this way, the ICBAS appropriately provides the solution paths and the predicted paths for a player during billiard game. The detail methods are described in the following sections.

Figure 2.

Flowchart of the proposed ICBAS algorithm to automatically provide both solution paths and the predicted paths. ICBAS: Intelligent Carom Billiards Assistive System.

Ball and cue stick detection

In this section, the detection method of three balls and a cue stick is introduced based on the image processing techniques. For robust detection to illumination change and image noise, the ICBAS first detects the pool table bed from the contour image and sets the region of interest (ROI) as the quadrilateral of the table bed. Within the ROI, three ball positions are obtained by the circle Hough transform (CHT). Next, the color of each ball is detected by averaging color robustly against the light reflection on the surface of the ball. For detecting the direction of the cue stick, the ICBAS first sets the ROI around the cue ball and detects the center position of the cue stick within the ROI. Finally, the direction of the cue stick is determined as the vector from the center of the cue stick to the center of the cue ball. The detail process is described as follows.

The image from the monocular camera is preprocessed as shown in Figure 3. First, the original RGB image is converted into the HSV image, where V, S, and H values of the HSV color model are calculated from R, G, and B values of the RGB color model as in equations. (1), (2) and (3), respectively.

V = max (R, G, B)

S = \{\begin{array}{l} \frac{(V - min (R, G, B))}{V} & if V \neq 0 \\ 0 & if V = 0 \end{array}

H = \{\begin{array}{l} \frac{(G - B) \times 60}{S} & if V = R \\ \frac{120 + (B - R) \times 60}{S} & if V = G \\ \frac{240 + (R - G) \times 60}{S} & if V = B \end{array}

where H is set as (H + 360) for $H < 0$ . In this case, $0 \leq H \leq 179$ , $0 \leq S \leq 255$ , and $0 \leq V \leq 255$ . Next, the HSV image is thresholded for detecting the green table bed as in equation (4).

(T_{l}^{H}, T_{l}^{S}, T_{l}^{V}) \leq (H, S, V) \leq (T_{u}^{H}, T_{u}^{S}, T_{u}^{V})

where $T_{l}^{K}$ and $T_{u}^{K}$ for $K = {H, S, V}$ are lower and upper threshold values, respectively. For detecting the green table bed, the actual threshold values of $(T_{l}^{H}, T_{l}^{S}, T_{l}^{V})$ and $(T_{u}^{H}, T_{u}^{S}, T_{u}^{V})$ are determined as (80, 100, 10) and (100, 255, 200), respectively. The hue value H is determined within the table green color range with a small offset value. However, the lower threshold of the saturation S is less than half and the upper threshold is the maximum, so the range of the saturation S is very wide. Furthermore, the range of the brightness V covers almost the entire range. As a result, it is possible to detect the pool table without being greatly affected by illumination change. From the binary image, the contours are detected by the border following technique¹⁹ as shown in Figure 3.

Figure 3.

Image preprocessing for image noise reduction and contour extraction for detecting the pool table bed and balls.

For detecting the pool table bed, the contours are first approximated as polygons by using the Douglas–Peucker approximation.²⁰ Since the largest quadrilateral indicates the table bed as shown in the contour extraction image of Figure 3, detecting the table bed is like finding the largest quadrilateral among the polygons. The quadrilaterals with four corner points are extracted among the polygons and their sizes are compared to find the largest quadrilateral. In this case, we assume the size of the table bed in the camera image is larger than $\frac{1}{4}$ of the whole image size and thus set the initial size of the quadrilateral as $\frac{W_{Img}}{2} \times \frac{H_{Img}}{2}$ , where $W_{Img}$ and $H_{Img}$ are the width and height of the image. In this way, the four corner points of the table bed in the image coordinates of ${I}$ can be obtained as $^{I} P_{1},^{I} P_{2},^{I} P_{3}, and^{I} P_{4}$ in Figure 4, where the superscripts indicate the coordinates. The obtained corner points in ${I}$ and the physical corner points, $^{O} P_{1},^{O} P_{2},^{O} P_{3}, and^{O} P_{4}$ , in the object coordinates of ${O}$ are related each other by homography between them as in equation (5).

s_{i}^{O} P_{i} = H^{I} P_{i} for i = 1, 2, 3, 4

where $H$ is the $3 \times 3$ homography transformation matrix, s_i is the scale factor, and $^{I} P_{i}$ and $^{O} P_{i}$ for $i = 1, 2, 3, 4$ are represented as ${[^{I} x_{i}^{I} y_{i} 1]}^{T}$ and ${[^{O} x_{i}^{O} y_{i} 1]}^{T}$ in homogeneous coordinates, respectively. The homography transformation matrix H is obtained by minimizing the back-projection error of E in equation (6).²¹

\begin{matrix} E = \sum_{i} ({(^{O} x_{i} - \frac{h_{11}^{I} x_{i} + h_{12}^{I} y_{i} + h_{13}}{h_{31}^{I} x_{i} + h_{32}^{I} y_{i} + h_{33}})}^{2} \\ + {(^{O} y_{i} - \frac{h_{21}^{I} x_{i} + h_{22}^{I} y_{i} + h_{23}}{h_{31}^{I} x_{i} + h_{32}^{I} y_{i} + h_{33}})}^{2}) \end{matrix}

where $h_{i j}$ for $i = 1, 2, 3$ and $j = 1, 2, 3$ are elements of H. In this case, the denominator, $(h_{31}^{I} x_{i} + h_{32}^{I} y_{i} + h_{33})$ , is the scale factor of s_i in equation (5). The corresponding physical points are set as the corner points of the actual pool table without wooden rails as in equations (7) –(10).

^{O} P_{1} = [\begin{matrix} W_{W} \\ W_{W} \\ 1 \end{matrix}]

^{O} P_{2} = [\begin{matrix} W_{W} \\ W_{T} - W_{W} \\ 1 \end{matrix}]

^{O} P_{3} = [\begin{matrix} L_{T} - W_{W} \\ W_{T} - W_{W} \\ 1 \end{matrix}]

^{O} P_{4} = [\begin{matrix} L_{T} - W_{W} \\ W_{W} \\ 1 \end{matrix}]

where W_T and L_T are the width and length of the pool table, and W_W is the width of the wooden rail. Using the obtained homography transformation matrix $H$ , any point $^{I} P$ in ${I}$ can be transformed to point $^{O} P$ in ${O}$ as in equation (11).

s^{O} P = H^{I} P

where s is the scale factor.

Figure 4.

Four corner detection of the pool table bed in the image coordinates of ${I}$ and homography transformation from the image coordinates of ${I}$ to the object coordinates of ${O}$ .

Next, we detect three balls from the contour image of Figure 3 as shown in Figure 5. We first apply the Gaussian smoothing filter to the contour image for reducing noise and small details, where the two-dimensional (2D) Gaussian filter is defined in equation (12).

G (x, y) = \frac{1}{2 π σ^{2}} e^{- \frac{x^{2} + y^{2}}{2 σ^{2}}}

where σ is the standard deviation of the Gaussian distribution, and the mean is (0, 0). Using equation (12), the Gaussian kernel is computed as an $n \times n$ matrix, where σ is set as $((\frac{n}{2} - 1) \times 0.3 + 0.8)$ . By convolving the contour image with the Gaussian kernel, the smoothed image can be obtained as in the first image of Figure 5. Next, we find circles from the smoothed image by using the circle Hough transform (CHT).¹⁰ The CHT is basically based on the voting procedure like the original Hough transform (HT).^22,23 However, the CHT differs from the HT in that the parameter space of the CHT is 3D whereas that of the HT is 2D. Mathematically, the equation of a circle in x–y plane is given by equation (13).

{(x - a)}^{2} + {(y - b)}^{2} = r^{2}

where $(a, b)$ is the center of the circle, and r is the radius. For point $(x, y)$ in the image, the CHT finds all possible parameters, a, b, and r, according to equation (13), and votes each grid cell $(a, b, r)$ in the 3D parameter space a–b–r (or Hough space), where voting means that the voting number of the cell increases by 1. Among the voted grid cells in the a–b–r space, the grid cell whose voting number is the local maxima is selected to determine the resultant circle parameters. In this case, we define the minimum distance $D_{min}$ among circles to remove the falsely detected circles around the true one. In other words, if the distances among the grid cells with the local maxima are less than $D_{min}$ , all of the grid cells are removed except the one grid cell with the largest voting number. We also define the minimum radius $R_{min}$ and the maximum radius $R_{max}$ of the circle, taking the actual radius of the ball into consideration. In other words, the radius of the detected circle should be in the range from $R_{min}$ to $R_{max}$ . As a result, the detected circles for three balls are shown in the second image of Figure 5, where $^{I} P_{B 1},^{I} P_{B 2},

and^{I} P_{B 3}$ are their center positions in ${I}$ . Next, the colors of the balls are determined by averaging the colors of the ${(N + 1)}^{2}$ points around the center position $(x_{c}, y_{c})$ of the ball as in equation (14) for robust color detection to light reflection as shown in Figure 6.

C (x_{c}, y_{c}) = (\bar{R}, \bar{G}, \bar{B})

where

\bar{R} = \frac{1}{{(N + 1)}^{2}} \sum_{j = 0}^{N} \sum_{i = 0}^{N} R (X_{i}, Y_{j})

\bar{G} = \frac{1}{{(N + 1)}^{2}} \sum_{j = 0}^{N} \sum_{i = 0}^{N} G (X_{i}, Y_{j})

\bar{B} = \frac{1}{{(N + 1)}^{2}} \sum_{j = 0}^{N} \sum_{i = 0}^{N} B (X_{i}, Y_{j})

where $R (X_{i}, Y_{j})$ , $G (X_{i}, Y_{j})$ , and $B (X_{i}, Y_{j})$ are red, green, and blue colors at $(X_{i}, Y_{j})$ position, respectively, and $(X_{i}, Y_{j})$ is determined as $(x_{c} - W_{c} + i \times Δ d, y_{c} - W_{c} + j \times Δ d)$ . In this case, N is $⌊\frac{2 W_{c}}{Δ d}⌋$ , where $⌊⌋$ discards the decimal places. If $\bar{R}, \bar{G}, and \bar{B}$ at $(x_{c}, y_{c})$ satisfy the condition in equation (18), the circle with the center position $(x_{c}, y_{c})$ is determined as the red ball.

\bar{R} \geq \bar{G} \times 1.5 and \bar{R} \geq \bar{B} \times 1.5

If the condition in equation (18) is not satisfied but the condition in equation (19) is satisfied, the circle is determined as the yellow ball.

\bar{R} \geq \bar{B} \times 1.5 and \bar{G} \geq \bar{B} \times 1.5

If both conditions in equations (18) and (19) are not satisfied, the circle is determined as the white ball. In this way, we can determine the positions of the red, yellow, and white balls, $^{I} P_{R},^{I} P_{Y}, and^{I} P_{W}$ , as shown in the third image of Figure 5.

Figure 5.

Three ball position detection based on the circle Hough transform (CHT), and ball color detection robustly against light reflection on the surface of balls.

Figure 6.

Averaging ball color detection: (a) light reflection problem on the glossy surface of the ball and (b) ball color detection by averaging the colors of the ${(N + 1)}^{2}$ points around the ball center position $(x_{c}, y_{c})$ .

Next, the three ball positions in the image coordinates of ${I}$ are transformed to the physical positions in the object coordinates of ${O}$ by the homography transformation matrix H in equation (5) as follows.

^{O} P_{K} = H^{I} P_{K} for K = W, Y, R

The resultant ball positions, $^{O} P_{W},^{O} P_{Y}, and^{O} P_{R}$ , in ${O}$ are depicted in Figure 7 and will be used as the ball positions in the following sections.

Figure 7.

Detected ball positions, $^{O} P_{W},^{O} P_{Y}, and^{O} P_{R}$ , in the object coordinates of ${O}$ .

Next, the cue stick is detected as shown in Figure 8. For detecting the cue stick, the HSV image of Figure 8(a) is thresholded by the thresholding values in equation (21).

(20, 20, 0) \leq (H, S, V) \leq (20, 80, 255)

Similar to the table detection, the binary image of the cue stick can be extracted without being greatly affected by illumination change using the HSV color model. The Gaussian smoothing filter with the size of $3 \times 3$ and the operations of erosion and dilation are sequentially applied to the thresholded image for noise reduction. Next, we define the ROI with the width of W_S at the cue ball position because the cue stick detection is needed only for predicting the path of the cue ball, where the cue ball is the white ball. In this case, the square with the width of W_B is subtracted from the ROI to remove the falsely detected points of the cue ball because the color of the cue stick is almost same as the white ball. The center position of the object points in the ROI is obtained as the center position $^{I} P_{S}$ of the cue stick. Using the homography transform in equation (11), we can obtain the cue center position of $^{O} P_{S}$ and the cue ball position of $^{O} P_{W}$ in the object coordinates of ${O}$ , where $^{O} P_{W}$ is obtained previously as in Figure 7. Eventually, we can find the direction of the cue stick as the vector $\vec{^{O} P_{S}^{O} P_{W}}$ . Similar to the white cue ball, the direction of the cue stick can be obtained for the yellow cue ball as $\vec{^{O} P_{S}^{O} P_{Y}}$ . According to the direction of the cue stick, we can predict the cue ball path under the conditions such as perfectly elastic collision between balls and the same angle of incidence and reflection of the ball.

Figure 8.

Cue stick detection: (a) definition of the ROI in the original image and (b) the cue stick position of $^{I} P_{S}$ and the white cue ball position of $^{I} P_{W}$ in the binary thresholded image. ROI: region of interest.

PCA-based ball motion detection

For appropriately providing the solution paths and the predicted paths without interfering player’s game, the new paths must be automatically recalculated and provided for a player whenever the player hits the cue ball and then all of the balls stop. In this section, the detection method for the moment when a player hits the cue ball and balls are moving and when all of the balls stop is proposed. In this case, we assume that the normal motion of the balls is linear. Of course, the motion of the cue ball may not be linear at the moment when a player hits the ball with spin, but the motion of the ball eventually becomes linear after the spin disappears. Thus, we define the linear motion constraints to detect the ball motion robustly against falsely detected ball positions. The false detection of the ball positions may occur due to image noise or occlusion by the player preparing the next billiard shot. Thus, the PCA is applied to the trajectory of the ball to robustly determine whether the ball motion satisfies the linear motion constraints. If the ball trajectory satisfies the linear motion constraints, it is determined that the ball has started to move. Next, it is determined that the ball has stopped if the variances of the ball positions are small enough. In this case, the variances of the ball positions can be obtained by the principal components. The detail method is described as follows.

For detecting the moment when a ball moves or stops, we apply the PCA to the n latest ball positions, $p (t), p (t - 1), ..., p (t - (n - 2)), p (t - (n - 1))$ , in the object coordinates of ${O}$ . For convenience of explanation, let the ball positions $p (t), ..., p (t - (n - 1))$ be $(x_{1}, y_{1}), \dots, (x_{n}, y_{n})$ , respectively. The ball positions can be represented as an $n \times 2$ matrix $X$ in equation (22).

X = [\begin{matrix} x_{1} & y_{1} \\ x_{2} & y_{2} \\ ⋮ \\ x_{n} & y_{n} \end{matrix}]

where the mean $\bar{X}$ of ${x_{1}, x_{2}, \dots, x_{n}}$ and the mean $\bar{Y}$ of ${y_{1}, y_{2}, \dots, y_{n}}$ are defined in equations (23) and (24), respectively.

\bar{X} = E (X) = \frac{1}{n} \sum_{i = 1}^{n} x_{i}

\bar{Y} = E (Y) = \frac{1}{n} \sum_{i = 1}^{n} y_{i}

With these means, the center position of the n ball positions is determined as $P_{c} (\bar{X}, \bar{Y})$ . Next, the covariance between two dimensions for X and Y can be obtained based on Bessel’s correction as in equation (25).

σ_{x y} = cov (X, Y) = \frac{1}{n - 1} \sum_{i = 1}^{n} (x_{i} - \bar{X}) (y_{i} - \bar{Y})

where $σ_{x y}$ is equal to $σ_{y x}$ because the covariance satisfies the commutative law. The covariance between one dimension and itself is the variance. For each dimension, the variance is obtained as in equations (26) and (27).

σ_{x x} = cov (X, X) = \frac{1}{n - 1} \sum_{i = 1}^{n} (x_{i} - \bar{X}) (x_{i} - \bar{X})

σ_{y y} = cov (Y, Y) = \frac{1}{n - 1} \sum_{i = 1}^{n} (y_{i} - \bar{Y}) (y_{i} - \bar{Y})

From equation (25) to equation (27), the covariance matrix $C$ can be defined in equation (28).

C = [\begin{matrix} σ_{x x} & σ_{x y} \\ σ_{y x} & σ_{y y} \end{matrix}]

where $C$ is a $2 \times 2$ symmetric matrix, and its eigenvalue λ and the eigenvector V can be obtained from equation (29).

C V = λ V

where two eigenvectors, V ₁ and V ₂, are the principal components of $X$ and are mutually perpendicular (or orthogonal) as shown in Figure 9. The eigenvalues $λ_{1}$ and $λ_{2}$ are variances in the directions of V ₁ and V ₂ respectively, where $λ_{1} \geq λ_{2}$ . In Figure 9, the center position $P_{c} (t)$ at time t is determined as $(\bar{X}, \bar{Y})$ from equations (23) and (24).

Figure 9.

PCA application to n consecutive ball positions from $(x_{1}, y_{1})$ to $(x_{n}, y_{n})$ , where $P_{c} (t)$ is a center position at time t, and V ₁ and V ₂ are eigenvectors for eigenvalues $λ_{1}$ and $λ_{2}$ respectively $(λ_{1} \geq λ_{2})$ .

First, we define the condition for ball stop detection with the largest variance $λ_{1}$ as in equation (30) when the ball has been moving.

\sqrt{λ_{1}} < D_{S}

where $\sqrt{λ_{1}}$ is the largest standard deviation of $X$ , and D_S is defined by considering the maximum position error caused by image noise as shown in Figure 10. The fact that all of three balls satisfy the condition in equation (30) means that all of the balls have stopped. At this time, their center positions are prepared for the next steps such as finding the solution paths and predicting the real-time path of the cue ball. In this case, the obtained center positions for the next step are no longer updated because slight change of the ball positions may be caused by image noise or occlusion by a player preparing the next billiard shot. The next update will be performed when a player hits the cue ball and all of the balls stop again.

Figure 10.

Condition for ball stop detection when the ball has been moving.

Next, as shown in Figure 11, we define the conditions for ball motion detection caused by player’s hitting as in equations (31) and (32) when all of the balls have stopped according to equation (30). First, for removing the abnormal ball motion change caused by falsely detected ball positions and detecting true ball motion by actual player’s hitting, we detect only linear motion of the ball by using the condition in equation (31).

\sqrt{λ_{1}} > D_{M 1} and \sqrt{λ_{2}} < D_{M 2}

where $D_{M 1}$ is the minimum displacement for true ball motion detection, and $D_{M 2}$ is the admissible displacement error in the perpendicular direction to ball motion. For linear ball motion, $D_{M 2}$ is defined much smaller than $D_{M 1}$ and near zero. Next, we define the condition for ball moving at almost constant speed as in equation (32).

\frac{D_{CH}}{D_{CT}} < 1.1

where $D_{CH}$ is the distance between the center position $P_{c} (t)$ and the position of the heading point $P (t)$ , and $D_{CT}$ is the distance between the center position $P_{c} (t)$ and the position of the tail point $P (t - (n - 1))$ . In other words, the condition in equation (32) means that the moving distances of the ball before and after the center position are almost the same. In fact, $D_{CH}$ is smaller than $D_{CT}$ and the left side of equation (32) is less than 1 because the speed of the ball decreases in time. However, we determine the right side of equation (32) as 1.1 instead of 1 considering the detection error of the ball positions. If the abrupt change of the heading point is occurred by image noise or occlusion, X cannot satisfy the conditions in equation (31) or equation (32). In this way, the false detection for the ball moving can be prevented. When at least one of three balls satisfies both conditions in equations (31) and (32), it is determined that the ball motion occurs.

Figure 11.

Condition for ball motion detection when the ball has stopped.

Systematic solution path acquisition of the cue ball

The basic rule of the three-cushion billiards among games within Carom billiards is that the cue ball hit by a player must hit two object balls and hit the second object ball after contacting at least three cushions. For finding the solution path of the cue ball that satisfies the rule, we use the five and half system (or the diamond system)¹⁴ that is the most representative technique for Carom billiards as shown in Figure 12, where the spin amount is basically set to the maximum at the middle height of the cue ball. If a player hit the cue ball, the cue ball begins to move from the starting point of S to the aiming point of A. After contacting the first cushion on the long rail, the cue ball contacts the second cushion on the short rail. After contacting the second cushion, the cue ball moves to the finishing point of F on the third cushion. After contacting the third cushion, the cue ball finally moves to the object balls and hits the balls. The five and half system defines the relationship among these three points of S, A, and F by the following formula in equation (33).

A = S - F

For the given starting point S and finishing point F, the aiming point of A can be calculated by equation (33), where S, A, and F are scalar.

Figure 12.

The five and half system to systematically find the solution path of the cue ball satisfying the three-cushion rule.

In the five and half system, for calculating the formula in equation (33), the scalar values of the marker points on the wooden rail are defined nonlinearly and differently for S, A, and F as shown in Figure 13. In this case, $(S - F)$ must be greater than or equal to zero because A is greater than or equal to zero.

Figure 13.

Scalar value definition of the marker points on the wooden rail for three parameters, S, A, and F of the five and half system.

For finding the solution path of the cue ball, we first find the three cushion line to hit the object balls. For given finishing points, three cushion lines are determined as shown in Figure 14. In this case, we can find the three cushion line of the finishing point F of 20 to hit the object balls. The three cushion line between each two lines is calculated by linear interpolation.

Figure 14.

Definition of the three cushion lines for finishing points.

Next, for the given position of the cue ball, we find the starting point S and the aiming point A that satisfy the formula in equation (33) as shown in Figure 15. For the given finishing point F of 20, the starting point S and the aiming point A are determined by equation (33) as 40 and 20, respectively. In this way, we can find the solution path of the cue ball. During billiard game, a player just hits the cue ball for the aiming point A and then the cue ball moves along the solution path to hit the object balls.

Figure 15.

Acquisition of the starting point of S and the aiming point of A for the given finishing point of F.

So far we find the solution path of the cue ball in the case of two object balls attached. In the case of two object balls separated, the solution path can be found as shown in Figure 16. In this case, we find the solution path for the cue ball to hit the first object ball and then to hit the second object ball after contacting three cushions. To do so, we first find the three cushion line to hit the second object ball. Next, we define the ghost ball right next to the first object ball by considering the collision point between the cue ball and the first object ball. In this case, we regard the ghost ball as the cue ball and calculate the starting point S and the aiming point A similar to the attached object ball case.

Figure 16.

Solution path acquisition in the case of two object balls separated.

Finally, we determine the direction of the cue ball so that its tangent after collision with the first object ball points to the aiming point A of 20. If we assume that the collision between balls is perfectly elastic, the collision can be described as shown in Figure 17.^24,25 The aiming velocity $\vec{v_{a}}$ of the cue ball is divided into the velocity $\vec{v_{o}}$ of the first object ball and the velocity $\vec{v_{c}}$ of the cue ball after the collision. The direction of $\vec{v_{o}}$ is determined parallel with the vector $\vec{P_{CB} P_{OB}}$ where $P_{CB}$ and $P_{OB}$ are the center points of the cue ball and the object ball, respectively. The vector $\vec{v_{c}}$ on the tangent line is determined perpendicular to $\vec{v_{o}}$ . Using the angle $θ_{o}$ between $\vec{v_{a}}$ and $\vec{v_{o}}$ , the angle $θ_{c}$ between $\vec{v_{a}}$ and $\vec{v_{c}}$ can be calculated by equation (34).

θ_{c} {= 90}^{\circ} - θ_{o}

When the vector $\vec{v_{c}}$ is coincided with the tangent line, the overlap width d_t between the two balls in the direction of $\vec{v_{a}}$ can be calculated by geometry. According to the overlap width corresponding to $θ_{o}$ and $θ_{c}$ , the fractional aiming table can be defined as shown in Figure 18 based on the fractional aiming system.²⁶ During billiard game, if a player just hits the cue ball with the given overlap width, the cue ball moves along the solution path after hitting the first object ball and finally hits the second object ball.

Figure 17.

Perfectly elastic collision between the cue ball and the first object ball.

Figure 18.

Overlap width d_t between the object ball and the cue ball for the desired angles of them after perfectly elastic collision between them.

When applying the five and half system, we can find up to two solution paths depending on which ball is set as the first object ball. In addition, for each case, we can find up to four solution paths by symmetrically transforming the ball positions vertically or horizontally. Furthermore, we can find more paths by anticipating the path after contacting the fourth cushion, where the cue ball after the fourth cushion simply moves at the same incidence and reflection angles since the amount of the spin of the cue ball is almost gone. In this way, the solution paths can be found and displayed in the guidance monitor. Although we use only the five and half system for finding the solutions paths in this article, much more solution paths can be found by various kinds of billiard techniques such as the plus system, the maximum English system, the double rail system, the reverse English system, and the no English system.

Experimental results

Experimental setup

For verifying the feasibility of the proposed ICBAS, experiments are conducted in a 60% reduced pool table as shown in Figure 19. For detecting the balls and the cue stick, a monocular camera, Logitech C615 webcam is installed 1.78 m above the pool table at a 27° from the ceiling with a beam projector. Using the camera detection results, the control PC calculates the predicted path and projects it onto the pool table bed by the beam projector. Also, the PC calculates the solution paths for three-cushion billiards and displays them in the guidance monitor which has a direction controller for the monitor to head for a player. The direction controller is implemented using a dinamixel AX-12+ servo motor with an Arduino nano. The Arduino receives commands to control the servo motor from the PC by serial communication. The detail parameters for experiments are described in Table 1.

Figure 19.

Experimental setup for the proposed ICBAS that not only displays the solution paths of the cue ball in the guidance monitor but also projects the predicted path changed by the direction of the cue stick onto the pool table bed in real time.

Table 1.

Parameters for experiments.

Parameters	Descriptions
$W_{Img} = 1920 pixels$	Width of the camera image
$H_{Img} = 1080 pixels$	Height of the camera image
$W_{T} = 880 mm$	Width of the pool table
$L_{T} = 1600 mm$	Length of the pool table
$W_{W} = 30 mm$	Width of the wooden rail
$R_{B} = 29 mm$	Radius of the ball
$n = 11$	Gaussian kernel matrix size of $(n \times n)$ for table bed detection
$D_{min} = 35 pixels$	Minimum distance to remove the falsely detected circles for CHT
$R_{min} = 15 pixels$	Minimum radius of the detected circle for CHT
$R_{max} = 50 pixels$	Maximum radius of the detected circle for CHT
$W_{S} = 240 pixels$	Width of ROI for cue stick detection
$W_{B} = 80 pixels$	Width of rejection square for cue stick detection
$N_{S} = 10$	Number of ball positions for PCA-based ball motion detection
$D_{S} = 3 mm$	Admissible position error for ball stop detection
$D_{M 1} = 30 mm$	Minimum displacement for ball motion detection
$D_{M 2} = 1.5 mm$	Admissible displacement error in the perpendicular direction to the ball motion

ROI: region of interest; PCA: principal component analysis.

Results of automatic solution path providing

The ICBAS calculates the solution paths of the cue ball and displays the paths in the guidance monitor. The trajectories of three balls during billiard game are shown in Figure 20, where the trajectories of the white, yellow, and red balls are depicted as blue, red, and green points, respectively. In Figure 20(a), all of the balls stop before the player hits the white ball as the cue ball. In Figure 20(b), as soon as the player hits the white ball, the ball moves too fast to get several positions of the ball. In Figure 20(c), the red ball begins to move after colliding with the white ball. In this case, the red ball moves as fast as the white ball, so it gets almost no position for the red ball. However, from Figure 20(d), the positions of the white and red balls are sufficiently obtained to confirm their linear motion. In Figure 20(e), the whole trajectories of the three balls are shown until all of the balls stop after player’s hitting.

Figure 20.

Three ball trajectories during billiard game where the trajectories of the white, yellow, and red balls are depicted as blue, red, and green points, respectively: (a) All of the balls stop before player’s hitting. (b) The white ball is moving too fast at the moment of player’s hitting. (c) After colliding with the white ball, the red ball moves quickly, either. (d) The movement of both white and red balls can be confirmed through their trajectories. (e) Whole trajectories of all three balls can be confirmed until all of the balls stop after the player hits the white ball as shown in (b).

During billiard game in Figure 20, the ball motion is detected as shown in Figure 21. In this case, only N_S latest positions for each ball are depicted where N_S is 10 as in Table 1, and the circle indicates the center position $P_{c} (t)$ . The black solid line from the center position indicates the vector V ₁ of the first principal axis with variance $λ_{1}$ , and the gray line indicates the vector V ₂ of the second principal axis with variance $λ_{2}$ although it is sometimes short and thus not well visible. Before all of the balls stop as in Figure 20(a), only the yellow ball moves whereas the other balls stop, and the trajectories of the balls are shown in Figure 21(a). In Figure 21(b), three balls are judged to be stopped because all of them satisfy the ball stop condition in equation (30). That is, if the variances of the three ball positions are small enough, we judge that there is no motion of the balls or all of the balls stop. After the player hits the white ball as shown in Figure 20(b), the white ball begins to move, and the red ball also begins to move due to the collision with the white ball. However, in this case, the motion of the white ball does not satisfy the linear ball motion conditions because the condition of $\sqrt{λ_{2}} < D_{M 2}$ is not satisfied, where $\sqrt{λ_{2}}$ and $D_{M 2}$ are 77.27 and 1.5, respectively. In Figure 21(d), both linear ball motion conditions of $\sqrt{λ_{1}} > D_{M 1}$ and $\sqrt{λ_{2}} < D_{M 2}$ in equations (31) and (32) are satisfied, where $\sqrt{λ_{1}}$ , $\sqrt{λ_{2}}$ , and $D_{M 1}$ are 86.55, 1.36, and 30, respectively. At this time, the ICBAS eventually judges that the motion of the balls occurs. Otherwise, the ICBAS may incorrectly judge the motion of the ball by falsely detected positions of the balls due to image noise or occlusion by a player preparing the next shot. After the balls move like in Figure 20(e), all of the balls are judged to be stopped again because the trajectories of three balls satisfy the ball stop condition in equation (30) as shown in Figure 21(e). In this way, the ICBAS can detect the moment when all of the balls stop or when the motion of the balls occurs.

Figure 21.

Ball motion detection results: (a) Yellow ball is moving with $\sqrt{λ_{1}} = 6.44$ and $\sqrt{λ_{2}} = 1.93$ at $t = 11$ . (b) Yellow ball stops with $\sqrt{λ_{1}} = 2.94$ and $\sqrt{λ_{2}} = 1.42$ at $t = 41$ . (c) White ball is moving after player’s hitting with $\sqrt{λ_{1}} = 130.86$ and $\sqrt{λ_{2}} = 77.27$ at $t = 241$ but the linear ball motion constraints are not satisfied because of the large value of $\sqrt{λ_{2}}$ . (d) White ball is moving with $\sqrt{λ_{1}} = 86.55$ and $\sqrt{λ_{2}} = 1.36$ at $t = 256$ and the linear ball motion constraints are satisfied. (e) All of the balls stop with $\sqrt{λ_{1}} = 2.97$ and $\sqrt{λ_{2}} = 1.01$ for white ball, $\sqrt{λ_{1}} = 1.33$ and $\sqrt{λ_{2}} = 0.83$ for yellow ball, and $\sqrt{λ_{1}} = 1.45$ and $\sqrt{λ_{2}} = 0.83$ for red ball at $t = 453$ .

Figure 22 shows the case that the white ball is falsely detected due to the occlusion by a player preparing the next billiard shot. In Figure 22(a), all of three balls stop and the motion of three balls satisfy the ball stop condition in equation (30) since their variances are small enough. In Figure 22(b), the white ball begins to be occluded by the arm of the player and the abrupt change of the white ball position occurs because the ICBAS incorrectly recognizes the player’s arm as the white ball. In this case, the trajectory of the white ball cannot satisfy the constant speed condition in equation (32) due to the abrupt change of the ball position. Thus, the ICBAS does not judge that the motion of the balls occurs. In Figure 22(c), several falsely detected positions are shown due to the motion of the player. In this case, the trajectory of the white ball cannot satisfy the linear motion condition of $\sqrt{λ_{2}} < D_{M 2}$ in equation (31), where the gray line from the center position represents the value of $\sqrt{λ_{2}}$ and is larger than $D_{M 2}$ of 1.5 mm in Table 1. Thus, the ICBAS maintains the judgment that all of the balls stop. In this way, the ICBAS can prevent from judging incorrectly the ball motion due to the occlusion. In Figure 22(d), the white ball appears and thus its variance becomes small enough again.

Figure 22.

False detection of the white ball positions due to the occlusion by a player preparing the next billiard shot: (a) All of the balls stop and their trajectories satisfy the ball stop condition in equation (30). (b) The white ball is occluded by the arm of the player preparing the next billiard shot and the abrupt change of the white ball position occurs. Due to the abrupt change of the ball position, the constant ball speed condition in equation (32) cannot be satisfied. (c) The white ball is still occluded and there are several positions falsely detected. In this case, the linear ball motion condition in equation (31) is not satisfied because the gray line represents the value of $\sqrt{λ_{2}}$ and is larger than $D_{M 2}$ of 1.5 mm. (d) The occluded white ball appears again and the variance of its positions becomes small enough again.

Whenever all of the balls are judged to be stopped, the solution paths are calculated and displayed in the guidance monitor. In Figure 23, the result of two solution paths for the white cue ball is depicted. In the first case, the yellow ball is the first object ball and the red ball is the second object ball. By using the five and half system, the solution path can be found and displayed together with the tip-point for maximum spin and the overlap width between the cue ball and the first object ball. In the second case, the red ball is the first object ball and the yellow ball is the second object ball. In this case, the path after contacting the fourth cushion is first estimated to hit the second object ball, where the path after the fourth cushion satisfies the condition of the same angle of incidence and reflection of the ball because the spin of the cue ball almost gone. For the fourth cushion of the estimated path, the solution path is found by the five and half system. The resultant solution path is also displayed with the tip-point and the overlap width.

Figure 23.

Solution paths with the tip-point and the overlap width for the white cue ball, displayed in the guidance monitor.

In Figure 24, two solution paths for the yellow cue ball are depicted. In the first case, the white ball is the first object ball and the red ball is the second object ball. The solution path can be found by using the five and half system and displayed with the tip-point and the overlap width like the first case in Figure 23. In the second case, although the white ball is the first object ball and the red ball is the second object ball as in the first case, we estimate the path after contacting the fourth cushion and find the solution path for the fourth cushion like the second case in Figure 23. The resultant solution path is also displayed with the tip-point and the overlap width. During billiard game, a player can refer to the solution paths with the tip-point for the desired ball spin and the overlap width displayed in the guidance monitor.

Figure 24.

Solution paths with the tip-point and the overlap width for the yellow cue ball, displayed in the guidance monitor.

Results of real-time path projection

The ICBAS detects the three ball positions and the cue stick direction and projects the cue ball path according to the ball configuration and the cue stick direction as shown in Figure 25. In Figure 25(a), only the predicted path of the cue ball is projected because of no collision with the object balls before the third cushion. On the other hand, in Figure 25(b), both paths of the cue ball and the object ball are projected together because the cue ball collides with the object ball before the third cushion, where the collision occurs between the first cushion and the second cushion. Also, it can be confirmed that the direction of the guidance monitor displaying the solution paths of the cue ball is continuously controlled by detecting the cue stick position to head for the player during player’s preparing the next billiard shot.

Figure 25.

Real-time paths of the cue ball and the object ball projected on the pool table bed according to the direction of the cue stick: (a) projected cue ball path and (b) projected cue ball and object ball paths.

The actual motion of the balls after player’s hitting is shown in Figure 26. In Figure 26(a), the predicted paths of the cue ball and the object ball are shown before player’s hitting. In Figure 26(b), it can be confirmed that the actual movement of the balls from 1 to 9 are almost same with the predicted path.

Figure 26.

Actual motion of the balls after player’s hitting: (a) predicted cue ball path and (b) actual movement from 1 to 9 after player’s hitting.

Conclusions

In this article, the ICBAS has been proposed to provide both the solution paths and the predicted paths of the cue ball for a novice player of Carom Billiards to easily enjoy his or her own game. Whenever the ICBAS detects the moment when the player hits the cue ball and all of the balls stop, the ICBAS automatically recalculates both the solution paths and the predicted paths of the cue ball. In this case, the motion of the balls is robustly detected by the PCA under the linear ball constraints against the false detection for the ball positions due to image noise or occlusion by the player preparing billiard shot. The solution paths are systematically calculated by using the five and half system and displayed in the guidance monitor that is continuously controlled to head for the player during billiard game. The predicted paths are calculated and projected by the beam projector onto the pool table bed in real time according to the change of the cue stick direction. The player could make a successful billiard shot conveniently comparing the predicted path with the solution paths. Through the ICBAS, more and more people are expected to enjoy billiards and be healthy. Furthermore, the proposed ICBAS can be applied to a humanoid robot player as well as a human player. A humanoid robot player has been already developed for playing simple Pool and Snooker.^27
–29 The ICBAS enables a humanoid player to play the more complicated Carom Billiards. In future work, we will expand the ICBAS to automatically determine the optimal solution path among several solution paths by considering the degrees of freedom and kinematic constraints of the humanoid robot player.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed the receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Basic Science Research Programs (NRF-2019R1A6A1A09031717 and NRF-2018R1D1A1B07049270) through the National Research Foundation of Korea (NRF) funded by the Ministry of Education.

ORCID iD

Jaebyung Park

Supplemental material

Supplemental material for this article is available online.

References

Hsu

C-C

Tsai

H-C

Chen

H-T

, et al. Computer-assisted billiard self-training using intelligent glasses. In: 2017 14th international symposium on pervasive systems, algorithms and networks and 2017 11th international conference on frontier of computer science and technology and 2017 Third international symposium of creative computing (ISPAN-FCST-ISCC), Exeter, UK, 21–23 June 2017.

Gao

Zhu

Liang

, et al. Design of the multiple neural network compensator for a billiard robot. In: Proceedings of 2015 IEEE 12th international conference on networking, sensing and control, Taipei, Taiwan, 9–11 April 2015.

Gao

Zhan

, et al. Dynamic modeling based on fuzzy neural network for a billiard robot. In: Proceedings of 2016 IEEE 13th international conference on networking, sensing and control, Mexico City, Mexico, 28–30 April 2016.

Fragkiadaki

Agrawal

Levine

, et al. Learning visual predictive models of physics for playing billiards. In: Fourth international conference on learning representations (ICLR), San Juan, Puerto Rico, 2–4 May 2016.

Jebara

Eyster

Weaver

, et al. Stochastics: augmenting the billiards experience with probabilistic vision and wearable computers, In: International symposium on wearable computers, Cambridge, MA, USA, 13–14 October 1997, pp. 138–145.

Uchiyama

Saito

. AR display of visual aids for supporting pool games by online markerless tracking. In: The 17th international conference on artificial reality and telexistence, Esbjerg, Jylland, Denmark, 28–30 November 2007, pp. 172–179.

Shih

Chu

. A vision based interactive billiard ball entertainment system. In: The First IEEE international workshop on digital game and intelligent toy enhanced learning, Jhongli City, Taiwan, 26–28 March 2007, pp. 200–202.

Paolis

LTD

Aloisio

Pulimeno

. A simulation of a billiards game based on marker detection. In: 2009 Second international conferences on advances in computer–human interactions, Cancun, Mexico, 1–7 February 2009, pp. 148–151.

Takahashi

Kasai

Suzuki

. Support system for pocket billiards. In: SICE annual conference 2008, Japan, 20–22 August 2008, pp. 3233–3236.

10.

Ballard

DHT

. Generalizing the Hough transform to detect arbitrary shapes. Pattern Recogn 1981; 13(2): 111–122.

11.

Yang

Price

Cohen

, et al. Object contour detection with a fully convolutional encoder–decoder network. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016.

12.

Maninis

K-K

Pont-Tuset

Arbeláez

, et al. Convolutional oriented boundaries: from image segmentation to high-level tasks. IEEE Trans Pattern Anal Mach Intell 2018; 40(4): 819–833.

13.

Jian

Gui

. Object and contour detection with an architecture-fusion network. In: 2021 IEEE 33rd international conference on tools with artificial intelligence (ICTAI), Washington, DC, USA, 1–3 November 2021.

14.

Be Good at Billiards—the Diamond System, http://www3.sympatico.ca/eric.perreault/diamond_system_en.html#3 (1998, accessed 11 January 2019).

15.

Balen

. A virtual billiard assistant. Master’s Thesis, University of Twente, 2009.

16.

. Line segments detection with scale analysis: a principal component analysis based approach. In: 2009 IEEE international conference on intelligent computing and intelligent systems, Shanghai, China, 20–22 November 2009, pp. 43–47.

17.

Lee

Park

. Hand-eye LRF-based iterative plane detection method for autonomous robotic welding. Int J Adv Robot Syst 2015; 12(172): 1–14.

18.

Camera Calibration Toolbox for Matlab, http://www.vision.caltech.edu/bouguetj/calib_doc (2015, accessed 11 January 2019).

19.

Suzuki

Abe

. Topological structural analysis of digitized binary images by border following. Comput Vis Graphics Image Process 1985; 30(1): 32–46.

20.

Douglas

Peucker

. Algorithms for the reduction of the number of points required for represent a digitized line or its caricature. Cartogr J 1973; 10: 112–122.

21.

Sihombing

Nugroho

Wibirama

. Perspective rectification in vehicle number plate recognition using 2D–2D transformation of planar homography. In: 2015 international conference on science in information technology, Yogyakarta, Indonesia, 27–28 October 2015, pp. 237–240.

22.

Jain

Kasturi

Schunck

. Machine vision. New York, NY: McGraw-Hill Science, 1995.

23.

Park

Lee

. Precise 3D lug pose detection sensor for automatic robot welding using a structured-light vision system. Sensors 2009; 9: 7550–7565.

24.

Chou

C-W

Tien

M-C

J-L

. Billiards wizard: a tutoring system for broadcasting nine-ball billiards videos. In: 2009 IEEE international conference on acoustics, speech and signal processing, Taipei, Taiwan, 19–24 April 2009, pp. 1921–1924.

25.

Landry

J-F

Dussault

J-P

Mahey

. A heuristic-based planner and improved controller for a two-layered approach for the game of billiards. IEEE Trans Comput Intell AI Games 2013; 5(4): 325–336.

26.

Fractional Aiming System, https://billiards.colostate.edu/faq/aiming/fractional (2014, accessed 12 August 2022).

27.

KHL

Martin

Baldwin

. Snooker robot player—20 years on. In: 2007 IEEE symposium on computational intelligence and games, Honolulu, HI, USA, 1–5 April 2007.

28.

Pastor

Kalakrishnan

Chitta

, et al. Skill learning and task outcome prediction for manipulation. In: 2011 IEEE international conference on robotics and automation, Shanghai, China, 9–13 May 2011.

29.

Nierhoff

Leibrandt

Lorenz

, et al. Robotic billiards: understanding humans in order to counter them. IEEE Trans Cybern 2016; 46(8): 1889–1899.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB