Real-time 3D particle tracking using ultrafast electron beam X-ray computed tomography

Abstract

Ultrafast electron beam X-ray computed tomography (UFXCT) is a fast tomographic imaging technique used, e.g., for investigations of highly dynamic multiphase flows. In the last years, UFXCT was enhanced with the capability of real-time image acquisition and reconstruction. Using those capabilities, a new feedback-loop to a positioning unit was realized which allows for real-time repositioning of the scanner based on the images contents. By traversing the scanner, and hence its imaging planes, vertically, an object moving up- and/or downwards in the imaging region can be tracked and visualized. Using a phantom object moving with predetermined trajectories, the tracking latency, trackable object velocities and positioning accuracy was evaluated. Further, a latency compensation approach was developed that enhances the tracking performance.

Introduction

Fast imaging techniques are essential for investigation of multiphase flows which can be found in chemical reactors, fluid transport systems, distillation columns, power plants, and many more industrial processes. Modelling and simulating these inherently complex flows still require thorough validation using experimental data of sufficient temporal and spatial resolution (Brennen, 2005). Particularly non-invasive techniques are of high interest as these do not interfere with the flow structures (Reinecke et al., 1998). As a broadly applicable example, the focus is set here on dense bubbly flows of air in water because this model system is often used as a base case to gain a better understanding of the hydrodynamics involved (Rzehak et al., 2017). Tracking (at least) a single gas bubble within a bubble swarm over time would give valuable insights into bubble-bubble interactions.

Unfortunately, optical measurement systems, such as high-speed cameras, are not able to reveal gas structures in the core of dense bubbly flows. Though electrically-based tomography systems (Xie et al., 1995) are able to resolve media with different electrical properties with high temporal resolution their limited spatial resolution rejects them for dense bubbly flow investigations. Furthermore, both magnetic resonance imaging (MRI) and conventional X-ray methods with their limited temporal resolution (Tayler et al., 2012) and scanning region (Clarke et al., 2023) are also excluded from the list of suitable measurement systems. Contrary, ultrafast electron beam X-ray computed tomography (UFXCT) provides high frame rates of up to 8000 cross-sectional images per second at a spatial resolution of about 1.5 mm. Hence, it was already used for various investigations of multiphase flows, such as fluidized beds (Barthel et al., 2015; Bieberle et al., 2012), bubble columns (Lau et al., 2018), silo discharge (Stannarius et al., 2019), and packed bed reactors (Zalucky et al., 2017). Until 2023, the UFXCT scanner could only be used in a fixed spatial position for a maximum total scanning time of 30 s after which the internal detector memory was completely filled. After each scan, the acquired raw data had to be downloaded, reconstructed to cross-sectional images, and analyzed successively. Thus, any flow scenario had to be studied by sequentially scanning multiple positions along the direction of the flow, such as published by Farmani et al. (2021); Kipping et al. (2021); Neumann-Kipping et al. (2020); Sohr et al. (2019).

In 2023, the detector data acquisition and image processing were enhanced to provide real-time imaging capability by using the so-called Real-time Image Stream Algorithms (RISA) software (Windisch et al., 2023b) and streaming out acquired data immediately (Windisch et al., 2021). The next logical step is to also analyze the acquired images immediately and to use the extracted information for real-time control applications, such as CT scanner positioning. In that way, tracking of particles, especially air bubbles or tracer particles, moving in opaque media within large test sections would become possible. Possible control concepts were already explained in detail by Windisch et al. (2020).

In this work, we present the UFXCTs capability of dynamically tracking a moving particle during its vertical movement in real-time solely based on its acquired image data. Moreover, the systems capability to resolve the particle velocity, position and size during its movement is evaluated. Further, the latency and tracking performance of the entire electro-mechanical system is analysed to assess viable applications. Finally, a latency compensation technique is proved to be a viable method to improve overall tracking performance.

Methods

UFXCT system

Ultrafast electron beam X-ray computed tomography is based on electromagnetically scanning an electron beam along a tungsten target providing a rapidly moving X-ray source (see Figure 1). Two statically arranged rings of detector pixels acquire the X-ray intensity in two distinct imaging planes which are 10.4 mm apart. From the intensity data, cross-sectional images can be reconstructed. The system allows for imaging frequencies of up to 8000 Hz and a spatial resolution down to 1.5 mm.

Figure 1.

Ultrafast X-ray electron beam computed tomography scanner components.

UFXCT data processing

As introduced by Windisch et al. (2023b), acquired detector data is streamed directly to a processing computer and processed in a GPU-based processing pipeline. Therein, multiple processing steps (a.k.a. stages) are connected into the data processing logic required. For the feeback-loop introduced in this work, a pipeline as shown in Figure 2 is used. Note that the Trajectory calculation processing stage is newly developed for online particle tracking.

Figure 2.

Data processing pipeline configuration used for dynamic real-time positioning. Gray boxes refer to newly developed stages for this work. Refer to Windisch et al. (2023a) and Windisch et al. (2023b) for details on the reconstruction pipeline and object recognition, respectively.

During a scan, incoming data is fed into the processing pipeline which reconstructs a stream of cross-sectional images. At the same time, the EIPController continuously provides the latest actual scan position given by the positioning unit (described below). The scan position is attached as meta data to each cross-sectional image such that all consecutive processing steps can access the cross-sectional image together with its scan position. This way, two-dimensional objects extracted from the image data (see Windisch et al., 2023a for details) can be stitched together into three-dimensional objects basing on the respective scanning position. By tracking such an object moving through both imaging planes of the UFXCT scanner, the object trajectory can be calculated and, hence, its future movement extrapolated using a model predictive control approach, as described by Windisch et al. (2020). The predicted movement is used to find the optimal next position for the CT scanner to track the object during its vertical movement. By feeding back this determined target position to the positioning unit, the control loop is completed.

To initially evaluate the positioning performance independent of the object recognition, a slightly modified data processing pipeline is used. Instead of calculating the trajectory based on the acquired image data, a predefined position for each frame is read from a file. This trajectory for the UFXCT scanner is then passed to the positioning unit allowing latency evaluation between setting a known trajectory and the respective positioning feedback.

Positioning system

The positioning unit, shown in Figure 3, is driven by a geared motor (SEW Eurodrive CMP63M series) controlled by a motor controller (SEW Eurodrive MDX61B series). Two parallel drive belts are used to guide and move the positioning sled vertically. Counterweights are used to balance the weight of the scanner of $m_{\mathrm{tot}}\approx 700\mathrm{kg}$ . A wire position sensor is attached directly to the positioning sled for the most accurate position feedback excluding stretching and torsion of the drive train. An overview of the positioning unit’s parameters is given in Table 1.

Figure 3.

Positioning unit and experimental setup of the hardware phantom. Left: Photography of the setup in the UFXCT scanner. Right: Schematics (not to scale). The phantom object (orange) is attached to a string driven by a stepper motor (M). A photoelectric barrier (red) is used to calibrate the objects start position.

Table 1.

Positioning unit parameters.

Parameter	Sym.	Value
Acceleration	a	$\left|a\right|\le 500\mathrm{mm}/s^2$
Velocity	v	$\left|v\right|\le 500\mathrm{mm}/s$
Position (height)	s	$770\mathrm{mm}\le s\le 2980\mathrm{mm}$
Sensor accuracy	$\mathrm{\Delta }{s}_{\mathrm{sens}}$	$\pm 0.3\mathrm{mm}$
Control frequency	$f_{\mathrm{pos}}$	$200\mathrm{Hz}$
Rated motor power	$P_{\mathrm{pos}}$	$1.5\mathrm{kW}$

The positioning units electronics are shown in Figure 4. An Arduino-based SPS board (Controllino Maxi) is used to control (release brake signal, safety relay) and to monitor (limit switches, interlock state) the SEW motor controller. The positioning state (set position, current position, set velocity, current velocity, software upper/lower limit) is transmitted using the EtherNet/IP protocol with cyclic communication at a requested packet interval of 5 ms. The target position is updated in a control routine implemented in IPOS. The control loop in the motor controller then drives the servo motor accordingly.

Figure 4.

Positioning unit electronics. The SPS controller orchestrates the release of movement whilst the target position and positioning velocity are provided via EtherNet/IP protocol.

Experimental setup

The investigations are split into two main experiments: first, the overall positioning accuracy and latency are determined with pre-known movement profiles fed into the pipeline as described above. Here, low frequency square wave signals are used for the target position to determine both the accuracy and latency. Each of the experiments is repeated 20 times. Scans are performed with 2000 fps, i.e. 1000 image pairs per second.

Second, measurements on tracking an artificially-controlled moving object are performed. For this, a polyamide tracer object is placed on a nylon string and moved upwards/downwards with a known velocity (cf. Figure 3). The UFXCT system shall then recognize the tracer object, determine its speed and reposition the scanner accordingly. With these scans, the real-world performance of particle tracking using UFXCT is determined. The experiments were performed both with a tracer object that is larger than the distance between both imaging planes (18 mm length) and one which is short enough to temporarily disappear between both imaging planes during its movement (7 mm length).

The tracer object is moved using a stepper motor controlled by a Lang GmbH ECO-STEP motor controller. Its set speed is transmitted via RS232. The tracer object itself can be moved vertically for 1690 mm with a maximum speed of 500 mm/s.

Evaluation procedure

As shown in Figure 2, for each image, the current position and set position are stored for later evaluation. From the acquired position time-series, the positioning offset, overshoot and latency are extracted as shown in Figure 5.

Figure 5.

Definition of evaluated parameters shown on a schematic position profile.

Note, that the latency is defined as the time between setting the target position and the first perception of a changed position. It therefore inherently includes not only the processing and communication delay, but also the time it takes the sled to physically move far enough for a change to be registered by the position sensor. Before the sled starts to move, however, the drive train belt expansion $\mathrm{\Delta }{s}_{\mathrm{belt}}$ needs to be taken into account. It is calculated using

$$\mathrm{\Delta }{s}_{\mathrm{belt}}\left(F\right)=F·\frac{{\epsilon }_{\max }}{F_{\max }}·\frac{L_1·L_2}{L_1+L_2}$$

(1)

with the driving force $F=2·m_{\mathrm{tot}}·a_{\max }=700\mathrm{N}$ , maximum allowed belt stretching $\epsilon =0.5\mathrm{\%}$ , maximum allowed belt force $F_{\max }=3400\mathrm{N}$ , and the distance between the sled and belt deflection rollers $L_1$ and $L_2$ , respectively. For the worst-case position of $L_1=(L_1+L_2)/2=1537\mathrm{mm}$ , the belt expansion $\mathrm{\Delta }{s}_{\mathrm{belt}}$ equates to 0.79 mm. Further, the gear play in the drive train creates a gap which needs to be closed before the sled starts to move. Converting the angular gap to a linear gap $\mathrm{\Delta }{s}_{\mathrm{gear}}$ using

$$\mathrm{\Delta }{s}_{\mathrm{gear}}=\pi ·d_{\mathrm{drive}}·\mathrm{\Delta }{\alpha }_{\mathrm{gear}}/360°$$

(2)

with the angular gear play $\mathrm{\Delta }{\alpha }_{\mathrm{gear}}=0.6°$ and the effective drive wheel diameter $d_{\mathrm{drive}}=61.2\mathrm{mm}$ yields $\mathrm{\Delta }{s}_{\mathrm{gear}}=0.32\mathrm{mm}$ . Hence, before starting to move, the drive train ”absorbs” a total distance $\mathrm{\Delta }{s}_{\mathrm{drive}}=\mathrm{\Delta }{s}_{\mathrm{belt}}+\mathrm{\Delta }{s}_{\mathrm{gear}}=1.11\mathrm{mm}$ of the input movement. On top of that, the sled needs to move twice the position sensor resolution $\mathrm{\Delta }{s}_{\mathrm{sens}}$ to guarantee registering the position change. In total, a worst-case input distance of $\mathrm{\Delta }{s}_{\mathrm{total}}=\mathrm{\Delta }{s}_{\mathrm{drive}}+2·\mathrm{\Delta }{s}_{\mathrm{sens}}=1.71\mathrm{mm}$ needs to be covered before registering a change. This results in a ”mechanical” worst-case latency of

$$\mathrm{\Delta }{t}_{\mathrm{mech}}=\sqrt{\frac{2·\mathrm{\Delta }{s}_{\mathrm{total}}}{a_{\max }}}=82.7\mathrm{ms}.$$

(3)

The cyclical communication with a packet interval of 5 ms adds 10 ms (5 ms each for sending and receiving data) to the worst case latency. The processing speed of the CPU in the motion controller does not add significant latency. In total, a positioning latency $\mathrm{\Delta }{t}_{\mathrm{pos}}$ between 56.2 ms (assuming closed gear gap, no distance to next sensor increment and no communication delay) and 92.7 ms is to be expected.

Based on the calculated latencies, UFXCT tracking capabilities can be estimated. Different tracking strategies were proposed by Windisch et al. (2020). We used both the single imaging-plane tracking (SIPT) strategy and the dual imaging-plane re-tracking (DIPRT) strategy. In short, SIPT aims at keeping the object visible within one imaging plane at all times to acquire an accurate trajectory. Contrary, DIPRT aims at acquiring more cross-sectional images along the objects extent by moving the UFXCT scanner such that both scanning planes are positioned alternatingly above and below the object. This way, full scans of the object during its movement are achieved which allow for a more detailed reconstruction of the objects three-dimensional shape. In Windisch et al. (2020), maximum delays between acquiring an image and realizing an acceleration change were estimated no longer than 20 ms. With the now estimated mechanical delay of at least 56.2 ms, simulations show that stable SIPT cannot be achieved using a controller unaware of the delay. Hence, compensation for a fixed delay is introduced into the control algorithm and will also be investigated.

The compensation works as follows: The control algorithm uses a receding horizon approach to generate an optimal sequence of movements such that the positioning unit reaches the tracked particle in the shortest amount of time possible (see Windisch et al., 2020 for details). With each newly acquired cross-sectional image, this optimal sequence of movements is updated based on the system state (position and velocity of the scanner and the particle, respectively) and only the first movement of the sequence is transmitted to the positioning unit. I.e. with cross-sectional images arriving at an interval of $\mathrm{\Delta }{t}_{\mathrm{acq}}$ and the current image acquired at time $t_{\mathrm{now}}$ , the optimal movement for $t_{\mathrm{now}}+\mathrm{\Delta }{t}_{\mathrm{acq}}$ is transmitted to the positioning unit. This approach, however, does not take into account that the optimal movement will only be realized with a delay $\mathrm{\Delta }{t}_{\mathrm{pos}}$ , i.e. at time $t_{\mathrm{now}}+\mathrm{\Delta }{t}_{\mathrm{pos}}$ . Hence, to compensate for the delay, the optimal movement for $t_{\mathrm{now}}+\mathrm{\Delta }{t}_{\mathrm{ctrl}}$ is selected from the sequence of movements and transmitted to the positioning unit. If $\mathrm{\Delta }{t}_{\mathrm{ctrl}}=\mathrm{\Delta }{t}_{\mathrm{pos}}$ , the scanner should move ideally as long as the sequence of optimal movements does not change.

As evaluation criteria for how well these strategies can be realized, we use the total time the object is visible for SIPT and the number of full scans achieved in DIPRT.

Results and discussion

Accuracy

The positioning controller includes a hold controller, which assures that the target position is reached without offset in all configurations. Regarding the overshoot of the target position, a clear difference between moving upwards or downwards is identified (cf. Figure 6 (left)). The larger overshoot for downwards movement is due to the weight imbalance between the positioning sled and the counter weights. The counter weights are slightly lighter than the sled to have the drive train prestressed for upwards movement. Consequently, when moving downwards quickly, the tension bias is released and leads to higher overshooting. The PI-controllers tuning was optimized to compensate for the pretension in the highest movement velocity during upwards movement. The resulting lower overshoot for upwards movement can be seen clearly in Figure 6 (left).

Figure 6.

Left: Acquired overshoot when reaching the target position for different maximum velocities. Right: Determined latencies for different maximum velocities. Error bars show minimum/maximum values.

Latency

The acquired latencies shown in Figure 6 (right) confirm the theoretic considerations. Further, a slight increase in latency and latency spread can be seen for lower velocities stemming from the fact that it takes longer for the position sensor to register the change in position.

Delay compensation

To evaluate delay compensation, we tracked the hardware phantom using SIPT strategy with different control delays $\mathrm{\Delta }{t}_{\mathrm{ctrl}}$ . To evaluate tracking performance using the delay compensation, the evaluation criterion is the fraction of the total time the object is visible during the tracking phase. That is, after the scanning position has caught up to the object position past the initial scan. For SIPT, the ideal value is 100%.

Results are shown in Figure 7. The fraction of time visible shows a clear dependence on the assumed output delay $\mathrm{\Delta }{t}_{\mathrm{ctrl}}$ with a maximum around 73 ms which fits well in the expected range. It can also be seen that the fraction of time visible becomes more unstable around the optimal value. This is likely because the actual positioning delay $\mathrm{\Delta }{t}_{\mathrm{pos}}$ is not a fixed value but dependent on current belt tension and gear play. Still, the achieved fraction of time visible around the optimal delay shows a clear improvement at up to 2.6 times that of the uncompensated control (i.e. $\mathrm{\Delta }{t}_{\mathrm{ctrl}}=0$ ).

Figure 7.

Achieved fraction of time the object is visible vs. assumed output delay.

Tracking strategy evaluation

To evaluate the tracking strategies, tracer objects of 7 mm and 18 mm length were scanned using both control strategies. An example scan is shown in Figure 8. For SIPT, we evaluated the total time the object is visible during the tracking phase as explained previously. Note, however, that the results shown were acquired using the hardware phantom with its limited travel position range from 560 mm to 2250 mm whilst the possible scanning range of UFXCT is 770 mm to 2980 mm (cf. Figure 3). Leaving some headroom to decelerate the object, it can be scanned from 770 mm to 2200 mm vertical position. Using UFXCTs full scanning range would yield longer scanning times. However, for fast vertically moving objects, a significant portion of the total scanning range is needed to accelerate the scanner and catch up to the object.

Figure 8.

Example cross-sectional views of a scan using SIPT strategy. In the vertical cut of the image sequence (right side) the tracer object is visible in one of the scanning planes during the majority of the scan. The pulley visibile at the top end of the vertical cut is the upper limit for moving the tracer particle.

Figure 9 (top row) shows the achieved time the object was visible in one of the imaging planes. Comparing the results with a simulation using the same assumed output delay of 73 ms yields a very good fit. It can also be seen that the theoretical limits, i.e. for a system without any delay, cannot be achieved for the small object with average practical results reaching about 70% of the theoretical limit. Contrary, the larger objects total time visible is very close to the theoretical limit and shows only marginal reduction due to the delay.

Figure 9.

Results for total time visible for SIPT strategy and full scans achieved for DIPRT strategy using different object sizes. It can be seen that delay-compensated results (i.e. $\mathrm{\Delta }{t}_{\mathrm{ctrl}}=\mathrm{\Delta }{t}_{\mathrm{pos}}$ ) are close to the theoretically optimal performance whereas uncompensated control (i.e. $\mathrm{\Delta }{t}_{\mathrm{ctrl}}=0$ ) results in suboptimal performance for an object length of 7 mm.

Regarding the achieved full scans of the object (cf. Figure 9, bottom row), a dependency on the latency for both the small and the large object is identified. Further, the larger object cannot be scanned as often as the smaller one because it takes the scanner longer to overtake it. Overall, results show a good match to the simulations with a delay of $\mathrm{\Delta }{t}_{\mathrm{ctrl}}=\mathrm{\Delta }{t}_{\mathrm{pos}}=73\mathrm{ms}$ .

Object parameter extraction

The instantaneous velocity estimations during object tracking are within $\pm 10\mathrm{\%}$ of the actual velocity which has shown to be close enough to allow for a stable tracking. There are, however, some outliers in the instantaneous velocity estimates due to wrong system state estimates in the tracking algorithm (cf. Windisch et al., 2020). These can be corrected post-measurement but not during the scan itself. Still, the employed state machine assured stable structure tracking under all conditions; i.e. it was never lost by moving the scanner away from it indefinitely.

Regarding extraction of the structure’s length, no clear trend was identified. The actual object length of 7 mm was determined as $6.74\mathrm{mm}\pm 0.13\mathrm{mm}$ across all scans. The large objects length was determined as $17.39\mathrm{mm}\pm 0.17\mathrm{mm}$ . The systematic underestimation is likely due to the object segmentation from the images which can only identify the object after it penetrates the imaging planes for a small but non-zero depth. Second, the object size determination is sensitive to the scanning plane distance.

Discussion and outlook

In this work, we have shown that ultrafast X-ray electron beam computed tomography (UFXCT) with a controlled movable lift is capable of tracking a moving particle based on acquired image data. Both objects larger and smaller than the axial distance between the imaging planes were tracked successfully and their position, velocity, and size were determined. Using a fixed delay compensation in the control algorithm improved tracking performance regarding both the the total time the object is visible and the number of achieved full scans of the object. The covered axial range, speed and acceleration allows for new insights in a variety of use-cases such as, e.g., gas bubble investigations in dense bubbly flows, fluidized bed, particle tracking applications similar to CARPT, or hopper discharging. Future developments will focus on improving image binarization to track not only tracer objects but also structures based on the in-plane properties, such as position, size, and shape.