Comprehensive and Efficient Validation of Beam Modeling for a Proton Therapy System: Practical Considerations

Abstract

Introduction

Accurate beam modeling is essential for ensuring safe and effective proton therapy delivery. Before clinical implementation, pencil beam scanning systems require thorough validation to confirm that calculated dose distributions reliably reflect measured performance. This work outlines a practical approach to achieving comprehensive and efficient validation.

Methods

The beam model for a pencil beam scanning system was configured in the treatment planning system (TPS). Beam data including integrated depth dose, lateral profiles in air, and absolute outputs for various energies were measured and entered into the TPS following vendor recommendations. Validation tests were performed according to AAPM TG 185 and insights from other proton centers, adapted to our clinical requirements, time constraints, and regulations. The validation incorporated test cases from AAPM TG 350 draft report and included: 1) rectangular field dose distributions in water, 2) PDD measurements, 3) planar dose measurements using the DigiPhant detector with TG 350 test plans and clinical cases, and 4) end-to-end tests in animal tissue. TPS-calculated dose distributions, obtained using either the proton convolution superposition or Acuros Protons algorithms, were compared with corresponding measurements. A peer review from an institute with a similar proton treatment machine validated the machine output and our validation process.

Results

For rectangular targets with various ranges and modulation widths in water based on TG 185, TG 350 test plans, and clinical plans, ionization chamber and MatriXX PT planar dose measurements agreed with TPS calculations (point dose difference < 3%, planar dose 3%/3 mm > 95%). Range differences for animal tissues were within 3%. Independent peer output measurements agreed with our results within 1%.

Conclusion

TPS-calculated range and dose were in good agreement with measurements across multiple validation tests. The beam model for both PCS and Acuros PT has been validated and used clinically. Incorporating practical considerations is essential for achieving comprehensive and efficient beam commissioning and validation.

Keywords

proton therapy treatment planning system beam model commissioning validation

Highlights

Proton beam modeled in a treatment planning system for a Varian ProBeam proton system in China.

Comprehensive validation tests following principle of AAPM TG 185 report recommendations, with judicious design and implementation of tests to maximize commissioning efficiency while maintaining high beam modeling accuracy.

Preliminary test cases from AAPM TG 350 report, and end-to-end testing with animal tissues completed with results supporting high confidence in beam modeling accuracy.

Independent output measurement and review from a peer proton therapy center for non-IROC regions.

Validation test results and plan QA measurements for 47 patients in clinical trial, meet or surpass clinical expectations.

Introduction

Proton therapy represents one of the most advanced modalities for precise cancer radiotherapy, and has been increasingly used clinically by more and more cancer centers worldwide.¹ However, the accuracy of treatment planning systems (TPS) heavily relies on the proper commissioning and validation of the beam model, which remains a critical challenge in clinical implementation. Following the installation and technical commissioning of a proton therapy system by the manufacturer, and subsequent acceptance testing by the customer, the user institution is required to complete clinical commissioning of the beam model. Beam model accuracy is directly related to the accuracy of dose calculations of TPS,^2–5 and it is important to conduct comprehensive validation tests for the TPS to ensure its accuracy. In recent years, extensive research has been conducted on proton therapy commissioning, including dosimetric validation of scanned ion beams,⁶ commissioning of synchrotron-based systems with Monte Carlo treatment planning,⁷ and characterization of gantry-mounted accelerator systems.⁸ These studies, alongside others,^9–11 have established robust frameworks for beam modeling and clinical implementation, underscoring the need for novel approaches to advance the field. However, due to the variety of proton beam machines and TPS, available equipment and time from each institution, as well as the experience and training level of staff, there are currently no consensus on standard tests and procedures for beam model validation. AAPM TG 185 report specifies a systematic framework for PBS proton therapy commissioning, including required proton beam measurement, TPS dose modeling, and equipment needed, but it lacks standardized anatomical test cases and the recommended list of tests could be challenging for a center with limited beam time for commissioning. AAPM TG 350 report provides a library of clinically representative test cases and defines gamma criteria for dose validation. However, the report is pending publication and its application for clinical TPS validation has not been reported. Furthermore, in many countries other than the United States, independent proton machine output verification organizations such as IROC are either nonexistent or have limited operational capacity. As a result, the accuracy of machine output may rely exclusively on the physicist responsible for machine calibration.

In China, proton therapy systems face rigorous regulatory oversight distinct from Western countries. Prior to clinical operation, centers must pass two mandatory certifications: (1) Type Testing, a 3-month comprehensive evaluation covering mechanical safety, electromagnetic compatibility (EMC), imaging accuracy, and dosimetry (including anthropomorphic head and double-wedge phantom tests); and (2) Radiation Safety Control Assessment by provincial authorities typically completed within 3 days. China lacks independent auditing bodies like IROC for proton machine output verification. Centers must develop self-reliant, peer-validated commissioning strategies to ensure dosimetry accuracy.

Our institution operates a modern proton therapy system (ProBeam, Varian Medical System, Palo Alto, CA), with four treatment rooms, all equipped with full-rotating gantry systems and using the pencil beam scanning technique. The ProBeam system (Varian Medical Systems) consists of a cyclotron accelerator (70–244 MeV energy range), energy selection system, and four gantry rooms with pencil beam scanning nozzles (maximum field size 30 × 40 cm²).

This manuscript presents our experience in the commissioning and validation of the TPS beam model for this proton therapy system towards ensuring its safe and accurate clinical application, including customizing the selection of tests from TG 185, utilization of test cases from TG 350, conducting end-to-end validation using animal tissue phantoms, multi-TPS algorithm comparisons, and independent machine output verification by an outside institution.

While beam modeling validation principles are established, this work uniquely addresses challenges in resource-limited settings without IROC-equivalent oversight. To our knowledge, this represents one of the first comprehensive validation framework implemented for a ProBeam system in China, incorporating draft TG 350 test cases prior to formal publication and pioneering animal tissue end-to-end validation under clinical time constraints.

Materials and Methods

Beam Modeling

The treatment planning system (Eclipse version 16.1, Varian Medical Systems, Palo Alto, CA, USA) provides two dose calculation algorithms: the proton convolution superposition (PCS) and Acuros Protons (Acuros PT) dose algorithms. Previous results^12–15 have shown that when the single Gaussian model was used to describe the lateral fluence distribution of protons, the dose calculated in the TPS and measurements agreed well and met the clinical requirements when the low-dose envelope was corrected by field size factors. The implementation of the PCS algorithm in our TPS allows fine-tuning of the output correction for proton beam irradiation fields of various sizes, modulation widths and ranges. The single Gaussian model was selected when building the beam model in our institution. In Contrast, The Acuros PT algorithm solves the Linear Boltzmann Transport Equation (LBTE) using Monte Carlo statistical sampling, enabling explicit modeling of proton transport through heterogeneous media, thus providing higher accuracy in complex anatomical structures.¹⁶ The CT-number to stopping-power conversion was performed using the stoichiometric calibration as commonly used in other proton centers. In addition, a CT number to material density curve was also created in TPS using the CT tissue sample phantom.

Beam commissioning data required for model building of Eclipse includes:

1）Integral depth dose (IDD) curves in water for 19 energies (70-240 MeV in increments of 10 MeV plus the highest energy at 244 MeV), measured in a 3D water scanner, using a large-area parallel plate ionization chamber¹⁷ (StingRay, IBA Dosimetry) and validated against Monte Carlo simulation.

2）Absolute dose calibration at a 2 cm reference depth in a 10 cm × 10 cm scanned field for each of 19 energies described above, in accordance with IAEA TRS 398 Rev. 1 protocol, with a small-volume ionization chamber (PPC05) to normalize IDD curves.

3）Lateral fluence profiles in air, acquired at multiple positions along the beam axis using a high -resolution detector (Phoenix), both with and without range shifter. The beam data provided to the TPS, which forms the basis of the model, is derived from direct measurements of the beam spot at various energies at a gantry angle of 0^o.

Beam Model Validation

To assess the accuracy of the beam modeling, we conducted a series of validation tests as follows.

Independent Monte Carlo Calculation of Integral Depth Dose Curves

The open source Monte Carlo (MC) code package TOPAS¹⁸ was used for independent MC dose calculation. A beam model was built using the TOPAS MC toolkit. The modeling parameters of proton beams were first tuned and validated using measured in air profiles and integral depth dose curves for several single spot proton beams of low, medium and high energies. The model was then used to calculate all integral depth dose curves for all proton beams being commissioned and compared with the measurements.

Rectangular Targets in Water Phantom

Rectangular targets in a water phantom were used for the initial tests in this study. To improve the efficiency of commissioning with limited machine time while maintaining dosimetric integrity, we selectively implemented a representative subset of test cases from TG 185 report recommendations, as listed in Table 1. The field size selection was primarily constrained by the active area of our 2D detector array (MatriXX PT, 24.4 × 24.4 cm²). The targets in this paper are labeled as RxMyFSz, where Rx represents range of x cm, My modulation width of the spread-out Bragg Peak (SOBP) y cm, and FSz field size of z × z cm². Point dose and 2D planar dose were measured to compare with the calculated results of the TPS. The 2D dose distribution in DigiPhant was measured at 4 locations along the beam axis: the entrance region (3 cm from phantom surface), the proximal 90% range (R90), the center of SOBP, and the distal 50% range (R50). The relative depth dose distribution along beam direction was measured using a Zebra multi-layer ionization chamber.¹⁹ The measured and TPS calculated 2D dose distributions were compared using a gamma analysis with criteria 2%/2 mm, a 10% dose threshold, and global normalization, following the recommendation for uniform fields in homogeneous media.²⁰

Table 1.

Rectangular Targets in Water for TPS Dose Model Validation.

Range	SOBP Width	Side length of square field
(cm)	(cm)	(cm)
4	2,4	2,10
6	1,2,3,4,5	2,10
8	46,8	10
10	46,8	10
16	6,8,10	10
20	6,8,10,12	10
24	6,10,12	10,24
28	6,8,10,14	10,20
32	6,8	10,20

Test Cases from AAPM TG 350 Report

The TG 350 report provides test cases and comparison for clinical sites to evaluate the calculation accuracy of intensity modulated proton therapy beam models. While the TG 350 report is still under preparation, we used the test cases as a participating site to design treatment plans and perform the dose verification recommended by the task group, as shown in Figure 1. A virtual water phantom (30 cm × 30 cm × 20 cm) is used, with various target structures to mimic various clinical sites (eg, CSI, prostate, breast, C-shape). All materials in the phantom were assigned to water. The field angle selection in each plan was in accordance with values recommended by the task group. The plans were optimized to meet the target dose coverage and dose limit of organ at risk as specified by the task group. The verification plans were generated in the TPS by resetting all gantry angles to 270 degree or 0 degree (for shallow-depth targets only), and recalculating the dose. All measurement depths were determined by TG 350 guidelines. The DigiPhant water tank could not be used for the breast case at shallow depths due to the minimum water-equivalent thickness (WET) required for proper measurement. Specifically, the combined WET of the tank wall, the buildup layer of the MatriXX PT detector, and the detector holder is approximately 23.6 mm. Depths less than this value could not be accurately measured with this setup. The DigiPhant or a solid water phantom (for the breast case) were utilized with the MatriXX PT to measure planar dose distribution of each field at the recommended depths by the task group. Gamma analysis used 3%/3 mm criteria per TG 350 recommendations for inhomogeneous fields, accounting for increased complexity in dose gradients.

Figure 1.

Test Cases from TG 350. a: CSI Case, b: C-shape Case, c: Breast Case, d: Head and Neck (H&N) Case, e: Prostate Case, f: Pelvic Node Case.

End-to-end Test Using Animal Tissues

End-to-end testing was performed with animal tissues through the entire process: CT scanning, contouring, plan design and optimization, and treatment delivery. Images and plan designing of a pig head and pig leg are illustrated in Figure 2. Animal tissues were placed in a custom PMMA container and the remaining volume of the container was filled with dry air. Along the beam direction, A water phantom was positioned distal to the animal tissue phantom, with the planning target partially or fully in the water phantom. This setup simulates clinical scenarios where protons pass through heterogeneous tissues (eg, bone, muscle in the pig head/leg) before reaching deep targets. The beam angle was set to 270°, and beam pass through the animal tissue to irradiate the target: for the pig head phantom, the proton energy range was 124.7–215.7 MeV (corresponding to ranges of 11.41-29.55 cm); for the pig leg phantom, the energy range was 72.2–194.74 MeV (corresponding to proton ranges of 4.31-24.79 cm). The time from purchasing the animal tissue to plan delivery is approximately 5 h. Range accuracy was verified by measuring 2D dose distributions at R80 and at depths scanned in 0.1 mm steps within ±2 mm of R80. The depth yielding optimal gamma agreement between measured and TPS-calculated distributions was used to validate the range prediction model. Before plan delivery, we took 2D kV images to guide the positioning, as shown in Figure 2c. An oblique gantry angle was used to allow for orthogonal KV imaging, to facilitate 3D alignment, without colliding with the treatment couch.

Figure 2.

End-to-end Testing with Animal Tissues: a. Dose Distribution of a Treatment Plan Performed on a Pig Head; b. Dose Distribution of a Treatment Plan Performed on a Pig Leg; c. Image-Guided Animal Tissue Alignment Using 2D kV Imaging Before Beam Delivery.

Clinical Cases

During the model validation phase, five clinically representative test proton treatment plans were designed using CT datasets from patients treated previously with other treatment modalities in our clinic, adhering to clinical requirements for head and neck (H&N), lung, abdomen, prostate, and rectum cancer cases (with 1 plan per clinical site). For disease sites with substantial heterogeneity (eg, lung cancer, head and neck with air cavities), the Acuros PT algorithm was selected due to its superior handling of interfaces and density variations to minimize calculation errors.²¹ In contrast, the PCS algorithm was used for other sites because of its balanced accuracy and computational efficiency, meeting clinical requirements for homogeneous or mildly heterogeneous tissues. The method of creating verification plans and dose measurement was the same as described in 2.2.3. Three measurement depths were selected for each field, in the entrance region, target region, and distal fall-off region respectively.

Similar to clinical trial requirements for novel medical technologies in other countries, China requires each proton therapy system to complete a regulatory-mandated clinical trial (NCT05914142) validating facility-specific safety before routine clinical implementation. In total 47 patients were enrolled under strict ethical oversight. The treatment sites, number of treatment fractions, and planned fields are summarized in Table 2. Patient-specific plan QA was performed for each plan at 2–3 depths for each field. This variation is in accordance with our standard operating procedure, which requires two mandatory measurements at one depth proximal to the target and one within it, while a third measurement in the dose fall-off region is optional, typically used for special cases such as high fractional dose treatment and/or per physician request. For the purpose of beam model validation, only the initial treatment plans (the first approved plan for each patient at the start of treatment) were analyzed.

Table 2.

Treatment Sites of 47 Patients in Clinical Trial and Number of Plans and Fields in Initial Plan.

Site	Number of Treatment Fractions			Number of Plans Used			Number of Fields/Initial Plan
Site	Median	Max	Min	Median	Max	Min	Median	Max	Min
Abdomen	25	28	10	2	4	2	3	4	3
Extremity	25	25	10	1	3	1	3	4	3
Intracranial	27	35	15	1	3	1	3	4	2
H&N	35	36	30	5	7	2	5	5	3
Pelvis	28	30	25	3	3	1	3	3	2
Spine(sacrum)	24	35	20	3	4	2	3	3	3
Thorax	28	30	10	3	4	2	3	4	3

Peer Review

Since there is no national standard laboratory available to independently verify machine output in our country, we requested an independent peer review from Hefei Ion Medical Center (HIMC) where a similar Varian ProBeam proton therapy system was installed. A thorough review of our output measurement process and calculation was conducted. In addition, the absolute dose in water for two SOBP plans (R15M10FS10 and R20M10FS10) was measured independently by HIMC physicists using their own ionization chamber (CC13, the type both institutions had for better comparison) and compared with our data.

Results and Analysis

Beam Data Analysis

The measured beam data showed that the integrated depth dose (IDD) curves agreed very well with both PSTAR²² reference data and Monte Carlo simulations. Discrepancies in proton range (R80) and point-to-point dose were within 1 mm and 1%, respectively. The beam spot size in air at the isocenter in the X and Y directions was energy-dependent, ranging from  6.20±0.05 mm (X) and 6.08±0.03 mm (Y) among all gantry angles at 70 MeV to 3.41±0.08 mm (X) and 3.45±0.07 mm at 244 MeV, with less than 6% variation between the X and Y directions, while well within our commissioning tolerance of 10%, furthermore, use of a 57 mm water-equivalent range shifter improved the symmetry of the spot size. Finally, the absolute dose measured at a 2 cm reference depth revealed an energy-dependent trend: the dose per monitor unit decreased rapidly from 0.421 Gy/MU at 70 MeV to 0.342 Gy/MU at 160 MeV, then stabilized with a slight increase up to 0.345 Gy/MU at 244 MeV.

Beam Model Validation Results

Rectangular Targets Dose Distribution in Water

By fine tuning the output beam model with correction factors as described in 2.1, the degree of agreement between calculated and measured doses was further improved, especially in the results of small fields. The average gamma passing rate for all fields at 2%/2 mm criteria was 98.8%. Figure 3 shows the horizontal and vertical dose distribution comparison between measured and calculated values at the center of the SOBP plane for target R20M10FS10, with a gamma passing rate (2%/2 mm) of 100%.

Figure 3.

Comparison of Lateral Dose Profiles at the SOBP Center Plane for Proton Beam R20M10FS10 in the X (Left) and Y (Right) Directions.

The relative depth dose distribution along the beam direction was measured using the Zebra. The comparison of depth dose distributions for R10M8FS10 and R20M10FS10 is shown in Figure 4. The disagreements in range and SOBP width were both less than 1 mm.

Figure 4.

The Depth Dose Comparison of Beams R10M8FS10 and R20M10FS10 Between Measurement with the Zebra and TPS Calculation.

TG 350 Test Cases

All TG 350 test cases were evaluated using the recommended gamma criteria of 3%/3 mm, with an average passing rate of 98.9% across all fields, as shown in Table 3, meeting or exceeding the TG 350 report's clinical acceptability threshold of 95%. Gamma passing rates were greater than 95% at all measurement depths in a breast case when a range shifter was used. Figure 5 shows the verification result for C-shape target at 12 cm (WET) depth, with a gamma passing rate of 100%.

Figure 5.

Comparison of Lateral Dose Profiles at a Depth of 12 cm in the Verification Plan for C-Shape Target Test Plan in X (a) and Y (b) Directions.

Table 3.

Planar Dose Comparison Results for Targets from TG 350 Report.

Target	Field ID	Gantry Angle in Treatment Plan	Gantry Angle in QA Plan	Measurement WET Depth (cm)	Gamma Analysis (3%/3 mm)
CSI	1	180°	270°	7, 16	99.8%, 99.7%
	2	180°	270°	5, 10	99.6%, 99.5%
C-shape	1	140°	270°	10	99.4%
	2	0°	270°	12	100%
	3	220°	270°	10	100%
Breast	1	0°	0°	2, 4, 6, 8*	100%, 99.8%, 100%, 95.5%*
H&N	1	180°	270°	6, 12	99.7%, 100%
	2	20°	270°	9, 13	100%, 99.7%
	3	340°	270°	9, 12	100%, 99.2%
Prostate	1	90°	270°	9, 14	100%, 97.6%
	2	270°	270°	14, 17.4*	98.9%, 94.9%
Pelvic node	1	90°	270°	10.8, 20*	99.5%, 95.2%
	2	270°	270°	10.8, 20*	98.5%, 91.5%*

Note: * represents measurement depth located in distal fall-off region.

Clinic Plan Dose Verification

Table 4 presents the planar dose verification results of the clinically representative test proton treatment plans using the DigiPhant system, which show the percentage of points passing the gamma criteria(3%/3 mm). The average for all fields was 98.8%, and the average passing rate in the distal fall-off region (97.9%) was slightly smaller than that in the entry (98.9%) and target regions (99.7%). For heterogeneous disease sites, Monte Carlo is recommended for the final dose calculation.²¹ Therefore, we used the Acuros PT algorithm to calculate the final dose for lung cancer and some head and neck tumors. Figure 6 shows the dose profile comparison of the PCS and Acuros PT algorithms in head and neck sites, and the results demonstrate that doses in the target and penumbra region were well matched.

Figure 6.

Dose Profile Calculated by PCS and Acuros PT in Head and Neck Case.

Table 4.

Average Gamma Analysis Results of all Fields at 3 Depths for 5 Plans of Various Sites.

Site	Number of fields	Algorithm for final dose calculation	Passing rate(3%/3 mm)
Site	Number of fields	Algorithm for final dose calculation	Entrance region	Target region	Fall-off region(~R50)
Base of skull	4	PCS	97.2%	100%	100%
Lung	3	Acuros PT	100%	100%	97.3%
Retroperitoneal sarcoma	4	PCS	100%	98.5%	97.9%
Prostate	2	PCS	98.15%	100%	95.65%
Rectum	3	PCS	99%	100%	98.6%

In addition, we performed beam interruption testing on one test plan (Lung case). One beam was manually interrupted and then resumed with other beams delivered without beam interruption. Measured plane dose distribution with beam interruption was compared with the plan delivered without any beam interruption. The percentage of points passing the gamma criteria(1%/1 mm) was 99.5%. This test demonstrated that the OIS (Oncology Information System) and beam control system functioned well in the case of beam interruption.

The patient specific plan QA results for the proton clinical trial are shown in Figure 7. A total of 462 planar doses distributions at various depths, including in the beam entrance, inside the target and in the distal falloff region were measured for the initial treatment plans for all patients. The average passing rates of gamma analysis using the 3%/3 mm criteria were 99.1%, 98.9%, and 98.2% for depths in the entrance, the target and the distal fall off regions, respectively. A total of 91.3% (422/462) of measurements achieved a gamma passing rate ≥95%, 8.44% (39/462) of measurements exhibited passing rates between 90% and 95%, and 53.9% of these 39 intermediate-pass-rate datasets (21/39) were localized to the distal fall-off region. Only one measurement had a passing rate below 90%, with the minimum passing rate recorded at 88.9%. Notably, this single low-pass-rate measurement was also confined to the distal fall-off region.

Figure 7.

Gamma Analysis Result of Each Treatment Field in Proton Plans as a Function of Depth for the 47 Patients Treated During the Clinical Trial.

End-to-end Testing with Animal Tissue

For the 2D dose distribution comparison at R80 depth in the fall-off zone, all gamma passing rate are over 95% with the 3%/3 mm criteria, as shown in Table 5. The results show that, for both PCS and Acuros PT, the relative range accuracy, as estimated by the shift required to produce the maximum passing rate, was better than 3%, by comparing the 2D doses calculated at R80 and adjacent depths with the measurement results. This test provided confirmation on the accuracy of the CT calibration and beam model.

Table 5.

Gamma Analysis in Distal Fall-off Region of Animal Tissues

Animal tissue	Depth R80 (mm)	Algorithm for final dose calculation	Passing rate (3%/3 mm)	Moving distance at highest passing rate (mm)
Pig Head	263.6	PCS	97.97%	−1.4
Pig Head	263.6	Acuros PT	99.60	−0.6
Pig leg	206.4	PCS	100%	−2.6
Pig leg	206.4	Acuros PT	99.25%	0

Peer Review

The results of the independent verification show that the deviation of the dose in water at the center of two SOBPs is with in 1%, as shown in Table 6.

Table 6.

Comparison of Dose Measurement Results for Two SOBP Plans Between our Institution (GCCC) and the Peer Proton Center Physicists (HIMC).

	R15M10FS10		R20M10FS10
Chamber (CC13)	HIMC	GCCC	HIMC	GCCC
Meas. Charge (nC)	6.2745	6.3535	6.3705	6.4465
N_D,w (Gy/C)	0.2725	0.2681	0.2725	0.2681
Meas. Dose (Gy)	1.829	1.818	1.868	1.854
(PEER-OUR)/OUR	0.61%		0.76%

Discussion

We validated our beam model and TPS comprehensively with measurements of absolute doses in rectangular body targets in water, 2D dose distribution perpendicular to the beam incident direction for TG 350 test plans and selected clinical plans, peer review from another institution with a similar proton therapy machine, and end-to-end testing with animal tissues. The calculations and measurements demonstrated that a single Gaussian function adequately represents the fluence of a pencil beam in air when adjusted for field size effects, consistent with findings reported in prior studies.^12,23 While in most previous studies¹²^24–26 only one or two methods were used to validate the beam model, in our study, we validated the beam model comprehensively using 4 methods, among them, the use of TG 350 test cases was the first to report. Our TG 350 validation results (average gamma passing rate 98.9%) exceeded the baseline criteria of 95% or higher from the report, demonstrating the robustness of our beam model.

Our comprehensive validation prepared the TPS well for clinical use. A clinical trial was conducted for 47 patients after TPS commissioning and validation, with excellent patient specific plan QA results, further demonstrating the accuracy of the beam model.

Accurate assessment of range uncertainty is critical in particle therapy.²⁷ Currently, no consensus exists regarding the estimation of range uncertainty, and different uncertainty parameters have been employed in proton treatment planning across different institutions.²⁸ The testing of range uncertainty using animal tissues of pig head and pig leg shows that the relative deviation of range was less than 3%, which was thus used as the range uncertainty in our clinical plan optimization and robustness evaluation. The good agreement in dose distribution between the TPS calculations and MatriXX PT measurements showed that the beam model was modeled within acceptable accuracy.

Commissioning a proton beam machine requires a lot of beam time and human efforts in a typically limited time frame. Furthermore, in countries such as China, commissioning and validation tests may often need to compete with other tests that may be required by government regulations tests. Therefore, when planning and executing the commissioning, it is crucial to account for available beam time, equipment, and other resources, as well as clinical needs, clinical implementation timelines, and other practical considerations. Guided by these practical factors, we determined the types and quantities of validation tests, third-party independent verification requirements, and other aspects to ensure comprehensive and efficient commissioning. In addition, the work for commissioning needs to be flexible and should be adjusted with changes in need and resources during the process.

In regions without centralized independent dose verification bodies such as IROC, peer-to-peer institutional collaboration plays a critical role in ensuring beam output accuracy and cross-validation. In our study, we invited a domestic proton therapy center operating the same Varian ProBeam system to independently verify absolute output using their own measurement equipment. The close agreement between their measurements and ours (within 1%) provided additional confirmation of our output calibration and commissioning process by a third party. This peer-review model could serve as a practical surrogate for independent validation in countries like China, where standardized national auditing systems for proton therapy are not yet established. Future efforts may benefit from formalizing such inter-institutional partnerships to enhance QA transparency and reproducibility in non-IROC regions.

It is noteworthy that the validation framework and findings presented in this study might be specific to the Varian ProBeam proton therapy system and the Eclipse TPS. The applicability of the validation test methods and results for other proton therapy platforms or TPS with distinct beam delivery mechanisms, modeling algorithms, or commissioning workflows remains unverified. In addition, the practical considerations and validation tests were developed under the operational context of the authors’ institution, including available equipment (eg, DigiPhant, MatriXX PT), staffing expertise, and time constraints during clinical commissioning. Additionally, the validation framework relies on institution-specific resources (eg, DigiPhant, MatriXX PT), which may not be universally available. Similarly, the peer-review model for output verification has been adopted by one domestic proton centers using IBA system, demonstrating its vendor-agnostic value. While our validation focused on ProBeam, the emphasis on balancing regulatory compliance, unpublished standards (TG 350), and heterogeneous tissue testing (animal phantoms) provides a blueprint for centers in regions with similar resource or auditing constraints.

These constraints highlight the need for adaptable validation protocols that balance accuracy with resource accessibility. Therefore, for each new proton therapy center, the validation tests need to be adapted with practical considerations based on institution-specific resource allocations, technological infrastructures, clinical priorities, workflows, and/or regulatory requirements.

It is important to note that while we tried to be as comprehensive as possible before clinical implementation, some validation tests for certain complex clinical scenarios - such as non-coplanar beams, the effect of range shifter in various conditions, the effect of couch on both dosimetry and collision, are important but not fully tested due to hardware limitations. These limitations need to be considered during clinical implementation, and further tests should be performed if possible.

Conclusion

This study provides a framework for comprehensive validation of the Varian ProBeam beam model, ensuring its safe and accurate application in clinical proton therapy. Our findings highlight the importance of rigorous validation processes in achieving high treatment accuracy while taking account of available resources and other practical considerations to also achieve high efficiency. The measured proton ranges and doses were found to be in good agreement with the calculations from the planning system. The beam model can be safely applied in clinical proton therapy.

Footnotes

Acknowledgements

The authors want to thank the engineer team from Varian and the physics team and other staff members in the Radiation Oncology department at the Guangzhou Concord Cancer Center for their support during acceptance test and commissioning.

ORCID iD

Yuanshui Zheng

Ethical Statement

Ethics approval is not required because all animal tissues used in this study were bought from meats at grocery store. The QA data are dosimetric data only from patients during the clinical trial which has been approved by NCT05914142. No live animal or human being was involved in this study.

Author Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Yajun Jia, Yifeng Yang, Zhangmin Li, Zuofeng Li and Yuanshui Zheng. Manuscript was drafted by Yajun Jia and edited by Yuanshui Zheng. All authors commented on the manuscript. All authors read and approved the final manuscript.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Paganetti

Beltran

Both

, et al. Roadmap: proton therapy physics and biology. Phys Med Biol. 2021;66(5). 10.1088/1361-6560/abcd16

Zhu

Poenisch

Lii

, et al. Commissioning dose computation models for spot scanning proton beams in water for a commercially available treatment planning system. Med Phys. 2013;40(4):041723. 10.1118/1.4798229

Saini

Cao

Bowen

, et al. Clinical commissioning of a pencil beam scanning treatment planning system for proton therapy. International Journal of Particle Therapy. 2016;3(1):51–60. https://doi/10.14338/ijpt-16-0000.1

Pidikiti

Patel

Maynard

, et al. Commissioning of the world’s first compact pencil-beam scanning proton therapy system. Radiation Oncology Physics. 2018;10(1):94–105.10.1002/acm2.12225

Langner

Eley

Dong

, et al. Comparison of multi-institutional Varian ProBeam pencil beam scanning proton beam commissioning data. J Appl Clin Med Phys. 2017;18(3):96–107. https://doi/10.1002/acm2.12078

Mirandola

Molinelli

Vilches Freixas

, et al. Dosimetric commissioning and quality assurance of scanned ion beams at the Italian National Center for Oncological Hadrontherapy. Med Phys. 2015;42(9):5287–5300. 10.1118/1.4928397

Azcona

J-D

Aguilar

Perales

, et al. Commissioning of a synchrotron-based proton beam therapy system for use with a Monte Carlo treatment planning system. Radiat Phys Chem. 2022;204(11):110708. 10.1016/j.radphyschem.2022.110708

Kang

Pang

. Commissioning and beam characterization of the first gantry-mounted accelerator pencil beam scanning proton system. Med Phys. 2020;47(8):3496–3510. 10.1002/mp.13972

Vilches-Freixas

Unipan

Rinaldi

, et al. Beam commissioning of the first compact proton therapy system with spot scanning and dynamic field collimation. Br J Radiol. 2020;93(1107):20190598. 10.1259/bjr.20190598

10.

Gomà

Henkner

Jäkel

, et al. ESTRO-EPTN radiation dosimetry guidelines for the acquisition of proton pencil beam modelling data. Phys Imaging Radiat Oncol. 2024;31:100621. 10.1016/j.phro.2024.100621

11.

Gillin

Sahoo

Zhu

, et al. Commissioning of the discrete spot scanning proton beam delivery system at the University of Texas M.D. Anderson Cancer Center, Proton Therapy Center, Houston. Med Phys. 2010;37(1):154–163. 10.1118/1.3259742

12.

Zhu

Poenisch

Lii

, et al. Commissioning dose computation models for spot scanning proton beams in water for a commercially available treatment planning system. Med Phys. 2013;40(4):041723. 10.1118/1.4798229

13.

Sahoo

Poenisch

Zhang

, et al. 3D Treatment planning system-Varian Eclipse for proton therapy planning. Med Dosim. 2018;43(2):184–194. 10.1016/j.meddos.2018.03.006

14.

Shen

Liu

Stoker

, et al. An efficient method to determine double Gaussian fluence parameters in the eclipse™ proton pencil beam model. Med Phys. 2016;43(12):6544. 10.1118/1.4967485

15.

Sawakuchi

Zhu

Poenisch

, et al. Experimental characterization of the low-dose envelope of spot scanning proton beams. Phys Med Biol. 2010;55(12):3467–3478. 10.1088/0031-9155/55/12/013

16.

Lin

Huang

Kang

, et al. A benchmarking method to evaluate the accuracy of a commercial proton monte carlo pencil beam scanning treatment planning system. J Appl Clin Med Phys. 2017;18(2):44–49. doi: 10.1002/acm2.12043

17.

Bäumer

Koska

Lambert

, et al. Evaluation of detectors for acquisition of pristine depth-dose curves in pencil beam scanning. J Appl Clin Med Phys. 2015;16(6):151–163. 10.1120/jacmp.v16i6.5577

18.

Perl

Shin

Schumann

Faddegon

Paganetti

. TOPAS: An innovative proton Monte Carlo platform for research and clinical applications. Med Phys. 2012;39(11):6818–6837. 10.1118/1.4758060

19.

Vai

Mirandola

Magro

, et al. Characterization of a MLIC detector for QA in scanned proton and carbon Ion beams. Int J Part Ther. 2019;6(2):50–59. https://doi/10.14338/IJPT-19-00064.1 .

20.

Farr

Moyers

Allgower

, et al. Clinical commissioning of intensity-modulated proton therapy systems: Report of AAPM task group 185. Med Phys. 2021;48(1):e1–e30. doi: 10.1002/mp.14546

21.

Guidelines for the Use of Hadron Radiation Therapy in NCI-Sponsored Cooperative Group Clinical Trials Updated September 4, 2019; http://irochouston.mdanderson.org/rpc/home_page/Proton_guidelines.htm；2019 [accessed 11 December 2022].

22.

National institute of Standards and Technology Physical Meas. Laboratary, 2009; https://physics.nist.gov/PhysRefData/Star/Text/PSTAR.html; [accessed 5 March 2022].

23.

Shang

Evans

Rahman

Lin

. Beam characteristics of the first clinical 360° rotational single gantry room scanning pencil beam proton treatment system and comparisons against a multi-room system. J Appl Clin Med Phys. 2020;21(9):266–271. 10.1002/acm2.12984

24.

Saini

Cao

Bowen

, et al. Clinical commissioning of a pencil beam scanning treatment planning system for proton therapy. Int J Part Ther. 2016;3(1):51–60. 10.14338/ijpt-16-0000.1

25.

Huang

Kang

Souris

, et al. Validation and clinical implementation of an accurate Monte Carlo code for pencil beam scanning proton therapy. J Appl Clin Med Phys. 2018;19(5):558–572. 10.1002/acm2.12420

26.

Bäumer

Geismar

Koska

, et al. Comprehensive clinical commissioning and validation of the RayStation treatment planning system for proton therapy with active scanning and passive treatment techniques. Phys Med. 2017;43:15–24. 10.1016/j.ejmp.2017.09.136

27.

Zheng

Kang

Zeidan

, et al. An end-to-end assessment of range uncertainty in proton therapy using animal tissues. Phys Med Biol. 2016;61(22):8010–8024. https://doi/10.1088/0031-9155/61/22/8010

28.

Paganetti

. Range uncertainties in proton therapy and the role of Monte Carlo simulations. Phys Med Biol. 2012;57(11):R99–117. https://doi/10.1088/0031-9155/57/11/R99 Phys Med Biol, 2012, 57(11): R99-117. https://doi/10.1088/0031-9155/57/11/R99.