Sage Journals: Discover world-class research

Abstract

This study aimed to compare the 3D skull models reconstructed from computed tomography (CT) images using three different open-source software with a commercial software as a reference. The commercial Mimics v17.0 software was used to reconstruct the 3D skull models from 58 subjects. Next, two open-source software, MITK Workbench 2016.11, 3D Slicer 4.8.1 and InVesalius 3.1 were used to reconstruct the 3D skull models from the same subjects. All four software went through similar steps in 3D reconstruction process. The 3D skull models from the commercial and open-source software were exported in standard tessellation language (STL) format into CloudCompare v2.8 software and superimposed for geometric analyses. Hausdorff distance (HD) analysis demonstrated the average points distance of Mimics versus MITK was 0.25 mm. Meanwhile, for Mimics versus 3D Slicer and Mimics versus InVesalius, there was almost no differences between the two superimposed 3D skull models with average points distance of 0.01 mm. Based on Dice similarity coefficient (DSC) analysis, the similarity between Mimics versus MITK, Mimics versus 3D Slicer and Mimics versus InVesalius were 94.1, 98.8 and 98.3%, respectively. In conclusion, this study confirmed that the alternative open-source software, MITK, 3D Slicer and InVesalius gave comparable results in 3D reconstruction of skull models compared to the commercial gold standard Mimics software. This open-source software could possibly be used for pre-operative planning in cranio-maxillofacial cases and for patient management in the hospitals or institutions with limited budget.

Keywords

3D reconstruction computed tomography standard tessellation language superimposition open-source software craniofacial reconstruction surgical planning

Introduction

Advances in craniofacial imaging have placed the importance on the 3D reconstruction of the skull model for medical applications. This technology has provided new possibilities to visualize complex medical data through generation of 3D skull models which were used for basic cranial education for medical students,¹ surgical training for surgeons,² pre-operative planning,^3,4 facial contouring surgery,⁵ forensic medicine and dentistry,⁶ computer-assisted surgery,⁷ maxillofacial prosthesis⁸ and craniofacial reconstruction.^9–12

Craniofacial reconstruction is commonly performed following head or facial trauma and on cancer patients who have lost part of the bony structures following tumour surgery. In current practice, the reconstruction of craniofacial defects is normally based on bone graft which is shaped to fit the defect. However, clinically, bone graft is limited to a small defect as the graft is taken from the patient’s own bone. With 3D skull model derived from CT data, pre-surgical planning can be done to fabricate an implant from compatible biomaterials such as titanium mesh or methylmethacrylate.¹³ Using this technique, bigger and complex defect of the skull can be repaired.

The use of 3D reconstruction and 3D model reduces the possibility of errors during surgery, improves fit, and provides better implant stability.¹⁴ In addition, the main advantage of 3D reconstruction is the possibility to simulate the surgical approach in the operating theatre, usually the supine position with lateral rotation of the head.¹⁵ 3D reconstruction also allows simulation of the various surgical approaches in order to select a minimal invasive approach during pre-operative planning.

The emerging of open-source software has shown an increasing trend^16,17 because of the main advantage of being free and easily available, which significantly lowers the entry barrier to using it. Abdullah et al.¹⁸ compared 3D reconstruction of segmented mandible between commercial software and open-source MITK software on five CT images. Results of the geometric differences showed average errors of two superimposed 3D models were less than 1% using Hausdorff distance (HD). However, these comparisons were performed on part of craniofacial region but do not cover the whole skull.

Most of the skull segmentation studies have utilised commercial software to create the meshed model or 3D model of the skull from patients’ CT data; for example, Mimics software,^19–21 CATIA software²² and Maxilim® software.²³ The 3D models created using skull segmentation software can be used for pre-operative planning or to design cranial implants.²⁴ However, most of the software mentioned in the literature is either commercial software or built in-house which were out of reach to researchers without big budgets or facilities.

In the medical imaging field, an accurate, reliable, and yet low-cost 3D imaging software for 3D reconstruction of the skull is crucial as it would have an impact on patient management that an appropriate software for skull segmentation is desirable. In this study, three open-source software, MITK, 3D Slicer and InVesalius were selected for the analysis based on their robustness, visualizations, reliability, and ease of use. Furthermore, this software was able to segment the 2D images and reconstructed them into 3D exportable models that are in STL files. An STL file describes the surface geometry of an object which can be sent to the 3D printers for printing of the skull.

The MITK software was available since 2003.²⁵ It has now evolved into a software system that can cover all steps of a clinical workflow including data retrieval, image analysis, diagnosis, treatment planning, intervention support and treatment control 26. The interface is easy to use, and the steps are user-friendly, especially to researchers or clinicians without an image processing background. The 3D Slicer software is a cross-platform and open-source software and features a flexible and mature plugin system. The community is very large and active. Compared to MITK, its approach is generally more application-centric though this has been addressed by the possibility of running modules on their own without the application on main window. On the other hand, the InVesalius software was made fully available as a free and open source via the Public Software Portal in November 2007.

The aim of this study is to assess the accuracy of 3D skull models reconstructed with commercial software and three open-source software. Geometric analyses were utilised to assess the differences and similarities between two skull models.

Methods

A sample population from January 2010 until December 2016 with CT head scans was obtained from the Neurosurgery Department, Hospital Universiti Sains Malaysia (USM), Kubang Kerian, Kelantan. Convenience sampling was used for this research. Data were collected from archived images of patients from the Picture Archiving and Communication System (PACS) server at the Radiology Department, Hospital USM. Ethical application was approved by the Ethics and Research Committee USM, reference number USMKK/PPP/JEPeM (259.3[2]).

Fifty-eight patients who had undergone CT scans with the CT images of 1-mm slice thickness and a matrix of 512 × 512 pixels were selected for this study. They were scanned using the Siemens Somatom Definition AS+ 128-slice (Siemens, Erlangen, Germany) at the Radiology Department, Hospital USM. The scans were acquired with 120 kVp. The slice thickness was 1.0 mm. For each patient, we reconstructed the scans with the same set of reconstruction parameters where the field of view (FOV) was 256 × 256 mm². The pixel size = FOV/matrix (number of pixels) = 256 × 256/512 × 512 = 0.25 mm.

The CT data in DICOM format were imported into the medical imaging software (Materialise Mimics ver 17.0, MITK Workbench 2016.11, 3D Slicer 4.8.1 and InVesalius 3.1) for 3D reconstruction process. All four software were used to construct the 3D images from 2D cross-sectional images that were retrieved from the PACS server. Automatic segmentation of the skull was applied to create the 3D model using thresholding method. All four software went through similar steps in 3D reconstruction process, which is loading the DICOM format, segmentation using thresholding method (226 Hounsfield Unit) for 3D skull model creation as well as post-processing to eliminate the noises. In order to eliminate the noises, region growing method was used in Mimics and MITK, while largest island method and largest surface selection were applied in 3D Slicer and InVesalius, respectively. Finally the obtained 3D model was exported to standard tessellation language (STL) format as shown in Figure 1.

Figure 1.

Summary of steps involved in 3D reconstruction of skull model for all four software.

All the data obtained in STL format were compared for accuracy analysis using analysis of variance (ANOVA) followed by Dunnet tests and geometric analyses (HD and DSC). The Statistical Package for the Social Science (SPSS) version 24.0 was used for statistical analyses in this study. Test of normality and homogeneity were carried out and confirmed using Kolmogorov-Smirnov test. One-way ANOVA was used to test for evidence of statistical differences in craniometric measurements of reconstructed 3D skull models. Dunnett tests were performed to find differences between each experimental group and the control group. A statistical significance value was set at p < 0.05.

In summary, four software; a commercial Mimics software and three open-source software (MITK, 3D Slicer and InVesalius) were utilised to reconstruct the 3D skull models of 58 patients as shown in Figure 2.

Figure 2.

The 3D skull models produced using different software: (a) mimics, (b) MITK, (c) inVesalius and (d) 3D slicer are shown.

The accuracy of the 3D skull model reconstructed using the three open-source software (MITK, 3D Slicer and InVesalius) were compared with 3D skull model reconstructed using the commercial Mimics software. The comparison was made using metrics based on 3D geometry, which were HD and Dice similarity coefficient (DSC) using the open-source CloudCompare v2.8 software following the steps in Abdullah et al.²⁶ study as shown in Figure 3.

Figure 3.

Geometric analyses of HD and DSC using CloudCompare v2.8 software.

Results

The 3D skull models segmented using Mimics software were used as the gold standard in this study. Similar segmentation processes were performed using MITK, 3D Slicer and InVesalius software. Results of the volume of 3D skull models segmented using four different software analysed using one-way ANOVA are shown in Table 1.

Table 1.

The 3D skull volume measurements using four different software.

Software	n	3D skull volume (mm³)
Software	n	Mean (SD)
Mimics	58	588,840.24 (79,936.06)
MITK	58	650,463.55 (87,274.28)
3D slicer	58	589,470.79 (79,896.06)
InVesalius	58	593,989.67 (81,229.70)

There were no statistically significant differences between volumes of the 3D skull models for all four software determined by one-way ANOVA [F (3, 228) = 1.606, p = 0.189].

Dunnett test comparing the volume of 3D skull models generated from three open-source software (MITK, 3D Slicer and InVesalius) against the control 3D skull models reconstructed using the commercial Mimics software is shown in Table 2.

Table 2.

Dunnett test of 3D skull volume.

Variable	Significant level, p
Variable	Mimics versus MITK	Mimics versus 3D slicer	Mimics versus InVesalius
Volume	0.147	>0.999	0.974

There were no significant differences in comparisons between Mimics and MITK, Mimics and 3D Slicer and Mimics and InVesalius with p > 0.05.

Geometric analyses using HD and DSC are shown in Tables 3 and 4.

Table 3.

Geometric analysis using HD (mm) of skull models.

Patient	Mimics versus MITK	Mimics versus 3D slicer	Mimics versus InVesalius
1	0.241	0.004	0.006
2	0.248	0.001	0.009
3	0.262	0.004	0.007
4	0.244	0.003	0.007
5	0.261	0.004	0.008
6	0.274	0.024	0.008
7	0.242	0.001	0.013
8	0.259	0.008	0.008
9	0.264	0.001	0.007
10	0.263	0.003	0.005
11	0.240	0.005	0.008
12	0.244	0.005	0.008
13	0.245	0.004	0.005
14	0.246	0.008	0.009
15	0.241	0.008	0.008
16	0.253	0.007	0.011
17	0.255	0.005	0.010
18	0.241	0.007	0.008
19	0.248	0.008	0.008
20	0.255	0.005	0.006
21	0.261	0.005	0.006
22	0.248	0.005	0.006
23	0.243	0.006	0.006
24	0.241	0.003	0.006
25	0.244	0.005	0.007
26	0.256	0.005	0.006
27	0.266	0.008	0.009
28	0.268	0.005	0.006
29	0.266	0.005	0.005
30	0.262	0.006	0.009
31	0.248	0.009	0.015
32	0.242	0.005	0.006
33	0.249	0.008	0.015
34	0.257	0.009	0.013
35	0.272	0.007	0.007
36	0.241	0.006	0.006
37	0.247	0.009	0.015
38	0.263	0.008	0.015
39	0.265	0.005	0.007
40	0.275	0.008	0.015
41	0.242	0.008	0.011
42	0.241	0.005	0.015
43	0.274	0.008	0.014
44	0.277	0.008	0.008
45	0.269	0.009	0.010
46	0.266	0.008	0.019
47	0.268	0.008	0.017
48	0.262	0.009	0.015
49	0.243	0.005	0.008
50	0.241	0.005	0.006
51	0.251	0.005	0.017
52	0.255	0.009	0.015
53	0.263	0.005	0.008
54	0.247	0.008	0.016
55	0.241	0.004	0.009
56	0.240	0.003	0.007
57	0.251	0.008	0.012
58	0.258	0.005	0.010
Mean (SD)	0.254 (0.011)	0.006 (0.002)	0.009 (0.004)

Table 4.

Geometric analysis using DSC (mm) of skull models.

Patient	Mimics versus MITK	Mimics versus 3D slicer	Mimics versus InVesalius
1	0.945	0.987	0.986
2	0.933	0.977	0.988
3	0.932	0.980	0.995
4	0.929	0.992	0.987
5	0.903	0.955	0.912
6	0.930	0.990	0.992
7	0.920	0.980	0.965
8	0.908	0.988	0.979
9	0.952	0.996	0.995
10	0.955	0.988	0.991
11	0.962	0.997	0.993
12	0.942	0.995	0.990
13	0.933	0.995	0.995
14	0.946	0.998	0.997
15	0.948	0.996	0.996
16	0.955	0.998	0.998
17	0.938	0.995	0.991
18	0.946	0.995	0.991
19	0.944	0.978	0.962
20	0.952	0.996	0.991
21	0.862	0.950	0.891
22	0.952	0.996	0.991
23	0.944	0.982	0.983
24	0.955	0.995	0.992
25	0.956	0.998	0.996
26	0.932	0.958	0.883
27	0.933	0.986	0.971
28	0.952	0.998	0.991
29	0.951	0.992	0.988
30	0.950	0.987	0.962
31	0.952	0.996	0.994
32	0.944	0.966	0.937
33	0.956	0.998	0.996
34	0.941	0.996	0.992
35	0.923	0.962	0.982
36	0.955	0.998	0.995
37	0.954	0.998	0.994
38	0.938	0.977	0.965
39	0.948	0.992	0.990
40	0.947	0.992	0.988
41	0.944	0.988	0.989
42	0.936	0.993	0.988
43	0.938	0.993	0.988
44	0.952	0.998	0.995
45	0.951	0.995	0.996
46	0.949	0.995	0.994
47	0.922	0.952	0.984
48	0.955	0.998	0.996
49	0.954	0.998	0.994
50	0.946	0.988	0.983
51	0.935	0.986	0.984
52	0.955	0.998	0.996
53	0.954	0.995	0.994
54	0.938	0.977	0.963
55	0.942	0.995	0.992
56	0.944	0.998	0.996
57	0.933	0.966	0.987
58	0.937	0.995	0.992
Mean (SD)	0.941 (0.016)	0.988 (0.013)	0.983 (0.024)

0: no overlap; 1: complete overlap.

Based on HD analysis, the average points distance from Mimics versus MITK were 0.25 mm while for Mimics versus 3D Slicer and Mimics versus InVesalius, there were almost no differences between the two 3D skull models with the average points distance at 0.01 mm.

Based on DSC analysis, the mean (SD) DSC was 0.941 (0.2) or 94.1% similar for MITK versus Mimics, and the mean (SD) DSC was 0.988 (0.01) or 98.8% similar for 3D Slicer versus Mimics, and mean DSC 0.983 (0.2) or 98.3% similar for InVesalius versus Mimics.

Discussion

In this study, the commercial Mimics software as well as open-source software of MITK, 3D Slicer and InVesalius were used to reconstruct 3D skull models from CT scan data. The volume of the 3D skull model reconstructed using the software were initially analysed prior to the geometric analyses using HD and DSC. The volume of the skulls generated from the open-source software were statistically insignificant as compared to the commercial software, which indicating that open-source software could generate accurate 3D models. It should be noted that anatomical structure includes internal cavities and intricate details that volumetric measurement is essential to represent the accurate size and shape of the respected skulls.

The commercial Mimics software used in this study was set as a reference or “gold standard.” It has been reported that HD value less than 3 mm and DSC value above 0.80 were considered adequate and accurate.^27,28 The HD value from this study for Mimics versus MITK and Mimics versus 3D slicer were 0.25 and 0.01 mm, respectively. The results suggest that the 3D skull models reconstructed using the open-source MITK software is accurate, or similar to the gold standard. However, the 3D skull models reconstructed using the open-source 3D Slicer software were more accurate as compared to MITK. Another study²⁶ compared 50 skull models between Mimics versus InVesalius software, the HD value was 0.01 mm which is compatible to the results of 3D Slicer software.

The 3D skull models reconstructed from MITK, 3D Slicer and InVesalius have DSC values of 0.941, 0.988 and 0.983 when analysed against the gold standard. This showed that the skull models were 94.1, 98.8 and 98.3% similar to the gold standard, respectively. These findings showed that 3D skull models reconstructed from the open-source 3D Slicer software were the most accurate compared to other two software. Similarly, Abdullah et al.,²⁶ compared the 3D reconstructed skulls using Mimics and InVesalius on 50 CT images with 97.6% similarity. reAlnaser et al.²⁹ evaluated seven open-source software for lung segmentation; two of the software were MITK and 3D Slicer. They evaluated the software based on the criteria such as functionality, usability, quality of segmentation and 3D export where the software must be able to export in STL format. Based on their study, MITK scored better than 3D Slicer. They reported MITK as a relatively easy to use and also regarded MITK as more user friendly than 3D Slicer. This study also revealed that MITK is more user-friendly than 3D Slicer. However, there are many learning resources available for 3D slicer which make it beneficial for beginners.

It should be noted that the cost of the commercial software is expensive, and often out of reach for university researchers, especially from public universities with limited grant. Moreover, the purchasing process was quite tedious and took many months for the public university before the software can be utilised, as there were lots of red tape involved. Once purchased, it can only be used by one user at one time (for one license). This would be a major setback for a big group of researchers in a university where they have to queue and wait for their turn to use the software, as the additional license would incur more cost to the university.

One of the advantages of the three open-source software utilised in this study is that the segmentation and post-processing of the images can be applied by an automatic process without manual intervention; therefore, human errors were minimised. Thus, these methods are reliable, repeatable and reproducible by other researchers from other research institutions. However, the involvement of clinicians is necessary for complex cases to identify the region of interest for better clinical outcomes.

Although 58 skulls were successfully reconstructed, limitation such as the presence of CT artefacts of noises required additional processing (Figure 4). In Mimics and MITK software, region growing method was performed to isolate the skull from other unrelated objects. Meanwhile, the largest island method was used in 3D Slicer to isolate the skull. These different methods were used as each software has its algorithm to isolate the different types of hard tissues or objects with a similar range of HU. Region growing method was carried out to select the region of interest by selecting only the connected region with similar pixels value. It removed the detached pixels from the segmentation. This is to ensure that the exact part of interest was selected. On the other hand, largest island algorithm determined the biggest group of connected pixels. Both region growing and largest island methods were useful tools after thresholding method to isolate the skull from other unrelated objects such as gantry and noises.

Figure 4.

Unrelated objects (in red circles) that need to be removed.

Region growing method was carried out to select the region of interest by selecting only the connected region with similar pixels value. It removed the detached pixels from the segmentation. This is to ensure that the exact part of interest was selected. On the other hand, largest island algorithm determined the biggest group of connected pixels. Both region growing and largest island methods were useful tools after thresholding method to isolate the skull from other unrelated objects such as gantry and noises. Therefore, the algorithm provided in this two open-source software is sufficient to segment the 3D skull models and produced equivalent results when compared with the commercial software. Meanwhile, an almost similar study was conducted where the accuracy of craniomaxillofacial region of mandibular jaw was assessed using Mimics as well as four software of Invesalius, ITK-Snap, Dolphin 3D and Slicer 3D.³⁰ Their results also revealed that the software were able to define the contour of the mandibular bone which indicated the reliability of the software.

Conclusions

In conclusion, this study confirmed that the alternative open-source software, MITK, InVesalius and 3D Slicer gave comparable results in 3D reconstruction of skull models compared to the commercial gold standard Mimics software. This open-source software could possibly be used for pre-operative planning in cranio-maxillofacial cases and for patient management in the hospitals or institutions with limited budget.

Footnotes

Acknowledgements

We would like to thank Mr. Firdaus Daud from the Radiology Department, Hospital USM for his helpful assistance in this project.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was made possible with funding from TDC Holdings Sdn Bhd through Universiti Sains Malaysia (304.PPSG.6150194/T152).

ORCID iD

Abdul Manaf Abdullah

Data availability statement

The data used to support the findings of this study are available from the corresponding author upon request.

References

Chen

Pan

, et al. The role of three-dimensional printed models of skull in anatomy education: a randomized controlled trail. Sci Rep 2017; 7: 575.

de Notaris

Palma

Serra

, et al. A three-dimensional computer-based perspective of the skull base. World Neurosurg 2014; 82: s41–s48.

Giannopoulos

Steigner

George

, et al. Cardiothoracic applications of 3-dimensional printing. J Thorac Imaging 2016; 31: 253–272.

Abdullah

Rajion

Omar

. Biomodelling of cranio-maxillofacial implant applying open source software. J Teknol 2015; 76: 57–61.

Yim

Nguyen

Kim

. Facial contouring surgery with custom silicone implants based on a 3D prototype model and CT-scan: a preliminary study. Aesthetic 2015; 39: 418–424.

Katsumura

Sato

Ikawa

, et al. “High-precision, reconstructed 3D model” of skull scanned by conebeam CT: reproducibility verified using CAD/CAM data. Leg Med (Tokyo) 2016; 18: 37–43.

Ritacco

Di Lella

Mancino

, et al. 3D printed models and navigation for skull base surgery: case report and virtual validation. Stud Health Technol Inform 2015; 216: 1025.

Jazayeri

Kang

Masri

, et al. Advancements in craniofacial prosthesis fabrication: a narrative review of holistic treatment. J Adv Prosthodont 2018; 10: 430–439.

Jardini

Larosa

Filho

, et al. Cranial reconstruction: 3D biomodel and custom-built implant created using additive manufacturing. J Cranio-Maxillofac Surg 2014; 42: 1877–1884.

10.

Maduri

Viaroli

Levivier

, et al. Use of a 3D skull model to improve accuracy in cranioplasty for autologous flap resorption in a 3-year-old child. Pediatr Neurosurg 2017; 52: 351–355.

11.

Park

Lim

Yun

, et al. Cranioplasty enhanced by three-dimensional printing: custom-made three-dimensional-printed titanium implants for skull defects. J Craniofac Surg 2016; 27: 943–949.

12.

Schebesch

Hohne

Gassner

, et al. Preformed titanium cranioplasty after resection of skull base meningiomas–a technical note. J Craniomaxillofac Surg 2013; 41: 803–807.

13.

Jalbert

Boetto

Nadon

, et al. One-step primary reconstruction for complex craniofacial resection with PEEK custom-made implants. J Cranio-Maxillofac Surg 2014; 42: 141–148.

14.

Saldarriaga

JFI

Velez

Posada

, et al. Design and manufacturing of a custom skull implant. Am J Eng Appl Sci 2011; 4: 169–174.

15.

Bonne

Dubrulle

Risoud

, et al. How to perform 3D reconstruction of skull base tumours. Eur Ann Otorhinolaryngol Head Neck Dis 2017; 134: 117–120.

16.

Ganry

Hersant

Quilichini

, et al. Use of the 3D surgical modelling technique with open-source software for mandibular fibula free flap reconstruction and its surgical guides. J Stomatol Oral Maxillofac Surg 2017; 118: 197–202.

17.

Sander

McGoldrick

Helms

, et al. Three-dimensional printing of X-ray computed tomography datasets with multiple materials using open-source data processing. Anat Sci Educ 2017; 10: 383–391.

18.

Abdullah

Omar

Pritam

HMH

, et al. Comparison of 3D reconstruction of mandible for pre-operative planning using commercial and open-source software. AIP Conf Proc 2016; 1791: 020001.

19.

Moiduddin

Darwish

Al-Ahmari

, et al. Structural and mechanical characterization of custom design cranial implant created using additive manufacturing. Electron J Biotechnol 2017; 29: 22–31.

20.

Bogu

Kumar

Kumar Khanara

. Modelling and structural analysis of skull/cranial implant: beyond mid-line deformities. Acta Bioeng Biomech 2017; 19: 125–131.

21.

Rotaru

Stan

Florian

, et al. Cranioplasty with custom-made implants: analyzing the cases of 10 patients. J Oral Maxillofac Surg 2012; 70: e169–e176.

22.

Chrzan

Urbanik

Karbowski

, et al. Cranioplasty prosthesis manufacturing based on reverse engineering technology. Med Sci Monit 2012; 18: MT1–MT6.

23.

Jonkergouw

van de Vijfeijken

Nout

, et al. Outcome in patient-specific PEEK cranioplasty: a two-center cohort study of 40 implants. J Craniomaxillofac Surg 2016; 44: 1266–1272.

24.

Kim

Hong

Park

, et al. Customized cranioplasty implants using three-dimensional printers and polymethyl-methacrylate casting. J Korean Neurosurg Soc 2012; 52: 541–546.

25.

Nolden

Zelzer

Seitel

, et al. The medical imaging interaction toolkit: challenges and advances: 10 years of open-source development. Int J Comput Assist Radiol Surg 2013; 8: 607–620.

26.

Abdullah Johari

Abdullah Abdul

Hadi

, et al. Comparison of STL skull models produced using open-source software versus commercial software. Rapid Prototyp J 2019; 25: 1585–1591.

27.

Hvid

Elstrom

Jensen

, et al. Accuracy of software-assisted contour propagation from planning CT to cone beam CT in head and neck radiotherapy. Acta Oncol 2016; 55: 1324–1330.

28.

Polan

Brady

Kaufman

. Tissue segmentation of computed tomography images using a random forest algorithm: a feasibility study. Physics Med Biol 2016; 61: 6553–6569.

29.

Alnaser

Gong

Moeller

. Evaluation of open-source software for the lung segmentation. Curr Dir Biomed Eng 2016; 2: 515–518.

30.

Lo Giudice

Ronsivalle

Grippaudo

, et al. One step before 3D printing-evaluation of imaging software accuracy for 3-dimensional analysis of the mandible: a comparative study using a surface-to-surface matching technique. Materials (Basel) 2020; 13: 2798.

Three-dimensional analysis of reconstructed skulls using three different open-source software versus commercial software