Introduction and validation of an automatic and low-cost three-dimensional facial imaging system: a comparison to direct anthropometry and Vectra H1

Introduction

Three-dimensional (3D) facial imaging has been widely accepted as a powerful tool for surgeons to obtain information on facial soft-tissue morphology. Anthropometric analysis based on 3D imaging outperforms direct measurement and two-dimensional (2D) photogrammetry in many ways (1-3). The capturing process only takes seconds, and the system generates a high-resolution and fully textured 3D facial image shortly after. The 3D image can be stored and exported in multiple formats. It supports linear, angular, surface area, and volumetric measurements.

There are many commercial 3D scanners in the market. They can be categorized as nonportable devices like 3dMDface system (3dMD Inc., Atlanta, GA, USA) and Vectra XT (Canfield Scientific, Inc., Parsippany, NJ, USA) and handheld devices like Vectra H1/H2 (Canfield Scientific, Inc.) and Artec Eva (Luxembourg). All of the above 3D scanners have been validated to be accurate and reliable in different clinical scenarios (4-6). However, commercial 3D scanners are constantly questioned for their cost-to-benefit ratio (7). Nonportable 3D scanners are too costly for private practitioners, and a handheld 3D camera can cost more than $15,000 in Asian countries. Though some companies provide a lower price option for fewer functions in the analysis module, the high price still impedes many clinicians from getting access to advanced technology.

To address this problem, attempts have been made to develop low-cost 3D scanners. One good solution is to develop a smartphone-based 3D facial scanning system. Mai et al. reviewed studies that developed and validated mobile device-compatible 3D scanners and concluded that their accuracy was clinically acceptable though not as good as professional scanning systems (8). Further study demonstrated a third-party iPhone application (ScandyPro, New Orleans, LA, USA) could be accurate to within 0.5 mm when compared to Vectra H1 (9). The software was free to download, and unlimited exportations within 1 week, 1 month, and 1 year were $1.99, $5.99, and $49.99, respectively. Other studies applied smartphone-based 3D imaging systems in scanning external ears (10,11) and palatal defect models (12).

In this study, we introduce an automatic, low-cost, and nonportable 3D facial imaging system and validate its accuracy and reliability for clinical use.

Methods

Study population

This study was designed to enroll 15 participants (8 males and 7 females). There were no specific criteria for enrollment. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The Institutional Review Board of Peking Union Medical College Hospital provided written ethical approval for this study (No. I-22PJ676). All volunteers signed the informed consent and agreed on their images and anthropometric data to be used for analysis.

3D imaging using the novel 3D facial imaging system and Vectra H1 handheld camera

Cristom-F is a novel nonportable 3D facial scanner based on phase-shifting fringe analysis. The scanner mainly consists of three camera lenses (A, B, and C), light-emitting diodes, and a power supply (Figure 1). When the white structured light is projected onto an object, the fringe patterns get distorted by the uneven surface. The deformed fringe patterns can be captured and demodulated to achieve a 3D digital reconstruction of an object. Camera lens A and C capture point cloud data to provide geometric information, and camera lens B collects color and texture. Scanning is performed three times from the frontal view, the left oblique view, and the right oblique view. The scanner is installed on a sliding track and driven by a motor (Figure 2). The capturing process is automated and takes less than 20 seconds (watch the video, Video S1). A complete set of 3D point cloud data is obtained at each scan and processed to create a 3D mesh (13). The entire scanning process eventually generates three 3D meshes (Figure 3A). A step called global adjustment is performed in the specially designed software (RECAM 3D imaging system) to fuse the three meshes into one according to the surface curvature (Figure 3B) (14). Finally, color and texture are integrated into the 3D mesh to complete a 3D image (Figure 3C). The whole process is automated and does not require any manipulation from the operator. When scanning a number of participants at a time, Cristom-F can outperform many other commercial devices for it saves much time and effort.

Figure 1 The appearance and the internal structure of the Cristom-F facial imaging system. (A) The scanner is about 30 cm long, 16 cm wide, and 10 cm tall. (B) The scanner consists of three camera lenses, a white structured light emitter, and a power supply. The manufacturer authorizes the use of this figure.

Figure 2 The novel 3D facial scanner is installed on a sliding track and driven by a motor to move automatically. The imaging process takes less than 20 seconds. Please also see the Video S1.

Figure 3 The image processing procedures. (A) Scanning is performed three times for each participant. Each scanning generates one 3D mesh. (B) Global adjustment is performed in the specially designed software to fuse the three meshes into one according to the surface curvature. (C) Color and texture are integrated into the 3D mesh to complete a 3D image.

Facial scanning was done in an outpatient consulting room. All participants wear a surgical cap to wrap their hair and fully exposed their faces with no make-up or jewelry. Calibration was necessary before scanning the first participant. All participants were asked to keep in neutral head position and try not to move during the scanning. One clinician supervised the scanning process without any manipulation.

All 15 participants were scanned again to obtain another set of 3D images using the handheld Vectra H1 3D camera in the same environment.

Direct measurement as the gold standard

A digital caliper was used to perform direct measurement as the gold standard. Nineteen landmarks, as defined by Farkas (15), were manually located onto each 3D image (Figure S1). To describe the horizontal (X-axis direction), vertical (Y-axis), and sagittal scale (Z-axis) of a 3D image, this study selected 18 anthropometric parameters defined by the landmarks (Table 1). There were seven horizontal parameters including ocular width, intercanthal width, and biocular width in the upper 1/3 face, midface width and nose width in the middle 1/3 face, and philtrum and mouth width in the lower 1/3 face. There were six vertical parameters including forehead height in the upper 1/3 face, nose height, nose bright length and morphological face height in the middle 1/3 face, and mouth height and lower face height in the lower 1/3 face. Finally, there were five facial depth parameters including upper facial third depth and orbito-tragial depth in the upper 1/3 face, maxillary depth in the middle 1/3 face, and labio-tragial depth and lower face depth in the lower 1/3 face. The measurement of the 18 parameters was manually taken using a digital caliper.

Evaluation of the 3D images generated by the novel imaging system

One author (Y.C.) measured the aforementioned parameters on 3D models generated by the Cristom-F 3D facial scanner in Geomagic Wrap 2017 (Geomagic, Inc., Research Triangle Park, NC, USA). The results were compared to the gold standard (results from the caliper measurement) to evaluate the validity of the novel device.

The same author took a second measurement of all the parameters 1 week apart. The first and the second measurement were compared to perform the intra-observer reproducibility test. Another author (Z.L.) measured the parameters on 3D models once again separately. The measurement of two different researchers was compared to perform the inter-observer reproducibility test.

Virtual models generated by the two 3D imaging systems were compared. All 3D images were trimmed appropriately to remove unnecessary parts. Two 3D images of the same participant were aligned using the best-fit algorithm in the software. Deviation analysis was performed to detect the average distance between the two 3D images.

Statistical analysis

The accuracy of the Cristom-F 3D facial scanner was evaluated by comparing the 3D photogrammetric results to the gold standard. Mean absolute difference (MAD) and relative error measurement (REM) were used to describe the difference. Further, The Pearson’s product moment correlation coefficient (PPMCC) was used to test for consistency.

For bias, Bland-Altman analysis was performed to test for the agreement of the measuring results given by the two 3D imaging systems. Before the analysis, Shapiro-Wilk test was performed to confirm the normal distribution, and Levene Statistic was performed to confirm the homogeneity of variance.

Intra- and inter-observer Reproducibility was evaluated by calculating the intraclass correlation coefficient (ICC). ICC higher than 0.75 suggested good reproducibility.

Statistical analysis was performed using SPSS 23.0 (IBM, New York, USA). The level of statistical significance was set at 0.0028 (0.05/18) after Bonferroni correction.

Results

The 3D photogrammetric results were highly correlated with the direct measurement results, as the Pearson’s product-moment correlation coefficient (r) was higher than 0.95 (P<0.001) for all 18 anthropometric parameters (Table 2). The MADs were less than 2 mm, and the REMs were no larger than 3.5%.

The bias between the 3D photogrammetric results and the direct measurement results was shown by the Bland-Altman plots. In the upper one-third face, for each parameter, 93% (14/15) of the differences were within the 95% limits of agreement margin, and the largest difference within the 95% limits of agreement (shown in black dot) was less than the clinical acceptance of 2 mm (Figure 4). In the middle one-third face, for each parameter, all the differences were within the 95% limits of agreement margin. For nose width, nose height, nose bridge length, morphological face height, and maxillary depth, the largest difference within the 95% limits of agreement (shown in black dot) was less than 2 mm (Figure 5). In the lower one-third face, for philtrum width, all the differences were within the 95% limits of agreement margin; for mouth width, mouth height, lower face height, and lower face depth, 93% (14/15) of the differences were within the 95% limits of agreement margin; for labio-tragial depth, 86.7% (13/15) of the differences were within the 95% limits of agreement margin. For each parameter, the largest difference within the 95% limits of agreement margin (shown in black dot) was less than 2 mm (Figure 6).

Figure 4 Bland-Altman analysis of parameters on the upper one-third face. SD, standard deviation; en, the point at the inner commissure of the eye fissure; ex, the point at the outer commissure of the eye fissure; tr, the midline point at the hairline; n, the most posterior midline point at the nasofrontal region; t, the notch at the superior margin of each tragus.

Figure 5 Bland-Altman analysis of parameters on the middle one-third face. SD, standard deviation; t, the notch at the superior margin of each tragus; al, the most lateral point on each alar contour; n, the most posterior midline point at the nasofrontal region; sn, the midline point at the junction of the nasal septum and the upper lip; prn, the most prominent point at the nasal tip; gn, the most inferior midline point on the chin.

Figure 6 Bland-Altman analysis of parameters on the lower one-third face. SD, standard deviation; cph, the point on each elevated margin of the philtrum; ch, the point located on each lateral oral commissure; ls, the midline point representing the mucocutaneous vermilion border of the upper lip; li, the midline point representing the mucocutaneous vermilion border of the lower lip; sn, the midline point at the junction of the nasal septum and the upper lip; prn, the most prominent point at the nasal tip; gn, the most inferior midline point on the chin; t, the notch at the superior margin of each tragus.

The ICC values of the intra- and inter-observer reliability tests were all higher than 0.8 (Table 3), indicating good reproducibility of measurement on 3D models obtained by the Cristom-F facial imaging system.

Further, deviation analysis was performed to detect the average distance between the 3D images obtained by the novel system and by the Vectra H1. The results were shown in the heat map (Figure 7). The mean distance of all 3D model pairs was 0.15±0.87 mm.

Figure 7 Heat map analysis results of 15 pairs of 3D virtual models. The average distance between the 3D virtual models was 0.15 mm. The average RMS was 0.71 mm. This image is published with the participants’ consent. D, distance; RMS, root mean square.

Discussion

This study introduced and validated an automatic, low-cost, and user-friendly 3D facial imaging system. After rigid comparison with the gold standard and Vectra H1 3D camera, the system was proved to be accurate and reliable for clinical use.

In recent years, 3D imaging has been intensively used in plastic and reconstructive surgery. It generates 3D facial models which enable surgeons to perform presurgical simulation and postoperative evaluation quantitatively (16-20). The main advantages of 3D imaging and 3D photogrammetry include quick capture, good precision, low requirements for the participants, and allowing for advanced morphological analysis. The common 3D facial imaging devices are already good enough to fulfill most clinical demands, but there is room for improvement.

Commercial 3D facial scanners are costly. In mainland China, one handheld Vectra H2 3D camera bundled with Vectra Analysis Module usually costs more than $15,000. Nonportable 3D cameras such as 3dMDface and Vectra XT are more expensive. The high price is the main cause many institutions do not use 3D imaging technology in their clinical practice. Few multi-centered studies use 3D imaging as the evaluation tool. Though there are inexpensive and easy-to-use 3D scanners such as Structure Sensor (Occipital, San Francisco, USA), most of them are not specially designed to do facial scanning and the precision of the 3D models does not meet clinical needs. The authors have invented a 3D facial imaging system running in an Apple IOS environment (21). It enables patients to do 3D facial scanning on their own so that a remote follow-up can become a reality. However, the system is not without limitations. Some patients have trouble with the scanning process, though in the presence of an instructional video, leading to unsuccessful scanning or low-quality 3D images. Additionally, it must work on iPhone X/iPad Pro (2^nd generation) or higher. Patients without the required devices are currently unable to do 3D scanning by themselves. In short, the system is a good alternative for remote follow-up but may not be the first choice in the consulting room.

Another drawback of the commonly used commercial portable 3D facial scanners is that the capturing process highly relies on human operation. The camera should be positioned properly to generate qualified images. This requires standardized training and experience. Beginners are prone to make mistakes like holding the camera at the wrong level or forgetting to turn on the ranging light.

Therefore, this study introduces an automatic, low-cost, and highly accurate 3D facial scanner for clinical use. The scanner underwent a rigid test for its accuracy. When comparing 3D photogrammetric results with the gold standard, the authors adopted 18 anthropometric parameters to evaluate the scanner’s accuracy. In the upper one-third face, the MADs of the six parameters were all less than 1.2 mm; in the middle one-third face, the MADs of the six parameters were less than 1.5 mm; in the lower one-third face, the MADs of all the six parameters were less than 1.4 mm. Bland-Altman analysis indicated good consistency between the two methods. For the 17 parameters except for midface width, the largest differences within the 95% limits of agreement margin were within the 2.0 mm clinical acceptance.

Midface width, defined as the horizontal distance between the two tragi, is the least accurate parameter. It may result from the slight distortion of the ears. 3D imaging is seldom used in the anthropometric analysis of the auricle, as the edge of the 3D virtual model is subjected to distortion.

Many portable 3D devices generate a 3D image by stitching three 2D photos from different angles. The automatic stitching algorithms work on human faces with no facial expression or obvious deformity, which limits portable 3D imaging devices from being applied in dynamic analysis and facial deformity evaluation. Currently, facial dynamic analysis based on 3D images can be achieved via four-dimensional imaging technology, which can be simply defined as “a time sequence of 3D facial images” (22). 3D imaging of faces with severe deformity is usually achieved by using nonportable devices like 3dMDface or Vectra XT. Both methods require expensive hardcore. Our novel device has the potential to provide a cost-effective solution by using multiple synchronized cameras positioned at different angles to do continuous scanning. Future studies will evaluate the availability of using the modified imaging system.

Conclusions

This study introduced and validated a novel, nonportable 3D facial imaging system. 3D virtual models generated by the novel device are highly accurate and reproducible. The novel system not only greatly reduces the cost of 3D imaging but enables an automatic capturing process without manual operation.

Acknowledgments

The authors would like to thank Mr. Qiang Zhang and Mr. Junya Zhang for their support.

Funding: This study was funded by National Key R&D Program of China (No. 2020YFE0201600), CAMS Innovation Fund for Medical Sciences (No. 2020-I2M-C&T-A-004), and CAMS Innovation Fund for Medical Sciences (No. CIFMS 2021-I2M-1-003).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-22-900/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The Institutional Review Board of Peking Union Medical College Hospital provided written ethical approval for this study (ethical approval No. I-22PJ676). All volunteers signed the informed consent and agreed on their images and anthropometric data to be used for analysis.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Weinberg SM, Scott NM, Neiswanger K, Brandon CA, Marazita ML. Digital three-dimensional photogrammetry: evaluation of anthropometric precision and accuracy using a Genex 3D camera system. Cleft Palate Craniofac J 2004;41:507-18. [Crossref] [PubMed]
2. Wong JY, Oh AK, Ohta E, Hunt AT, Rogers GF, Mulliken JB, Deutsch CK. Validity and reliability of craniofacial anthropometric measurement of 3D digital photogrammetric images. Cleft Palate Craniofac J 2008;45:232-9. [Crossref] [PubMed]
3. Jacobs RAPlastic Surgery Educational Foundation DATA Committee. Three-dimensional photography. Plast Reconstr Surg 2001;107:276-7. [Crossref] [PubMed]
4. Heike CL, Cunningham ML, Hing AV, Stuhaug E, Starr JR. Picture perfect? Reliability of craniofacial anthropometry using three-dimensional digital stereophotogrammetry. Plast Reconstr Surg 2009;124:1261-72. [Crossref] [PubMed]
5. Gibelli D, Pucciarelli V, Cappella A, Dolci C, Sforza C. Are Portable Stereophotogrammetric Devices Reliable in Facial Imaging? A Validation Study of VECTRA H1 Device. J Oral Maxillofac Surg 2018;76:1772-84. [Crossref] [PubMed]
6. Koban KC, Perko P, Etzel L, Li Z, Schenck TL, Giunta RE. Validation of two handheld devices against a non-portable three-dimensional surface scanner and assessment of potential use for intraoperative facial imaging. J Plast Reconstr Aesthet Surg 2020;73:141-8. [Crossref] [PubMed]
7. Weissler JM, Stern CS, Schreiber JE, Amirlak B, Tepper OM. The Evolution of Photography and Three-Dimensional Imaging in Plastic Surgery. Plast Reconstr Surg 2017;139:761-9. [Crossref] [PubMed]
8. Mai HN, Lee DH. Accuracy of Mobile Device-Compatible 3D Scanners for Facial Digitization: Systematic Review and Meta-Analysis. J Med Internet Res 2020;22:e22228. [Crossref] [PubMed]
9. Rudy HL, Wake N, Yee J, Garfein ES, Tepper OM. Three-Dimensional Facial Scanning at the Fingertips of Patients and Surgeons: Accuracy and Precision Testing of iPhone X Three-Dimensional Scanner. Plast Reconstr Surg 2020;146:1407-17. [Crossref] [PubMed]
10. Farook TH, Rashid F, Jamayet NB, Abdullah JY, Dudley J, Khursheed Alam M. A virtual analysis of the precision and accuracy of 3-dimensional ear casts generated from smartphone camera images. J Prosthet Dent 2022;128:830-6. [Crossref] [PubMed]
11. Nightingale RC, Ross MT, Cruz RLJ, Allenby MC, Powell SK, Woodruff MA. Frugal 3D scanning using smartphones provides an accessible framework for capturing the external ear. J Plast Reconstr Aesthet Surg 2021;74:3066-72. [Crossref] [PubMed]
12. Farook TH, Jamayet NB, Asif JA, Din AS, Mahyuddin MN, Alam MK. Development and virtual validation of a novel digital workflow to rehabilitate palatal defects by using smartphone-integrated stereophotogrammetry (SPINS). Sci Rep 2021;11:8469. [Crossref] [PubMed]
13. Point Cloud Library. Available online: https://github.com/PointCloudLibrary/pcl
14. Holz D, Ichim AE, Tombari F, Rusu RB, Behnke S. Registration with the Point Cloud Library: A Modular Framework for Aligning in 3-D. IEEE Robotics & Automation Magazine 2015;22:110-24. [Crossref]
15. Farkas LG. Anthropometry of the Head and Face. New York, NY, USA: Raven Pr, 1994.
16. Day KM, Kelley PK, Harshbarger RJ, Dorafshar AH, Kumar AR, Steinbacher DM, Patel P, Combs PD, Levine JP. Advanced Three-Dimensional Technologies in Craniofacial Reconstruction. Plast Reconstr Surg 2021;148:94e-108e. [Crossref] [PubMed]
17. Miller TR. Long-term 3-Dimensional Volume Assessment After Fat Repositioning Lower Blepharoplasty. JAMA Facial Plast Surg 2016;18:108-13. [Crossref] [PubMed]
18. De Greve G, Malka R, Barnett E, Robotti E, Haug M, Hamilton G, Lekakis G, Hellings PW. Three-Dimensional Technology in Rhinoplasty. Facial Plast Surg 2022;38:483-7. [Crossref] [PubMed]
19. Jacono AA, Malone MH, Talei B. Three-Dimensional Analysis of Long-Term Midface Volume Change After Vertical Vector Deep-Plane Rhytidectomy. Aesthet Surg J 2015;35:491-503. [Crossref] [PubMed]
20. Jain M, Savage NE, Spiteri K, Snell BJ A. 3-Dimensional Quantitative Analysis of Volume Loss Following Submental Cryolipolysis. Aesthet Surg J 2020;40:123-32. [Crossref] [PubMed]
21. Chong Y, Liu X, Shi M, Huang J, Yu N, Long X. Three-dimensional facial scanner in the hands of patients: validation of a novel application on iPad/iPhone for three-dimensional imaging. Ann Transl Med 2021;9:1115. [Crossref] [PubMed]
22. Shujaat S, Khambay BS, Ju X, Devine JC, McMahon JD, Wales C, Ayoub AF. The clinical application of three-dimensional motion capture (4D): a novel approach to quantify the dynamics of facial animations. Int J Oral Maxillofac Surg 2014;43:907-16. [Crossref] [PubMed]