Publications
2023
JH, Moon; JM, Lee; JA, Park; H, Suh; SJ, Lee
Reliability statistics every orthodontist should know Journal Article
In: Seminars in Orthodontics, 2023, ISSN: 1073-8746.
Abstract | Links | BibTeX | Tags: Bland-Altman plot, Confidence ellipse, Confidence interval, Reliability
@article{Suh2023,
title = {Reliability statistics every orthodontist should know},
author = {Moon JH and Lee JM and Park JA and Suh H and Lee SJ},
url = {https://doi.org/10.1053/j.sodo.2023.12.004},
doi = {10.1053/j.sodo.2023.12.004},
issn = {1073-8746},
year = {2023},
date = {2023-12-29},
journal = {Seminars in Orthodontics},
abstract = {It is essential to conduct a reliability examination even if the method was considered reliable in the past, as it may not be reliable in a new study conducted by different researchers using different materials. The current article highlights the importance of reliability examination in orthodontic studies and explains which assessment methods are more appropriate than others. Several fallacies in reporting and interpreting reliability are also discussed. In addition, the article presents examples of reliability examination for one-, two-, and three-dimensional data using graphic visualization in a tutorial format.},
keywords = {Bland-Altman plot, Confidence ellipse, Confidence interval, Reliability},
pubstate = {published},
tppubtype = {article}
}
It is essential to conduct a reliability examination even if the method was considered reliable in the past, as it may not be reliable in a new study conducted by different researchers using different materials. The current article highlights the importance of reliability examination in orthodontic studies and explains which assessment methods are more appropriate than others. Several fallacies in reporting and interpreting reliability are also discussed. In addition, the article presents examples of reliability examination for one-, two-, and three-dimensional data using graphic visualization in a tutorial format.
2022
Ghowsi, A; Hatcher, D; Suh, H; Wiled, D; Castro, W; Krueger, J; Park, J; Oh, H
Automated landmark identification on one cone beam computed tomography: Accuracy and reliability Journal Article
In: Angle Orthodontist, vol. 92, pp. 642-654, 2022.
Abstract | Links | BibTeX | Tags: 3D landmark identification, AAOF, accuracy, Automated, CBCT, Landmark error, Reliability
@article{Oh2022b,
title = {Automated landmark identification on one cone beam computed tomography: Accuracy and reliability},
author = {A Ghowsi and D Hatcher and H Suh and D Wiled and W Castro and J Krueger and J Park and H Oh},
url = {https://pubmed.ncbi.nlm.nih.gov/35653226/},
doi = {10.2319/122121-928.1},
year = {2022},
date = {2022-06-02},
urldate = {2022-06-02},
journal = {Angle Orthodontist},
volume = {92},
pages = {642-654},
abstract = {Objectives: To evaluate the accuracy and reliability of a fully automated landmark identification (ALI) system as a tool for automatic landmark location compared with human judges.
Materials and methods: A total of 100 cone-beam computed tomography (CBCT) images were collected. After the calibration procedure, two human judges identified 53 landmarks in the x, y, and z coordinate planes on CBCTs using Checkpoint Software (Stratovan Corporation, Davis, Calif). The ground truth was created by averaging landmark coordinates identified by two human judges for each landmark. To evaluate the accuracy of ALI, the mean absolute error (mm) at the x, y, and z coordinates and mean error distance (mm) between the human landmark identification and the ALI were determined, and a successful detection rate was calculated.
Results: Overall, the ALI system was as successful at landmarking as the human judges. The ALI's mean absolute error for all coordinates was 1.57 mm on average. Across all three coordinate planes, 94% of the landmarks had a mean absolute error of less than 3 mm. The mean error distance for all 53 landmarks was 3.19 ± 2.6 mm. When applied to 53 landmarks on 100 CBCTs, the ALI system showed a 75% success rate in detecting landmarks within a 4-mm error distance range.
Conclusions: Overall, ALI showed clinically acceptable mean error distances except for a few landmarks. The ALI was more precise than humans when identifying landmarks on the same image at different times. This study demonstrates the promise of ALI in aiding orthodontists with landmark identifications on CBCTs.},
keywords = {3D landmark identification, AAOF, accuracy, Automated, CBCT, Landmark error, Reliability},
pubstate = {published},
tppubtype = {article}
}
Objectives: To evaluate the accuracy and reliability of a fully automated landmark identification (ALI) system as a tool for automatic landmark location compared with human judges.
Materials and methods: A total of 100 cone-beam computed tomography (CBCT) images were collected. After the calibration procedure, two human judges identified 53 landmarks in the x, y, and z coordinate planes on CBCTs using Checkpoint Software (Stratovan Corporation, Davis, Calif). The ground truth was created by averaging landmark coordinates identified by two human judges for each landmark. To evaluate the accuracy of ALI, the mean absolute error (mm) at the x, y, and z coordinates and mean error distance (mm) between the human landmark identification and the ALI were determined, and a successful detection rate was calculated.
Results: Overall, the ALI system was as successful at landmarking as the human judges. The ALI's mean absolute error for all coordinates was 1.57 mm on average. Across all three coordinate planes, 94% of the landmarks had a mean absolute error of less than 3 mm. The mean error distance for all 53 landmarks was 3.19 ± 2.6 mm. When applied to 53 landmarks on 100 CBCTs, the ALI system showed a 75% success rate in detecting landmarks within a 4-mm error distance range.
Conclusions: Overall, ALI showed clinically acceptable mean error distances except for a few landmarks. The ALI was more precise than humans when identifying landmarks on the same image at different times. This study demonstrates the promise of ALI in aiding orthodontists with landmark identifications on CBCTs.
Materials and methods: A total of 100 cone-beam computed tomography (CBCT) images were collected. After the calibration procedure, two human judges identified 53 landmarks in the x, y, and z coordinate planes on CBCTs using Checkpoint Software (Stratovan Corporation, Davis, Calif). The ground truth was created by averaging landmark coordinates identified by two human judges for each landmark. To evaluate the accuracy of ALI, the mean absolute error (mm) at the x, y, and z coordinates and mean error distance (mm) between the human landmark identification and the ALI were determined, and a successful detection rate was calculated.
Results: Overall, the ALI system was as successful at landmarking as the human judges. The ALI's mean absolute error for all coordinates was 1.57 mm on average. Across all three coordinate planes, 94% of the landmarks had a mean absolute error of less than 3 mm. The mean error distance for all 53 landmarks was 3.19 ± 2.6 mm. When applied to 53 landmarks on 100 CBCTs, the ALI system showed a 75% success rate in detecting landmarks within a 4-mm error distance range.
Conclusions: Overall, ALI showed clinically acceptable mean error distances except for a few landmarks. The ALI was more precise than humans when identifying landmarks on the same image at different times. This study demonstrates the promise of ALI in aiding orthodontists with landmark identifications on CBCTs.
JH, Moon; JM, Lee; JA, Park; H, Suh; SJ, Lee
Reliability statistics every orthodontist should know Journal Article
In: Seminars in Orthodontics, 2023, ISSN: 1073-8746.
@article{Suh2023,
title = {Reliability statistics every orthodontist should know},
author = {Moon JH and Lee JM and Park JA and Suh H and Lee SJ},
url = {https://doi.org/10.1053/j.sodo.2023.12.004},
doi = {10.1053/j.sodo.2023.12.004},
issn = {1073-8746},
year = {2023},
date = {2023-12-29},
journal = {Seminars in Orthodontics},
abstract = {It is essential to conduct a reliability examination even if the method was considered reliable in the past, as it may not be reliable in a new study conducted by different researchers using different materials. The current article highlights the importance of reliability examination in orthodontic studies and explains which assessment methods are more appropriate than others. Several fallacies in reporting and interpreting reliability are also discussed. In addition, the article presents examples of reliability examination for one-, two-, and three-dimensional data using graphic visualization in a tutorial format.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
It is essential to conduct a reliability examination even if the method was considered reliable in the past, as it may not be reliable in a new study conducted by different researchers using different materials. The current article highlights the importance of reliability examination in orthodontic studies and explains which assessment methods are more appropriate than others. Several fallacies in reporting and interpreting reliability are also discussed. In addition, the article presents examples of reliability examination for one-, two-, and three-dimensional data using graphic visualization in a tutorial format.
Ghowsi, A; Hatcher, D; Suh, H; Wiled, D; Castro, W; Krueger, J; Park, J; Oh, H
Automated landmark identification on one cone beam computed tomography: Accuracy and reliability Journal Article
In: Angle Orthodontist, vol. 92, pp. 642-654, 2022.
@article{Oh2022b,
title = {Automated landmark identification on one cone beam computed tomography: Accuracy and reliability},
author = {A Ghowsi and D Hatcher and H Suh and D Wiled and W Castro and J Krueger and J Park and H Oh},
url = {https://pubmed.ncbi.nlm.nih.gov/35653226/},
doi = {10.2319/122121-928.1},
year = {2022},
date = {2022-06-02},
urldate = {2022-06-02},
journal = {Angle Orthodontist},
volume = {92},
pages = {642-654},
abstract = {Objectives: To evaluate the accuracy and reliability of a fully automated landmark identification (ALI) system as a tool for automatic landmark location compared with human judges.
Materials and methods: A total of 100 cone-beam computed tomography (CBCT) images were collected. After the calibration procedure, two human judges identified 53 landmarks in the x, y, and z coordinate planes on CBCTs using Checkpoint Software (Stratovan Corporation, Davis, Calif). The ground truth was created by averaging landmark coordinates identified by two human judges for each landmark. To evaluate the accuracy of ALI, the mean absolute error (mm) at the x, y, and z coordinates and mean error distance (mm) between the human landmark identification and the ALI were determined, and a successful detection rate was calculated.
Results: Overall, the ALI system was as successful at landmarking as the human judges. The ALI's mean absolute error for all coordinates was 1.57 mm on average. Across all three coordinate planes, 94% of the landmarks had a mean absolute error of less than 3 mm. The mean error distance for all 53 landmarks was 3.19 ± 2.6 mm. When applied to 53 landmarks on 100 CBCTs, the ALI system showed a 75% success rate in detecting landmarks within a 4-mm error distance range.
Conclusions: Overall, ALI showed clinically acceptable mean error distances except for a few landmarks. The ALI was more precise than humans when identifying landmarks on the same image at different times. This study demonstrates the promise of ALI in aiding orthodontists with landmark identifications on CBCTs.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Objectives: To evaluate the accuracy and reliability of a fully automated landmark identification (ALI) system as a tool for automatic landmark location compared with human judges.
Materials and methods: A total of 100 cone-beam computed tomography (CBCT) images were collected. After the calibration procedure, two human judges identified 53 landmarks in the x, y, and z coordinate planes on CBCTs using Checkpoint Software (Stratovan Corporation, Davis, Calif). The ground truth was created by averaging landmark coordinates identified by two human judges for each landmark. To evaluate the accuracy of ALI, the mean absolute error (mm) at the x, y, and z coordinates and mean error distance (mm) between the human landmark identification and the ALI were determined, and a successful detection rate was calculated.
Results: Overall, the ALI system was as successful at landmarking as the human judges. The ALI's mean absolute error for all coordinates was 1.57 mm on average. Across all three coordinate planes, 94% of the landmarks had a mean absolute error of less than 3 mm. The mean error distance for all 53 landmarks was 3.19 ± 2.6 mm. When applied to 53 landmarks on 100 CBCTs, the ALI system showed a 75% success rate in detecting landmarks within a 4-mm error distance range.
Conclusions: Overall, ALI showed clinically acceptable mean error distances except for a few landmarks. The ALI was more precise than humans when identifying landmarks on the same image at different times. This study demonstrates the promise of ALI in aiding orthodontists with landmark identifications on CBCTs.
Materials and methods: A total of 100 cone-beam computed tomography (CBCT) images were collected. After the calibration procedure, two human judges identified 53 landmarks in the x, y, and z coordinate planes on CBCTs using Checkpoint Software (Stratovan Corporation, Davis, Calif). The ground truth was created by averaging landmark coordinates identified by two human judges for each landmark. To evaluate the accuracy of ALI, the mean absolute error (mm) at the x, y, and z coordinates and mean error distance (mm) between the human landmark identification and the ALI were determined, and a successful detection rate was calculated.
Results: Overall, the ALI system was as successful at landmarking as the human judges. The ALI's mean absolute error for all coordinates was 1.57 mm on average. Across all three coordinate planes, 94% of the landmarks had a mean absolute error of less than 3 mm. The mean error distance for all 53 landmarks was 3.19 ± 2.6 mm. When applied to 53 landmarks on 100 CBCTs, the ALI system showed a 75% success rate in detecting landmarks within a 4-mm error distance range.
Conclusions: Overall, ALI showed clinically acceptable mean error distances except for a few landmarks. The ALI was more precise than humans when identifying landmarks on the same image at different times. This study demonstrates the promise of ALI in aiding orthodontists with landmark identifications on CBCTs.
2023 |
JH, Moon; JM, Lee; JA, Park; H, Suh; SJ, Lee: Reliability statistics every orthodontist should know. In: Seminars in Orthodontics, 2023, ISSN: 1073-8746. (Type: Journal Article | Abstract | Links | BibTeX | Tags: Bland-Altman plot, Confidence ellipse, Confidence interval, Reliability)@article{Suh2023,It is essential to conduct a reliability examination even if the method was considered reliable in the past, as it may not be reliable in a new study conducted by different researchers using different materials. The current article highlights the importance of reliability examination in orthodontic studies and explains which assessment methods are more appropriate than others. Several fallacies in reporting and interpreting reliability are also discussed. In addition, the article presents examples of reliability examination for one-, two-, and three-dimensional data using graphic visualization in a tutorial format. |
2022 |
Ghowsi, A; Hatcher, D; Suh, H; Wiled, D; Castro, W; Krueger, J; Park, J; Oh, H: Automated landmark identification on one cone beam computed tomography: Accuracy and reliability. In: Angle Orthodontist, vol. 92, pp. 642-654, 2022. (Type: Journal Article | Abstract | Links | BibTeX | Tags: 3D landmark identification, AAOF, accuracy, Automated, CBCT, Landmark error, Reliability)@article{Oh2022b,Objectives: To evaluate the accuracy and reliability of a fully automated landmark identification (ALI) system as a tool for automatic landmark location compared with human judges. Materials and methods: A total of 100 cone-beam computed tomography (CBCT) images were collected. After the calibration procedure, two human judges identified 53 landmarks in the x, y, and z coordinate planes on CBCTs using Checkpoint Software (Stratovan Corporation, Davis, Calif). The ground truth was created by averaging landmark coordinates identified by two human judges for each landmark. To evaluate the accuracy of ALI, the mean absolute error (mm) at the x, y, and z coordinates and mean error distance (mm) between the human landmark identification and the ALI were determined, and a successful detection rate was calculated. Results: Overall, the ALI system was as successful at landmarking as the human judges. The ALI's mean absolute error for all coordinates was 1.57 mm on average. Across all three coordinate planes, 94% of the landmarks had a mean absolute error of less than 3 mm. The mean error distance for all 53 landmarks was 3.19 ± 2.6 mm. When applied to 53 landmarks on 100 CBCTs, the ALI system showed a 75% success rate in detecting landmarks within a 4-mm error distance range. Conclusions: Overall, ALI showed clinically acceptable mean error distances except for a few landmarks. The ALI was more precise than humans when identifying landmarks on the same image at different times. This study demonstrates the promise of ALI in aiding orthodontists with landmark identifications on CBCTs. |