3D Surface Mesh Estimation Challenge
Surface estimation is crucial in industrial ultrasound image analysis as it separates the scanned object's surface from background noise and artifacts such as spurious refractions and other reverberations. Accurate surface estimation not only provides better visualization, but also ensures increased accuracy in measurements crucial for NDT tasks such as defect/flaw identification, detection, and characterization. This training dataset consists of 89 volumetric industrial ultrasound images; each is a separate scan containing a piece (or connected pieces) of steel pipe, with or without an object inside the pipe, and with or without debris/dirt debris/dirt at the bottom of the pipe. Corresponding surface meshes (5) are provided as reference estimations, which were manually cleaned by an experienced data analyst.
For questions regarding this dataset challenge, please email Mengliu Zhao at cvpr2023.dl.ultrasound@gmail.com
Competition Rules:
- All participants should register for this competition with their real names, affiliations (including department, full name of university/institute/company, and country), websites, titles, e-mails, and experience with ultrasound images. Incomplete and redundant registrations will be removed without notice.
- All participants must submit a complete solution to this competition with a public GitHub repository. A complete solution includes a Docker container (tar file) and a qualified technical paper with clear illustrations and citations.
- All participants should agree that the submitted Docker containers and papers can be publicly available to the community on the competition website, and organizers can use the information provided by the participants for final analysis, including scores, predicted labels, and papers.
- Participants are not allowed to register multiple teams and accounts (only listed names in the signed document will be considered). Participants from the same research group are also not allowed to register multiple teams. Organizers retain the right to disqualify such participants.
- Participants agree to the Terms of Access attached to the downloaded dataset. Redistribution or transferring of competition data or data links is not allowed during or after the competition. Participants should use the data only for this competition and publication in CVPR 2023. If the participant wants to access, redistribute, or transfer the Dataset for any other purposes, he or she should contact the DL-UIA committee for permission.
- Participants should develop deep learning-based methods only using given annotations. Any extra manual interventions (e.g., manually annotating the unlabeled images) are not allowed. For a fair comparison, participants should post the used external data or pre-trained models (freely available) links in the submitted technical report.
Evaluation
Participants must submit the URL of their GitHub repository through a designated Google Form.
The evaluation process for this challenge will be rigorous and comprehensive. Our team will run the participant's code and conduct a thorough analysis of its performance using two metrics. The Chamfer distance will be the primary ranking metric, which is a widely used method for measuring the similarity between two sets of points. Additionally, we will consider the direct Hausdorff metric as a secondary ranking metric, which measures the maximum distance between two sets of points.
See the demo notebook for more details of the metrics
Prize
1st Place: 5000 CAD
2nd Place: 3000 CAD
3ed Place: 1000 CAD
Timeline
- Competition Start: January 17, 2023
- Competition challenge closes: May 30, 2023
- Competition winner announcement: On the day of the workshop
