IDA Challenge Overview

ICME26 Grand Challenge IDA*: Cross-Scenario Defect Detection and Fine-Grained Severity Grading for High-Precision Manufacturing

*IDA: Industrial Defect Analysis

Dataset Evaluation Submission Leaderboard

Overview

The quality control of Printed Circuit Board Assemblies (PCBA) and high-density electronic components represents a cornerstone of modern manufacturing. As electronic devices become increasingly miniaturized, the automated inspection of solder joints, connector pins, and substrate surfaces faces unprecedented challenges. However, a critical bottleneck restricts the massive deployment of current Industrial AI: the lack of Cross-Scenario Generalization Ability and Fine-Grained Severity Grading Capability.

In real-world workflows, production lines and sensing environments frequently change. Conventional models often suffer from catastrophic performance degradation when transferred from one production line to another unseen domain. When transferred to another unseen scenario, existing benchmarks predominantly focus on binary decisions, lacking fine-grained assessment ability of the severity, which is crucial for better optimizing industrial pipelines.

IDA 2026 establishes two distinct competition tracks:

Track 1: Cross-Scenario Defect Detection

This track focuses on the adaptability of vision models. Participants must develop models that can accurately detect, localize, and classify defects across different, unseen production lines and varying imaging conditions.

  • Goal: Maximize detection, classification, and localization performance in unseen domains while minimizing false alarms.
  • Key Challenge: Domain adaptation and Zero-Shot/Few-Shot generalization.

Track 2: Fine-Grained Severity Grading

This track focuses on the analytical depth of vision models. Moving beyond simple detection, participants must analyze detected defects to identify their specific mechanism and assign an industry-standard severity level (Acceptable, Marginal NG (Not Good), NG, or Gross NG).

  • Goal: Accurately categorize defect types and grade severity based on visual evidence.
  • Key Challenge: Fine-Grained Classification and Ordinal Regression.

The goal is to advance AI systems from mere “defect finders” (Track 1) to intelligent “risk assessors” (Track 2) for electronics assembly.

Dataset

Dataset details will be released soon.

Evaluation Criteria

We define separate scoring metrics for each track. Crucially, both tracks require strong capabilities in Defect Localization and Classification, but with different weighting priorities.

Track 1: Cross-Scenario Defect Detection

The ranking for Track 1 is determined by the Detection Score \(S_{Track1}\). This score emphasizes the model's ability to find and recognize defects in new domains while controlling false alarms.

$$ S_{Track1} = 0.3 \times S_{loc} + 0.3 \times S_{cls} + 0.4 \times S_{screen} $$

  • Localization (\(S_{loc}\)): Mean Intersection over Union (mIoU).
  • Classification (\(S_{cls}\)): Macro-F1 Score for defect type identification.
  • Screening Efficiency (\(S_{screen}\)): composite of Image-Level Recall and Specificity to penalize false alarms in unseen domains.

Track 2: Fine-Grained Severity Grading

The ranking for Track 2 is determined by the Risk Assessment Score \(S_{Track2}\). While severity grading is the primary goal, accurate localization and classification are prerequisites for valid reasoning.

$$ S_{Track2} = 0.2 \times S_{loc} + 0.2 \times S_{cls} + 0.6 \times S_{grade} $$

  • Severity Grading (\(S_{grade}\)): Quadratic Weighted Kappa (QWK).
  • Localization (\(S_{loc}\)) & Classification (\(S_{cls}\)): ensures the severity grade is assigned to the correct defect instance and type.

Submission Format

Participants can choose to compete in one or both tracks.

  • Track 1 Submission: JSON with image_id, bbox, defect_type, confidence.
  • Track 2 Submission: JSON with image_id, bbox, defect_type, severity_level.

Timeline

The detailed timeline will be announced soon.

Awards

A total prize pool of $9,600 USD will be awarded across two tracks.

Champion

$3,000

Runner-up

$1,000

Third Place

$800

Organizers

  • Wei Sun, wsun@cee.ecnu.edu.cn
  • Weixia Zhang, zwx8981@sjtu.edu.cn
  • Linhan Cao, caolinhan@sjtu.edu.cn
  • Xiongkuo Min, minxiongkuo@sjtu.edu.cn
  • Xiaoping Zhang, xpzhang@ieee.org
  • Patrick Le Callet, patrick.lecallet@univ-nantes.fr
  • Guangtao Zhai, zhaiguangtao@sjtu.edu.cn