pcb-fault-detection

Project Overview

This repository hosts the core machine learning components for an AI-driven optical Printed Circuit Board (PCB) inspection system. Its primary purpose is to develop a computer vision-based inspection tool that significantly reduces inspection time and minimizes costly field returns, a capability especially critical in high-volume EMS (Electronics Manufacturing Services) environments. It includes all necessary code for data acquisition, preprocessing, model training, and evaluation for PCB fault and component detection, as well as artifacts from successful training runs.

This project serves as a companion to the PCB Fault Detection UI, which provides the user interface and is available at pcb_fault_detection_ui.

Demo and Application

The ./test_images directory contains the images used in the demo. You can view the full demonstration on YouTube here: YouTube Demo Link. For detailed instructions on setting up and using the desktop application, please refer to the pcb_fault_detection_ui repository.

AI Project Lifecycle & Methodology

For a detailed explanation of the methodology and steps followed, please refer to the subsequent sections. Much of this content has been adapted from the Markdown cells within the Jupyter Notebooks used for data pre-processing and model training.

Step 1: Problem Scoping

AI-Driven Optical PCB Inspection

• Category: General problem
• Description: Manual PCB inspections are time-consuming and error-prone. This project aims to develop a computer vision-based inspection tool to reduce inspection time and avoid costly field returns—especially critical in high-volume EMS environments.

Step 2: Data Acquisition

We have gathered various types of data from the following sources:

These data sources, detailing defects on copper tracks, were used to train the CopperTrack model.

• DsPCBSD+
• MIXED PCB DEFECT DETECTION
• PCB_DATASET

These datasets contain annotated images of PCBs highlighting actual components. This data can be used to identify missing components by comparing an image with the output of a known good PCB.

• pcb-oriented-detection
• WACV pcb-component-detection
• FICS-PCB
  • Also at TRUST-HUB
  • And Roboflow
• PCB-Vision (Zenodo Download)
• CompDetect Dataset

This dataset contains annotated images of PCBs with missing soldered components. It was used in the final YouTube demo but not for model training.

• PCB defects.v2i.yolov11

Step 3: Data Preprocessing & Exploration

For the Copper Track Defect Detection Model

DsPCBSD+ Dataset

This is the most comprehensive of all datasets, as it contains the most classes. Therefore, its class names were used for the entire aggregated dataset.

Due to inconsistent class naming across datasets, the following mappings (from the DsPCBSD+ Dataset) were applied to standardize labels:

DSCPBSD_MAP = {
		0: SHORT,
		1: SPUR,
		2: SPURIOUS_COPPER,
		3: OPEN,
		4: MOUSE_BITE,
		5: HOLE_BREAKOUT,
		6: SCRATCH,
		7: CONDUCTOR_FOREIGN_OBJECT,
		8: BASE_MATERIAL_FOREIGN_OBJECT,
}

MIXED PCB DEFECT DATASET

From: Mendeley Data

It contains similar data but is labeled differently, thus necessitating label mapping.

MIXED_PCB_DEFECT_DATASET_MAPPING = {
		0: MISSING_HOLE,
		1: MOUSE_BITE,
		2: OPEN,
		3: SHORT,
		4: SPUR,
		5: SPURIOUS_COPPER,
}

PCB_DATASET

From Kaggle by The Open Lab on Human Robot Interaction of Peking University.

It is in Pascal VOC format and thus required conversion to YOLO format.

PCB_DATASET_MAPPING = {
		"missing_hole": MISSING_HOLE,
		"mouse_bite": MOUSE_BITE,
		"spurious_copper": SPURIOUS_COPPER,
		"short": SHORT,
		"spur": SPUR,
		"open_circuit": OPEN,
}

Sample Images

For the Component Detection model

Class names to integer mapping:

@enum.verify(enum.UNIQUE, enum.CONTINUOUS)
class Component(enum.IntEnum):
		battery = 0
		button = 1
		buzzer = 2
		capacitor = 3
		clock = 4
		connector = 5
		diode = 6
		display = 7
		fuse = 8
		heatsink = 9
		ic = 10
		inductor = 11
		led = 12
		pads = 13
		pins = 14
		potentiometer = 15
		relay = 16
		resistor = 17
		switch = 18
		transducer = 19
		transformer = 20
		transistor = 21

CompDetect Dataset

From: Roboflow Universe.

This dataset is useful for component detection; however, Roboflow does not allow direct download of the original, higher-quality images. For example, if you go to Roboflow CompDetect v23 and attempt to "Download Dataset" in formats like YoloV11, the image quality is significantly degraded. Additionally, it contains many augmented images that are not needed, as we perform augmentation ourselves during training.

(This dataset appears to be copied from another source, but this is not specified on the dataset page, and I could not find the original source during my data gathering.)

Consequently, the steps taken to download this dataset are:

• Go to Roboflow CompDetect
• Select Fork Project
• Fill in the API_KEY and PROJECT_ID in my_secrets.py
• Run roboflow_download.py, which saves all images to ./temp_images
• Run roboflow_save_labels.py, which saves all data to ./temp_data. Note that this data is in Roboflow JSON API response format and requires manual conversion to YOLO format.

FICS-PCB

Paper: FICS-PCB: A Multi-Modal Image Dataset

Download from: TRUST-HUB

The provided link leads to a ~79GB dataset, with images stored in individual ZIP files and annotations presented in a challenging-to-parse mix of CSV (for class labels) and JSON (for bounding box positions).

However, during my data gathering, I also discovered the dataset mirrored at Roboflow Universe.

Therefore, we employed the same methodology as with the CompDetect dataset, placing both datasets' images and labels into ./temp_images and ./temp_data respectively, and then parsing them simultaneously into the YOLO format.

Additionally, the CompDetect dataset includes some, but not all, images from the WACV dataset. Therefore, these extra images required removal. Fortunately, their filenames matched the originals, simplifying the filtering process.

PCB-Vision

Paper at arXiv

Download from Zenodo (11GB).

This dataset presented unique challenges, necessitating significant preprocessing. As a result, we dedicated a separate Jupyter Notebook, ./pcb-components-detection-datasets/preprocess-pcb-vision.ipynb, to convert the dataset into a YOLO-compatible format, which is then stored at ./pcb-components-detection-datasets/PCBVisionYolo.

The original dataset specified classes using a grayscale image mask, where each pixel in the mask was assigned a value indicating the presence of a specific component.

• 0 = Nothing
• 1 = IC (represented below as red)
• 2 = Capacitor (represented below as green)
• 3 = Connectors (e.g., DIMM, GPU PCIe, excluding berg strips or screw terminals) (represented below as blue)

Unfortunately, this format is not compatible with YOLO, necessitating conversion.

The preprocess-pcb-vision.ipynb file performs the following pre-processing steps:

• Straighten the images: Some PCBs are tilted, which is problematic as it can lead to issues with YOLO (e.g., loose bounding boxes), even if it might be acceptable for training. (Furthermore, data augmentation is performed during training.). For this, we use PCB mask files from the dataset which specify which pixels of the image have the PCB. This is implemented by the get_pcb_rotation function, which performs the following steps:
  • Takes in a PCB mask as input.
  • Smooths out any irregularities.
  • Finds the largest contour (i.e., the most prominent shape, regardless of its irregularity) in the image to isolate the PCB.
  • Finds the smallest rotated bounding box to determine the bounds of the PCB's contour.
  • Then, rotate_copy rotates the image (and component mask) by the angle of this rotated bounding box to straighten the PCB.
• From the component mask, for each component type, all separate component contours are found, and then all bounding boxes that fit those contours are derived. These are then saved as labels for YOLO.
• The images are often very dark; therefore, basic color correction is performed by clipping the values in all color channels to be less than their 97.5th percentile.

Final result for this dataset:

WACV: pcb-component-detection

From Chia-Wen Kuo's Research Page.

This dataset is in the Pascal VOC format, requiring conversion to YOLO format.

pcb-oriented-detection

From: Kaggle.

It contains oriented bounding boxes, which necessitated conversion to regular bounding boxes.

The file performs the following preprocessing steps:

• Finds the tightest regular bounding boxes that fit the oriented bounding boxes and saves them.
• Here, the images are also dark; consequently, the same basic color correction as before is performed: clipping the values in all color channels to be less than their 97.5th percentile.

It contains many more classes, often redundant, necessitating a comprehensive class mapping.

Tiling

Many datasets contain very tiny bounding boxes for small SMD components. To facilitate better model training by making these components comparatively larger, these images were split into larger tiles, and their bounding box annotations were adjusted accordingly.

This process is analogous to Slicing Aided Hyper Inference (SAHI); however, SAHI is exclusively used for inference during deployment, not training. Therefore, this process is referred to as Tiling to differentiate it from Slicing, as Tiling is performed during training. While this concept is likely not new, limited existing code implementations were found online.

It works as follows:

• It reads an image and its corresponding bounding boxes.
• Identifies the smallest dimensions among all bounding boxes within an image.
• Determines the necessary tile size to ensure that the smallest bounding box occupies at least 3% of the tile's area.
• Crops the image into tiles, ensuring at least 10% overlap between them and redistributing the overlap to minimize clipping at the edges.

Sample images from consolidated & cropped dataset

Step 4: Modeling

Here, we trained two YOLOv11 nano models, one for each dataset/task, using the Ultralytics library. After iterative training, evaluation, and fine-tuning of model parameters and datasets, the final models were developed.

Step 5: Evaluation

Track Defect Detection Model

Here, each model was evaluated in four distinct ways:

• on the entire dataset
• on the entire dataset, treated as a single class (i.e., by combining all problem types into one).
• on the large PCB images dataset
• on the large PCB images dataset, with a single class

Evaluation Metrics used

• box-p (Precision): The proportion of correct positive predictions out of all positive predictions made by the model.
• box-r (Recall): The proportion of actual positive instances that were correctly identified by the model.
• box-f1 (F1-score): The harmonic mean of precision and recall, balancing both metrics.
• box-map (Mean Average Precision - IoU threshold range 0.5 to 0.95): The average of Average Precisions calculated across multiple IoU thresholds (0.5 to 0.95) and all classes.
• box-map50 (Mean Average Precision - IoU threshold 0.5): The Mean Average Precision specifically calculated when a detected box is considered correct if its IoU with a ground truth box is $\ge 0.5$.
• box-map75 (Mean Average Precision - IoU threshold 0.75): The Mean Average Precision specifically calculated when a detected box is considered correct if its IoU with a ground truth box is $\ge 0.75$.

The best model has the following evaluation output:

		box-p	box-r	box-f1	box-map	box-map50	box-map75
	class_name
Full	Short	0.75861	0.63978	0.69415	0.34406	0.67146	0.29448
	Spur	0.76153	0.53222	0.62655	0.34406	0.67146	0.29448
	Spurious copper	0.61793	0.64026	0.62890	0.34406	0.67146	0.29448
	Open	0.70308	0.70270	0.70289	0.34406	0.67146	0.29448
	Mouse bite	0.69113	0.55098	0.61315	0.34406	0.67146	0.29448
	Hole breakout	0.81272	0.96760	0.88342	0.34406	0.67146	0.29448
	Conductor scratch	0.57191	0.55369	0.56265	0.34406	0.67146	0.29448
	Conductor foreign object	0.58090	0.48402	0.52805	0.34406	0.67146	0.29448
	Base material foreign object	0.67865	0.75600	0.71524	0.34406	0.67146	0.29448
	Missing hole	0.73747	0.41842	0.53391	0.34406	0.67146	0.29448
Full (Single Class)	Combined	0.77179	0.65834	0.71057	0.37722	0.74098	0.32281
Large PCBs	Short	0.43373	0.35714	0.39173	0.17780	0.39143	0.11074
	Spur	0.51660	0.23636	0.32433	0.17780	0.39143	0.11074
	Spurious copper	0.47770	0.42857	0.45181	0.17780	0.39143	0.11074
	Open	0.66475	0.43056	0.52262	0.17780	0.39143	0.11074
	Mouse bite	0.50888	0.40000	0.44792	0.17780	0.39143	0.11074
	Missing hole	0.59376	0.47629	0.52858	0.17780	0.39143	0.11074
Large PCBs (Single Class)	Combined	0.59917	0.42411	0.49666	0.19223	0.43753	0.11418

The best model demonstrates strong performance on the full dataset, with only a slight decrease in performance on the large PCB dataset (as indicated by its F1-vs-Confidence curve).

Confusion Matrix

Normalized Confusion Matrix

Precision-vs-Confidence Curve

Recall-vs-Confidence Curve

F1-vs-Confidence Curve

Precision-vs-Recall Curve

Results

The model performs commendably. Its F1-vs-Confidence curve on the general dataset is notably wide and high, indicating robust performance across a broad range of confidence values, or probability thresholds. Its confusion matrix shows high values, though it occasionally produces false positives, particularly evident in the "background" row.

The optimal probability threshold for this model is 0.25 (25%), which will be used for final deployment.

Thus, this is designated as the final model for deployment: Model CopperTrack

Component Detection Model

Here, we evaluate the model in two ways:

• on the entire dataset
• on the entire dataset, with a single class (by combining all component types as the same)

The best results are as follows:

		box-p	box-r	box-f1	box-map	box-map50	box-map75
	class_name
Full	battery	0.48056	0.71429	0.57457	0.38478	0.61576	0.41728
	button	0.88862	0.36275	0.51519	0.38478	0.61576	0.41728
	buzzer	0.91834	0.84615	0.88077	0.38478	0.61576	0.41728
	capacitor	0.85284	0.62589	0.72195	0.38478	0.61576	0.41728
	clock	0.60928	0.41667	0.49489	0.38478	0.61576	0.41728
	connector	0.60877	0.48309	0.53870	0.38478	0.61576	0.41728
	diode	0.68364	0.55175	0.61066	0.38478	0.61576	0.41728
	display	0.44909	0.70000	0.54715	0.38478	0.61576	0.41728
	fuse	0.59291	0.80000	0.68106	0.38478	0.61576	0.41728
	ic	0.71568	0.83037	0.76877	0.38478	0.61576	0.41728
	inductor	0.47205	0.39218	0.42842	0.38478	0.61576	0.41728
	led	0.68681	0.49583	0.57590	0.38478	0.61576	0.41728
	pads	0.31419	0.05019	0.08656	0.38478	0.61576	0.41728
	pins	0.21008	0.31959	0.25352	0.38478	0.61576	0.41728
	potentiometer	0.71059	0.40948	0.51956	0.38478	0.61576	0.41728
	relay	0.79690	1.00000	0.88697	0.38478	0.61576	0.41728
	resistor	0.86273	0.65675	0.74578	0.38478	0.61576	0.41728
	switch	0.87072	0.68504	0.76680	0.38478	0.61576	0.41728
	transistor	0.75751	0.66376	0.70754	0.38478	0.61576	0.41728
Full (Single Class)	Combined	0.84495	0.66326	0.74316	0.42653	0.73219	0.44990

Confusion Matrix

Normalized Confusion Matrix

Precision-vs-Confidence Curve

Recall-vs-Confidence Curve

F1-vs-Confidence Curve

Precision-vs-Recall Curve

Results

Similarly, this model also demonstrates strong performance. Its F1-vs-Confidence curve on the general dataset is very wide and high, indicating robust performance across a wide range of confidence values, or probability thresholds. Its confusion matrix similarly shows high values, though it occasionally produces false positives, as observed in the "background" row.

The optimal probability threshold for this model is 0.322 (approximately 32%). We will round this down to 30% for final deployment.

Step 6: Model Deployment

Export the models to ONNX for deployment.

See pcb_fault_detection_ui for the final deployed desktop application.

Also, view the demo on YouTube here: YouTube Demo Link. The ./test_images directory contains the images used in the demo.