Image-Segmentation-with-Clustering

Introduction

A typical digital image processing system can be divided into three phases: image pre-processing, image analysis, and decision-making phase [1]. The pre-processing phase includes image enhancement, noise filtering, and image compression. Image segmentation belongs to the analysis phase before the decision-making phase as shown in Figure 1. In general, image segmentation is the initial step before performing a higher-level operation such as object detection, tracking, and classification.

Figure 1. Digital image processing steps

The supervised approach (classification) classifies the data based on the model obtained from the training of labeled data. The supervised approach highly depends on the labeled training data and often require large number of training data set and high computational power during the model training phase. On the other hand, the unsupervised approach (clustering) groups the data without any the prior data training. This approach is relatively simpler to implement with lower computational cost. However, the accuracy of this technique highly depends on the initialization and the number of clusters chosen.

Examples: K-Means clustering, Fuzzy C-Means clustering, Mean shift clustering, Support Vector Machine (SVM),

Clustering VS Classification

The advantages and disadvantages of each segmentation techniques are summarized in table below:

There are many clustering techniques that can be implemented for image segmentation. In this work, the K-Means clustering, the Fuzzy C-Means (FCM) clustering, and Mean Shift clustering have been chosen as candidate algorithms. The advantages and disadvantages for each techniques can be found in table below:

Comparison between different clustering techniques

Image Preprocessing

Image preprocessing has a significant impact on the clustering result especially when dealing with a highly detailed image and an image with complex structure. There are many image preprocessing techniques. Here, color space transforming, noise filtering, contrast stretching, image filling, and features adding have been performed.

Color space transforming

Color space transforming has a direct impact on the image structure. Different color spaces can emphasize different features in an image, making it easier to perform image segmentation based on colors. In this work, the HSV (Hue, Saturation, Value) color space has been selected as it help to better separate intensity values related to brightness of the image. In addition , the CIELAB (Luminance, a chrominance, b chrominance) color space has also been chosen to separate and group color together. It is not recommended to rely on one color space as this important features may be removed from some image. This issue can be fixed by features adding. The effect of transforming color space is shown in Figure 4. Color space transforming can be done in MATLAB using function rgb2hsv() and rgb2lab()

Figure 4: Image in different color space

Noise filtering

Noise in the image can be reduced by applying an image filter before the image processing. There are various filtering methods that can be applied to an image such as Averaging Filter, Gaussian lowpass filter, Circular Averaging Filter, and Prewitt filter. After testing various filter, the Circular Averaging Filter (Disk Filter) has been chosen as it significantly reduces the image noise without removing the important features as shown in Figure 5. The Circular Averaging Filter can be done using MATLAB function fspecial() with imfilter() function

Figure 5: Image with different noise filtering

Contrast stretching

Enhancing image’s contrast can significantly bring out the image features such as edges and color intensity. The goal of contrast stretching is to increase the dynamic range of an image so that it spans the full range of possible intensity values, from minimum to maximum. This technique works very well for image in the HSV color space as shown in Figure 6.
Contrast stretching can be performed in MATLAB using function imadjust()

Figure 6: The effect of contrast stretching on image in LAB and HSV color space

Image Filling

Image filling is a technique used to fill holes or missing regions in an image. This process is to increase the completeness of each segment of the image. The effect of image filling can be seen in Figure 7. Image filling can be done in MATLAB using function imfill()

Figure 7: The effect of image filling on image in LAB and HSV color space

Features adding

In this work, all the aforementioned preprocessing methods are combined before feeding into the clustering algorithm as shown in Figure 8. An image has been transformed into HSV and LAB color space and undergoes noise filtering, contrast stretching and image filling. However, features adding should be used carefully, as adding more features does not necessarily give better result.

Figure 8: Overall image preprocessing

The aforementioned methods have been custom coded in preproc() function:

Image-Segmentation-with-Clustering/Main_Clustering.m

Lines 322 to 376 in 7786566

	%% Preprocessing function
	function test0 = preproc(test0, filterOpt, colorSpace)
	% ==================================================================
	% Gassian Option
	sigma = 2; % Define the standard deviation for the Gaussian filter
	kernelSize = 5; % Define the size of the Gaussian filter kernel
	% ==================================================================
	% Filtering
	switch filterOpt
	case 0
	case 1
	% Gaussian filter
	h = fspecial('gaussian', kernelSize, sigma);
	test0 = imfilter(test0, h);
	case 2
	% Disk filter
	h = fspecial('disk', 5);
	test0 = imfilter(test0, h);
	case 3
	test0 = imgaussfilt(test0, 5);
	case 4
	h = fspecial('sobel');
	test0 = imfilter(test0, h);
	case 5
	h = fspecial('prewitt');
	test0 = imfilter(test0, h);
	case 6
	h = fspecial('disk', 5);
	test0 = imfilter(test0, h);
	h2 = fspecial('sobel');
	test0 = imfilter(test0, h);
	end
	% Convert to other color space
	switch colorSpace
	case 'none'
	case 'hsv'
	test0 = rgb2hsv(test0);
	case 'lab'
	test0 = rgb2lab(test0);
	case 'gray'
	test0 = rgb2gray(test0);
	case 'lin'
	test0 = rgb2lin(test0);
	end
	% test0(:,:,1)=ones(sz_test(j, 1:2))*0;
	% Perform contrast stretching
	test0 = imadjust(test0, stretchlim(test0), [0 1]);
	% Preform image filling
	test0 = imfill(test0,"holes");
	end

Evaluation Algorithm

As the ground truth and the segmented image may have different number of clusters, it is necessary to create a cluster matching algorithm before evaluating the segmenting performance. The overall algorithm is as follow:

In this work, the Jaccard similarity has been used and the similarity threshold has been set to be ≥40%. This algorithm has been custom coded in a function binaryMatch() in MATLAB.

Image-Segmentation-with-Clustering/Main_Clustering.m

Lines 379 to 466 in 43d9bd4

	%% Cluster matching function
	function [Binar, T] = binaryMatch(GTseg_test, segIMG, sz_test, numPic, showPlot)
	% GTseg_test: Ground Truth Image (n x m)
	% segIMG: Segmented Image (n x m x 3)
	% sz_test: size of the testing image (numPic x n x m x 3)
	% numPic : index of the current image
	segGT = reshape(GTseg_test(:,numPic), sz_test(numPic,1:2));
	segIMG2 = rgb2gray(segIMG);
	idxSegGT = unique(segGT);
	idxSegIMG = unique(segIMG2);
	colName = {};
	rowName = {};
	for i = 1:length(idxSegGT)
	for j = 1:length(idxSegIMG)
	% Pre-allocation;
	BW_gt = zeros(sz_test(numPic,1:2));
	BW_img = zeros(sz_test(numPic,1:2));
	% Locate each unique segments
	idxYes_img = segIMG2 == idxSegIMG(j);
	idxYes_gt = segGT == idxSegGT(i);
	BW_gt(idxYes_gt) = 1;
	BW_img(idxYes_img) = 1;
	% Calculate similarity for all combination
	% between GT segments and obtained segments
	mat(i,j) = jaccard(BW_gt, BW_img);
	if i == 1
	colName = [colName {['IMG_seg' num2str(j)]}];
	end
	end
	rowName = [rowName {['GT_seg' num2str(i)]}];
	end
	% Put all similarities into a table
	T = array2table(mat, 'VariableNames', colName,'RowNames', rowName);
	if showPlot
	figure
	sgtitle(['Image ' num2str(numPic) ': Binary matching (white = active segment)'], 'FontSize', 20)
	end
	similarThresh = 0.4; % Keep segment with >= 40% similarity
	for j = 1:length(idxSegIMG)
	% Pre-allocated the image
	BW_gt = zeros(sz_test(numPic,1:2));
	BW_img = zeros(sz_test(numPic,1:2));
	% Select segment pair with highest similarity OR >= 40% similarity
	idxMax = find( (mat(:,j)==max(mat(:,j))) | (mat(:,j) >= similarThresh));
	% If >1 segments obtained, combine them
	for k = 1:length(idxMax)
	dum = segGT == idxSegGT(idxMax(k));
	BW_gt = BW_gt + dum;
	end
	% Map the segment into the pre-allocated image
	idxYes2 = segIMG2 == idxSegIMG(j);
	BW_img(idxYes2) =1 ;
	if showPlot
	subplot(2, length(idxSegIMG),j)
	imshow(BW_gt)
	set(gca, 'FontSize', 14)
	if length(idxMax) > 1
	title({['Ground Truth: cluster ' num2str(j)],...
	['(' num2str(length(idxMax)) ' segments combined)']})
	else
	title(['Ground Truth: cluster ' num2str(j)])
	end
	subplot(2, length(idxSegIMG),j+length(idxSegIMG))
	imshow(BW_img)
	title(['Segmented image: cluster ' num2str(j)])
	set(gca, 'FontSize', 14)
	end
	Binar.seg(j).GT = BW_gt;
	Binar.seg(j).IMG = BW_img;
	end
	end

Figure 10: Clusters matching algorithm scheme

Figure 11: Segmentation evaluation scheme for each image

Results

Segmentation Results

Figure 14: Segmentation results for image 1

Figure 15: Segmentation results for image 2

Figure 16: Segmentation results for image 3

Figure 17: Segmentation results for image 4

Figure 18: Segmentation results for image 5

Evaluation Results

Here, only the cluster matching of the FCM technique has been plotted since the clusters from Mean Shift clustering are highly detailed and the number of clusters also vary drastically.

Figure 19: Cluster matching result for image 1 with FCM-based segmentation

Figure 20: Cluster matching result for image 2 with FCM-based segmentation

Figure 21: Cluster matching result for image 3 with FCM-based segmentation

Figure 22: Cluster matching result for image 4 with FCM-based segmentation

Figure 23: Cluster matching result for image 5 with FCM-based segmentation

Figure 24: Segmentation performance scores for FCM and Mean shift clustering technique on five images

Evaluation Results Discussion

It can be seen from Figure 19 - Figure 23 that the clusters matching algorithm works very well in matching the obtained segments with ground truth segments, making the evaluated scores highly reliable. According to Figure 24,by evaluating both techniques for 5 images, FCM clustering technique gave higher performance as expected.

Nonetheless, as shown Figure 25, after evaluating for 200 images, the Mean Shift clustering techniques gave higher precision and accuracy than FCM. This implies the result from Mean Shift clustering has low number of False Positive (FP) data. This is because the number of clusters for the FCM clustering has been fixed to three clusters which is a very strict constraints, while the number of cluster in Mean Shift clustering is determined automatically. In spite of that, FCM clustering still has the highest average recall, indicating that FCM has higher number of True Positive (TP) data and lower False Negative (FN) data.

Figure 25: Segmentation performance evaluated for 200 images

Conclusion

K-means clustering technique should only be used if the computational cost is at concern. Fuzzy C-Means (FCM) clustering gives the most consistent performance if a prior information of the image is known, or if the number of segmentations can be approximated. Lastly, Mean Shift clustering technique has the highest flexibility and is most suitable for segmenting an unknown image.

Image preprocessing can significantly improve the performance for any segmentation techniques. However, it should be done carefully as some process may remove important image features resulting in a wrong segmentation.

References

[1] T. Verma and N. Patel, "Data Clustering: Algorithms and Applications", Springer, 2020.

[2] S. Kumar. “C-Means Clustering Explained” Builtin. https://builtin.com/data-science/c-means (accessed Feb. 5, 2023)

[3] S. Ghosh and S. Kumar Dubey “Comparative Analysis of K-Means and Fuzzy C-Means Algorithms”, IJACSA, 2013