Contents
Introduction
In this article, we will show you how to do image segmentation in OpenCV Python by using multiple techniques. We will first explain what is image processing and cover some prerequisite concepts. And then we will go through different techniques and implementations one by one.
What is Image Segmentation?
Image segmentation is an image processing task in which the image is segmented or partitioned into multiple regions such that the pixels in the same region share common characteristics.
There are two forms of image segmentation:
- Local segmentation – It is concerned with a specific area or region of the image.
- Global segmentation – It is concerned with segmenting the entire image.
Types of Image Segmentation Approaches
- Discontinuity detection – This is a method of segmenting a picture into areas based on discontinuity. This is where edge detection comes in. Discontinuity in edges generated due to intensity is recognized and used to establish area borders. Examples: Histogram filtering and contour detection.
- Similarity detection – A method of segmenting a picture into sections based on resemblance. Thresholding, area expansion, and region splitting and merging are all included in this methodology. All of them split the image into sections with comparable pixel counts. Based on established criteria, they divide the picture into a group of clusters with comparable features. Example: Kmeans, Colour detection/classification.
- Neural network approach – For the goal of decision making, neural network-based segmentation algorithms replicate the learning techniques of the human brain. This approach is widely used in segmenting medical images and separate them from the background. A neural network is made up of a vast number of linked nodes, each with its own weight.
Pre-requisite Concepts
i) K-Means Algorithm
K-means is a clustering algorithm that is used to group data points into clusters such that data points lying in the same group are very similar to each other in characteristics.
K-means algorithm can be used to find subgroups in the image and assign the image pixel to that subgroup which results in image segmentation.
ii) Contour Detection
Contours can be simply defined as curves/polygons formed by joining the pixels that are grouped together according to intensity or color values.
OpenCV provides us with inbuilt functions to detect these contours in images. Contour detection is generally applied on binary images(grayscale images) after edge detection or thresholding(or both) has been applied to them.
ii) Masking
The application of masks (which are binary images with only 0 or 1 as pixel values) to transform a picture is known as masking. The pixels (of the picture) that coincide with the zero in the mask are turned off when the mask is applied to it.
iv) Color Detection
Detection and classification of colors by using their RGB colorspace values are known as color detection. For example:
R G B Red = (255, 0, 0) Green = (0, 255, 0) Blue = (0, 0, 255) Orange = (255, 165, 0) Purple = (128, 0, 128)
Image Segmentation in OpenCV Python
We will be looking at the following 4 different ways to perform image segmentation in OpenCV Python and Scikit Learn –
- Image Segmentation using K-Means
- Image Segmentation using Contour Detection
- Image Segmentation using Thresholding
- Image Segmentation using Color Masking
We will be using the below image to perform image segmentation with all the techniques.
1. Image Segmentation using K-means
i) Importing libraries and Images
Import matplotlib, numpy, OpenCV along with the image to be segmented.
import matplotlib as plt import numpy as np import cv2 path = 'image.jpg' img = cv2.imread(path)
ii) Preprocessing the Image
Preprocess the image by converting it to the RGB color space. Reshape it along the first axis to convert it into a 2D vector i.e. if the image is of the shape (100,100,3) (width, height, channels) then it will be converted to (10000,3). Next, convert it into the float datatype.
img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB) twoDimage = img.reshape((-1,3)) twoDimage = np.float32(twoDimage)
iii) Defining Parameters
Define the criteria by which the K-means algorithm is supposed to cluster pixels.
The ‘K’ variable defines the no of clusters/groups that a pixel can belong to (You can increase this value to increase the degree of segmentation).
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0) K = 2 attempts=10
iv) Apply K-Means
The K variable randomly initiates K different clusters and the ‘center’ variable defines the center of these clusters. The distance of each point from these centers is computed and then they are assigned to one of the clusters. Then they are divided into different segments according to the value of their ‘label variable’.
ret,label,center=cv2.kmeans(twoDimage,K,None,criteria,attempts,cv2.KMEANS_PP_CENTERS) center = np.uint8(center) res = center[label.flatten()] result_image = res.reshape((img.shape))
Output:
2. Image Segmentation using Contour Detection
i) Importing libraries and Images
Import OpenCV, matplotlib, numpy and load the image to memory.
import cv2 import matplotlib.pyplot as plt import numpy as np path = 'image.jpg' img = cv2.imread(path) img = cv2.resize(img,(256,256))
ii) Preprocessing the Image
- Convert the image to grayscale.
- Compute the threshold of the grayscale image(the pixels above the threshold are converted to white otherwise zero).
- Apply canny edge detection to the thresholded image before finally using the ‘cv2.dilate’ function to dilate edges detected.
- Also Read – Learn Image Thresholding with OpenCV
- Also Read – OpenCV Tutorial – Erosion and Dilation of Image
gray = cv2.cvtColor(img,cv2.COLOR_RGB2GRAY) _,thresh = cv2.threshold(gray, np.mean(gray), 255, cv2.THRESH_BINARY_INV) edges = cv2.dilate(cv2.Canny(thresh,0,255),None)
Output:
iii) Detecting and Drawing Contours
- Use the OpenCV find contour function to find all the open/closed regions in the image and store (cnt). Use the -1 subscript since the function returns a two-element tuple.
- Pass them through the sorted function to access the largest contours first.
- Create a zero pixel mask that has equal shape and size to the original image.
- Draw the detected contours on the created mask.
cnt = sorted(cv2.findContours(edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)[-2], key=cv2.contourArea)[-1] mask = np.zeros((256,256), np.uint8) masked = cv2.drawContours(mask, [cnt],-1, 255, -1)
iv) Segmenting the Regions
In order to show only the segmented parts of the image, we perform a bitwise AND operation on the original image (img) and the mask (containing the outlines of all our detected contours).
Finally, Convert the image back to RGB to see it segmented(while being comparable to the original image).
dst = cv2.bitwise_and(img, img, mask=mask) segmented = cv2.cvtColor(dst, cv2.COLOR_BGR2RGB)
Output:
3. Image Segmentation using Thresholding
i) Importing libraries and Images
Import numpy, scikit-image, matplotlib, and OpenCV.
import numpy as np import matplotlib.pyplot as plt from skimage.filters import threshold_otsu import cv2 path ='image.jpg' img = cv2.imread(path)
ii) Preprocessing the Image
Convert the image to the RBG color space from BGR in order to finally convert it to grayscale.
img_rgb=cv2.cvtColor(img,cv2.COLOR_BGR2RGB) img_gray=cv2.cvtColor(img_rgb,cv2.COLOR_RGB2GRAY)
iii) Segmentation Process
Create a “filter_image” function that multiplies the mask (created in the previous section) with the RGB channels of our image. Further, they are concatenated to form a normal image.
Next, we calculate the threshold (thresh) for the gray image and use it as a deciding factor i.e. values lying below this threshold are selected and others are discarded. This creates a mask-like (img_otsu) image that can later be used to segment our original image.
Finally, apply the “filter_image” function on the original image(img) and the mask formed using thresholding (img_otsu)
def filter_image(image, mask): r = image[:,:,0] * mask g = image[:,:,1] * mask b = image[:,:,2] * mask return np.dstack([r,g,b]) thresh = threshold_otsu(img_gray) img_otsu = img_gray < thresh filtered = filter_image(img, img_otsu)
Output:
4. Segmentation using Color Masking
i) Import libraries and Images
Import OpenCV and load the image to memory.
import cv2 path ='image.jpg' img = cv2.imread(path)
ii) Preprocessing the Image
OpenCV default colorspace is BGR so we convert it to RGB. Next, we convert it to the HSV colorspace.
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) hsv_img = cv2.cvtColor(rgb_img, cv2.COLOR_RGB2HSV)
iii) Define the Color Range to be Detected
Define the RGB range for the color we want to detect. Use the OpenCV in range function to create a mask of all the pixels that fall within the range that we defined. It will later help to mask these pixels.
light_blue = (90, 70, 50)
dark_blue = (128, 255, 255)
# You can use the following values for green
# light_green = (40, 40, 40)
# dark_greek = (70, 255, 255)
mask = cv2.inRange(hsv_img, light_blue, dark_blue)
iv) Apply the Mask
Use the bitwise AND operation to apply our mask to the query image.
result = cv2.bitwise_and(img, img, mask=mask)
Output:
