4 Image Segmentation Techniques in OpenCV Python

Image Segmentation Techniques in OpenCV Python


In this article, we will show you how to do image segmentation in OpenCV Python by using multiple techniques. We will first explain what is image processing and cover some prerequisite concepts. And then we will go through different techniques and implementations one by one.

What is Image Segmentation?

Image Segmentation Techniques in OpenCV Python

Image segmentation is an image processing task in which the image is segmented or partitioned into multiple regions such that the pixels in the same region share common characteristics.

There are two forms of image segmentation:

  1. Local segmentation – It is concerned with a specific area or region of the image.
  2. Global segmentation – It is concerned with segmenting the entire image.

Types of Image Segmentation Approaches

  1. Discontinuity detection – This is a method of segmenting a picture into areas based on discontinuity. This is where edge detection comes in. Discontinuity in edges generated due to intensity is recognized and used to establish area borders. Examples: Histogram filtering and contour detection.
  2. Similarity detection – A method of segmenting a picture into sections based on resemblance. Thresholding, area expansion, and region splitting and merging are all included in this methodology. All of them split the image into sections with comparable pixel counts. Based on established criteria, they divide the picture into a group of clusters with comparable features. Example: Kmeans, Colour detection/classification.
  3. Neural network approach – For the goal of decision making, neural network-based segmentation algorithms replicate the learning techniques of the human brain. This approach is widely used in segmenting medical images and separate them from the background. A neural network is made up of a vast number of linked nodes, each with its own weight.

Pre-requisite Concepts

i) K-Means Algorithm

K-means is a clustering algorithm that is used to group data points into clusters such that data points lying in the same group are very similar to each other in characteristics.

Deep Learning Specialization on Coursera

K-means algorithm can be used to find subgroups in the image and assign the image pixel to that subgroup which results in image segmentation.

K-means Algorithm
K-means Algorithm visualization

ii) Contour Detection

Contours can be simply defined as curves/polygons formed by joining the pixels that are grouped together according to intensity or color values.

OpenCV provides us with inbuilt functions to detect these contours in images. Contour detection is generally applied on binary images(grayscale images) after edge detection or thresholding(or both) has been applied to them.

Contour detection with OpenCV
Contour detection with OpenCV

ii) Masking

The application of masks (which are binary images with only 0 or 1 as pixel values) to transform a picture is known as masking. The pixels (of the picture) that coincide with the zero in the mask are turned off when the mask is applied to it.

Mask applied to an image
In order: (Mask, query image, result image)

iv) Color Detection

Detection and classification of colors by using their RGB colorspace values are known as color detection. For example:

          R   G   B    
Red =    (255, 0, 0)
Green =  (0, 255, 0)
Blue =   (0, 0, 255)
Orange = (255, 165, 0)
Purple = (128, 0, 128)

Image Segmentation in OpenCV Python

We will be looking at the following 4 different ways to perform image segmentation in OpenCV Python and Scikit Learn –

  1. Image Segmentation using K-Means
  2. Image Segmentation using Contour Detection
  3. Image Segmentation using Thresholding
  4. Image Segmentation using Color Masking

We will be using the below image to perform image segmentation with all the techniques.

Query image
Query image

1. Image Segmentation using K-means

i) Importing libraries and Images

Import matplotlib, numpy, OpenCV along with the image to be segmented.

import matplotlib as plt
import numpy as np
import cv2
path = 'image.jpg'
img = cv2.imread(path)

ii) Preprocessing the Image

Preprocess the image by converting it to the RGB color space. Reshape it along the first axis to convert it into a 2D vector i.e. if the image is of the shape (100,100,3) (width, height, channels) then it will be converted to (10000,3). Next, convert it into the float datatype.

img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
twoDimage = img.reshape((-1,3))
twoDimage = np.float32(twoDimage)

iii) Defining Parameters

Define the criteria by which the K-means algorithm is supposed to cluster pixels.

The ‘K’ variable defines the no of clusters/groups that a pixel can belong to (You can increase this value to increase the degree of segmentation).

criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 10, 1.0)
K = 2

iv) Apply K-Means

The K variable randomly initiates K different clusters and the ‘center’ variable defines the center of these clusters. The distance of each point from these centers is computed and then they are assigned to one of the clusters. Then they are divided into different segments according to the value of their ‘label variable’.

center = np.uint8(center)
res = center[label.flatten()]
result_image = res.reshape((img.shape))


Image Segmentation using K-Means Python OpenCV
K-Means Output

2. Image Segmentation using Contour Detection

i) Importing libraries and Images

Import OpenCV, matplotlib, numpy and load the image to memory.

import cv2
import matplotlib.pyplot as plt
import numpy as np
path = 'image.jpg'
img = cv2.imread(path)
img = cv2.resize(img,(256,256))

ii) Preprocessing the Image

  1. Convert the image to grayscale.
  2. Compute the threshold of the grayscale image(the pixels above the threshold are converted to white otherwise zero).
  3. Apply canny edge detection to the thresholded image before finally using the ‘cv2.dilate’ function to dilate edges detected.
gray = cv2.cvtColor(img,cv2.COLOR_RGB2GRAY)
_,thresh = cv2.threshold(gray, np.mean(gray), 255, cv2.THRESH_BINARY_INV)
edges = cv2.dilate(cv2.Canny(thresh,0,255),None)


Contour Detection Pre processing

iii) Detecting and Drawing Contours

  1. Use the OpenCV find contour function to find all the open/closed regions in the image and store (cnt). Use the -1 subscript since the function returns a two-element tuple.
  2. Pass them through the sorted function to access the largest contours first.
  3. Create a zero pixel mask that has equal shape and size to the original image.
  4. Draw the detected contours on the created mask.
cnt = sorted(cv2.findContours(edges, cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE)[-2], key=cv2.contourArea)[-1]
mask = np.zeros((256,256), np.uint8)
masked = cv2.drawContours(mask, [cnt],-1, 255, -1)

iv) Segmenting the Regions

In order to show only the segmented parts of the image, we perform a bitwise AND operation on the original image (img) and the mask (containing the outlines of all our detected contours).

Finally, Convert the image back to RGB to see it segmented(while being comparable to the original image).

dst = cv2.bitwise_and(img, img, mask=mask)
segmented = cv2.cvtColor(dst, cv2.COLOR_BGR2RGB)


Image Segmentation using Contour Detection OpenCV Python
Contour Detection output

3. Image Segmentation using Thresholding

i) Importing libraries and Images

Import numpy, scikit-image, matplotlib, and OpenCV.

import numpy as np
import matplotlib.pyplot as plt
from skimage.filters import threshold_otsu
import cv2
path ='image.jpg'
img = cv2.imread(path)

ii) Preprocessing the Image

Convert the image to the RBG color space from BGR in order to finally convert it to grayscale.


iii) Segmentation Process

Create a “filter_image” function that multiplies the mask (created in the previous section) with the RGB channels of our image. Further, they are concatenated to form a normal image.

Next, we calculate the threshold (thresh) for the gray image and use it as a deciding factor i.e. values lying below this threshold are selected and others are discarded. This creates a mask-like (img_otsu) image that can later be used to segment our original image.

Finally, apply the “filter_image” function on the original image(img) and the mask formed using thresholding (img_otsu)

def filter_image(image, mask):
    r = image[:,:,0] * mask
    g = image[:,:,1] * mask
    b = image[:,:,2] * mask
    return np.dstack([r,g,b])

thresh = threshold_otsu(img_gray)
img_otsu  = img_gray < thresh
filtered = filter_image(img, img_otsu)


Image Segmentation using Thresholding in OpenCV Python

4. Segmentation using Color Masking

i) Import libraries and Images

Import OpenCV and load the image to memory.

import cv2
path ='image.jpg'
img = cv2.imread(path)

ii) Preprocessing the Image

OpenCV default colorspace is BGR so we convert it to RGB. Next, we convert it to the HSV colorspace.

rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
hsv_img = cv2.cvtColor(rgb_img, cv2.COLOR_RGB2HSV)

iii) Define the Color Range to be Detected

Define the RGB range for the color we want to detect. Use the OpenCV in range function to create a mask of all the pixels that fall within the range that we defined. It will later help to mask these pixels.

light_blue = (90, 70, 50)
dark_blue = (128, 255, 255)
# You can use the following values for green
# light_green = (40, 40, 40)
# dark_greek = (70, 255, 255)
mask = cv2.inRange(hsv_img, light_blue, dark_blue)

iv) Apply the Mask

Use the bitwise AND operation to apply our mask to the query image.

result = cv2.bitwise_and(img, img, mask=mask)


Color masking (blue)
Color masking (blue)

Image Segmentation in OpenCV Python




Please enter your comment!
Please enter your name here