EE-5356 DIGITAL IMAGE PROCESSING
Metric Structural Similarity (SSIM)
Ajaykumar Sharma
A perceptual image quality assessment metric-structural similarity (SSIM)
This project is related to structural similarity index (SSIM) which represents perceptual image quality based on the structural information. SSIM is an objective image quality metric and is superior to traditional quantitative measures such as MSE and PSNR. This project demonstrates the SSIM based image quality assessment and illustrates its validity in terms of human visual perception. It will be very helpful to understand SSIM and its applications by reviewing the paper listed below.
(a)A general form of SSIM is
,
[Note that >0, β>0 and γ >0 are parameters used to adjust the relative importance of the three components].
where are image patches and
, , and
l(x,y) is luminance comparison (eq. 6), c(x,y) is contrast comparison (eq. 9), and s(x,y) is structural comparison (eq. 10). are constants.x, y, x, y, xy are defined in Eqs.14, 15, 16 in the reference paper. Gaussian weighting function has the form ,
For more information, students are referred to the paper, Z. Wang etal. "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004, which can be downloaded from (Also from IEEE Xplore).
Write a Matlab function as my_ssimwhosethe inputs are two images and,, the outputs are SSIM_metric and SSIM_mapbetween the two images. The SSIM_metric is the mean value ofSSIM_mapwhich is computed by a local moving window (11x11 pixels).You can download the Matlab implementation of the SSIM at as a reference.
(b)Set , as in the paper and apply your function to the different distorted Lena images (512x512) with the same mean square error (MSE). The test images can be downloaded at the link “Universal image quality index” at Compute the SSIM_metricand show theSSIM_map. You will find that the SSIM_metric is more correlated to human perception of quality compared with MSE.
(c)Fix as in (b) and set as 1. Choose any 5 pairs of , for example
[] , [],[],[],[] and apply your function to the distorted Lena images (You can use all or parts of the images). Then find the to produce the SSIM_metric most correlated with your perception of the quality. There is no right answer, which is totally dependent on your own opinion of the quality.
SSIM was introduced in JVT Hannover (Germany) meeting April 2008. Document is referred to JVT-AB31 (W.S.Kim, Z. Li, P. Pahalawatta,and A.M. Tourapis from Dolby Lab, Burbank CA)
Matlab Code:
%%Project Part a
clear all;
close all;
clc;
%% Reading Images
img1=imread('Image1.gif');
img2=imread('Image3.gif');
figure(1);
subplot(2,2,1);
imshow(img1);
title('Orginal Lena Image');
subplot(2,2,2);
imshow(img2);
title('Distorted Lena Image which has noise');
[x,y]=size(img1);
img1 = double(img1);
img2 = double(img2);
%% Defining the alpha and beta images which are as defined in the paper
alpha=2;
beta=2;
gamma=2;
L=length(x);
K1=0.01;
K2=0.03;
%% Defining the window with guassian filter
window = fspecial('gaussian', 11, 2);%
C1 = (K1*L)^2;
C2 = (K2*L)^2;
C3=C2/2;
window = window/sum(sum(window));
%% Obtaining the 2-D filter
mu_x = filter2(window, img1, 'valid');
mu_y = filter2(window, img2, 'valid');
%% Calculating the mux and muy values
mu1_sq = mu_x.*mu_x;
mu2_sq = mu_y.*mu_y;
mu1_mu2 = mu_x.*mu_y;
%% Calculating the variance of the filter
var_x_sq = filter2(window, img1.*img1, 'valid') - mu1_sq;
var_y_sq = filter2(window, img2.*img2, 'valid') - mu2_sq;
var_x=sqrt(var_x_sq);
var_y=sqrt(var_y_sq);
var_xy = filter2(window, img1.*img2, 'valid') - mu1_mu2;
%% Equations I,c,s
I=(((2.*(mu_x)).*(mu_y))+C1)./(mu_x.^2+mu_y.^2+C1);
c=((2.*var_x_sq.*var_y_sq)+C2)./(var_x_sq.^2+var_y_sq.^2+C2);
s=((var_xy+C3)./((var_x.*var_y+C3)));
%% Calculating the Structural Similarity index value
ssim_map =(I.^alpha).*(c.^beta).*(s.^gamma);
mssim = mean2(ssim_map)
figure(1);
subplot(2,2,3);
imshow(max(0, ssim_map).^4);title('SSIM-MAP output');
Output:
OBSERVATIONS:
The compression ratios for the images are:
Lena image= 4.8574
goldhill image= 2.2445
Salt and Pepper / 0.4598 / 225.36
Additive Gaussian noise / 0.1884 / 225.18
Multiplicative speckle noise / 0.2580 / 224.14
Mean shift algorithm / 0.9788 / 224.99
Contrast Stretching / 0.7115 / 225.09
Blurred image / 0.1392 / 224.13
Jpeg compressed image / 0.1537 / 214.11
For goldhill image:
Type of distortion / SSIM Values / MSE valuesSalt and Pepper / 0.7085 / 120.21
Additive Gaussian noise / 0.3481 / 121.12
Multiplicative speckle noise / 0.4443 / 121.42
Mean shift algorithm / 0.9857 / 121.00
Contrast Stretching / 0.8093 / 120.90
Blurred image / 0.1721 / 121.93
Jpeg compressed image / 0.2998 / 117.47
Project Part b:
Matlab Code:
clear all;
close all;
clc;
%% Reading Images
img1=imread('Image1.gif');
img2=imread('Image8.gif');
figure(1);
subplot(2,2,1);
imshow(img1);
title('Orginal Lena Image');
subplot(2,2,2);
imshow(img2);
title('Distorted Lena Image which has noise');
[x,y]=size(img1);
img1 = double(img1);
img2 = double(img2);
%% Defining the alpha and beta images which are as defined in the paper
alpha=1;
beta=1;
gamma=1;
L=length(x);
K1=0.01;
K2=0.03;
%% Defining the window with guassian filter
window = fspecial('gaussian', 11, 1.5);%
C1 = (K1*L)^2;
C2 = (K2*L)^2;
C3=C2/2;
window = window/sum(sum(window));
%% Obtaining the 2-D filter
mu_x = filter2(window, img1, 'valid');
mu_y = filter2(window, img2, 'valid');
%% Calculating the mux and muy values
mu1_sq = mu_x.*mu_x;
mu2_sq = mu_y.*mu_y;
mu1_mu2 = mu_x.*mu_y;
%% Calculating the variance of the filter
var_x_sq = filter2(window, img1.*img1, 'valid') - mu1_sq;
var_y_sq = filter2(window, img2.*img2, 'valid') - mu2_sq;
var_x=sqrt(var_x_sq);
var_y=sqrt(var_y_sq);
var_xy = filter2(window, img1.*img2, 'valid') - mu1_mu2;
%% Equations I,c,s
I=(((2.*(mu_x)).*(mu_y))+C1)./(mu_x.^2+mu_y.^2+C1);
c=((2.*var_x_sq.*var_y_sq)+C2)./(var_x_sq.^2+var_y_sq.^2+C2);
s=((var_xy+C3)./((var_x.*var_y+C3)));
%% Calculating the Structural Similarity index value
ssim_map =(I.^alpha).*(c.^beta).*(s.^gamma);
mssim = mean2(ssim_map)
figure(1);
subplot(2,2,3);
imshow(max(0, ssim_map).^4);
title('SSIM-MAP output');
MSE=0;
for x=1:512
for y=1:512
MSE=MSE+(img1(x,y)-img2(x,y))^2/(512^2);
end
end
Output:
OBSERVATIONS:
Type of distortion / SSIM Values / MSE valuesSalt and Pepper / 0.5961 / 225.36
Additive Gaussian noise / 0.2444 / 225.18
Multiplicative speckle noise / 0.3148 / 224.14
Mean shift algorithm / 0.9890 / 224.99
Contrast Stretching / 0.8388 / 225.09
Blurred image / 0.1829 / 224.13
Jpeg compressed image / 0.1980 / 214.11
For Goldhill image:
Type of distortion / SSIM Values / MSE valuesSalt and Pepper / 0.8066 / 120.21
Additive Gaussian noise / 0.4571 / 121.12
Multiplicative speckle noise / 0.5443 / 121.42
Mean shift algorithm / 0.9927 / 121.00
Contrast Stretching / 0.8903 / 120.90
Blurred image / 0.2622 / 121.93
Jpeg compressed image / 0.3757 / 117.47
Project Part C:
clear all;
close all;
clc;
%% Reading Images
img1=imread('Image1.gif');
img2=imread('Image2.gif');
figure(1);
subplot(2,2,1);
imshow(img1);
title('Orginal Lena Image');
subplot(2,2,2);
imshow(img2);
title('Distorted Lena Image which has noise');
[x,y]=size(img1);
img1 = double(img1);
img2 = double(img2);
%% Defining the alpha and beta images which are as defined in the paper
alpha=1;
beta=1;
gamma=2;
L=length(x);
K1=0.01;
K2=0.03;
%% Defining the window with guassian filter
window = fspecial('gaussian', 11, 2);%
C1 = (K1*L)^2;
C2 = (K2*L)^2;
C3=C2/2;
window = window/sum(sum(window));
%% Obtaining the 2-D filter
mu_x = filter2(window, img1, 'valid');
mu_y = filter2(window, img2, 'valid');
%% Calculating the mux and muy values
mu1_sq = mu_x.*mu_x;
mu2_sq = mu_y.*mu_y;
mu1_mu2 = mu_x.*mu_y;
%% Calculating the variance of the filter
var_x_sq = filter2(window, img1.*img1, 'valid') - mu1_sq;
var_y_sq = filter2(window, img2.*img2, 'valid') - mu2_sq;
var_x=sqrt(var_x_sq);
var_y=sqrt(var_y_sq);
var_xy = filter2(window, img1.*img2, 'valid') - mu1_mu2;
%% Equations I,c,s
I=(((2.*(mu_x)).*(mu_y))+C1)./(mu_x.^2+mu_y.^2+C1);
c=((2.*var_x_sq.*var_y_sq)+C2)./(var_x_sq.^2+var_y_sq.^2+C2);
s=((var_xy+C3)./((var_x.*var_y+C3)));
%% Calculating the Structural Similarity index value
ssim_map =(I.^alpha).*(c.^beta).*(s.^gamma);
mssim = mean2(ssim_map)
figure(1);
subplot(2,2,3);
imshow(max(0, ssim_map).^4);title('SSIM-MAP output');
Output:
For Alpha=1, beta=1, gamma=2
OBSERVATIONS:
Type of distortion / SSIM ValuesSalt and Pepper / 0.4817
Additive Gaussian noise / 0.2187
Multiplicative speckle noise / 0.2864
For goldhill image:
Type of distortion / SSIM ValuesSalt and Pepper / 0.7265
Additive Gaussian noise / 0.3975
Multiplicative speckle noise / 0.4874
For Alpha=1, beta=1, gamma=3
OBSERVATIONS:
Type of distortion / SSIM ValuesSalt and Pepper / 0.4464
Additive Gaussian noise / 0.1754
Multiplicative speckle noise / 0.2411
For goldhill image:
Type of distortion / SSIM ValuesSalt and Pepper / 0.6951
Additive Gaussian noise / 0.3203
Multiplicative speckle noise / 0.4109
For Alpha=1, beta=1, gamma=4
OBSERVATIONS:
Type of distortion / SSIM ValuesSalt and Pepper / 0.4215
Additive Gaussian noise / 0.1454
Multiplicative speckle noise / 0.2084
For goldhill image:
Type of distortion / SSIM ValuesSalt and Pepper / 0.6718
Additive Gaussian noise / 0.2648
Multiplicative speckle noise / 0.3530
For Alpha=1, beta=2, gamma=1
OBSERVATIONS:
Type of distortion / SSIM ValuesSalt and Pepper / 0.4976
Additive Gaussian noise / 0.2339
Multiplicative speckle noise / 0.3065
For goldhill image:
Type of distortion / SSIM ValuesSalt and Pepper / 0.7428
Additive Gaussian noise / 0.4333
Multiplicative speckle noise / 0.5291
For Alpha=1 , beta= 3, gamma=1
OBSERVATIONS:
Type of distortion / SSIM ValuesSalt and Pepper / 0.4758
Additive Gaussian noise / 0.2054
Multiplicative speckle noise / 0.2789
For goldhill image:
Type of distortion / SSIM ValuesSalt and Pepper / 0.7243
Additive Gaussian noise / 0.3827
Multiplicative speckle noise / 0.4845
For lena image.
Type of distortion / SSIM Values / Q valuesSalt and Pepper / 0.4598 / 0.5915
Additive Gaussian noise / 0.1884 / 0.3723
Multiplicative speckle noise / 0.2580 / 0.4257
Mean shift algorithm / 0.9788 / 0.9893
Contrast Stretching / 0.7115 / 0.9356
Blurred image / 0.1392 / 0.3233
Jpeg compressed image / 0.1537 / 0.2861
For goldhill image
Type of distortion / SSIM Values / Q valuesSalt and Pepper / 0.7085 / 0.8077
Additive Gaussian noise / 0.3481 / 0.6009
Multiplicative speckle noise / 0.4443 / 0.6634
Mean shift algorithm / 0.9857 / 0.9928
Contrast Stretching / 0.8093 / 0.9506
Blurred image / 0.1721 / 0.4779
Jpeg compressed image / 0.2998 / 0.4857
Conclusions:
Thus we have calculated the Structural Similarity index of various distorted images. From the above observations we see that even though the MSE remains the same in most of the cases the MSSIM is different depending on the contrast and quality of the image.
References:
1)Z. Wang, et al., “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600–612, Apr. 2004.
2)Z. Wang, L. Lu and A. C. Bovik, "Video quality assessment using structural distortion measurement", IEEE International Conference on Image Processing, Vol: 3, Page(s): 65 -68, Rochester, NY, September 22-25, 2002.
3)The SSIM Index for Image Quality Assessment
4)W. Malpica and A. Bovik, “Range image quality assessment by structural similarity”, IEEE ICASSP 2009, 19-24 April 2009.
5)J. Zujovic, T.N. Pappas and D.L. Neuhoff, “Structural similarity metrics for texture analysis and retrieval”, IEEE ICIP 2009, Cairo, Egypt, Nov. 2009 (paper MP.L1.5)
6)B.M.K. Aswathappa and K.R. Rao, “Rate-Distortion Optimization using Structural Information in H.264 Strictly Intra-frame Encoder”, Southeastern Symposium on Systems Theory, Tyler, TX, March 2010.
7)X. Shang, “Structural similarity based image quality assessment: pooling strategies and applications to image compression and digit recognition,” M.S. Thesis, EE Department, The University of Texas at Arlington, Aug. 2006.