1. INTRODUCTION 1. 简介

Under a hazy environment, scattering particles in the atmosphere medium interact with light. Therefore, the scattered atmospheric light coming from the illumination sources superimposes on the attenuated target radiation emanating from scene objects to form a light veil. This phenomenon degrades contrast and visibility of the interested scene [1,2], limiting the performance of hazy imaging [3], object detection [4], lane detection [5], and atmospheric and oceanic sensing [6,7]. Hence, there are urgent demands for techniques improving the quality of such degraded images, especially dehazing techniques used in practical application scenarios. Many techniques of restoring and improving hazy image quality have been proposed in archival literature, and this continues to be a timely subject [8–15]. Among those, polarized imaging is an effective means because of its stable characteristics for avoiding light scattering and absorption [9,12–15]. Schechner et al. used the polarization characteristics of scattered light to separate background scattering light and target radiation light, then to realize dehazing processing in the outdoor scenes [12]. However, the need for human interaction in atmospheric parameter estimation (in particular, the choice of bias factor) and the acquisition of polarization images has long plagued the wide application of polarization dehazing methods. Liang et al. proposed the methods combined with various image processing technologies, such as image fusion, hyperspectral imaging, and histogram stretching [16–19]. Though these methods prominently increase image performance, they use ponderous acquisition devices and a time-consuming process to acquire the desired hazy-free images. Liu et al. used the wavelet transform to carry out a multi-scale analysis of the image, then dehazing and denoising the low- and high-frequency components, respectively, and finally reconstructing the hazy-free image by the inverse transformation [20–22]. However, the noise of the processed image is relatively obvious, and it is only suitable for short-distance scenes.
在雾霾环境下，大气介质中的散射粒子与光相互作用。因此，来自照明源的散射大气光叠加在从场景物体发出的衰减的目标辐射上，形成光幕。这种现象会降低感兴趣场景的对比度和可见度 [1, 2]，限制模糊成像 [3]、物体检测 [4]、车道检测 [5] 以及大气和海洋传感 [6, 7] 的性能。因此，迫切需要提高此类退化图像质量的技术，特别是在实际应用场景中使用的去雾技术。档案文献中已经提出了许多恢复和改善模糊图像质量的技术，这仍然是一个及时的主题[8-15]。其中，偏振成像是一种有效的手段，因为它具有避免光散射和吸收的稳定特性[9,12-15]。谢赫纳等人。利用散射光的偏振特性将背景散射光和目标辐射光分离，从而实现室外场景的去雾处理[12]。然而，大气参数估计（特别是偏置因子的选择）和偏振图像获取中人为交互的需求长期以来一直困扰着偏振去雾方法的广泛应用。梁等人。提出了与各种图像处理技术相结合的方法，例如图像融合、高光谱成像和直方图拉伸[16-19]。尽管这些方法显着提高了图像性能，但它们使用笨重的采集设备和耗时的过程来获取所需的无雾图像。刘等人。 利用小波变换对图像进行多尺度分析，然后分别对低频和高频分量进行去雾和去噪，最后通过逆变换重建无雾图像[20-22]。但处理后的图像噪声比较明显，仅适用于近距离场景。

Division of focal plane (DoFP) systems simultaneously recorded the light intensity of four different polarization states on an image during a single capture [23]. However, this type of imaging sensor captures only partial polarization information on a single pixel, resulting in reduced spatial resolution output. Interpolation methods are proposed to recover the missing spatial and polarization information. Nevertheless, even the most advantageous interpolation method has failed to achieve information recovery without the noise influence [24]. Since the captured image contains important features of different frequency components, the low-frequency parts mainly represent the basic framework of the image information, and the high-frequency parts mainly include the salient feature information, such as the detail edges, texture distribution, and mutating noise of the image [25,26]. Therefore, it is necessary to extract and analyze the captured raw hazy image on different frequency scales.
焦平面分割 (DoFP) 系统在单次捕获期间同时记录图像上四种不同偏振状态的光强度 [23]。然而，这种类型的成像传感器仅捕获单个像素上的部分偏振信息，导致空间分辨率输出降低。提出了插值方法来恢复丢失的空间和偏振信息。然而，即使是最有利的插值方法也无法在没有噪声影响的情况下实现信息恢复[24]。由于捕获的图像包含不同频率成分的重要特征，因此低频部分主要代表图像信息的基本框架，高频部分主要包括显着特征信息，如细节边缘、纹理分布等。改变图像的噪声[25, 26]。因此，有必要在不同频率尺度上提取和分析捕获的原始模糊图像。

To overcome the aforementioned problems, we first analyze the physical essence of the signal and system by converting the captured raw hazy image to the frequency domain. Then, we targeted and optimized the most advantageous residual interpolation process [24], so that the spatial resolution could be maintained without the noise influence using the optimized residual interpolation (ORI) algorithm before dehazing. Third, we propose a generalized polarimetric dehazing method based on spatial frequency division and fusion (SFDF) for far-field and dense hazy image. The proposed SFDF method includes the following steps: (1) the captured raw hazy image is extracted to obtain four polarization sub-images; (2) the sub-images are upsampled by a proposed ORI algorithm; (3) calculate the “best polarization” and “worst polarization” images (𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$, respectively); (4) the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ are decomposed into the low- and high-frequency parts by using non-subsampled contourlet transform (NSCT) [27], respectively; (5) the low-frequency parts are polarization dehazed; (6) the high-frequency parts are fused and enhanced by the Laplacian pyramid [28]; (7) the clear dehazing image without spatial resolution loss is reconstructed by inverting NSCT transform. The proposed SFDF method is compared with polarization difference method (PD method, proposed by Schechner et al. [12]) and the polarization angle (PA method, proposed by Liang et al. [13]) by evaluating the standard deviation (STD), entropy, contrast (C), and average gradient (AG) [23,29,30]. Moreover, we discuss the influence of bias factor on the quality of dehazing image by traversing the empirical value. The experimental results in different environmental conditions indicate that the proposed SFDF algorithm outperforms the existing schemes in dense hazy weather more than kilometer distances. The robust advantage without manual choice of bias factor causes the proposed method to have a broader prospect in practical application scenarios.

2. POLARIZATION DEHAZING MODEL
2. 偏振去雾模型

As shown in Fig. 1, the widely used physical model of hazy image formation was proposed by Schechner et al. [12]. When imaging through the atmosphere, the detector senses two sources: the scattered atmospheric light coming from the illumination sources, e.g., the Sun (the airlight) and the attenuated target radiation emanating from scene objects (the direct transmission). The airlight superimposes on the direct transmission, degrading contrast and visibility of the interested scene. As a result, the captured image becomes blurred. Assuming that the scattering medium is homogeneous, the irradiance of the captured image is expressed as [12]
如图 1 所示，广泛使用的模糊图像形成物理模型是由 Schechner 等人提出的。 [12]。当通过大气成像时，探测器感测两个源：来自照明源（例如太阳）的散射大气光和从场景物体发出的衰减目标辐射（直接透射）。空气光叠加在直接传输上，降低了感兴趣场景的对比度和可见度。结果，所拍摄的图像变得模糊。假设散射介质是均匀的，捕获图像的辐照度表示为[12]

 

Fig. 1. Schematic diagram of the hazy image formation physical model.
图1 雾化成像物理模型示意图。

Download Full Size | PDF
下载全尺寸| PDF

After eliminating transmittance by using 𝐴=𝐴∞(1−𝑡)$A=A_{\mathrm{\infty }}(1-t)$, the object radiance (hazy-free image) can be inverted from Eq. (1) [12]:

3. SPATIAL FREQUENCY DIVISION AND FUSION (SFDF) ALGORITHM
3. 空间频分融合（SFDF）算法

The clear and hazy outfield images are captured by the linear polarization camera (U.S.A., FLIR Systems, Inc., BLACKFLY SBFS-U3-51S5P; image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$; chroma, mono). In order to analyze the physical essence of signal and system, the raw hazy images are converted to the frequency domain by Fourier transform. The value of each pixel in the Fourier spectrum corresponds to the gray difference between adjacent points in the raw image [25,26]. The raw images under different hazy concentrations and the corresponding frequency domains are shown in Fig. 2. The hazy concentration was characterized by the visibility index (VI) [10]. The VI is proposed to judge the blur degree of the hazy image and is defined as

 

Fig. 2. (a)–(d) Clear and hazy outfield images captured by the linear polarization camera. (e)–(h) Corresponding frequency domain of images in (a)–(d).
图 2.(a)–(d) 由线性偏振相机捕获的清晰和模糊的外场图像。 (e)–(h) (a)–(d) 中图像的相应频域。

Download Full Size | PDF
下载全尺寸| PDF

As shown in Figs. 2(e)–2(h), the overall spectrum distribution of the clear and hazy outfield image is similar. However, the scene boundary of the clear image capture in the Sun is sharp, and the gray values between adjacent points are quite different. Thus, the frequency domain image has more bright points (contains more high-frequency components). On the contrary, the scene boundary of the hazy image is fuzzy, and the corresponding frequency domain image has more dark points (contains more low-frequency components). With the increase of hazy concentration, the frequency domain image contains more black points. Those results verify that the low-frequency components of the outfield image mainly characterize the hazy background, while the high-frequency components primarily denote the detailed object information [25,26]. This property is used as a prior (or knowledge) to restore the hazy-free image.
如图所示。如图2(e)–2(h)所示，清晰和模糊外场图像的整体光谱分布相似。然而，在阳光下捕获的清晰图像的场景边界很清晰，并且相邻点之间的灰度值差异较大。这样，频域图像就有更多的亮点（包含更多的高频成分）。相反，有雾图像的场景边界模糊，对应的频域图像有更多的暗点（包含更多的低频成分）。随着雾浓度的增加，频域图像包含更多的黑点。这些结果验证了外场图像的低频分量主要表征模糊背景，而高频分量主要表示详细的物体信息[25, 26]。该属性用作恢复无雾图像的先验（或知识）。

 

Fig. 3. Schematic diagram of the proposed SFDF method. The numerical labels are, respectively, represented as 1, the raw hazy image; 2, four polarization sub-images; 3, four polarization sub-images after ORI upsampling; 4, the 𝐼⊥$I^{\mathrm{\perp }}$ image; 5, the 𝐼‖$I^{\Vert }$ image; 6, normalized high-frequency parts of the 𝐼⊥$I^{\mathrm{\perp }}$ image (here are 36 images); 7, low-frequency part of the 𝐼‖$I^{\Vert }$ image; 8, normalized high-frequency parts of the 𝐼⊥$I^{\mathrm{\perp }}$ images (here are 36 images); 9, low-frequency part of the 𝐼‖$I^{\Vert }$ images; 10, high-frequency images fused by Laplace pyramid (here are 36 images); 11, low-frequency image after polarization dehazing; 12, hazy-free image after dehazing by the SFDF method; 13, white block partially enlarged in the raw hazy image; 14, white block partially enlarged in the low-frequency part of the 𝐼⊥$I^{\mathrm{\perp }}$ image; 15, white block partially enlarged in the low-frequency part of the 𝐼‖$I^{\Vert }$ image. (Note: normalization is merely for better display here. This step is not carried out in actual operation.)

Download Full Size | PDF
下载全尺寸| PDF

According to the above analysis, this paper proposes a SFDF method. As shown in Fig. 3, the proposed algorithm is divided into seven steps: extracting four polarization sub-images, ORI upsampling, calculating the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images, NSCT decomposition, polarization dehazing, Laplace pyramid fusion, and NSCT inverse transform.

Step 1: Extracting Four Polarization Sub-Images
步骤1：提取四个偏振子图像

The high-resolution and multi-polarization hazy image is captured by the linear polarization camera. The raw hazy image contains four polarization states of 0°, 45°, 90°, and 135°. The four polarization sub-images are extracted from the hazy image.
线偏振相机捕获高分辨率、多偏振模糊图像。原始模糊图像包含 0°、45°、90° 和 135° 四种偏振态。从模糊图像中提取四个偏振子图像。

Step 2: ORI Upsampling 第 2 步：ORI 上采样

The four polarization sub-images are interpolated by using an ORI algorithm to avoid spatial resolution loss, respectively. The diagram of the ORI algorithm is shown in Fig. 4. The four polarization sub-images are bilinear interpolated to generate tentative estimates of 0°, 45°, 90°, and 135° (𝑞𝑖0$q_{i0}$, 𝑞𝑖45$q_{i45}$, 𝑞𝑖90$q_{i90}$, 𝑞𝑖135$q_{i135}$). The pixel value differences between the four polarization sub-images (𝐼0$I_0$, 𝐼45$I_{45}$, 𝐼90$I_{90}$, 𝐼135$I_{135}$) and those tentatively estimated (𝑞𝑖0$q_{i0}$, 𝑞𝑖45$q_{i45}$, 𝑞𝑖90$q_{i90}$, 𝑞𝑖135$q_{i135}$) are used to calculated the residuals (𝐼0−𝑞𝑖0$I_0-q_{i0}$, 𝐼45−𝑞𝑖45$I_{45}-q_{i45}$, 𝐼90−𝑞𝑖90$I_{90}-q_{i90}$, 𝐼135−𝑞𝑖135$I_{135}-q_{i135}$). We used the guided filter to perform linear transform on the bilinear interpolated high-resolution image; hence, the tentative estimate images after smoothing and edge preserving are obtained [31].

 

Fig. 4. Schematic diagram of the ORI algorithm.
图4.ORI算法示意图。

Download Full Size | PDF
下载全尺寸| PDF

It is worth noting that the interpolation processes of four polarization images can be performed simultaneously. To reduce the noise error and improve accurately the image spatial resolution, we propose an optimization operation in the ORI algorithm (marked with the red dashed rectangle in Fig. 4): the average intensity of four nearest neighboring pixels in the original hazy image is estimated as a new light intensity value. Then the average intensity image is selected as the guiding filter for edge preservation.
值得注意的是，四个偏振图像的插值过程可以同时进行。为了减少噪声误差并准确提高图像空间分辨率，我们在ORI算法中提出了一种优化操作（图4中用红色虚线矩形标记）：估计原始模糊图像中四个最近邻像素的平均强度作为新的光强度值。然后选择平均强度图像作为边缘保留的引导滤波器。

Step 3: Calculate for the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ Images

In the polarization dehazing model of Eqs. (1) and (2), the key is to separate airlight from direct transmission. We assume that the direct transmission is non-polarized, and the airlight is partially linearly polarized (this model does not consider circularly polarized light). The energy of direct transmission is evenly distributed between two orthogonal polarization components, and the slight intensity variations of different polarization images are mainly due to the airlight. In particular, the brightness of the collected polarized image reaches the brightest and darkest due to the largest and smallest impact of airlight. This polarized characteristic can be an initial cue for the estimate of airlight parameters.
在方程的偏振去雾模型中。 （1）和（2），关键是将空气光与直接传输分开。我们假设直接透射是非偏振的，空气光是部分线偏振的（该模型不考虑圆偏振光）。直接传输的能量均匀分布在两个正交偏振分量之间，不同偏振图像的轻微强度变化主要是由于空气光造成的。特别是，由于空气光的影响最大和最小，采集到的偏振图像的亮度达到最亮和最暗。这种偏振特性可以作为估计空气光参数的初始线索。

In order to get the polarization information of airlight, the “best polarization” and “worst polarization” images (𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$, respectively) need to be acquired from the ORI images. According to the polarization principle [21], the Stokes vectors (𝑆0$S_0$, 𝑆1$S_1$, 𝑆2$S_2$) are obtained and used to calculate the degree of linear polarization (𝑃$P$):

Step 4: NSCT Decomposition of Images
步骤4：图像的NSCT分解

Before processing the different frequency parts separately, the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images need to be decomposed for low- and high-frequency parts. Considering the advantages of NSCT in maintaining the image texture and shaping the feature extraction [27], this paper selected it to decompose the acquired image of the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images. The NSCT consists of two parts: non-subsampled pyramid (NSP) and nonsubsampled directional filter bank (NSDFB). When the NSCT method is used to process the image, a low-pass subband containing the background and ∑𝐽𝑗=12𝑘𝑗${\sum }_{j=1}^J{2}^{k_j}$ high-pass subbands containing target details will be obtained, where 𝑗$j$ represents the number of decomposition layers of NSP and 𝑘$k$ represents the number of directions of NSDFB. Herein, we define the low-pass subband as the low-frequency part and high-pass subbands as the high-frequency part. We set parameters as 𝑗=4$j=4$ and 𝑘=[2,3,3,4]$k=[2,3,3,4]$. Then a low-frequency part and the number of 22+23+23+24=36$2^2+2^3+2^3+2^4=36$ high-frequency parts of the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images are obtained, respectively. In order to clearly show the details, we normalized the high-frequency parts to [0, 255]. (Note: normalization is merely for better display here. This step is not carried out in actual operation).

As shown in Fig. 3, compared with the original image, the decomposed low-frequency image is fuzzy because of the loss of details. However, because the images inserted in the flow chart are compressed, the low-frequency part seems the same as the original image. In order to clearly show the difference between the low-frequency image and the original image, we locally enlarge the white block diagram, while all high-frequency parts images are shown in the form of block diagrams.
如图3所示，与原始图像相比，分解后的低频图像由于细节的丢失而变得模糊。然而，由于流程图中插入的图像被压缩，低频部分看起来与原始图像相同。为了清楚地显示低频图像与原始图像的差异，我们对白色框图进行局部放大，而所有高频部分图像以框图的形式显示。

 

Fig. 5. (a) Raw hazy image (VI = 15, corresponds to the slightly hazy; image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). (b) Four polarization sub-images of 0°, 45°, 90°, and 135° (image resolution, 1224pixels×1024pixels$1224\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 1024\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). (c) ORI images (image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). (d) and (e) Low-frequency parts of the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images, respectively (image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). (f) Dehazing image of the low-frequency parts (image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$).

Download Full Size | PDF
下载全尺寸| PDF

Step 5: Polarization Dehazed for the Low-Frequency Parts
步骤5：低频部分的偏振去雾

Since the low-frequency part of the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images contains haze information, we perform the polarization dehazing process for the low-frequency part in this step. The slight intensity variations of the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images are caused by the polarized characteristic of airlight, which can be initial cues for the estimate of airlight parameters (𝐴∞$A_{\mathrm{\infty }}$ and the degree of linear polarization of the airlight 𝑃𝐴$P_A$). With regard to 𝐴∞$A_{\mathrm{\infty }}$ and 𝑃𝐴$P_A$, we assume a homogeneous distribution of scattering particles in the atmosphere, so 𝐴∞$A_{\mathrm{\infty }}$ and 𝑃𝐴$P_A$ remain constant in the image. Their values can be estimated by using the region of the sky in the image [12]:

In the absolute dehazing image, the intensity values of the sky area are naturally zero. The dehazing image would be the dark vision of the sky area and loses the field depth. These phenomena make the dehazing image strange and artificial. If the dehazing images are meant for human inspection, it is preferable to avoid these phenomena. This is easy to achieve by multiplying 𝑃$P$ with a bias factor 𝜀$\epsilon$ (𝑃→𝜀𝑃$P\to \epsilon P$) [12].

Step 6: Fusion and Enhance for High-Frequency Parts
第6步：高频部分的融合和增强

When decomposing the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images by using the NSCT method, 36 high-frequency parts of the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images are obtained, respectively. In order to get more details of the scene, we choose the Laplace pyramid method [28] to fuse the two groups of high-frequency images. The high-frequency images of preparing fusion are convoluted by the Gaussian kernel (3×3$3\times 3$), and then the convoluted image is downsampled to obtain image 𝐺0$G_0$; we repeat the convolution and downsampling operations on the image 𝐺0$G_0$ to obtain 𝐺1$G_1$, iterating many times to form a Gaussian pyramid 𝐺0$G_0$, 𝐺1,…,𝐺𝑁$G_1,\dots ,G_N$. 𝐺0$G_0$ is the bottom layer of the Gaussian pyramid, and 𝐺𝑁$G_N$ is the top layer of the pyramid. The image 𝐺𝑖$G_i$ is upsampled, and Gaussian convolution is performed to obtain the predicted image; we subtract the predicted image from the image 𝐺𝑖+1$G_{i+1}$ of the Gaussian pyramid to obtain a series difference images of LP1,…,LP𝑁${\mathrm{L}\mathrm{P}}_1,\dots ,{\mathrm{L}\mathrm{P}}_N$ (the Laplace pyramid). The largest absolute value of the window coefficient (the window set to 3×3$3\times 3$) fusion rules is selected for the fused Laplacian pyramid. For the fused Laplacian pyramid, it is recursive from the top layer to the bottom layer, and finally the fused image can be obtained. This Laplacian pyramid method is used to fuse two groups of 36 high-frequency parts one by one. Finally, 36 fused high-frequency parts are obtained.

Step 7: Reconstruction of the Hazy-Free Image
步骤7：重建无雾图像

After polarization dehazing and fusion processing for low- and high-frequency parts, respectively, the hazy-free image with high spatial resolution is reconstructed by inverting the NSCT transform. For the perfect reconstruction, filters should satisfy the Bezout identity and constraint as follows [27,33]:
分别对低频和高频部分进行偏振去雾和融合处理后，通过NSCT变换逆变换来重建高空间分辨率的无雾图像。为了完美重建，滤波器应满足 Bezout 恒等式和约束，如下所示 [27, 33]：

Herein, 𝐺0(𝑧)$G_0(z)$ and 𝐺1(𝑧)$G_1(z)$ are the low- and high-pass synthesis filters, respectively. 𝐻0(𝑧)$H_0(z)$ are 𝐻1(𝑧)$H_1(z)$ is the low- and high-pass decomposition filters of NSPFB, respectively.

 

Fig. 6. (a) Raw hazy image (VI = 15, corresponds to the slightly hazy; image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). Dehazing results by using the (b) PD method (image resolution, 1224pixels×1024pixels$1224\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 1024\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$), (c) PA method (image resolution, 1224pixels×1024pixels$1224\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 1024\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$), and (d) proposed SFDF method (image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). (e)–(h) Respective enlarged local areas corresponding to the white blocks in (a). (I)–(l) Corresponding frequency domain image of (a)–(d).

Download Full Size | PDF

4. EXPERIMENTS AND ANALYSIS

A. Verify the Effectiveness of the Proposed SFDF Algorithm in the Actual Outfield Images

In order to verify the effectiveness of the proposed method, two groups of actual outfield images are selected. The VIs are 15 (corresponding to the slightly hazy) and 4 (corresponding to the dense hazy), respectively.

The raw hazy image and four polarization sub-images before and after ORI upsampling are shown in Figs. 5(a)–5(c). The low-frequency parts of the 𝐼⊥$I^{\mathrm{\perp }}$ and 𝐼‖$I^{\Vert }$ images are shown in Figs. 5(d) and 5(e), respectively. Obviously, although the brightness of the 𝐼⊥$I^{\mathrm{\perp }}$ is darker than that of 𝐼‖$I^{\Vert }$, the contrast is larger. The dehazing image of the low-frequency parts is shown in Fig. 5(f). Compared with Figs. 5(d)–5(f), the feature contrast and layer sense of the dehazing image are greatly improved. The detail contrast of nearby buildings is evidently enhanced, and the mountains and buildings in the distance are also clearly visible. These results show that we have achieved remarkable dehazing processing in the low-frequency part of the image.

 

Fig. 7. (a) Raw hazy image (VI = 4, corresponds to the dense hazy; image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). Dehazing results by using the (b) PD method (image resolution, 1224pixels×1024pixels$1224\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 1024\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$), (c) PA method (image resolution, 1224pixels×1024pixels$1224\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 1024\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$), and (d) proposed SFDF method (image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$). (e)–(h) Respective enlarged local areas corresponding to the white blocks in (a). (I)–(l) Corresponding frequency domain image of (a)–(d).

Download Full Size | PDF

 

Fig. 8. (a) and (b) are the histogram distributions of Figs. 6 and 7, respectively.

Download Full Size | PDF

 

Fig. 9. (a) and (b) are the color hazy image 1 and its dehazing image processed by SFDF method, respectively. (c) and (d) are the color hazy image 2 and its dehazing image processed by SFDF method, respectively. (e)–(h) Frequency domain images according to (a)–(d), respectively. (i) Intensity distributions of RGB color channels of (a) and (b). (j) Intensity distributions of RGB color channels of (c) and (d).

Download Full Size | PDF

The comparison between the hazy and dehazing results are shown in Fig. 6. Although the details of the scene target in Fig. 6(b) are the clearest, the noises are amplified at the same time. The edges of the buildings and the trees behind the buildings are completely submerged in the amplified noise. However, the visual effect of the Fig. 6(d) is better than Fig. 6(b). Besides, the image definition in Fig. 6(d) is significantly higher than that of Fig. 6(c) with compressed noise. The edges of the buildings and the trees behind them are also visible. The distances marked in Fig. 6(e) are measured by a long-range laser rangefinder (China, Shanghai Jiangtu Technology Co., Ltd., LDS-05157; measuring range, 5 km; accuracy, 0.5 m). From the comparison dehazing image by using the SFDF method and hazy image, the sight distance of the scene has been greatly improved and recovered. Besides, the resolution of the reconstructed image in Fig. 6(d) is consistent with that of the raw hazy image. The results demonstrate that the SFDF method restores hazy-free image with advantages in definition, compressed noise, and spatial resolution. The frequency domain images are given in Figs. 6(i)–6(l). Compared with Fig. 6(i), the high-frequency information in Fig. 6(j) is increased, but the overall brightness is also brighter, which indicate that the noises are amplified while the hazes are removed. However, only the high-frequency information increases in Figs. 6(k) and 6(l), and the image in Fig. 6(k) has more high-frequency information. The results prove again that the SFDF method restores hazy-free images with advantages in definition and compressed noise.

Table 1. Quantitative Comparisons Corresponding to Fig. 6

View Table | View all tables in this article

Table 2. Quantitative Comparisons Corresponding to Fig. 7

View Table | View all tables in this article

Figure 7 shows another hazy image (VI = 4, corresponding to dense haze; image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$) and its dehazing results. In the dense hazy weather, only a vague building in the distance of 1423 m can be seen, and the only faint outline of the building in the distance of 2162 m can be distinguished. However, the visual effect of the dehazing image by using the SFDF method is very significant, and the detail visual range is effectively improved from 1423 to 2162 m [as shown in Fig. 7(d)]. By observing the corresponding local enlarged areas, we can see that the window details degraded in dense haze are restored with high quality. In particular, the object features such as street lamps and roof, as well as the window details of buildings in the distance, are distinctly clear in Fig. 7(h).

 

Fig. 10. (a)–(f) Dehazing image of the low-frequency part with bias factor 𝜀=1$\epsilon =1$, 1.4, 1.7, 2, 2.3, 2.6. (g)–(l) Clear image after SFDF polarization dehazing with bias factor 𝜀=1$\epsilon =1$, 1.4, 1.7, 2, 2.3, 2.6 (corresponding to the scene of Fig. 6).

Download Full Size | PDF

The frequency domain images shown in Figs. 7(h)–7(j) indicate that the SFDF method’s result has significantly bright points. It can effectively remove the dense haze and suppress the noise as well in the long range. It is worth noting that the image in Fig. 7(j) has the most bright spots, and the locations of the bright spot are in random distribution, which indicates that the contrast and target information in Fig. 7(b) are not well highlighted. These results show that the SFDF method can be used to remove haze in dense hazy weather.

In order to quantitatively explain the effect of the proposed SFDF method, the histogram distributions of Figs. 6 and 7 are given in Fig. 8. The histogram distribution ranges of the hazy image are narrow and concentrated in the middle of the gray level, which indicates that the contrasts and dynamic ranges of hazy images are unusually small and contain less object information. By comparison, the histogram distribution ranges of the hazy-free images are expanded evenly to the whole gray level range by using the SFDF method. In particular, the histogram distributions of the dense hazy image shown in Fig. 8(b) only concentrate on the gray level of 112–191, while the histogram distribution ranges of the SFDF result uniformly expand to the whole gray level of 0–255. Those comparison results quantitatively verify the effectiveness of the proposed SFDF algorithm.

Tables 1 and 2 show the quantitative comparisons of hazy and dehazing images. It can be seen that almost each index value of the SFDF method is obviously highest. More specifically, the STD, Entropy, C, and AG of the SFDF method in Table 2 increase by 287.96%, 29.84%, 344.16%, and 629.84%, respectively, compared with the hazy image. The numerical results are consistent with the image subjective evaluation of Figs. 6 and 7, and they quantitatively verify that the image quality is significantly improved after processing by the SFDF method.

 

Fig. 11. Relationship between the bias factor 𝜀$\epsilon$ and (a) STD. (b) Entropy. (c) C. (d) AG of the dehazing images (corresponding to the scene of Fig. 6) by using PD and SFDF method, respectively. Inset, the enlarged view of the blue rectangle.

Download Full Size | PDF

B. Verify the Effectiveness of the Proposed SFDF Algorithm in the Color Images

Considering that an object in the atmosphere provides unique spectral the light transmission and the scattering media is deeply affected by wavelength 𝜆$\lambda$, in most practical applications, the processing of color images must be considered. Herein, we study the effectiveness of the proposed SFDF algorithm on the color image. The color hazy images are photographed by another linear polarization camera (U.S.A., FLIR Systems, Inc., BLACKFLY SBFS-U3-51S5PC; image resolution, 2448pixels×2048pixels$2448\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}\times 2048\mathrm{p}\mathrm{i}\mathrm{x}\mathrm{e}\mathrm{l}\mathrm{s}$; chroma, color). By processing the RGB channels independently, the color dehazing image can be obtained. The dehazing results and corresponding histogram distributions of each color channel by using the SFDF method are performed in Fig. 9.

As can be seen from Fig. 9, the visibility of color dehazing images has improved significantly. Compared with the hazy images, the details of the branches or cars in hazy-free images can be easily distinguished, and the outline of the mountains or architecture in the distance can be clearly seen. The visual range of the image is greatly improved. The intensity distributions of the RGB color channels of the hazy images are narrow and concentrated in the middle of the gray level. However, they are basically extended to the range of 0–255 after dehazing. In addition, the frequency domain images in Figs. 9(e)–9(h) show that the dehazing images have more high-frequency components than hazy images. These results verify the effectiveness of the proposed SFDF algorithm in the color images.

5. DISCUSSIONS OF SFDF ALGORITHM

A. Discussion on the Algorithm Robustness

In most of the polarization dehazing methods, the accurate determination of bias factor 𝜀$\epsilon$ is the key to ensure the dehazing effectiveness. In order to explore the influence of bias factor in the SFDF algorithm, we scan carefully the empirical value of 𝜀$\epsilon$ and obtain a series of dehazing images by using the PD and proposed SFDF method.

 

Fig. 12. Interpolated results of ORI and RI algorithm corresponding to Fig. 6. (a)–(d) are the “ground truth image,” the “low-resolution image,” the “RI interpolated image,” and the “ORI interpolated image,” respectively. (e)–(h) are corresponding enlarged images of white block 1 in (a)–(d), respectively. (i)–(l) are corresponding enlarged image of white block 2 in (a)–(d), respectively.

Download Full Size | PDF

Figures 10(a)–10(f) draw the dehazing image of the low-frequency parts with bias factor 𝜀=1$\epsilon =1$, 1.4, 1.7, 2, 2.3, 2.6, respectively. Figures 10(g)–10(l) draw the clear image after SFDF polarization dehazing with bias factor 𝜀=1$\epsilon =1$, 1.4, 1.7, 2, 2.3, 2.6, respectively (corresponding to the scene of Fig. 6). The intensity of the sky region of the low-frequency part comes from the airlight. When the haze is completely removed [𝜀=1$\epsilon =1$, corresponding to Fig. 10(a)], the sky region of the low-frequency part appears thorough black. This phenomenon is exactly consistent with the theory [13]. As the bias factor gradually increases, the haze is gradually retained, so that the sky area becomes brighter and brighter. When the parameter reaches a certain value [this scene corresponds to 𝜀=1.7$\epsilon =1.7$, as shown in Fig. 10(c)], the haze is retained to a suitable extent. In this case, the dehazing effect is the best. When the parameters increase gradually, the haze was gradually over-retained, and the dehazing effect gradually decreased. According to 𝑃→𝜀𝑃$P\to \epsilon P$ and Eqs. (5)–(8), the influence of bias factor directly affects the accuracy of an airtight estimation. Therefore, the dehazing image in the low-frequency parts are obviously affected by this parameter. On the contrary, the clear image after SFDF polarization dehazing has no visual change no matter how the parameter changes, as shown in Figs. 10(g)–10(l). The results of Fig. 10 indicate that the bias factor only affects the low-frequency part and hardly affects the clear image after SFDF polarization dehazing.

 

Fig. 13. Interpolated results of ORI and RI algorithm corresponding to Fig. 7. (a)–(d) are the “ground truth,” the “low-resolution image,” the “RI interpolated image,” and the “ORI interpolated image,” respectively. (e)–(h) are corresponding enlarged image of white block 1 in (a)–(d), respectively. (i)–(l) are corresponding enlarged image of white block 2 in (a)–(d), respectively.

Download Full Size | PDF

As drawn in Fig. 11, in order to quantitatively study the relationship between the bias factor 𝜀$\epsilon$ and dehazing image, the STD, Entropy, C, and AG of the dehazing images by using the PD and proposed SFDF method are calculated, respectively (corresponding to the scene of Fig. 6). When 𝜀$\epsilon$ increases, the overall trend of STD, C, AG curves processed by the PD method is declining with great fluctuation, and the overall trend of Entropycurve presents a parabola. This is because the influence of bias factor directly affects the accuracy of the airtight estimation. However, the index curve of the same scene processed by the SFDF method not only keeps a robustness horizontal trend, but is also always larger than that of the PD method. Those results indicate that no matter what value of 𝜀$\epsilon$ is, the quality of a hazy-free image reconstructed by using the SFDF method would not be affected, and is always better than that of the PD. This is because the factor 𝜀$\epsilon$ only acts on the low-frequency part, and the influence is greatly weakened after fusing the high-frequency parts. This robust advantage without human interaction causes the proposed SFDF algorithm to have a broader prospect in practical application scenarios.

B. Discussion on the Advantages of ORI Algorithm

The image interpolation results generated using the ORI algorithm and RI algorithm [24] are compared and discussed in this section. In order to evaluate the effectiveness of interpolation algorithm, a ground truth image, as shown in Fig. 12(a), is first calculated using the four polarization sub-images of 0°, 45°, 90°, and 135°. The ground truth is then downsampled by a factor of 0.5 to obtain “the low-resolution images,” as shown in Fig. 13(b). The RI and ORI algorithm are then applied on the downsampled low-resolution images to obtain the “RI interpolated image” and the “ORI interpolated image”, respectively.

The image interpolation results of the ORI and RI algorithm corresponding to Fig. 6 are shown in Fig. 12. Figures 12(e)–2(h) and 12(i)–12(l) show the detail of the areas indicated by the numbered white blocks in Fig. 12(a), respectively. Figure 13 shows the image interpolated results generated using the ORI and RI algorithm, corresponding to Fig. 7. It can be seen that, compared with the “low-resolution image,” the “RI interpolated image” and the “ORI interpolated image” effectively improve the clarity of the image. Moreover, compared with the “RI interpolated image,” the “ORI interpolated image” reduces burrs of the building edges, branch details, and lamp outline.

In order to quantitatively compare the performance of the ORI and RI algorithm, we analyze the interpolated results in terms of objective MSE and PSNR. The MSE [24,25] and PSNR [26] results in Tables 3 and 4 illustrate that the performance of the ORI algorithm outperforms that of RI algorithm.

Table 3. MSE Performance Comparison for RI and ORI Algorithm

View Table | View all tables in this article

Table 4. PSNR Performance Comparison for RI and ORI Algorithm

View Table | View all tables in this article

6. CONCLUSION

Polarization dehazing can achieve good reconstruction results of hazy-free images in extreme bad weather. In this work, we present a robust and efficient polarization method to reconstruct the hazy-free image in far-field and dense hazy weather. In addition, we optimize the most competitive interpolation algorithm before dehazing to maintain the image spatial resolution. The experimental results applied to gray scale and color haze images with different hazy conditions highlight that the dehazing effect of the proposed SFDF method is more significant than the existing schemes. In particular, the STD, Entropy, C, and AGof the dehazing image by using the proposed SFDF method increased by 287.96%, 29.84%, 344.16%, and 629.84%, respectively, in the dense haze with a VI of 4. Finally, the algorithm robustness of the bias factor effect is discussed. The numerical calculation results approve that the proposed method can not only restore hazy-free image with advantages in definition, compressed noise, and spatial resolution but can also get rid of the influence of the bias factor. This SFDF method has the potential for applying a far-field and dense hazy image. Considering the complexity and diversity of targets (the polarization state of each target quite different) and evaluating the overall error of the outfield scene, we chose the average filter (𝑆0$S_0$) in this paper. However, it may be misrepresented for an outfield scene with a high degree of polarization and objects directly illuminated by the Sun [34]. We will optimize the proposed SFDF method in future research so that it is also applicable in the outfield scene with a high degree of polarization and objects directly illuminated by the Sun.