对抗网络实现单幅RGB重建高光谱图像

Hyperspectral images reconstruction using adversarial networks from single RGB image

  • 摘要: 高光谱成像能够提供比普通RGB图像更全的光谱信息,在监测自然环境变化、农业植被土壤分类等具有广泛的应用。从单幅RGB图像重建高光谱信息是严重欠约束问题,传统重建算法需要增加光学组件或已知相机光谱响应,在实际应用中往往不能满足要求。针对此问题,提出一种端到端对抗生成网络,设计一种改进残差结构作为对抗网络的基本模块,使用多尺度特征金字塔融合局部和全局特征并捕获像素空间上下文信息;提出了新的WNet网络,利用局部边缘图像引导模型学习到高频信号,进一步提升了高光谱重建精度。实验结果表明:无论是高光谱图像数据合成的RGB图像以及普通相机拍摄的真实RGB图像,所提方法的高光谱重建效果在定量和定性评价指标上均优于已有的代表性方法,对比稀疏字典算法,均方误差和相对均方误差分别降低了45%和50%。

     

    Abstract: Hyperspectral imaging can provide more spectral information than an ordinary RGB camera. The spectral information has been beneficial to numerous applications, such as monitoring natural environment changes and classifying plants and soils in agriculture. The hyperspectral images reconstruction from a single RGB image is severely unconstrained problem. Previous methods need additional components or the spectral response by commercial camera. An end-to-end conditional generative adversarial network was proposed with modified residual network as backbone. The feature pyramid was used inside the network and a scale attention module was designed to fuse local and global information. In order to provide more accurate solution, another distinct architecture was proposed, named WNet. Experiments manifested the superiority of the proposed method over other representative methods in terms of quality and quantity. Experiments used both synthesized RGB images using public hyperspectral data and real-world image by ordinary camera demonstrate that proposed method outperforms the state-of-the-art. The WNet drops 45% and 50% in terms of RMSE and relative RMSE on the ICVL dataset than sparse coding.

     

/

返回文章
返回