Classification of Rethinking Hyperspectral Images using 2D and 3D CNN with Channel and Spatial Attention: A Review
by Muhammad Ahsan Aslam 1,* 
, Muhammad Tariq Ali 2, Sunwan Nawaz 1, Saima Shahzadi 3, Muhammad Ali Fazal 2
1 Institute of Computer Science, Khwaja Fareed University of Engineering & Information Technology, RYK, 64200, Pakistan
2 IT Department, Khwaja Fareed University of Engineering & Information Technology, RYK, 64200, Pakistan
3 Computer Science Department, University of Agriculture, Faisalabad, 38000, Pakistan
* Author to whom correspondence should be addressed.
Journal of Engineering Research and Sciences, Volume 2, Issue 4, Page # 22-32, 2023; DOI: 10.55708/js0204003
Keywords: Hyperspectral, Image classification, Deep learning, Convolutional neural network, Feature extraction, Machine Learning
Received: 09 September 2022, Accepted: 31 March 2023, Published Online: 28 April 2023
APA Style
Chicago/Turabian Style
IEEE Style
173 Downloads
Abstract
Full Text
References
Cited By
Metrics
Abstract
It has been demonstrated that 3D Convolutional Neural Networks (CNN) are an effective technique for classifying hyperspectral images (HSI). Conventional 3D CNNs produce too many parameters to extract the spectral-spatial properties of HSIs. A channel service module and a spatial service module are utilized to optimize characteristic maps and enhance sorting performance in order to further study discriminating characteristics. In this article, evaluate CNN’s methods for hyperspectral image categorization (HSI). Examined the replacement of traditional 3D CNN with mixed feature maps by frequency to lessen spatial redundancy and expand the receptive field. Evaluates several CNN stories that use image classification algorithms, elaborating on the efficacy of these approaches or any remaining holes in methods. How do improve those gaps for better image classification?
Full Text
References
- Yin, J.; Qi, C.; Chen, Q.; Qu, J. Spatial-Spectral Network for Hyperspectral Image Classification: A 3- D CNN and Bi-LSTM Framework. Remote Sens. 2021, 13, 2353, org/10.3390/rs13122353.
- Yan, H.; Wang, J.; Tang, L.; Zhang, E.; Yan, K.; Yu, ; Peng, J. A 3D Cascaded Spectral–Spatial Element Attention Network for Hyperspectral Image Classification. Remote Sens. 2021, 13, 2451, doi.org/10.3390/rs13132451.
- Pu, S.; Wu, Y.; Sun, X.; Sun, X. Hyperspectral Image Classification with Localized Graph Convolutional Filtering. Remote Sens. 2021, 13, 526, org/10.3390/rs13030526.
- Hong, D.; Yokoya, N.; Chanussot, J.; Zhu, X.X. An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing. IEEE Trans. Image Process. 2019, 28, 1923–1938, DOI: 1109/TIP.2018.2878958.
- Gudmundsson, Steinn, Thomas Philip Runarsson, and Sven Sigurdsson. “Support vector machines and dynamic time warping for time series.” 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). IEEE, 2008.Wang, Q.; Meng, Z.; Li, X. Locality adaptive discriminant analysis for spectral–spatial classification of hyperspectral images. IEEE Geosci. Remote Sens. Lett. 2017, 14, 2077–2081, DOI: 1109/IJCNN.2008.4634188.
- Wang, Q.; Lin, J.; Yuan, Y. Salient band selection for hyperspectral image classification via manifold ranking. IEEE Trans. Neural Netw. Learn. Syst. 2016, 27, 1279–1289, DOI: 1109/TNNLS.2015.2477537.
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp.770–778, DOI: 1109/CVPR.2016.90.
- Sharma, V.; Diba, A.; Tuytelaars, T.; Van Gool, L. Hyperspectral CNN for Image Classification & Band Selection, with Application to Face Recognition; KU Leuven, ESAT: Leuven, Belgium, 2016, org/10.3390/pr11020435.
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: New York, NY, USA, 2012; pp. 1097–1105, org/10.1145/3065386.
- Goodfellow, Ian, Yoshua Bengio, and Aaron Deep learning. MIT press, 2016.
- Albawi, Saad, Tareq Abed Mohammed, and Saad Al-Zawi. “Understanding of a convolutional neural network.” 2017 International Conference on Engineering and Technology (ICET). Ieee, 2017, DOI:1109/ICENGTECHNOL.2017.8308186.
- Schmidhuber, “Deep Learning in neural networks: An overview,” Neural Networks, vol. 61. pp. 85–117, 2015, doi.org/10.1016/j.neunet.2014.09.003.
- Zhao, H. Lu, S. Chen, J. Liu, and D. Wu, “Convolutional neural networks for time series classification,” J. Syst. Eng. Electron., vol. 28, no. 1, pp. 162–169, 2017, DOI: 10.21629/JSEE.2017.01.18.
- He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE International Conference on Computer Vision, 2016, vol. 11–18–Dece, pp. 1026–1034, doi.org/10.48550/arXiv.1502.01852.
- Aslam, M. A., Sarwar, M. U., Hanif, M. K., Talib, R., & Khalid, U. (2018). Acoustic classification using deep learning. Int. J. Adv. Comput. Sci. Appl, 9(8), 153-159, (DOI) : 14569/IJACSA.2018.090820.
- Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA, 21–26 July 2017; pp. 2261–2269, DOI: 1109/CVPR.2017.243.
- Hu, J., Kuang, Y., Liao, B., Cao, L., Dong, S., & Li, (2019). A multichannel 2D convolutional neural network model for task-evoked fMRI data classification. Computational intelligence and neuroscience, 2019, doi.org/10.1155/2019/5065214.
- Liu, B.; Yu, X.; Zhang, P.; Tan, X.; Yu, A.; Xue, Z. A semi-supervised convolutional neural network for
hyperspectral image classification. Remote Sens. Lett. 2017, 8, 839–848, org/10.1080/2150704X.2017.1331053. - Yang, X.; Ye, Y.; Li, X.; Lau, R.Y.; Zhang, X.; Huang, Hyperspectral Image Classification with Deep
Learning Models. IEEE Trans. Geosci. Remote Sens. 2018, 56, 5408–5423, DOI: 10.1109/TGRS.2018.2815613. - Hamida, A.B.; Benoit, A.; Lambert, P.; Amar, C.B. 3- D Deep Learning Approach for Remote Sensing Image Classification. IEEE Trans. Geosci. Remote Sens. 2018, 56, 4420–4434, DOI: 1109/TGRS.2018.2818945.
- Li, Y.; Zhang, H.; Shen, Q. Spectral–spatial classification of hyperspectral imagery with 3D convolutional neural network. Remote Sens. 2017, 9, 67, org/10.3390/rs9010067.
- He, M.; Li, B.; Chen, H. Multi-scale 3D deep convolutional neural network for hyperspectral image
In Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 17–20 September 2017; pp.3904–3908, DOI: 10.1109/ICIP.2017.8297014. - Luo, Y.; Zou, J.; Yao, C.; Zhao, X.; Li, T.; Bai, G. HSI-CNN: A Novel Convolution Neural Network for
Hyperspectral Image. In Proceedings of the 2018 International Conference on Audio, Language and Image Processing, Shanghai, China, 16–17 July 2018; 464–469, DOI: 10.1109/ICALIP.2018.8455251. - Fan, J.; Tan, H.L.; Toomik, M.; Lu, S. Spectral–spatial hyperspectral image classification using
super-pixel-based spatial pyramid representation. In Proceedings of the Image and Signal Processing
for Remote Sensing XXII, Edinburgh, UK, 26–29 September 2016; Volume 10004, pp.315–321, org/10.3390/rs12122033. - Yang, X., Zhang, X., Ye, Y., Lau, R. Y., Lu, S., Li, , & Huang, X. (2020). Synergistic 2D/3D convolutional neural network for hyperspectral image classification. Remote Sensing, 12(12), 2033, doi.org/10.3390/rs12122033.
- Wang, Q.; He, X.; Li, X. Locality and structure regularized low rank representation for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2018, 57, 911–923, DOI: 1109/TGRS.2018.2862899.
- Yuan, Y.; Feng, Y.; Lu, X. Projection-Based NMF for Hyperspectral Unmixing. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 2632–2643, DOI: 1109/JSTARS.2015.2427656.
- Li, W.; Du, Q.; Zhang, B. Combined sparse and collaborative representation for hyperspectral target Pattern Recognit. 2015, 48, 3904–3916, doi.org/10.1016/j.patcog.2015.05.024.
- Pan, B.; Shi, Z.; Xu, X. MugNet: Deep learning for hyperspectral image classification using limited samples. ISPRS J. Photogramm. Remote Sens. 2018, 145, 108–119, org/10.1016/j.isprsjprs.2017.11.003.
- Zhou, S.; Xue, Z.; Du, P. Semisupervised Stacked Autoencoder with Cotraining for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2019, 57, 3813–3826, DOI: 1109/TGRS.2018.2888485.
- Ghamisi, P.; Plaza, J.; Chen, Y.; Li, J.; Plaza, A.J. Advanced Spectral Classifiers for Hyperspectral Images: A review. IEEE Geosci. Remote Sens. Mag. 2017, 5, 8–32, DOI: 1109/MGRS.2016.2616418.
- Cao, K. Wang, G. Han, J. Yao, and A. Cichocki, “A robust pca approach with noise structure learning and spatial–spectral low-rank modeling for hyperspectral image restoration,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 11, no. 10, pp.3863–3879, 2018, DOI: 10.1109/JSTARS.2018.2866815.
- Hong, N. Yokoya, J. Chanussot, J. Xu, and X. X. Zhu, “Joint and progressive subspace analysis (jpsa) with spatial-spectral manifold alignment for semi-supervised hyperspectral dimensionality reduction,”
IEEE Trans. Cybern., vol. 51, no. 7, pp. 3602–3615, 2021, DOI: 10.1109/TCYB.2020.3028931. - Luo, T. Guo, Z. Lin, J. Ren, and X. Zhou, “Semisupervised hypergraph discriminant learning for dimensionality reduction of hyperspectral image,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 13, pp. 4242–4256, 2020, DOI: 10.1109/JSTARS.2020.3011431.
- Hong, X. Wu, P. Ghamisi, J. Chanussot, N. Yokoya, and X. X. Zhu, “Invariant attribute profiles: A spatial-frequency joint feature extractor for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sens., vol. 58, no. 6, pp. 3791–3808, 2020, doi: 10.1109/TGRS.2019.2957251.
- Rasti, D. Hong, R. Hang, P. Ghamisi, X. Kang, J. Chanussot, and J. Benediktsson, “Feature extraction for hyperspectral imagery: The evolution from shallow to deep: Overview and toolbox,” IEEE Geosci. Remote Sens. Mag., vol. 8, no. 4, pp. 60–88, 2020, DOI: 10.1109/MGRS.2020.2979764.
- Hamidian, S., Sahiner, B., Petrick, N., & Pezeshk, A. (2017, March). 3D convolutional neural network for automatic detection of lung nodules in chest CT. In Medical Imaging 2017: Computer-Aided Diagnosis (Vol. 10134, p. 1013409). International Society for Optics and Photonics, DOI: 1117/12.2255795.
- Paoletti, M. E., & Haut, J. M. (2021). Adaptable Convolutional Network for Hyperspectral Image Classification. Remote Sensing, 13(18), 3637, Dorg/10.3390/rs13183637.
- Le, H.; Borji, A. What are the receptive, effective receptive, and projective fields of neurons in convolutional neural networks? arXiv 2017, arXiv:1705.07049, org/10.48550/arXiv.1705.07049.
- Gao, H.; Zhu, X.; Lin, S.; Dai, J. Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation. arXiv 2019 ,arXiv:1910.02940, org/10.48550/arXiv.1910.02940.
- Araujo, A.; Norris, W.; Sim, J. Computing receptive fields of convolutional neural networks. Distill 2019, 4, e21, DOI: 23915/distill.00021.
- Xu, X., Li, J., & Plaza, A. (2016, July). Fusion of hyperspectral and LiDAR data using morphological component analysis. In 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), (pp.3575-3578). DOI: 1109/IGARSS.2016.7729926.
- Wang, W., Dou, S., & Wang, S. (2019). Alternately updated spectral–spatial convolution network for the classification of hyperspectral images. Remote Sensing, 11(15), 1794, org/10.3390/rs11151794.
- Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826), org/10.48550/arXiv.1512.00567.
- Zhu, L., Chen, Y., Ghamisi, P., & Benediktsson, J. A. (2018). Generative adversarial networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 56(9), 5046- 5063, doi:1109/TGRS.2018.2805286.
- Ghaderizadeh, S., Abbasi-Moghadam, D., Sharifi, A., Zhao, N., & Tariq, A. (2021). Hyperspectral image classification using a hybrid 3D-2D convolutional neural networks. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14, 7570-7588, DOI:1109/JSTARS.2021.3099118.
- Paoletti, J. Haut, J. Plaza, and A. Plaza, “Deep&dense convolutional neural network for hyperspectral image classification,” Remote Sens., vol. 10, 2018, Art. no. 1454, doi.org/10.3390/rs10091454.
- Gao, S. Lim, and X. Jia, “Hyperspectral image classification using convolutional neural networks and multiple feature learning,” Remote
Sens., vol. 10, no. 2, 2018, Art. no. 299, doi:10.3390/rs10020299. - Farooq, Umar, and Robert B. Bass. “Frequency Event Detection and Mitigation in Power Systems: A Systematic Literature Review.” IEEE Access 10 (2022), DOI: 10.1109/ACCESS.2022.3180349.
- Ahmad, M. F., et al. “Tracking system using artificial neural network for FPGA cart follower.” Journal of Physics: Conference Series. Vol. 1874. No. 1. IOP Publishing, 2021, DOI 10.1088/1742-6596/1874/1/01.
Cited By
Metrics
