Text-Based Traffic Panels Detection using the Tiny YOLOv3 Algorithm
by Saba Kheirinejad 1,* 
, Noushin Riahi 2, Reza Azmi 2
1 Center for Ubiquitous Computing, University of Oulu, Oulu, Finland
2 Department of Computer Engineering, Faculty of Engineering, Alzahra University, Tehran, Iran
* Author to whom correspondence should be addressed.
Journal of Engineering Research and Sciences, Volume 1, Issue 3, Page # 68-80, 2022; DOI: 10.55708/js0103008
Keywords: Intelligent transportation system, Deep learning, Convolutional neural networks, Tiny YOLOv3, Traffic signs, Traffic panels
Received: 16 January 2022, Revised: 04 March 2022, Accepted: 11 March 2022, Published Online: 17 March 2022
AMA Style
Kheirinejad S, Riahi N, Azmi R. Text-based traffic panels detection using the tiny yolov3 algorithm. Journal of Engineering Research and Sciences. 2022;1(3):68-80. doi:10.55708/js0103008
Chicago/Turabian Style
Kheirinejad, Saba, Noushin Riahi, and Reza Azmi. “Text-Based Traffic Panels Detection Using the Tiny yolov3 Algorithm.” Journal of Engineering Research and Sciences 1, no. 3 (2022): 68–80. https://doi.org/10.55708/js0103008.
IEEE Style
S. Kheirinejad, N. Riahi, and R. Azmi, “Text-based traffic panels detection using the tiny yolov3 algorithm,” Journal of Engineering Research and Sciences, vol. 1, no. 3, pp. 68–80, 2022.
339 Downloads
Abstract
Full Text
References
Cited By
Metrics
Abstract
Lately, traffic panel detection has been engrossed by academia and industry. This study proposes a new categorization method for traffic panels. The traffic panels are classified into three classes: symbol-based, text-based, and supplementary/additional traffic panels. Although few types of research have investigated text-based traffic panels, this type is considered in detail in this study. However, there are many challenges in this type of traffic panel, such as having different languages in different countries, their similarity with other text panels, and the lack of suitable quality datasets. The panels need to be detected first to obtain a reasonable accuracy in recognizing the text. Since there are few public text-based traffic panels datasets, this study gathered a novel dataset for the Persian text-based traffic panels all over the streets of Tehran-Iran. This dataset includes two collections of images. The first collection has 9294 images, and the latter has 3305 images. The latter dataset is more monotonous than the first one. Thus, the latter is utilized as the main dataset, and the first is used as an additional dataset. To this end, the algorithm uses the additional dataset for pre-training and the main datasets for training the network. The tiny YOLOv3 algorithm that is fast and has low complexity compared to the YOLOv3 is used for pre-training, training, and testing the data to examine the utility and advantages of the data. The K-fold cross-validation procedure is used to estimate the model’s skill on the new data. It achieves 0.973 for Precision, 0.945 for Recall, and 0.955 for Fmeasure.
Full Text
References
- A. Gudigar, S. Chokkadi and U. Raghavendra, “A review on automatic detection and recognition of traffic sign,” Multimedia Tools and Applications, vol. 75, no. 1, pp. 333 -364, 2016.
- J. Stallkamp, M. Schlipsing, J. Salmen and C. Igel, “The German Traffic Sign Recognition Benchmark: A multi-class classification competition,” in International joint conference on neural networks (IJCNN), 2016.
- F. Zaklouta and B. Stanciulescu, “Real-time traffic sign recognition in three stages,” Robotics and autonomous systems, vol. 62, no. 1, pp. 16-24, 2014.
- J. Greenhalgh and M. Mirmehdi, “Real-Time Detection and Recognition of Road Traffic Signs,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 4, pp. 1498 — 1506, 2012.
- S. Houben, J. Stallkamp, J. Salmen, M. Schlipsing and C. Igel, “Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark,” The 2013 international joint conference on neural networks (IJCNN), pp. 1-8, 2013.
- R. Timofte, K. Zimmermann and L. Van Gool, “Multi-view traffic sign detection, recognition, and 3D localisation,” Machine vision and applications, vol. 25, no. 3, pp. 633 – 647, 2014.
- Y. Yang, H. Luo, H. Xu and F. Wu, “Towards Real-Time Traffic Sign Detection and Classification,” IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 7, pp. 2022 – 2031, 2016.
- Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li and S. Hu, “Traffic-Sign Detection and Classification in the Wild,” in Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
- E. Ayoub, M. El Ansari and I. El Jaafari, “Traffic sign detection and recognition based on random forests,” Applied Soft Computing, vol. 46, pp. 805 — 815, 2016.
- Y. Yuan, Z. Xiong and Q. Wang, “An Incremental Framework for Video-Based Traffic Sign Detection, Tracking, and Recognition,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 7, pp. 1918 – 1929, 2017.
- U. Kamal, T. Islam Tonmoy, S. Das and M. K. Hasan, “Automatic Traffic Sign Detection and Recognition Using SegU-Net and a Modified Tversky Loss Function With L1-Constraint,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 4, pp. 1467 – 1479, 2020.
- C. Gamez Serna and Y. Ruichek, “Traffic Signs Detection and Classification for European Urban Environments,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 10, pp. 4388-4399, 2020.
- L. Wei, C. Xu, S. Li and X. Tu, “Traffic Sign Detection and Recognition Using Novel Center-Point Estimation and Local Features,” IEEE Access, vol. 8, pp. 83611-83621, 2020.
- D. Tabernik and D. Skoˇcaj, “Deep Learning for Large-Scale Traffic-Sign Detection and Recognition,” IEEE Transactions on Intelligent Transportation Systems , vol. 21, no. 4, pp. 1427 – 1440, 2020.
- F. Almutairy, T. Alshaabi, J. Nelson and S. Wshah, “ARTS: Automotive Repository of Traffic Signs for the United States,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 1, pp. 457 – 465, 2021.
- C. Dewi, R.-C. Chen, Y.-T. Liu, X. Jiang and K. D. Hartomo, “Yolo V4 for Advanced Traffic Sign Recognition With Synthetic Training Data Generated by Various GAN,” IEEE Access, vol. 9, pp. 97228-97242, 2021.
- X. Liu, X. Jiang, H. Hu, R. Ding, H. Li and C. Da, “Traffic Sign Recognition Algorithm Based on Improved YOLOv5s,” in 2021 International Conference on Control, Automation and Information Sciences (ICCAIS), 2021.
- Y. Sun and L. Chen, “Traffic Sign Recognition Based on CNN and Twin Support Vector Machine Hybrid Model,” Journal of Applied Mathematics and Physics, vol. 9, no. 12, pp. 3122-3142, 2021.
- Z. Wang, J. Wang, Y. Li and S. Wang, “Traffic Sign Recognition With Lightweight Two-Stage Model in Complex Scenes,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 2, pp. 1121-1131, 2022.
- S. Ahmed, U. Kamal and M. K. Hasan, “DFR-TSD: A Deep Learning Based Framework for Robust Traffic Sign Detection Under Challenging Weather Conditions,” IEEE Transactions on Intelligent Transportation Systems, pp. 1-13, 2021.
- Z. Nadeem, A. Samad, Z. Abbas and J. Massod, “Pakistani traffic-sign recognition using transfer learning,” in 2018 International Conference on Computing, Electronic and Electrical Engineering (ICE Cube), 2022.
- Z. Liu, M. Qi, C. Shen, Y. Fang and X. Zhao, “Cascade saccade machine learning network with hierarchical classes for traffic sign detection,” Sustainable Cities and Society, vol. 67, p. 102700, 2021.
- J. Wan, W. Ding, H. Zhu, M. Xia, Z. Huang, L. Tian, Y. Zhu and H. Wang, “An efficient small traffic sign detection method based on yolov3,” Journal of Signal Processing Systems, vol. 93, no. 8, pp. 899–911, 2021.
- D. Nienhüser, T. Gumpp, J. M. Zöllner and K. Natroshvili, “Fast and reliable recognition of supplementary traffic signs,” in Intelligent Vehicles Symposium, 2010.
- A. S. Puthon, F. Moutarde and F. Nashashibi, “Recognition of supplementary signs for correct interpretation of traffic signs,” in Intelligent Vehicles Symposium (IV), 2013.
- T. Wenzel, S. Brueggert and J. Denzler, “Additional Traffic Sign Detection — A Comparative Study,” in 2015 IEEE 18th International Conference on Intelligent Transportation Systems, 2015.
- T. Wenzel, S. Brueggert and J. Denzler, “Towards unconstrained content recognition of additional traffic signs,” in 2017 IEEE Intelligent Vehicles Symposium (IV), 2017.
- T. Wenzel, S. Brueggert and J. Denzler, “Additional traffic sign detection using learned corner representations,” in Intelligent Vehicles Symposium (IV), 2016.
- T. Weber, E. Ercelik, M. Ebert and A. Knoll, “Recognition amp; Evaluation of Additional Traffic Signs on the example of ’80 km/h when wet’,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC), 2019.
- O. Hamdoun, A. Bargeton, F. Moutarde, B. Bradai and L. Chanussot, “Recognition of End-of-Speed-Limit and Supplementary Signs for Improved Speed Limit Support,” in 15th World Congress on Intelligent Transport Systems and ITS America’s 2008 Annual Meeting, 2008.
- X. Rong, C. Yi and Y. Tian, “Recognizing text-based traffic guide panels with cascaded localization network,” in European Conference on Computer Vision, 2016.
- O. de , G. Henrique, F. A. da Silva, D. R. Pereira, L. L. de Almeida, A. O. Artero, A. F. Bonora and V. H. C. de Albuquerque, “Automatic Detection and Recognition of Text-Based Traffic Signs from images,” IEEE Latin America Transactions, vol. 16, no. 12, pp. 2947 – 2953, 2018.
- H. Eun, J. Kim, J. Kim and C. Kim, “Fast Korean Text Detection and Recognition in Traffic Guide Signs,” in 2018 IEEE Visual Communications and Image Processing (VCIP), 2018.
- R. Jain and D. Gianchandani, “A Hybrid Approach for Detection and Recognition of Traffic Text Sign using MSER and OCR,” in 2018 2nd International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), 2018 2nd International Conference on, 2018.
- W. Wu, X. Chen and J. Yang, “Detection of text on road signs from video,” IEEE Transactions on Intelligent Transportation Systems, vol. 6, no. 4, pp. 378 — 390, 2005.
- Á. González, L. M. Bergasa, J. J. Yebes and J. Almazán, “Text recognition on traffic panels from street-level imagery,” in Intelligent Vehicles Symposium, 2012.
- J. Greenhalgh and . M. Mirmehdi, “Recognizing Text-Based Traffic Signs,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 13, pp. 1360 — 1369, 2015.
- A. Gonzalez, L. M. Bergasa and J. J. Yebes, “Text Detection and Recognition on Traffic Panels From Street-Level Imagery Using Visual Appearance,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 1, pp. 228 — 238, 2014.
- J. Guo, R. You and L. Huang, “Mixed Vertical-and-Horizontal-Text Traffic Sign Detection and Recognition for Street-Level Scene,” IEEE Access, vol. 8, pp. 69413-69425, 2020.
- K. S. Boujemaa, M. Akallouch, I. Berrada, K. Fardousse and A. Bouhoute, “ATTICA: A Dataset for Arabic Text-Based Traffic Panels Detection,” IEEE Access, vol. 9, pp. 93937-93947, 2021.
- R. Bagi, T. Dutta, N. Nigam, D. Verma and H. P. Gupta, “Met-MLTS: Leveraging Smartphones for End-to-End Spotting of Multilingual Oriented Scene Texts and Traffic Signs in Adverse Meteorological Conditions,” IEEE Transactions on Intelligent Transportation Systems, pp. 1-10, 2021.
- H. Luo, Y. Yang, B. Tong, F. Wu and B. Fan, “Traffic sign recognition using a multi-task convolutional neural network,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 4, pp. 1100-1111, 2017.
- X. Peng, X. Chen and C. Liu, “Real-time Traffic Sign Text Detection Based on Deep Learning,” in Materials Science and Engineering, 2020.
- J. Zhang, Z. Xie, J. Sun, X. Zou and J. Wang, “A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection,” IEEE Access, vol. 8, pp. 29742-29754, 2020.
- Korghond, N. Khazaee and R. Safabakhsh, “AUT-UTP: Urban traffic panel detection and recognition dataset,” in 24th Iranian Conference on Electrical Engineering (ICEE), 2016.
- S. Kheirinejad, N. Riahi and R. Azmi, “Persian Text Based Traffic sign Detection with Convolutional Neural Network: A New Dataset,” in 10th International Conference on Computer and Knowledge Engineering (ICCKE), 2020.
- A. Soetedjo and K. Yamada, “A new approach on red color thresholding for traffic sign recognition system,” Journal of Japan Society for Fuzzy Theory and Intelligent Informatics, vol. 19, no. 5, pp. 457-465, 2007.
- A. Mogelmose, M. M. Trivedi and T. B. Moeslund, “Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 4, pp. 1484 – 1497, 2012.
- S. Maldonado-Bascón, S. Lafuente-Arroyo, P. Gil-Jimenez, H. Gómez-Moreno and F. López-Ferreras, “Road-Sign Detection and Recognition Based on Support Vector Machines,” IEEE Transactions on Intelligent Transportation Systems, vol. 8, no. 2, pp. 264 -278, 2007.
- Y. Li, S. Pankanti and W. Guan, “Real-time traffic sign detection: an evaluation study,” in 2010 20th International Conference on Pattern Recognition, 2010.
- C. G. Kiran, L. V. Prabhu and K. Rajeev, “raffic sign detection and pattern recognition using support vector machine,” in 2009 Seventh International Conference on Advances in Pattern Recognition, 2009.
- S. M. Bascón, J. A. Rodríguez, S. L. Arroyo, A. F. Caballero and F. López-Ferreras, “An optimization on pictogram identification for the road-sign recognition task using SVMs,” Computer Vision and Image Understandin, vol. 114, no. 3, pp. 373-383, 2010.
- P. Gil-Jimenez, H. Gomez-Moreno, P. Siegmann, S. Lafuente-Arroyo and S. Maldonado-Bascon, “Traffic sign shape classification based on Support Vector Machines and the FFT of the signature of blobs,” in 2007 IEEE Intelligent Vehicles Symposium, 2007.
- I. M. Creusen, R. G. Wijnhoven, E. Herbschleb and P. H. N.de, “Color exploitation in hog-based traffic sign detection,” in 2010 IEEE International Conference on Image Processing, 2010.
- Y. Wu, Y. Liu, J. Li, H. Liu and X. Hu, “Traffic sign detection based on convolutional neural networks,” in International joint conference on neural networks (IJCNN), 2013.
- R. Qian, B. Zhang, Y. Yue, Z. Wang and F. Coenen, “Robust chinese traffic sign detection and recognition with deep convolutional neural network,” in 11th International Conference on Natural Computation (ICNC), 2015.
- Z. Di, J. Zhang, D. Zhang, M. Bao, J. Cheng and K. Tang, “Traffic sign detection based on cascaded convolutional neural networks,” in 17th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD), 2016.
- L. Abdi and A. Meddeb, “Deep learning traffic sign detection, recognition and augmentation,” in Proceedings of the Symposium on Applied Computing, 2017.
- Z. Wang, J. Wang, Y. Li and S. Wang, “Traffic Sign Recognition With Lightweight Two-Stage Model in Complex Scenes,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 2, pp. 1121-1131, 2022.
- A. V. Reina, R. L. Sastre, S. L. Arroyo and P. G. Jiménez, “Adaptive traffic road sign panels text extraction,” in International Conference on Signal Processing, Robotics and Automation, 2006.
- A. Khodayari and M. Yousefi, “Design an Intelligent Driver Assistance System Based On Traffic Sign Detection with Persian Context,” International Journal of Automotive Engineering, vol. 6, no. 2, pp. 2138 -2147, 2016.
- Y. Zhu, M. Liao, M. Yang and W. Liu, “Cascaded Segmentation-Detection Networks for Text-Based Traffic Sign Detection,” IEEE Transaction Intelligent Transportation Systems, vol. 19, no. 1, pp. 209 — 219, 2018.
- H. Zhang, L. Qin, J. Li, Y. Guo, Y. Zhou, J. Zhang and Z. Xu, “Real-Time Detection Method for Small Traffic Signs Based on Yolov3,” IEEE Access, vol. 8, pp. 64145-64156, 2020.
- Z. Yi, S. Yongliang and Z. Jun, “An improved tiny-yolov3 pedestrian detection algorithm,” Optik, vol. 183, pp. 17 -23, 2019.
- W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, . C. Y. Fu and A. C. Berg, “Ssd: Single shot multibox detector,” in European conference on computer vision, 2016.
- X. Wang, A. Shrivastava and A. Gupta, “A-fast-rcnn: Hard positive generation via adversary for object detection,” in IEEE Conference on Computer Vision and Pattern Recognition, 2017.
- J. Redmon, S. Divvala, R. Girshick and A. Farhadi, “You only look once: Unified, real-time object detection,” in IEEE conference on computer vision and pattern recognition, 2016.
- J. Redmon and A. Farhadi, “YOLO9000: better, faster, stronger,” in IEEE conference on computer vision and pattern recognition, 2017.
- J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” in arXiv preprint arXiv, 2018.
- A. Bochkovskiy, C.-Y. Wang and H.-Y. M. Liao, “Yolov4: Optimal speed and accuracy of object detection,” arXiv preprint arXiv:2004.10934, 2020.
- J. Nelson and J. Solawetz, “Responding to the Controversy about YOLOv5,” https://blog.roboflow.com/yolov4-versus-yolov5/, 2020.
- M. Andreas, . M. M. Trivedi and . T. B. Moeslund, “Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey,” IEEE Transactions on Intelligent Transportation Systems, pp. 1484 — 1497, 2012.
- X. Baró, S. Escalera, J. Vitria, O. Pujol and P. Radeva, “Traffic Sign Recognition Using Evolutionary Adaboost Detection and Forest-ECOC Classification,” IEEE Transactions on Intelligent Transportation Systems, vol. 10, no. 1, pp. 113 – 126, 2009.
- M. Benallal and J. Meunier, “Real-time color segmentation of road signs,” in CCECE 2003-Canadian Conference on Electrical and Computer Engineering. Toward a Caring and Humane Technology, 2003.
- G. H. de Oliveira, F. A. da Silva, D. R. Pereira, L. L. de Almeida, A. O. Artero, A. F. Bonora and V. H. C. de Albuquerque, “Automatic Detection and Recognition of Text-Based Traffic Signs from images,” IEEE Latin America Transactions, vol. 16, no. 12, pp. 2947-2953, 2018.
- A.-S. Puthon, F. Moutarde and F. Nashashibi, “Recognition of supplementary signs for correct interpretation of traffic signs,” in 2013 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops, 2013.
- A. Ruta, F. Porikli, S. Watanabe and Y. Li, “In-vehicle camera traffic sign detection and recognition,” Machine Vision and Applications, vol. 22, no. 2, pp. 359 – 375, 2011.
- T. Wenzel, S. Brueggert and J. Denzler, “Towards unconstrained content recognition of additional traffic signs,” in 2017 IEEE Intelligent Vehicles Symposium (IV), 2017.
- M. Zadeh, M. T. Kasvand and C. Y. Suen, “Localization and recognition of traffic signs for automated vehicle control systems,” Intelligent Transportation Systems, vol. 3207, pp. 272-282, 1998.
Cited By
Metrics
