Deep convolution neural network model in problem of crack segmentation on asphalt images
https://doi.org/10.23947/1992-5980-2019-19-1-63-73
Abstract
Introduction. Early defect illumination (cracks, chips, etc.) in the high traffic load sections enables to reduce the risk under emergency conditions. Various photographic and video monitoring techniques are used in the pavement managing system. Manual evaluation and analysis of the data obtained may take unacceptably long time. Thus, it is necessary to improve the conditional assessment schemes of the monitor objects through the autovision.
Materials and Methods. The authors have proposed a model of a deep convolution neural network for identifying defects on the road pavement images. The model is implemented as an optimized version of the most popular, at this time, fully convolution neural networks (FCNN). The teaching selection design and a two-stage network learning process considering the specifics of the problem being solved are shown. Keras and TensorFlow frameworks were used for the software implementation of the proposed architecture.
Research Results. The application of the proposed architecture is effective even under the conditions of a limited amount of the source data. Fine precision is observed. The model can be used in various segmentation tasks. According to the metrics, FCNN shows the following defect identification results: IoU - 0.3488, Dice - 0.7381.
Discussion and Conclusions. The results can be used in the monitoring, modeling and forecasting process of the road pavement wear.
About the Authors
B. V. Sobol
Don State Technical University, Rostov-on-Don
Russian Federation
Russian Federation
Sobol, Boris V., Head of the Information Technologies Department, Dr.Sci. (Eng.), professor
1, Gagarin sq., Rostov-on-Don, 344000
A. N. Soloviev
Don State Technical University, Rostov-on-Don
Russian Federation
Russian Federation
Soloviev, Arkady N., Head of the Theoretical and Applied Mechanics Department, Dr.Sci. (Phys.-Math.), professor
1, Gagarin sq., Rostov-on-Don, 344000
P. V. Vasiliev
Don State Technical University, Rostov-on-Don
Russian Federation
Russian Federation
Vasiliev, Pavel V., Senior lecturer of the Information Technologies Department,
1, Gagarin sq., Rostov-on-Don, 344000
L. A. Podkolzina
Don State Technical University, Rostov-on-Don
Russian Federation
Russian Federation
Podkolzina, Lubov A., Post-graduate student of the Information Technologies Department,
1, Gagarin sq., Rostov-on-Don, 344000
References
1. Quality Management of Pavement Condition Data Collection / National Academies of Sciences, Engineering, and Medicine. — Washington : The National Academies Press, 2009. — 144 p. DOI: https://doi.org/10.17226/14325 .
2. Pavement Distress Analysis Using Image Processing Techniques / D. S. Mahler [et al.] // Computer-Aided Civil and Infrastructure Engineering. — 1991. — Vol. 6, iss. 1. — P. 1–14. DOI: https://doi.org/10.1111/j.1467-8667.1991.tb00393.x .
3. Tizhoosh, H. R. Locally adaptive fuzzy image enhancement / H. R. Tizhoosh, G. Krell, B. Michaelis // Computational Intelligence Theory and Applications. Fuzzy Days 1997. Lecture Notes in Computer Science. — Berlin ; Heidelberg : Springer, 1997. — Vol. 1226. — P. 272–276. DOI: https://doi.org/10.1007/3-540-62868-1_118 .
4. Crack Tree: Automatic crack detection from pavement images / Q. Zou [et al.] // Pattern Recognition Letters. — 2012. — Vol. 33. — P. 227–238. DOI: 10.1016/j.patrec.2011.11.004 .
5. Pavement crack detection based on saliency and statistical features / W. Xu [et al.] // IEEE International Conference on Image Processing. — 2013. — Melbourne : IEEE, 2013. — P. 4093–4097. DOI: https://doi.org/10.1109/ICIP.2013.6738843 .
6. Oliveira, H. CrackIT — An Image Processing Toolbox for Crack Detection and Characterization / H. Oliveira, P. L. Correia // IEEE International Conference on Image Processing — ICIP 2014. — Paris : IEEE, 2014. — P. 798–802. DOI: http://doi.org/10.1109/ICIP.2014.7025160 .
7. Automatic Road Crack Detection Using Random Structured Forests / Y. Shi [et al.] // IEEE Transactions on Intelligent Transportation Systems. — 2016. — Vol. 17, iss. 12. — P. 3434–3445. DOI: https://doi.org/10.1109/TITS.2016.2552248 .
8. Lee, B. J. Position-Invariant Neural Network for Digital Pavement Crack Analysis / B. J. Lee, H. Lee // Computer-Aided Civil and Infrastructure Engineering. — 2004. — Vol. 19, iss. 2. — P. 105–118. DOI: http://dx.doi.org/10.1111/j.1467-8667.2004.00341.x .
9. Sun, B.-C. Automatic Identification of Pavement Cracks Using Mathematic Morphology / B.-C. Sun, Y.-J. Qiu // First International Conference on Transportation Engineering. — Chengdu : ASCE, 2007. — P. 1783–1788. — DOI: https://doi.org/10.1061/40932(246)292 .
10. Chambon, S. Detection of road cracks with multiple images / S. Chambon // International Joint Conference on Computer Vision Theory and Applications, VISAPP. — Angers : Springer, 2010. — 7 p.
11. Zhang, L. Road crack detection using deep convolutional neural network / L. Zhang [et al.] // IEEE International Conference on Image Processing (ICIP-2016). — Phoenix : IEEE, 2016. — P. 3708–3712. DOI: https://doi.org/10.1109/ICIP.2016.7533052 .
12. Automated Crack Detection on Concrete Bridges / P. Prasanna [et al.] // IEEE Transactions on Automation Science and Engineering. — 2014. — Vol. 13, iss. 2. — P. 591–599. DOI: https://doi.org/10.1109/TASE.2014.2354314 .
13. An innovative methodology for detection and quantification of cracks through incorporation of depth perception / M.-R. Jahanshahi [et al.] // Machine Vision and Applications. — 2013. — Vol. 24. — P. 227–241. DOI: https://doi.org/10.1007/s00138-011-0394-0 .
14. Enhanced automatic detection of road surface cracks by combining 2D/3D image processing techniques / R. J. Medina [et al.] // IEEE International Conference on Image Processing (ICIP-2014). — Paris : IEEE, 2014. — P. 778–782. https://doi.org/10.1109/ICIP.2014.7025156 .
15. Automatic Bridge Crack Detection — A Texture Analysis-Based Approach / S. Chanda [et al.] // Artificial Neural Networks in Pattern Recognition (ANNPR) : Lecture Notes in Computer Science. — Cham : Springer, 2014. — Vol. 8774. — P. 193–203. DOI: https://doi.org/10.1007/978-3-319-11656-3_18 .
16. Asphalt Surfaced Pavement Cracks Detection Based on Histograms of Oriented Gradients / R. Kapela [et al.] // 22-nd International Conference Mixed Design of Integrated Circuits & Systems. — Moscow : ALT Linux, 2015. — P. 579–584. DOI: https://doi.org/10.1109/MIXDES.2015.7208590 .
17. Gradient-based learning applied to document recognition / Y. LeCun [et al.] // Proceedings of the IEEE. — 1998. — Vol. 86, iss. 11. — P. 2278–2324. DOI: https://doi.org/10.1109/5.726791 .
18. Ciresan, D. Multi-column deep neural networks for image classification / D. Ciresan, U. Meier, J. Schmidhuber // Proc. of Conference on Computer Vision and Pattern Recognition (CVPR). — Providence : IEEE, 2012. — P. 3642–3649. DOI: https://arxiv.org/abs/1202.2745 .
19. Multi-digit number recognition from street view imagery using deep convolutional neural networks / I. J. Goodfellow [et al.] // Proc. of International Conference on Learning Representations (ICLR). — Banff : Deep Learning, 2014. — P. 1–12. DOI: https://arxiv.org/abs/1312.6082 .
20. A committee of neural networks for traffic sign classification / D. Ciresan [et al.] // Proc. of International Joint Conference on Neural Networks (IJCNN). — San Jose : IEEE, 2011. — P. 1918–1921. DOI: http://doi.org/10.1109/IJCNN.2011.6033458 .
21. Arel, I. Deep machine learning — a new frontier in artificial intelligence research / I. Arel, D. C. Rose, T. P. Karnowski // IEEE Computational Intelligence Magazine. — 2010. — Vol. 5, iss. 4. — P. 13–18. DOI: http://doi.org/10.1128/IAI.02190-14 .
22. Simard, P. Y. Best practices for convolutional neural networks applied to visual document analysis / P. Y. Simard, D. Steinkraus, J. C. Platt // Proc. of International Conference on Document Analysis and Recognition (ICDAR). — Edinburgh : IEEE, 2003. — P. 958–963. DOI: http://doi.org/10.1109/ICDAR.2003.1227801 .
23. Going deeper with convolutions / C. Szegedy [et al.] // IEEE Conference on Computer Vision and Pattern Recognition (CVPR). — Boston : IEEE, 2015. — P. 1–9. DOI: http://doi.ieeecomputersociety.org/10.1109/CVPR.2015.7298594 .
24. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation / R. Girshick [et al.] // IEEE Conference on Computer Vision and Pattern Recognition. — Columbus IEEE, 2014. — P. 580–587. DOI: https://doi.org/10.1109/CVPR.2014.81 .
25. Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks / X. Chen [et al.] // IEEE Geoscience and Remote Sensing Letters. — 2014. — Vol. 11, iss. 10. — P. 1797–1801. DOI: 10.1109/ACPR.2013.33 .
26. Some, L. Automatic image-based road crack detection methods / L. Some. — Stockholm : Royal Institute of Technology, 2016. — 61 p.
27. Xie, D. Deep learning in visual computing and signal processing / D. Xie, L. Zhang, L. Bai // Applied Computational Intelligence and Soft Computing. — 2017. — Vol. 2017 (10). — P. 1–13. DOI: https://doi.org/10.1155/2017/1320780 .
28. Saar, T. Automatic Asphalt pavement crack detection and classification using Neural Networks / T. Saar, O. Talvik // 12th Biennial Baltic Electronics Conference. — Tallinn : IEEE, 2010. — P. 345–348. DOI: http://dx.doi.org/10.1109/BEC.2010.5630750 .
29. Meignen, D. One application of neural networks for detection of defects using video data bases: identification of road distresses / D. Meignen, M. Bernadet, H. Briand // Database and Expert Systems Applications : Proc. 8th International Workshop in Toulouse, France. — Berlin ; Heiderberg : Springer-Verlag, 1997. — P. 459–464. DOI: https://doi.org/10.1109/DEXA.1997.617332 .
30. Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection / K. Gopalakrishnan [et al.] // Construction and Building Materials. — 2017. — Vol. 157. — P. 322–330. DOI: https://doi.org/10.1016/j.conbuildmat.2017.09.110 .
31. Browne, M. Convolutional Neural Networks for Image Processing: An Application in Robot Vision / M. Browne, S. S. Ghidary // AI 2003: Advances in Artificial Intelligence. — Berlin ; Heidelberg : Springer, 2003. — Vol. 2903. — P. 641–652. DOI: https://doi.org/10.1007/978-3-540-24581-0_55 .
32. Dawood, T. Machine vision-based model for spalling detection and quantification in subway networks / T. Dawood, Z. Zhu, T. Zayed // Automation in Construction. — 2017. — Vol. 81. — P. 149–160. DOI: http://dx.doi.org/10.1016/j.autcon.2017.06.008 .
33. A survey and evaluation of promising approaches for automatic image-based defect detection of bridge structures / M. R. Jahanshahi [et al.] // Structure and Infrastructure Engineering: Maintenance, Management, Life-Cycle Design and Performance. — 2009. — Vol. 5, iss. 6. — P. 455–486. DOI: https://doi.org/10.1080/15732470801945930 .
34. Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network / A. Zhang [et al.] // Computer-Aided Civil and Infrastructure Engineering. — 2017. — Vol. 32, iss. 10. — P. 805–819. DOI: https://doi.org/10.1111/mice.12297 .
35. Badrinarayanan, V. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation / V. Badrinarayanan, A. Kendall, R. Cipolla // IEEE Transactions on Pattern Analysis and Machine Intelligence. — 2017. — Vol. 39, iss. 12. — P. 2481–2495. DOI: 10.1109/TPAMI.2016.2644615.
36. Ronneberger, O. U-Net: Convolutional Networks for Biomedical Image Segmentation / O. Ronneberger, P. Fischer, T. Brox // Medical Image Computing and Computer-Assisted Intervention — MICCAI 2015. — Cham : Springer, 2015. — Vol. 9351. — P. 234–241. DOI: 10.1007/978-3-319-24574-4_28.
37. How to Get Pavement Distress Detection Ready for Deep Learning? A Systematic Approach / M. Eisenbach [et al.] // IEEE International Joint Conference on Neural Networks (IJCNN). — Anchorage : IEEE. — 2017. — P. 2039–2047. DOI: http://doi.org/10.1109/IJCNN.2017.7966101 .
38. Shelhamer, E. Fully Convolutional Networks for Semantic Segmentation / E. Shelhamer, J. Long, T. Darrell // IEEE Transactions on Pattern Analysis and Machine Intelligence. — 2017. — Vol. 39, iss. 4. — P. 640– 651. DOI: 10.1109/TPAMI.2016.2572683.
39. Ioffe, S. Batch normalization: Accelerating deep network training by reducing internal covariate shift / S. Ioffe, C. Szegedy // Computing Research Repository. — 2015. — 9 p. DOI: abs/1502.03167.
40. Glorot, X. Understanding the difficulty of training deep feedforward neural networks / X. Glorot, Y. Bengio // AlSTATS. — 2010. — Vol. 9. — P. 249–256.
41. Diederik, P. K. Adam: A Method for Stochastic Optimization / P. K. Diederik, J. Ba // International Conference on Learning Representations. — Banff : IEEE, 2014. — 15 p.
Review
For citations:
Sobol B.V., Soloviev A.N., Vasiliev P.V., Podkolzina L.A. Deep convolution neural network model in problem of crack segmentation on asphalt images. Vestnik of Don State Technical University. 2019;19(1):63-73. https://doi.org/10.23947/1992-5980-2019-19-1-63-73
Views:
909
Email this article 