MultiExtractor Crack Full: Pros and Cons of Using a Cracked Version of MultiExtractor
Pavement damage is the main factor affecting road performance. Pavement cracking, a common type of road damage, is a key challenge in road maintenance. In order to achieve an accurate crack classification, segmentation, and geometric parameter calculation, this paper proposes a method based on a deep convolutional neural network fusion model for pavement crack identification, which combines the advantages of the multitarget single-shot multibox detector (SSD) convolutional neural network model and the U-Net model. First, the crack classification and detection model is applied to classify the cracks and obtain the detection confidence. Next, the crack segmentation network is applied to accurately segment the pavement cracks. By improving the feature extraction structure and optimizing the hyperparameters of the model, pavement crack classification and segmentation accuracy were improved. Finally, the length and width (for linear cracks) and the area (for alligator cracks) are calculated according to the segmentation results. Test results show that the recognition accuracy of the pavement crack identification method for transverse, longitudinal, and alligator cracks is 86.8%, 87.6%, and 85.5%, respectively. It is demonstrated that the proposed method can provide the category information for pavement cracks as well as the accurate positioning and geometric parameter information, which can be used directly for evaluating the pavement condition.
Multi Extractor Crack Full
In 2016, Zhang et al.  proposed a crack detection method based on deep learning. They trained a deep CNN based on supervised learning, proving the feasibility of combining deep learning with pavement crack recognition. In 2017, Zhao et al.  proposed a pavement crack detection method based on a CNN using images of different scales and taken at different angles for training, achieving the detection of cracks of various shapes. However, owing to road surface interference and noise, the detection accuracy of this system peaked at 82.5%. In 2017, Markus et al. developed the open dataset GAPs for the training of deep neural network and evaluated the pavement damage detection technology for the first time, which is of great significance [20, 21]. In 2018, Nhat-Duc et al.  established an intelligent method for the automatic recognition of pavement crack morphology; this study constructs a machine learning model for pavement crack classification that included multiple support vector machines and an artificial swarm optimization algorithm. Using feature analysis, a set of features is extracted from the image projection integral, which can significantly improve the prediction performance. However, the algorithm is complex and programming it becomes very difficult. In 2020, Zhaoyun Sun et al.  proposed a method to detect pavement expansion cracks with the improved Faster R-CNN, which can achieve accurate expansion crack location detection through the optimization model. The aforementioned studies only detect and classify pavement cracks and their location but cannot quantify certain crack characteristics, such as crack width and area. On the other hand, there are also many studies on crack segmentation. In 2018, Zhang and Wang  proposed CrackNet, which is an efficient architecture based on CNN to predict the class of each image pixel, but its network structure is related to input image size, which prevents the generalization of the method. In the same year, Sen Wang et al.  proposed to use the full convolutional networks (FCNs) to detect cracks and built the Crack-FCN model taking into account the shortcomings of the FCN model in the crack segmentation experiment and obtained a complete crack image. However, the highest accuracy obtained by their method is only 67.95%; thus, segmentation performance needs to be improved. In 2019, Piao Weng et al.  proposed a pavement crack segmentation method based on the VGG-U-Net model. It solves the problem of fracture in the crack segmentation result in complex background, but its training time is slightly longer and its efficiency is low. In 2020, Zhun Fan et al.  proposed an encoder-decoder architecture based on hierarchical feature learning and dilated convolution (U-HDN) detects cracks in an end-to-end manner. The U-HDN method can extract and fuse different context sizes and different levels of feature mapping, so it has high performance. In the same year, Zhun Fan et al.  proposed an ensemble of convolutional neural network based on probability fusion for automatic detection and measurement of pavement cracks, and the predicted crack morphology is measured by skeleton extraction algorithm. In summary, these previous studies only use the segmentation method, which cannot achieve accurate crack classification and location determination.
In this paper, a crack identification method based on a deep CNN fusion model is proposed. First, the image dataset is established, and the image noise in the dataset is filtered out to increase the contrast between road cracks and background. Next, the processed images are provided as input into an improved single shot multibox detector (SSD) crack detection model and an improved U-Net crack segmentation model for training. Then, the binary image of a crack obtained by the segmentation model is used to calculate the geometric parameters of the crack. By integrating the advantages of the two models, this pavement crack identification method can effectively overcome the single-model limitations of inaccurate positioning and imperfect information. The overall process flowchart is shown in Figure 1. The details of each step are discussed in Section 2.1.
Increasing the number of network layers can improve the accuracy of the network in identifying pavement cracks. Therefore, in this study, the feature extraction network structure Visual Geometry Group 16 (VGG16) in the SSD network model was replaced with a deep residual network to improve the pavement crack identification accuracy. The deep residual network [33, 34] solves this problem by fitting a residual map instead of the original map and by adding multiple connections between layers.
The structure of the U-Net network is simple, and the original U-Net network has crack segmentation accuracy problems. Therefore, the feature extraction network of the U-Net crack segmentation model was also replaced with a deep residual network to fully extract crack features and ensure crack segmentation accuracy. The specific improvement steps are similar to those of the SSD model. As shown in Table 3, the two basic network feature graph outputs match the network layers. After adjusting the corresponding layers of the feature extraction network, it is still necessary to adjust the network parameters through continuous training to optimize the crack segmentation effect. The improved crack segmentation model is shown in Figure 11.
In the training of U-Net crack segmentation model, the ReLU function is used as the activation function and the input data samples are regularized many times. Regularization adjusts the output value of each convolutional network layer to the same distribution, thereby avoiding a deviation or change in the distribution of feature vectors caused by network deepening. The segmentation model uses the upsampling method. That is, the feature map with the new size is obtained by the convolution inversion operation, and the feature map with the size corresponding to the convolution layer is added as the upsampling result. The segmentation network performs upsampling of the feature extraction network feature maps with sizes of , , , and , and the upsampling process combines the feature extraction network feature maps with sizes of , , , and ; this improves the segmentation network accuracy through multilevel joint learning.
The crack segmentation model uses the cascade mode of multiple residual elements, which can effectively extract the morphological characteristics of the pavement crack and improve the learning effectiveness of the neural network on the crack characteristics. A single crack segmentation method based on U-Net can provide the crack pixel location information, but it cannot classify the crack . Therefore, in this study, a fusion of two models was adopted to identify pavement crack images and to obtain the crack category, location information, and geometric parameters, thereby facilitating accurate quantification and evaluation of pavement cracks.
As shown in Figure 17, when only the SSD detection network is used, the number of cracks can be accurately obtained, but the crack width cannot be quantified. Use of only the U-Net segmentation model will lead to misjudgment of the number of cracks; e.g., a fractured crack may be misidentified as multiple cracks. The fusion of the detection and segmentation networks can avoid this phenomenon and ensure that the crack is identified as a single crack. Thus, the advantage of the fusion model is that it can accurately identify the number of cracks and ensure that cracks are quantified correctly.
The proposed fusion model adopts the following order: (1) detection, (2) segmentation. Because the detection network will obtain the result of a detection box, which contains one crack, and then segment that crack. This method can not only ensure crack number accuracy but also quantify the crack width information, which can prevent the misjudgment of crack number caused by using only the segmentation model. If the order is changed, i.e., segmentation first and detection second, in the presence of fractures in the cracks, the segmentation result will be misjudged into multiple cracks. Moreover, the segmented image is a binary image, which is not suitable for detection.
From the above analysis, it can be seen that crack features extracted by different convolutional layers in the deep convolutional network are not the same. Meanwhile, with the deepening of network layers, crack features extracted by the deep convolutional neural network evolve from low-order features to high-order features. Therefore, in order to perform a comprehensive study of crack features and improve the identification accuracy of pavement cracks, it is necessary to fully extract their features by increasing the convolutional network depth.