Evaluation of the elastic modulus of pavement layers using different types of neural networks models

Introduction. This paper studies the capability of different types of artificial neural networks (ANN) to predict the modulus of elasticity of pavement layers for flexible asphalt pavement under operating conditions. The falling weight deflectometer (FWD) was selected to simulate the dynamic traffic loads and measure the flexural bowls on the road surface to obtain the database of ANN models. Materials and Methods. Artificial networks types (the feedforward backpropagation, layer-recurrent, cascade backpropagation, and Elman backpropagation) are developed to define the optimal ANN model using Matlab software. To appreciate the efficiency of every model, we used the constructed ANN models for predicting the elastic modulus values for 25 new pavement sections that were not used in the process of training, validation, or testing to ensure its suitability. The efficiency measures such as mean absolute error (MAE), the coefficient of multiple determinations R, Root Mean Square Error (RMSE), Mean Absolute Percent Error (MAPE) values were obtained for all models results. Results. Based on the performance parameters, it was concluded that among these algorithms, the feed-forward model has a better performance compared to the other three ANN types. The results of the best four models were compared to each other and to the actual data obtained to determine the best method. Discussion and Conclusions. The differences between the results of the four best models for the four types of algorithms used were very small, as they showed the closeness between them and the actual values. The research results confirm the possibility of ANN-based models to evaluate the elastic modulus of pavement layers speedily and reliably for using it in the structural assessment of (NDT) flexible pavement data at the appropriate time.

displacement is known as the peak deflection, which occurs under the loading point. Traditional methods use the highest values of FWD deflections to back-calculate linear elastic modulus for each layer of pavement [10]. During the past few years, it has been observed that the pavement administrations use new and different methods of collecting and processing data for road maintenance [11]. Due to the rapid development of information technology and artificial intelligence, there have been multiple opportunities for implementation in developing pavements management systems.
More effective ways to address the problem of specifying (complex, non-linear, multivariate) parameters should be considered. ANN is one of the artificial intelligence techniques that provide solutions to classification and regression problems. It is known as one of the best techniques for data mining tasks. It has a framework for different machine learning algorithms to perform together with data inputs. ANN learns how to predict the output from a set of attributes. The algorithm learns to forecast during the training process, which must include data with a large domain, to avoid a problem of falling the expected data outside that range, which affects the validity of the results [12]. In addition, the frequency of sampling should be sufficient to learn correctly. It has been observed from a lot of research that ANN provided good accuracy in pavement performance prediction. The goal of ANN is to find solutions to problems in a similar manner that a human brain does [13].
In 2004, the authors submitted a formulation for the reverse calculation of the pavement layer modules using artificial neural networks (ANN). The research has shown that the proposed ANN method needs considerably less time computed than other methods such as layered elastic theory, equivalent layer thickness (ELT), and finite-element methods, respectively. The ANN is used in simulation at a high rate because it can learn complex nonlinear relationships [14].
Halil Ceylanet and the contributing authors (2008) developed back-calculation models based on artificial neural networks (ANNs) for predicting the elastic modulus of the Portland cement concrete (PCC) layer and the coefficient of subgrade reaction for the pavement foundation. ANN-based models have been trained to estimate the layer modules with deflective basin data (FWD) and the pavement structure thickness. The research indicates that the ANN models can predict the rigid module of paving layers with high precision [15].
More studies were conducted on the development of more accurate and effective models with algorithms of optimization and hybrid systems [16][17][18][19].
G. Beltrán and the contributing authors (2014) collected data from field tests to recalculate layer models by artificial neural network models. The results proved the efficiency of the ANN models in calculating the pavements parameters [20].
Maoyun Li and Hao Wang built a model for calculating the elasticity modulus for flexible pavement layers using both the genetic algorithm and the artificial neural network system based on the falling weight deflectometer data. The results of the ANN-GA model showed reasonable accuracy with the data registered in the LTPP test database, where there were no big differences between the predicted values of the elastic modulus for the asphalt surface layer and the registered in the LTPP data [21].
In this investigation, several analyses were performed to define the best possible architecture along with learning rules and the type of the ANN model to increase the forecasting capabilities of ANNs. The used database includes wide ranges of deflection values obtained from impact load tests conducted on existing three-layer pavement systems on the roads network by the State Company Russian Highways from 2014 to 2018. It was utilized as an experimental basis for training artificial neural network models.

Falling weight
Sensors pavement deflection bowl Sensor deflection

Materials and Methods
The properties of the used sections. In Table 1, the acceptable limits of the pavement layer parameters used in building models are mentioned for calculating the elasticity modulus for the pavement layers (surface, base, and subgrade). Back-calculation models based on ANNs approach. In this work, we used the back-propagation algorithm function to solve the problem of the nonlinear function mapping, where it has high efficiency between ANN algorithms [22][23][24][25]. Furthermore, ANN networks of this type are defined as the neural networks of multilayer feed-forward. The traditional architecture of artificial neural networks is preserved in this algorithm. The structure of this algorithm consists of inputs and outputs represented by neurons, and between them, there are connections used to transfer the weights given to each cell according to its effect. The backpropagating algorithm is characterized by its ability to change the neuron weights to reduce the differences between the goals and the output values of the algorithm using the error reduction technique [26]. The final set of node biases and connection weights is known when the error rate is reduced to permissible limits [27].
The network is trained by different algorithms with the training dataset.
-Feed-Forward Model The feed-forward network involves at least three layers (the input layer, the hidden layer, and the output layer), and it may increase to have more than one hidden layer according to the network needs. As it is clear from the name of the network, the information has one direction from the input to the hidden layer and then to the output layer, as in Figure 2.
-Layer Recurrent Model The structures of the recurrent neural network and the feed-forward network are similar, but the recurrent neural network is unique in that there is a specific feedback loop to each layer in the network except for the last layer, as shown in Figure 3. This feedback loop permits the network to have an unlimited dynamic reaction to incoming time series data.
-Cascade Forward Network Model The Cascade forward network is similar to a feed-forward network, with the only difference being that it includes a link from the input to each layer and from each layer to the following layers. As shown in Figure 4, the Cascade forward model produces links from the first to the second Layer, from Layer 2 to Layer 3, and from the first to the third Layer. These networks also provide input connections for all layers, where additional links can quickly improve the learning process of the network model.
-Elman Neural Network The Elman neural network structure overrides the feed-forward network by having a layer called the context layer in addition to the input, output and hidden layers. The function of the context layer is to store the output of the hidden layer in each finished cycle and reuse it as input to the hidden layer in the next iteration to ensure that patterns are generated over time, as shown in Figure 5. Elman networks also reduce the error rate in the outputs to the permissible limits by using the back-propagation feature, as is the case in the forward propagation network. Through the command of "nntool" in the Matlab program, we implemented the created ANN models. The data that was used in building the models belong to a group of asphalt sections of the M-4 highway of the Russian road network. The set of training data used in this study included the results of calculating the elastic moduli of the structural layers of nonrigid road pavements, carried out in a specialized software package supplied with an FWD Primax shock loading unit on 555 pavement sections, managed by the State Company Russian Highways. We used four types of artificial neural networks, which were (feed-forward, layer recurrent, cascade, and Elman) back-propagation to get the best results. Several cells were selected in the hidden layer for each type of model to study its effect on the training process. The program divided the data at random with 70 % for the training process, 15 % for verification, and 15 % for the testing process. Figure  6 shows the architecture of the artificial neural network model.
We carried out three steps to obtain the optimal number of neurons and hidden layers in ANN models: we trained the model with different hidden layers (first step), estimated the results of the performance model (second step), and compared the predicted values of the tested data to the target values (third step). The Models Evaluation. The correctness of the values of the prediction results for every model is calculated using the mean absolute error (MAE), the multiple determinations coefficient R 2 , the mean root square error (RMSE), and the mean absolute percentage error (MAPE), which are determined from the following formulas (1, 2 and 3): input data hidden layers output data Where A t is the actual value in period t; E t is the expected value in period t; and n is the number of the total period. With respect to the statistical indices, MAE, RMSE and MAPE, smaller values usually indicate higher accuracy results. In this analysis, the MAE, R 2 , RMSE and MAPE values for every model are obtained through comparing the predicted results against the actual values.
Results. Four types of NNA were developed with four different neuron numbers in the hidden layers to see which one is more suitable to use in the forecasting process. The total number of ANN models generated was sixteen. All models were trained under various conditions, including 10, 15, 17, and 20 neurons in the hidden layers. The values of the following MAE, R 2 , RMSE and MAPE indices for all models were calculated as shown in Table 2 to assess the model performance. The best four models that express the four types of artificial neural networks were selected based on the values of the analytical parameters mentioned in the previous table for the results of the models to compare them and know the extent of their impact. Whereas, the best results were due to the models with two hidden layers and 14-15-3 structure for all developed types. All models took the maximum number of repetitions to complete the training process, which were 1000 repetitions. But they differed in the periods taken for training, as training of the models ended after (54, 157, 67, and 173) seconds for the four models, respectively, as shown in Figures (6−9). Being aware of that, the training process stops if the maximum number of repetitions or the time rate is exceeded. The weights and biases of all parameters were modified to decrease the error between target values and network output throughout the training phase. Each neuron weight is adjusted based on its impact on the network result. These weights and biases were evaluated during training as the weights of the inputs to the hidden layer. Figures (10-13) illustrate the mean squared error values vs the number of iterations for the training process of the compared models, respectively. When using the feed-forward back-propagation model, the best validation performance was 16.68 at epoch 1000. Otherwise, the best validation performance for the layer recurrent, cascade and Elman backpropagation models were (11.52, 119,888 and 28,067) at epoch zero, respectively. In all of the curves, we observe the convergence of the test curves with the validation curves, which revealed that the test and validation curves are very similar. According to Figures (14 and 15), artificial neural networks are evaluated using a correlation function (R) between network result values against target data in feed-forward and cascading back-propagation models. Looking at the R-value of the models during the training, validation and testing phases, we notice a good agreement between the target values and the network results. Where we find that the slope of the line that represents the relationship between data and results is approximately equal to 1 in all stages of training, and this indicates the robustness of the models and the possibility of using them during the prediction process.   Target  Target  Target  Target   Target  Target  Target  Target  1000 2000 3000  Output~=1Target+0.13 Output~=1Target+0.69 Output~=1Target+0.37 Output~=1Target+0.026 Advanced Engineering Research 2021. Т. 21, № 4. С. 364−375. ISSN 2687−1653 The models that correctly represent the available data are selected during the stages of education, verification, and testing, based on the statistical parameters, as mentioned previously. It was required to determine if the models could forecast the elastic modulus for 25 segments that had not been utilized during model training and compare them to target values, along with knowing how much the network type affected its performance in terms of elastic modulus assessment. Table 3 shows the comparison of the elastic modulus target values for all layers (asphalt concrete, base, and subgrade) of 25 sections and the forecasting results values of the four ANN types (feed-forward, layer recurrent, cascade, and Elman) back-propagation models. A high degree of convergence can be shown between the four models when they are compared to each other and to actual data, as shown in Figure 16 (a, b, c). It is also understood from the results that the models can reduce the error during the process of predicting the elastic modulus, where the lowest value of the coefficient of multiple determination was "R 2 = 0.95" for the cascade back-propagation model. However, the best performance in the prediction process, as shown in Table 2, was for the feed-forward model with a 14-15-3 structure, which had the lowest values for the statistical parameters MAE, RMSE, and MAPE, and the largest value for the coefficient of multiple determination (R 2 ) for the three pavement layers (asphalt, base, and substrate). Discussion and Conclusion. The authors constructed sixteen models in the study. Every four models follow four different artificial neural network types (feed-forward, layer recurrent, cascade, and Elman back-propagation) to calculate the elastic modulus of the pavement layers subjected to dynamic load. The FWD test was selected to represent the dynamic load generated by the road traffic and to measure the reaction of the pavement. Matlab software was used to create ANN models utilizing deflection data from the (M4) highway in the Russian Federation Road network.
The differences between the results of the four best models for the four types of used algorithms were very small, as they showed closeness between them and the target values. The best results were for the feed-forward model with 15 neurons in the two-hidden layers to form the structure of the model 14-15-3, which produced the best values for the statistical coefficients.
There is no correlation between an increase or a decrease of neurons in the hidden layers and an improvement in models' performance. Instead, the decision is dependent on trial and error.
From the results, we find the capability of ANN models to predict the elastic modulus of flexible pavement correctly and use it in managing the pavement deteriorations.