Prediction of Rheological Parameters of Polymers by Machine Learning Methods

Introduction. Polymers are used in various industries, including the production of plastics, textiles, packaging materials, and more. Accurate prediction of the rheological parameters of polymers is a complex task that is important for optimizing production processes and creating products with desired properties.

Today, machine learning methods have gained great popularity in various fields, including chemistry and materials science, due to their ability to efficiently process and analyze large amounts of data. These methods make it possible to predict the properties of materials. In [1], a platform based on machine learning was described, and the integration of metrological support in the context of digital transformation was proposed. In [2], the local distribution of deformation, the development of plastic anisotropy, and fracture in additively manufactured alloys were predicted. The problems of developing measuring control regulators on digital platforms were formulated in [3]. An intelligent model for controlling the parameters of overlap joint welding was built in [4]. However, the issues of using machine learning methods to predict the rheological properties of polymers remain insufficiently investigated. This is caused by both technical and methodological difficulties, such as the heterogeneity of the polymer structure, their sensitivity to external conditions, and complex interactions between molecules during deformation.

Research in the field of rheological properties of polymers and composites using machine learning methods has great prospects in the construction industry [5]. For numerous polymers, the experimental data are well described by the generalized nonlinear Maxwell-Gurevich equation [6], which has the form for a uniaxial stress state [7]:

 (1)

where ε^* — creep deformation, f^* — stress function, σ — stress, E_∞ — module of high elasticity, — initial relaxation viscosity, m^* — velocity module.

Various intelligent machine learning models can be used to determine the rheological parameters of polymers, such as the initial relaxation viscosity (hereinafter just “viscosity”) and the velocity module [8, 9]. For example, one such model is a neural network that can be trained on generated datasets to determine optimal polymer parameters [10].

Prediction based on synthesized data is a fairly common practice, including for nonlinear optimization
methods [11, 12]. One of the ways to generate data is the use of Rosenbrock, Himmelblau, and Booth functions [13], which are effectively applied to test optimization methods such as gradient descent methods, genetic algorithms, and the Newton method. This approach was applied in [14], where a data set based on theoretical stress relaxation curves using the nonlinear Maxwell-Gurevich equation was generated to test the efficiency of various optimization methods.

In [15], several machine learning approaches were given to predict the durability of a reinforced concrete beam, such as a neural network of back propagation, linear and ridge regression, a decision tree, and a random forest. The input parameters of the study were both various characteristics of the material and their properties, depending on the environment (temperature, humidity). Finally, according to the results of the study, the back propagation model determined a more accurate forecast (85%), the average values (MAE) and MAPE were 1.13% and 14.5%, respectively.

Another approach to solving inverse problems of creep theory using the neural network method is based on training a model on large amounts of experimental data. In [16], a neural network model was developed, which was trained on data obtained as a result of long-term experiments on polymer materials, and successfully predicted the viscoelastic behavior of these materials. The data obtained from experiments on samples of various materials were used for the study.

Unlike the above-mentioned papers, the presented research is intended to promote the development of more accurate and reliable methods for predicting polymer properties, such as the k-nearest neighbor method and the support vector machine, which is important for various industries and science.

The research objective was to develop a predictive model based on artificial intelligence methods for analyzing the rheological properties of polymers. Previously, the authors had already used a machine learning algorithm based on gradient boosting CatBoost to process stress relaxation curves [17, 18]. CatBoost is one of the most powerful machine learning algorithms applicable to solving not only regression problems, but also classification and ranking problems [19].

The CatBoost method can be useful for solving some tasks, but it also has its limitations and disadvantages. In this regard, there is an interest in using other algorithms mentioned earlier [20] to solve the problem.

Materials and Methods. The generated data array is partially presented in Table 1. This array was formed on the basis of theoretical stress relaxation curves described by the Maxwell-Gurevich equation, according to the technique presented in [14]. The variation ranges of the velocity modulus and the initial relaxation viscosity in the generated array correspond to the real ranges for polyvinyl chloride in the temperature interval from 20° to 60°C. The total number of numerical experiments (n) was 30,000.

The data set consisted of five input variables and two output variables. Input variables (unit measure):
deformation — ε (%); stress at the beginning of the process — σ₀ (MPa); stress at the end of the process — σ_∞ (MPa); relaxation time — t_n (h); conditional end time of the process — t₉₅ (h). Output variables (unit measure): velocity
module — m^* (MPa); initial relaxation viscosity —  (in Table 1 and further, simply “viscosity”) (10⁶ MPa∙s).
Values σ₀, σ_∞, t_n и t₉₅ are schematically shown on the typical stress relaxation curve (Fig. 1).

The k-nearest neighbor (k-NN) algorithm is based on the similarity analysis of nearby objects. The k-NN method is in great demand for solving various types of machine learning tasks.

Formula (2) represents the general form of the algorithm, where w(i, x) — weight function evaluating the importance of the i-th neighbor.

 (2)

The maximum total weight can be achieved for several objects at the same time. The entropy of this process can be adjusted using nonlinear sequence w(i, x) = [i ≤ k]qⁱ (exponentially weighted k-nearest neighbor method) provided that 0 ≤ q ≤ 0.5.

Representing a fairly simple machine learning algorithm, k-NN is well applicable to solving classification and regression problems. The advantages of this method are ease of implementation, no need for pre-training of the model. It is used for all types of data, including categorical and numeric. Disadvantages: a tendency to over-training (provided that k is too small), poor performance with large amounts of data, it is not possible to take into account the relationship between the signs.

The support vector algorithm — support vector regression (SVR) — solves the problem of minimizing the sum of the mean absolute error. SVR is more resistant to outliers, unlike the least squares method, due to the regularization coefficient (C) and the “epsilon-insensitive tube” (ε). In this case, ε determines the width of the tube in which errors are ignored. Stochastic gradient descent is used to find the minimum of the function.  

The support vector machine learning algorithm is function F(x) of approximation and regularization of empirical risk, which converts training and test samples into output data for each object of the corresponding sample. Formula (3) represents the general form of the algorithm, (4) is a linearly separable sample, (5) is a linearly inseparable sample, where C — regularization coefficient, M_i(w, w₀) — scalar product of vectors (feature and support vector), w_i— weight coefficients.

Function  is a function of a pair of objects , п representable as a scalar product in some space H, for which transformation  takes place. Function  — kernel if  , provided that K is symmetric:  and nonnegative definite:  The regularization coefficient is determined by the sliding mode control method.

Advantages of the SVM method are as follows: high accuracy in classification problems in nonlinear spaces; ability to work with a large number of features (including categorical and numerical), generalize data (which provides applying the model to new data), work with data that are not linearly separable due to the use of kernel functions.

Disadvantages of the SVM method include inefficiency of working with large amounts of data; low interpretability of the model; the requirement to configure numerous parameters, such as the type of kernel (its parameters, regularization parameters), etc.

In this research, algorithms are developed in the Jupyter Notebook intelligent computing environment using machine learning methods.

The selection of such a parameter as the number of neighbors affects the generalizing ability of the developed model and is important for its correct operation. The most suitable algorithm for calculating distance based on data is Distance, in which the weights of objects are inversely proportional to their distance. Accordingly, in the case of closer neighbors of the query object, they have more influence than their neighbors located at a greater distance from the object.

The data set was divided into training and test samples in a ratio of 75/25. In turn, 20% of the training sample became validation. The sample size was: training — x_train = 20,400; test — x_test = 6,000; validation — x_eval = 3,600. For variables y_train, y_test, y_eval, the data were distributed in a similar way.

To build the k-nearest neighbor model, the following parameters were selected: number of neighbors, sheet size, interval, and weight function. The range and functionality of the values for the configurable parameters are shown in Table 2.

To build the SVR model, the following parameters were selected: kernel type, kernel order, regularization coefficient (quadratic regularizer), ε. The range and functional values for the adjustable parameters are presented in Table 3.

Research Results

Figure 2 shows the correlations between the variables.

The following types of linear correlations between individual input and output variables of the model can be noted:

• strong enough — between the variables “Deformation” and “Stress at the beginning” “Relaxation time” and “End time of the process” 
• average — between the variables “Deformation” and “Stress at the end”  “Stress at the beginning” and “Stress at the end” 
• weak — between the variables “End time of the process” and “Viscosity”  “Viscosity” and “Relaxation time” 

The presence of a moderate correlation between variables or its absence indicates only the absence of a linear relationship; therefore, it is possible to have a nonlinear relationship between variables.

Table 4 shows the statistical characteristics of the original data set.

The best parameters for the k-nearest neighbor model were determined as a result of 5-block cross-validation (Table 5).

Table 5

Best k-NN model parameters

Parameter	Number of neighbors (k)	Sheet size (n)	Interval (p)	Weight function (w)
	3	15	2	'distance'
m*	5	15	2	'distance'

The best parameters of the SVR model for viscosity parameters  (at the beginning of the relaxation process) and velocity module  were obtained empirically (Table 6).

Table 6

Best parameters of SVR model

Parameter	Kernel type	Kernel order	Quadratic regularizer	ε
	'rbf'	2	5	0.3
m*	'rbf'	3	6	0.3

The ratio between the real and predicted values for the k-NN model in terms of the parameters “Viscosity” and “Velocity modulus” is shown in Figures 3, 4.

The ratio between the real and predicted values for the SVR model according to the parameters “Viscosity” and “Velocity module” is shown in Figures 5, 6.

The metrics of the developed models of k-nearest neighbors and support vectors are presented in Tables 7 and 8, respectively.

Table 7

Metrics of the developed k-NN models

Parameter	MAE	MSE	RMSE	MAPE (%)	R2 train	R2 test
	1.8	6.8	2.6	5.9	1.00	0.98
m*	0.7	0.8	0.9	6.9	0.99	0.98

Table 8

Metrics of the developed SVR models

Parameter	MAE	MSE	RMSE	MAPE (%)	R2 train	R2 test
	1.67	5.75	1.67	8.92	0.98	0.97
m*	0.72	1.21	1.1	7.3	0.89	0.87

In addition to synthetic data, the developed models were also tested on real experimental data presented in [13]. Experimental relaxation curves of polyvinyl chloride were used for various temperatures in the range from 20° to 60°C. In Figure 7, the experimental stress values at different temperatures at different points in time are marked with felt-tip pens, and solid lines show stress relaxation curves based on values m^* and   predicted by the models.

Discussion and Conclusion. Figure 5 shows that the quality of prediction based on experimental data is quite high, specifically, for temperatures of 30°C, 50°C and 60°C. For other temperatures, the prediction quality is somewhat lower, which is due to the quality of the experimental curves themselves. It was necessary to extend the experiment time and wait for the curves to reach the horizontal asymptote.

In this research, the most preferred method is the support vector machine (SVM). This is due to the fact that SVM can process data with a large number of features, which is important for the analysis of rheological parameters of materials. In addition, SVM works with nonlinear dependences between features, it is applicable to solve the regression problem, which is required to determine the rheological parameters of materials.

However, the CatBoost method can also be effective in this task, especially, if there are categorical features in the data. In addition, CatBoost can process missing data, which can be important for analyzing rheological parameters of materials.

The k-nearest neighbor method is less preferable in this task due to its low efficiency in processing a large number of features, as well as the presence of problems with high data dimensionality.

In the course of the investigation, it has been shown that the use of machine learning methods makes it possible to effectively analyze and process large amounts of data, including information about the characteristics of polymers and their rheological properties. The model developed on the basis of such an analysis maintains high accuracy in predicting the rheological parameters of polyvinyl chloride, which is confirmed by the results of cross-validation and comparison to experimental data.

One of the key advantages of this approach is the ability to automate the process of predicting the rheological parameters of polymers, which reduces the time and cost of research and development of new materials. In addition, the model can be easily adapted to analyze other types of polymers and predict their properties.

As a result of this research, a predictive model has been developed to evaluate the rheological parameters of polyvinyl chloride using artificial intelligence methods based on data of its characteristics and rheological properties. The model demonstrates high prediction accuracy and can be used to optimize the production and development of new polymer-based materials.