Integration of Sensor Data and Mathematical Modeling of Underwater Robot Behavior Using a Digital Twin

Introduction. In recent years, there has been growing interest in digital twin (DT) technology. These are virtual models closely related to physical objects and designed to display, analyze and predict their behavior in real time [1]. DT are widely used to solve applied problems in industry, energy and transport, especially if it is necessary to control and manage objects under conditions of high uncertainty [2]. With the spread of autonomous robotic systems, the digital twin becomes an important tool for providing reliable and sustainable management. It allows for the formation of a predictive model of the object's behavior, taking into account both internal parameters and external actions [3].

Of particular interest is the use of DT in the control of underwater robotic complexes (URC), where operating conditions are significantly complicated by poor visibility, lack of precise navigation, communication delays, high inertia, and noise in sensor data. All this reduces the efficiency of traditional closed-loop control systems [4]. There are publications on digital twins in the public domain, but the topic is still insufficiently developed in terms of using DT in underwater systems. Existing studies, as a rule, either do not cover predictive control tasks or do not take into account the specificity of the underwater environment and typical sensor errors [5]. Numerous papers are characterized by significant gaps related to the lack of integrated solutions for building digital twins of underwater robotic systems focused on stable predictive control and state assessment under conditions of noisy and incomplete data.

A stable assessment of the current state of the underwater robotic complex (URC) is required, and this can be provided by integrating sensor data — accelerometry, gyroscopy, encoder measurements, and other sources. However, due to the instability and noise typical of inertial sensors, it is necessary to implement algorithms for filtering, correcting and calibrating data at different stages of operation [5].

It should also be noted that in the literature on digital modeling and control of robotic systems, authors often focus on static models or limited control scenarios (e.g., on navigation or position stabilization). At the same time, a comprehensive approach to predicting the dynamic behavior of URC using digital twins in real time has not been sufficiently developed.

Based on the aforesaid, the objective of this study is to develop and apply a method for predicting the dynamic behavior of the URC using a digital twin to improve the efficiency of controlling an autonomous system when performing underwater engineering work.

Materials and Methods. To reach the research objective, a comprehensive model was implemented, including a physicomathematical description of the motion, sensor architecture, and visualization.

Scientific research made it possible to solve the following problems:

• construction of a generalized mathematical model of the motion of the URC taking into account the external environment and sensor architecture;
• formalization of the behavior of the autonomous system taking into account inertial and control effects;
• implementation of a feedback control system based on classical approaches to automatic control theory;
• assessment of the accuracy of the digital twin by analyzing the discrepancies between the physical and virtual behavior of the URC based on experiments;
• visualization of the digital twin in the Unity software environment with the ability to compare the indicators of virtual and physical trajectories.

Within the framework of the presented work:

• a mathematical model of the URC movement is created;
• data from sensors are collected and calibrated;
• a digital twin is implemented in the Unity environment;
• the model is experimentally verified;
• the results are analyzed, and the accuracy of the model is assessed.

The motion of the URC is described on the basis of differential kinematics and equations that take into account water resistance, inertial characteristics of the system, and control actions. The model is based on a system of second-order motion equations modified to take into account hydrodynamic resistance and correction coefficients obtained experimentally. Assumptions are made about the rigidity of the housing, insignificant drift, and quasi-stationary motion.

The system state vector includes coordinates: x, y, z, and heading angle θ, as well as linear and angular rates. The formulas used in the introduction describe kinematic relationships, feedback, resistance model, and control signals. PID controllers with parameters empirically selected on the basis of the system response are used for control along the longitudinal, transverse, and vertical axes.

A modular sensor system is used to collect data in real time. Its elements:

• 38S6G5-B-G24N encoders, 2000 pulses per revolution — to estimate linear displacement;
• LIS3DH three-axis accelerometer (measurement range ±2g/±4g/±8g/±16g, I2C/SPI interfaces, sampling frequency up to 5.3 kHz) — to obtain acceleration and tilt data;
• gyroscope as part of the IMU module (inertial measurement unit) — to track angular speeds and orientation.

The sensor data was pre-filtered, calibrated (normalized relative to the zero position) and integrated into the digital twin model.

The key stage in building a digital twin is mathematical modeling of the movement of the underwater robotic complex. It allows predicting the behavior of the object in various environments [6]. This study considers a mobile URC with a rigid physical structure, moving in a flat (2D) or spatial (3D) coordinate system depending on the task scenario [7]. An independent drive for each of the tracks is used to control the movement, which allows for differential maneuvering [8].

To simulate the position and orientation of the URC on the plane, a kinematic model of a differential robot is used, based on the position of the center of mass (or center of geometry) of the robot [9]. Let x and y be the coordinates of the center of mass in the global coordinate system, θ — robot’s orientation angle (angle between the longitudinal axis and the OX axis in the global system).

(1)

(2)

(3)

where v — linear speed of the center of mass (determined by encoder readings); w — angular speed (obtained from a gyroscope or calculated by the difference in track speeds).

The linear and angular speeds are related to the individual speeds of the right v_R and left v_L tracks:

(4)

(5)

where L — robot base (distance between tracks).

The robot's behavior is significantly affected by the underwater environment: water resistance, lifting force, viscous forces, friction forces. These effects are taken into account in the form of generalized disturbance F_d(t) [10]. The system of differential equations takes the form:

(6)

where M — robot’s mass; — position radius vector; F_u(t) — control action from the drive system (determined by the PID controller or another algorithm); F_d(t) — environmental disturbances (determined experimentally or set empirically).

Within the framework of the layout and digital model, the resistance of the environment can be taken into account in the form of a damped term:

(7)

where k_u — coefficient of viscous resistance of the medium (adjustable parameter of the digital twin).

The physical model is equipped with encoders, therefore the positions and orientation of the URC in space are calculated through numerical integration of the speeds obtained from the left and right tracks:

(8)

(9)

where Δs_L, Δs_R — increments of distance according to encoder readings; Δs — increment of the center of mass;
Δθ — change in orientation.

We use the current orientation Δθ, to express the new position in the global coordinate system:

(10)

(11)

Mathematical modeling is a fundamental part of the development of a digital twin of an underwater complex [11]. The objective of modeling is to formalize the processes of movement [12], control and response to external actions with the possibility of further analysis of the stability and accuracy of execution of control actions [13].

The underwater robot is considered as a system with multiple inputs and outputs, located in an environment with a high level of inertia, delay and stochastic disturbances [14]. Control actions are implemented through thrust modules, and movements are tracked using encoders and inertial measurement units [15]. The model can be described as a system of second-order equations with position and speed feedback:

(12)

where M — mass matrix (includes inertial characteristics); D — damping matrix (taking into account viscous resistance of water); K — stiffness matrix (includes elastic forces in the case of mechanical constraints); x(t) — position and orientation vector in 3D space; u(t) — control action (signals to the motors); B — control distribution matrix.

For rotational motion, the Euler model is used:

(13)

where J — inertia tensor of the robot, ω(t) — angular speed (from the gyroscope), τ(t) — control moment.

When implementing a spot turn operation, as a rule, tracks with different directions of rotation are used. The difference in speeds on the right and left tracks sets the moment:

(14)

To set the angle of rotation, the expression is used:

(15)

The control system is implemented in the form of PID controllers for each axis (X, Y, Z):

(16)

where e(t) = x_reF(t) − x(t) — error between target and current position. Coefficients K_p, K_i, K_D are adjusted according to the optimality criterion.

The physical model of the robot is implemented on the basis of Arduino Mega 2560 Pro. Microcontroller — ATMega 2560-16AU, 16 MHz, 256 KB FLash, with 54 digital inputs/outputs (I/O), 16 analog inputs, USB-UART interface CH340G. The complex is connected to sensors and a drive control system. Data is collected and transmitted to the digital environment via Serial port in the format of combined time-stamped lines.

The virtual implementation of the digital twin is created in Unity. C#-scripts are used to interpret the data, display the movements, rotations and behavior of the object in a three-dimensional environment. Axes, contours, coordinate values and trajectories are visualized. A model of a translucent robot that does not interact with other objects is used, which allow us to concentrate on comparing movements.

A data processor and a filtering system are implemented on the Unity side. Integration with correction based on sensor data is applied to calculate angles and coordinates. Individual blocks are implemented as scripts with the ability to scale the project. Standard metrics are used to statistically assess accuracy: standard deviation, maximum deviation, and average error in coordinates and angles.

Research Results. To verify the model in a digital environment, movements were simulated under successive control actions. Then, the trajectories of rectilinear and semicircular motion were compared (Fig. 1 and 3). The readings of the digital robot were compared to the ideal trajectory. The root-mean-square analysis showed a discrepancy when moving in a straight line (Fig. 2) and in a semicircle (Fig. 4).

The quality of displaying the URC behavior in the digital twin was assessed during experiments that included linear motion, the spot turn, and the trajectory with turns. In parallel, data from real sensors was recorded and compared to the parameters of the model in Unity. The results of measurements and comparisons are presented in Table 2.

The digital twin was visualized in the Unity environment. This flexible 3D engine allowed us to embed data streams from physical sensors, display spatial movements, orientation, and also create scenarios for interaction with the external environment (Fig. 5).

Data from physical sensors (accelerometers, gyroscopes, encoders) was transmitted via a COM port (communications port) using a custom script in C#. Incoming data was divided into channels and displayed as parameters of the twin object.

The digital model in Unity used the same coordinate system and the same control structure as the implemented mathematical model. Its functions were:

• transmitting commands from the PID controller model to the virtual wheels;
• displaying position, tilt angle, speed;
• visual reproduction of movement based on integrated sensor data.

The solution allows creating a closed loop “model – reality – visualization” and provides consistency between the digital twin and the physical system (Fig. 6).

The elements of the software operation algorithm are listed below.

System Startup and Initialization. The physical robot is turned on and connected via USB to a computer that collects and analyzes data, runs the main program, and initializes additional scripts:

• responsible for maintaining communication and collecting data,
• providing visualization.

All this takes place in Unity, after which the system is ready for work.

Robot Initialization. After the program is launched, its connection with the physical robot is established. Consequently, calibration occurs, and then the sensors and modules are launched: gyroscope, accelerometer, encoders, voltmeter, ammeter. A control message about the robot's readiness for launch and the start of data transmission is sent to Unity. After that, the stage of waiting for commands starts.

Receiving Command from the User. The value of the user interface entered into the program is analyzed and processed. The message is passed to intermediate scripts to perform the actions of the digital and physical robots.

Digital Robot’s Command Response. The entered values and actions are recorded in the computer memory. Based on the embedded command execution algorithms and a mathematical model, the digital robot moves in digital space. At the same time, data on its position, speed and distance traveled, are recorded. After executing the command, the robot maintains its position and orientation in space.

Physical Robot’s Command Response. Data is transmitted to the Arduino Mega microcontroller. Here, it is analyzed and, depending on the entered command and value, scripts are activated that are responsible for the operation of sensors: left and right encoders, accelerometers, and gyroscopes. Only after that does movement start at the user's command. During the execution of the command, the robot records its position and orientation in space and transmits data to the user interface program, where the data is visualized using a 3D model linked to the physical robot.

Data Collection and Display. At this stage, data on position, tilt angles, distance traveled and other parameters are collected from the digital and physical robots and visualized in the user interface program. After that, the program waits for a new command.

The described algorithm is based on the classical principles of closed-loop feedback control systems. The use of data from sensors (accelerometers, gyroscopes, encoders) allows for the implementation of a measuring element in the general structural form of control systems. The mathematical model integrated into Unity acts as an observer, tracking the behavior of the system and providing visual verification of the correctness of command execution.

The system is designed taking into account:

• linearization of the model of the URS motion in small neighborhoods of the trajectory;
• PID control of the angle of rotation and speed;
• filtering of sensor data using simple sliding windows and zero calibration corrections.

Formulas (17)–(19) present the indicators that are used to analyze the accuracy of the digital twin.

Absolute position error (in mm):

(17)

Root mean square orientation error (in degrees):

(18)

Percentage deviation of the final position:

(19)

As a result of experiments and analysis of sensor readings, additional parameters are obtained that characterize the error of the developed system (Table 3).

The digital twin is resistant to minor fluctuations in the data due to the following factors:

• smoothing of input data;
• stability of the regulator implemented in the system;
• adaptation of the model to calibration values at the start of each cycle.

Discussion and Conclusion. A model of a digital twin of a robotic complex based on the integration of sensor data, mathematical modeling and 3D visualization has been developed and experimentally confirmed. Sustainable control and tracking of the behavior of the robotic complex have been implemented. The results of the comparative analysis of the digital model and the real system are presented. The results of the comparison have proven:

• a high degree of correspondence between the behavior of the digital twin and the real object;
• efficiency of the selected calibration and data processing algorithms;
• potential of using digital twins in debugging autonomous systems if physical testing is difficult.

The constructed model makes it possible to adequately assess the current state and predict the behavior of the URC under uncertainty. PID controllers provide stability during control, and visualization in Unity makes it possible to implement a digital twin as an interface for interaction with the system. Further development of this solution may be related to the implementation of adaptive and neural network control methods.