Placement of Multiple Virtual Objects in Physical Space in Augmented Reality Applications

Introduction. Rapid development of augmented reality (AR) technology opens up new opportunities in various fields — from entertainment to education and industrial applications [1][2]. However, despite considerable achievements, there are numerous problems that need to be solved, specifically, in the context of placing several virtual objects in a real-world environment. One of such problems is the optimal placement of virtual objects in augmented reality applications to provide for optimal and comfortable user experience [3–5].

This problem arose in connection with the need for the device to understand physical space. For effective placement of virtual objects in the real world, the application should be able to correctly interpret the material environment in which the user is located applying sensors and cameras of mobile devices [6].

This article presents a new approach to determining the optimal placement of virtual objects in physical space. This problem has some similarities with another close topic of generative contextual scene augmentation (CSA), where the key goal is to create a harmonious and convenient interaction between virtual and physical objects [7][8]. However, the approach proposed by the authors differs from the one mentioned, since it focuses on determining the optimal distance between objects using a monotone comfort function, while CSA takes into account the semantics of the scene, the context and the meaning of virtual objects.

Existing approaches to solving the problem of rational location of virtual objects are usually limited by assumptions about the form of the comfort function, and they do not always guarantee the optimality of the solution. This article proposes a new approach that is significantly different from those currently used. This makes it more flexible to find a rational arrangement of virtual objects and provides a universal solution for various scenarios and conditions. The term “optimal placement” is used for the following reasons: firstly, the final choice in the proposed recommendations is still at the discretion of the user; secondly, despite the fact that in the context of this work, the task of minimizing the objective function is solved, it contains elements of fuzzy sets.

Within the framework of this work, the concept of optimal placement of a set of virtual objects in a one-dimensional physical space is introduced. A model has been developed that allows solving the problem of optimal placement of a set of objects and results, regardless of the choice of the type of comfort function, in a system of linear equations for determining the optimal distances between objects. As an example, the solution to the problem for the case with two virtual objects is given.

The objective of this article is to propose and demonstrate a new approach to determining the optimal placement of virtual objects in physical space, to establish its applicability and efficiency. Mathematical formulations and methods for solving the optimal placement problem are presented, as well as examples of practical application of the results obtained. This will show new opportunities for improving the interaction between virtual and physical objects, as well as contribute to the development of the theory and practice of augmented reality.

Thus, this article is aimed at deepening the understanding of the problem of optimal placement of virtual objects in physical space and offering a new approach to its solution. The results of the study can be used to create more efficient and user-friendly virtual reality systems, as well as for further development of theory in this area.

Materials and Methods. Placing virtual objects in a real physical space is a task that arises in almost every AR/VR application. For all its simplicity, it can create challenges in case of insufficient attention to the issue of optimizing the placement of such objects, up to the complete refusal of a large number of potential users to work with the mentioned applications. The optimization problem is most acute when it is required to place several virtual objects at once in a given physical space. At the same time, even in the case of placing only one object, only recently the concept of “comfort” of its placement was formulated and a corresponding model was proposed [9], consisting of the following.

An object embedded in a three-dimensional physical area is presented as a rectangular parallelepiped with characteristic dimensions: l — length; d — width; h – height. At the same time, for each of X, Z, Y coordinates, the following concept of placement comfort is introduced. It is clear that the size of the free space should not be less than the size of the object, but, in addition, for each coordinate, the concept of comfortable distances from the object to the boundary of free space is introduced. For example, for X coordinate, we introduce the concept of comfortable distance on the left — and right — and, respectively, left and right comfort — and . Denote the distances to the left and right of the object to the boundary of free space and . We assume that , if and decreases to 0 when approaching zero. For example, for simplicity, let us take linear dependences and :

Dependence is similar (1). In exactly the same way, we introduce the concept of comfort on one side and on the other for Z and Y coordinates.

If the size of the free space horizontally is , then the problem of comfortable placement does not arise; and all problems appear when . In this case, the concept of comfort of the object placement is introduced, when comfort on the one hand is not obtained at the expense of comfort on the other hand. We introduce the target function of comfort:

By optimal placement, we will understand such placement, in which minimum is achieved. Obviously, this happens if .

As it was shown in [9], if dependences and are linear, the minimum of the target function (2) corresponding to the optimal placement of the object is attained at the following values and :

Formulas (3) are simple and convenient for optimal placement of a single virtual object. As noted in [9], their disadvantage is that they were derived for the case of linear (1) comfort functions

and . We show that they are always valid if functions and are the same function k(x), satisfying the condition that it increases monotonically for 0≤ x ≤1, and is equal to 1 at x >1. The type of such function is shown in Figure 1.

Figures 2 and 3 present additionally two more functions — cubic and linear, demonstrating similar described behavior.

Indeed, suppose we need to embed an object of size l with comfortable distances on the left and right into the space bounded by size L, and conditions are met. Then, if one-sided comforts are presentable in the form

,

where , and dependence k(x) satisfies the above conditions, then, from the minimum condition of the objective function (2) we obtain:

.

For the given nonlinear comfort function k(x), the resulting equation can be solved numerically by one of the known methods. However, since one-sided comforts are described by self-similar function k(x), from equation

,

we obtain relation

,

from which, taking into account equality

, equations follow (3).

Thus, it is shown that simple and rather convenient equalities (3), which provide embedding an object with optimal comfort, are valid for any one-sided comfort function k(x).

Research Results. In a real situation, there is a need to place several objects at once. In this case, it makes no sense to solve sequentially the problems of optimal placement of the first object, then the second, third, etc., since when placing the next object, a need arises to shift previously installed objects so that the placement is optimal for the totality of all objects. If condition is met, the objects in principle do not fit in the free space of length L. If

then the objects can be placed so that they do not interfere with each other. In reality, the problem of optimal placement arises if the following conditions are met:

In this case, we introduce the target function:

,

where — comfort of the i-th object determined by formula (2). Here, when there are two consecutive objects with numbers i and i+1, then their neighborhood will be comfortable if the distance between them is not less than , which corresponds to the new comfortable distances

, i=1,2,…,(n-1) since a comfortable distance to the wall is one thing, and to another object, from where something can be pushed, is quite another. The rational arrangement of embedded objects is determined by the minimum of the target function (5).

The minimum of the target function (5), taking into account (2), gives us a system of n equations:

.

From (6), it follows:

.

The system of equations (7) can be written in the following form:

That is, we have obtained n equations with respect to n unknowns , where — distance of the first object from the left edge of the embedding area, — distance between objects with numbers i and (i-1).

Since function k(x) is monotonic, system (8) is reduced to a linear system of equations, which does not depend on the type of the comfort function itself k(x).

System (9) can be solved by one of the known methods. However, due to the fact that the matrix of system (9) is highly sparse, the solution to the system can be found quite simply. For example, in the first (n-1) equations, it is possible to express by in each i-th equation, then, substituting this into the last equation, to obtain a linear equation with respect to . After that, moving from the first equation to (n-1)-th, we can consistently find

.

In the case of placing two virtual objects, n=2, system (9) takes the following form:

.

From system (10), we find:

where — distance between the first object and the left edge of space; — distance between the right edge of the first object and the left edge of the second object.

Consider example (12) when:

.

Since conditions (13) are met, then:

For optimal arrangement of two objects, we can use expressions (11), which give regardless of the form k(x) in this case

.

Here, the value of one-sided comfort depends on the type k(x). For linear dependence k(x), shown in Figure 3,

.

If dependence k(x) corresponds to Figure 1, we get the comfort value

.

Thus, the paper introduces the concept of optimal placement of a set of virtual objects in physical space. A model has been developed that provides solving the problem of optimal placement of a set of objects. It is shown that the solution to this problem does not depend on the type of monotone comfort function.

Discussion and Conclusion. The theoretical aspect of the important issue of optimal placement of a set of virtual objects in physical space (the problem often encountered in augmented reality applications) was considered. By proposing a new mathematical model and including fuzzy logic, we laid the foundation for an algorithm that could potentially help users find a rational and convenient location of virtual objects in their real environment.

The foundations laid in this study show that the proposed model effectively solves the problems associated with the placement of a set of virtual objects in a given physical space. By analyzing virtual planes and taking into account the distances between virtual objects and the edges of these virtual planes, our method provides optimal placement considering the linear dimensions of virtual objects and the comfort zone around them.

The results obtained contribute to the current development of augmented and mixed reality applications, providing a theoretical solution to the problem of optimal placement, which, in turn, can improve user interaction and overall satisfaction with the tools of the technology under discussion. Moreover, the possible applications of this research go beyond AR applications, they open up new avenues for research in the related fields, such as virtual reality, mixed reality, and spatial computing.

Considering the results of this study, future developments may be aimed at verifying the algorithm through empirical testing, enabling dynamic real-time adjustments based on user behavior, and exploring the integration of our approach into various XR application scenarios. As the area of augmented reality continues to evolve, we expect that our research will make a significant contribution to the development of the technology, inspiring its wide dissemination and further enriching the user experience.