An AR-based hybrid approach for facility layout planning and evaluation for existing shop floors

Abstract

Facility layout planning (FLP) has substantial impact on the various aspects of a manufacturing system. FLP is typically performed for new shop floors. However, enterprises nowadays are often faced with the need to reconfigure the existing shop floor layouts to synchronize the shop floor operations with the constantly changing production targets. FLP tasks for existing shop floors are often small in scale, e.g., adding or removing a couple of machines, but complex to address since a wide range of criteria need to be incorporated and the presence of the existing facilities imposes critical constraints. Current FLP approaches are not efficient to handle these issues. The development of computer-aided design (CAD) technology has provided feasible solutions to bridge this gap. In this paper, an augmented reality (AR)-based hybrid approach is proposed which facilitates on-site layout planning and evaluation in real time. By integrating AR technology with the mathematical modeling technique, the proposed approach allows the users to augment the shop floor with the facilities to be laid out, model the existing facilities to obtain their geometric data, and define the criteria and constraints to formulate the problems as multi-attribute decision-making (MADM) models. Two planning methods are employed to solve the MADM models, namely, information-aided manual planning and analytic hierarchy process (AHP)–genetic algorithm (GA)-based automatic planning. During the planning process, the criteria and the constraints are assessed in real time to provide immediate evaluation and feedback to facilitate decision-making.

Appendix The 3D positioning method

In this method, PTAM [11] is used for real-time camera tracking and feature point registration. PTAM provides the functions that allow the users to use the chessboard to calibrate the camera so as to obtain the intrinsic camera parameters. In the tracking mode, new feature points are registered into the points map, and the world-to-camera transformation matrix is calculated and updated for every frame. By making use of this matrix, the positioning method is used to register the POIs (Section 3), which are mostly points with little visual features that can be tracked by PTAM.

As the world-to-camera transformation, matrix M is known for every frame, for a point P in the world CS and its projections p in image CS is given as follows:

$$ P={M}_{\mathrm{A}}m{p}_{\mathrm{A}}={M}_{\mathrm{B}}m{p}_{\mathrm{B}} $$

(10)

where p _A and p _B are the coordinates of P in the image CS from two different frames A and B, respectively. M _A and M _B are the world-to-camera transformation matrices for frames A and B, and m is the camera-to-image transformation matrix. To implement this method, the 3D coordinate of a point in the world CS can be calculated if its 2D coordinates in two different frames can be obtained. By using this method to position a POI, the users only need to select (via mouse click) the POI from two different frames, as shown in Fig. 15.

Fig. 15

User-aided point positioning

This positioning process can be further simplified if the point to be positioned is on a known plane, e.g., the x–y (or y–z, z–x) plane, where the users will only need to position the point in one frame. This has great value for modeling the objects that are placed on the floor.