Evaluation of Deep Learning-Based Automatic Floor Plan Analysis Technology: An AHP-Based Assessment

Abstract

This study proposes a technology that allows automatic extraction of vectorized indoor spatial information from raster images of floor plans. Automatic reconstruction of indoor spaces from floor plans is based on a deep learning algorithm, which trains on scanned floor plan images and extracts critical indoor elements such as room structures, junctions, walls, and openings. The newly developed technology proposed herein can handle complicated floor plans which could not be automatically extracted by previous studies because of its complexity and difficulty in being trained in deep learning. Such complicated reconstruction solely from a floor plan image can be digitized and vectorized either through manual drawing or with the help of newly developed deep learning-based automatic extraction. This study proposes an evaluation framework for assessing this newly developed technology against manual digitization. Using the analytical hierarchy process, the hierarchical aspects of technology value and their relative importance are systematically quantified. The analysis suggested that the automatic technology using a deep learning algorithm had predominant criteria followed by, substitutability, completeness, and supply and demand. In this study, the technology value of automatic floor plan analysis compared with that of traditional manual edits is compared systemically and assessed qualitatively, which had not been done in existing studies. Consequently, this study determines the effectiveness and usefulness of automatic floor plan analysis as a reasonable technology for acquiring indoor spatial information.

1. Introduction

With the emergence of artificial intelligence (AI) and the Internet of things (IoT), there have been high expectations for real-time monitoring of smarter decisions, such as digital twins. Building 3D models of indoor structures is required for constructions that combine AI and IoT sensor technology. However, compared with the demand, indoor spatial data available are insufficient [1,2]. Accordingly, previous studies have proposed the recreation of indoor spaces from various data sources, such as light detection and ranging (LiDAR), building information modeling (BIM), computer aided design (CAD), and architectural drawings [3]. Digitalized plans from LiDAR, BIM, or CAD are certainly useful sources to recreate indoor spaces; however, there is an unignorable proportion of missing digital blueprints of old buildings, which results in considerable effort to draw the blueprint image into digital plans. It is possible to recreate indoor information through LiDAR systems. However, it needs to be executed at the site, owing to the high cost and time required; only a few buildings can be modeled using this method [3]. Therefore, floor plans have been reported as spatial data sources in recent studies because they are relatively affordable and accessible compared with other data [3,4].

Studies that generate indoor spatial information from floor plan images typically used two main approaches: rule-based methods and learning algorithms. In recent studies, the learning method has gained attention and exhibited high performance because it can automatically learn patterns from data and offer robustness in performance against noise [3,5]. However, the floor plans previously targeted in learning-based research fields were limited to simple and unified formats because it is favorable to compose test sets suitable for the training of deep/machine learning [5]. For practical use, floor plan images generally have considerable noise, auxiliary lines, and symbols. In these circumstances, Kim et al. [5] proposed a new technology based on deep learning algorithms to analyze complicated floor plans with extensive and complex indoor structures by employing style transfer and conditional generative adversarial networks (GANs). In this study, critical indoor spatial information such as room structures, junctions, walls, and openings could be extracted automatically and converted into a 3D data model that is compatible with a standard 3D data model, such as IndoorGML (Indoor Geographic Markup Language). Before the development of this technology, extracting indoor spatial information from floor plan images could be achieved through manual digitization. Manual measurements are time- and labor-demanding and therefore are either very expensive or unreliable when applied comprehensively on sites [6]. However, there is hardly any precedent study that comparatively evaluates automatic and manual digitalization of floor plans despite a growing body of literature introduced in the field of floor plan analysis. In fact, technology evaluation of existing studies on floor plan analysis was mainly based on internal comparisons between automatic floor plan analysis only.

This study expands the scope of the research of Kim et al. [5] by considering the value of the developed technology compared with traditional manual digitization. Accordingly, an assessment of the indoor information extraction technology is performed considering the technological characteristics and hierarchy of technology value. Considering the key factors for assessing the technology value, a quantitative evaluation method using the analytical hierarchy process (AHP) is presented. Through this study, an evaluation framework for assessing technologies that extract indoor spatial information from floor plan images can be developed, which has not been systematically demonstrated in existing literature. Additionally, using the relative importance of each criterion and factors of technology value derived from the AHP analysis and the scoring of each element, the distinction between automatic and manual extraction is investigated.

2. Background Study

2.1. Automatic Floor Plan Analysis

Floor plans are good sources for acquiring and processing information about indoor environments because of its high accessibility and low cost [3,5]. Research on automatic floor plan analysis has resulted in building models generated from scanned floor plans by converting raster images into vector information. Initially, conventional approaches have set up rules based on local features within specific plans; however, the rules are highly dependent on the format of the floor plan [4,7,8]. To resolve the dependency on specific formats, recent studies have applied deep/machine learning algorithms that can train the geometry of the patterns directly from the datasets [3,9,10]. However, in a learning-based approach, researchers tend to target a simplified format that is suitable for labeling and training. CVC, Rakuten, and Rent3D floor plans have been used in previous studies for automatic floor plan analysis [5] (Figure 1). However, considering practical applications, learning-based automatic extraction should be applied in more complex and diverse formats. Therefore, an electronic architectural information system (EAIS) dataset [11], which is composed of complicated, diverse architectural drawings of buildings, including fuzzy architectural drawings, was used for automatic floor plan analysis [5]. Through the findings of the study, crucial indoor spatial information that can formulate 2D and 3D indoor structures can be extracted automatically in a more sophisticated and diverse format.

2.2. Technology Assessment with AHP

The technology assessment commonly applies multi-criteria decision-making tools [12]. AHP is a commonly used technology assessment method that involves characterizing the factors of the evaluation, as well as minimizing the subjectivity in assessment. The method is called “analytical” and “hierarchical” because it allows complex decisional queries for necessary elements and involves the decomposition of the problem [13]. AHP was originally developed by Saaty [14] and is one of the most widely used mathematical methods for analyzing complex decision problems with multiple criteria [15,16]. AHP was employed because of its accuracy, simplicity, and theoretically robust capability for handling both numerical and non-numerical measurements [15]. This implies that if decision-making incorporates both qualitative and quantitative aspects, AHP can systematically solve it by decomposing the structure of the problem into hierarchies because people tend to make pairwise comparison judgments to develop priorities in each hierarchy [17]. AHP has evolved through several applications, including energy allocation, marketing decisions, technology selection, conflict resolution, project selection, and evaluation [17,18,19,20].

In the field of technology evaluation, AHP can be used to quantitatively determine the factors and weights appropriate for a given technology possessing several qualitative evaluation criteria. Previous studies utilized the AHP method for evaluating technology, considering performance potential [21] or tangible and intangible factors of technology [22]. Additionally, AHP was applied for selecting technology [20], ranking emerging technologies [18] and evaluating new technology [19]. To quantify intangible risks and benefits, Ordoobadi characterized both the positive and negative impacts of critical factors using AHP [20]. In addition, Cho and Lee conducted an assessment of decision making of new technology using the hierarchical structure of AHP considering technological, organizational, marketing, and business aspects [19].

In the construction technology field, previous studies adapted AHP methodology on ranking, decision making, and evaluating. Erdogan et al. utilized AHP methodology in construction management decision making, because most of construction management problem is based on multi-criteria decision making [23]. As a result, they suggested seven criteria for solving decision-making problems in construction; technical experience, performance record, financial stability, management and employee qualification, capacity, safety record, and operation and equipment. Prascevic and Prascevic applied fuzzy AHP to rank and selected construction management project [24]. A fuzzy AHP model is also used to assess the performance of the construction industry in the study conducted by Işik et al. [25]. In addition, Skibniewski and Chao evaluated advanced construction technology using AHP, assessing the intangible aspects of technology by considering both favorable and unfavorable evaluation factors in one framework [26]. A review paper by Darko et al. investigated 77 AHP-based papers from the construction sector from 2004 to 2014. They concluded that application of AHP in construction has gained increased attention because this method can analyze complex situations and make sound decisions [27]. Particularly, when assessing new technologies, the evaluation criteria vary according to the characteristics of each technology. Therefore, it is favorable to utilize the AHP method to assess the effectiveness of a new technology which does not possess any evaluation criteria. It is also reasonable to use the AHP method as it can transform qualitative elements into quantitative indicators. Therefore, in this study, to evaluate the automatic approach for converting floor plan images to vector indoor information, the AHP method was employed.

3. Research Framework

In this study, technology that constructs vectorized indoor spatial information from raster floor plan images is defined as floor plan analysis technology. This technology can be divided into two approaches: manual and automatic. The process of the study goes through three stages: (1) formulating a hierarchical model containing assessment criteria and factors; (2) refining the measures of criteria and factors and identifying the weights of criteria and factors through AHP; and (3) scoring both methods (i.e., manual and automatic). Figure 2 shows the manual and automatic approaches in terms of extracting indoor spatial information from floor plan images. In this study, EAIS data (a complicated floor plan with emphasis on practical value) were employed as the target floor plan image.

In phase 1, to construct the hierarchy of technology value, both literature reviews and focus group interviews (FGIs) were conducted, considering the characteristics of the technology for both methods. In phase 2, after deriving key factors and criteria and their hierarchy for technology evaluation, a survey on AHP was conducted with experts in the field of spatial information. With the relative weights derived from AHP, assessment of each method and comparison of their characteristics was conducted in phase 3. To score both methods (i.e., automatic and manual), an additional survey was conducted. The overall process is illustrated in Figure 3.

3.1. Hierarchy Structuring

The proposed model for the evaluation of floor plan analysis utilizes a hierarchy of evaluation elements, as shown in Figure 4. The hierarchy reflects the objective concerns in the decision-making process in developing a set of criteria to evaluate the technology value. The criteria and factors are derived mainly based on the Technology Valuation Program for the R&D Result Diffusion report by the Korea Institute of Science and Technology Evaluation and Planning (KISTEP) [28] and they are consistent with the characteristics of the present technology. In addition, an FGI was conducted among spatial information experts, including automatic floor plan engineers and manual editors to: (1) identify the distinctive differences between and the strengths and weaknesses of each technology and (2) determine the criteria and factors for floor plan analysis technology evaluation.

Table 1. Strengths and weaknesses of manual and automatic approach.

	Manual	Automatic
Strengths	Elaborate construction of indoor spatial information is possible according to the editor’s purpose. Suitable for building with a fewer number of drawing images. Compatible with drawings in relatively varied formats.	When applied to a large number of drawing images, time and financial costs are significantly lower. A relatively unified level of indoor spatial information is built because output is based on the deep learning model. The automation technology was built on an open-source basis; there is scalability of the technology as a future service.
Weaknesses	A paid drawing and editing program is required. High temporal and monetary costs (especially when applied to many drawings). Depends on the editor; the quality of indoor spatial information extraction and editing characteristics are likely to be non-uniform.	Only major building elements such as room structures, junctions, walls, and openings can be extracted. Extraction of other objects is possible, but additional training is required. High initial cost for deep learning network training (e.g., collection of various drawings, labeling work, and improvement of deep learning algorithms).

presents the strengths and weaknesses of the manual and automatic approaches that were concluded from the FGIs. In manual editing, a drawing editor engineer manually creates and edits based on a floor plan image to generate indoor spatial information in vector format. In automatic editing, floor plan images are trained using a deep learning algorithm, and the learned segmentation results are vectorized to generate indoor spatial information. The main advantage of the automatic approach is that once it is trained, it enables the processing of a vast number of drawing images with little marginal cost. However, only the major building elements like room structures, junctions, walls, and openings based on the information provided in the floor plan image can be extracted automatically. The manual edit can generate additional information that is not stated in the floor plan image, if necessary.

According to Barney [29], corporate resources must have the following four requirements: (1) value, (2) ratio, (3) non-importability, and (4) non-substitution to gain continuous competitive advantage and greater benefits over competitors. KISTEP [28] suggested an aspect of technology evaluation from Barney’s by combining value and rareness under technology superiority, non-imitability and non-substitution under technology exclusivity, and adding technology constraints.

Table 2. Aspects of technology valuation [30].

Aspect	Criteria	Factor
Technology superiority	Completeness *	Technical development phase, completeness of technology, commercialization potential
	Distinctiveness *	Novelty, originality, efficiency
	Applicability	Time to commercialize, scalability, scale of commercialization investment
	Transferability	Ease of technology transfer, cost of technology transfer, regulation of technology transfer
Technology exclusivity	Legal rights	Legal rights, defense of rights, scope of rights.
	Substitutivity *	Possibility of existence of similar technologies, possibility of new technologies to emerge
	Ease of technology protection	Difficulty of imitation, technology protection costs
	Supply and demand *	Demand from customers, supply from providers
Technology constraints	Market constraints	Restrictions in the sales market, restrictions on buyers of products, restrictions on producers and sellers
	Technical constraints	Requirement for purchasing raw materials, requirement for facility purchasing, technology improvement constraints, conformity with other technologies
	Social constraints	Environmental pollution possibility, social regulation, legal regulation
	Competitive constraints	Competitors’ responses, key success factors

* denotes the criteria that were selected for hierarchy in this study.

presents the aspects of technology evaluation and the criteria and factors suggested in [30].

Technology value assessment is primarily evaluated in three categories: technology superiority, technology exclusivity, and technology constraints [30]. Among them, the technology constraints assess the competitive and socioeconomic constraints that may arise during commercialization and utilization of the developed technology. This technology is developed on an open-source basis for public purposes rather than commercialization purposes; therefore, a suitable hierarchy of technology value assessment is constructed by selecting the appropriate detailed factors from technology superiority and technology exclusivity criteria.

Technology superiority assesses the superiority of the technology, to judge how unique it is compared with existing technologies. Factors that determine the superiority of this technology include technology completeness, differentiation, applicability, and transferability. Through FGI, experts stated that it would be better to judge the superiority of floor plan analysis technology in terms of completeness and differentiation rather than industry expansion, facility investment size, and technology transfer because the target technology is far from being developed for commercialization investment or technology transfer purposes. Therefore, this is reflected in the hierarchy structuring by considering the level of technology development, completeness of technology, novelty, originality, and efficiency.

Technology exclusivity assesses whether there is any difficulty in exercising exclusive ownership and using the technology, along with legal rights. As this technology was developed on an open-source basis for public purposes through national R&D projects, FGI concluded that it is better to examine it in terms of alternative possibilities and supply and demand aspects rather than considering legal rights and technology protection.

The classification, detail, and hierarchy of technology valuation are as follows. There are four types of criteria: completeness, distinctiveness, substitutivity, and supply and demand. The factors consist of a total of 10 categories: technical development phase, completeness of technology, commercialization potential under the completeness criteria, novelty, originality, efficiency under the distinctiveness criteria, possibility of existence of similar technologies, possibility of new technologies to emerge under the substitutivity criteria, and finally demand from customers, and supply from providers under the supply and demand criteria. The proposed model for the evaluation of floor plan analysis technology with a hierarchy of evaluation elements is shown in Figure 4, and the individual factors are defined in

Table 3. Definition of criteria and factors of hierarchy.

Criteria	Factors	Definition
Completeness	Technical development phase	Whether the degree of completion of technology development is high
	Completeness of technology	Whether it can be applied without complementary technology or supporting instructions
	Commercialization potential	Whether it is easy to commercialize the technology
Distinctiveness	Novelty	Whether there are many differences from the existing technology or whether they are brand-new
	Originality	Whether the knowledge in the technology is innovative
	Efficiency	Whether the developed technology is more efficient than the existing technology
Substitutivity	Possibility of existence of similar technologies	Degree of unlikelihood of similar technologies existing
	Possibility of new technologies to emerge	Degree of unlikelihood of new and replaceable technologies emerging
Supply and Demand	Demand from customers	Whether there are many consumers who need the technology
	Supply from providers	Whether there are little suppliers who have and develop the technology

.

3.2. Relative Weight Calculation

When all the elements (criteria and factors) referring to the same hierarchical level are compared in pairs, relative weights are assigned through the construction of the matrices of the comparison pairs. A pairwise comparison is used because the psychologist claimed that only two alternatives are easier and more accurate in expressing public opinion than simultaneously employing all alternatives [31]. For comparing an element in a group on one level of the hierarchy with respect to an element at the next higher level, an n × n matrix is constructed, where n is the number of elements in the group.

The result of the single comparison is an a_ij dominance coefficient, which expresses a measure of the relative importance of element i with respect to element j. A pairwise comparison was performed to determine the weight for prioritizing components, and a nine-point scale was used for relative evaluation (see

Table 4. Meaning of each scale value (Saaty scale) [32].

Scale Value	Meaning
1	Two elements are equally important
3	One element is a little more important than the others
5	One element is distinctly more important than the others
7	One element is very more important than the others
9	One element is absolutely more important than the others
2,4,6,8	Intermediate values between two adjacent judgments

).

The estimation of relative weights is derived by performing ⁿC₂ times of two-way comparison of n evaluation items at one level of decision makers, which can be used to construct the pairwise comparison matrix A_n×n. The square matrix a_ij is an estimate of the relative weight w_i/w_j of i for element j, and matrix A is a_ij = 1/a_ji, a reciprocal matrix where the elements in the main diagonal are all equal to 1. These square matrices allow for eigenvectors and eigenvalues, which are used as a means of determining priorities and measuring consistency in judgment, respectively. The key to the stratification analysis is the extent of transitive consistency that can be maintained, and this can be verified through the determination of the consistency ratio (CR), which divides the consistency index (CI) of the result using a random index (RI). CI is calculated as (λ_max − n)/(n − 1), where λ_max is the largest eigenvalue of the two-way comparison matrix, and n denotes the number of criteria being compared. Contrastingly, RI is the average value of the CIs of the comparative matrix constructed using random numbers from 1 to 9, which refers to the dimensional average random index of the matrix. In a two-way comparison matrix of stratification analysis, the relationship of λ_max ≥ n is always maintained. For comparison matrices with perfect consistency, λ_max = n and the greater the consistency, the closer the λ_max is to n; it is possible to measure the degree of consistency using equation (2). The smaller the value of CR, the higher its transitive consistency. Saaty judged that people performed a two-way comparison fairly consistently if the CR was within 10% (0.1), and an acceptable level of inconsistency was within 20% (0.2); however, a lack of consistency above 20% (0.2) required re-examination [33]. In this study, only CR results within 0.1 were utilized.

$$\mathrm{CR}=\mathrm{CI}/\mathrm{RI}$$

(1)

$$\mathrm{CR}=\mathrm{CI}/\mathrm{RI}=({\mathsf{\lambda }}_{\max }-\mathrm{n})/(\mathrm{n}-1)\times (1/\mathrm{RI})$$

(2)

3.3. Technology Value Evaluation

An evaluation of the floor plan analysis technology was performed using the relative weights derived from the AHP method. To assess both manual and automatic methods, an additional survey for scoring each technology was conducted with the same group of AHP survey candidates. By comparing the survey and applying the relative weights and scores of each factor, a comprehensive evaluation of floor plan analysis technology was derived.

4. Results

4.1. Process of the AHP

To acquire relative importance among the technology value criteria and factors, a survey was conducted with experts in the field of spatial information. Out of the 100 questionnaires, 84 responses were received, and 61 results with CR values below 0.1 were used for the AHP analysis.

Table 5. Demographic characteristics of the sample.

Category		Frequency	Rate
Affiliation	Government and public agency	10	16.39%
	University and research institute	37	60.66%
	Private company	14	22.95%
Career period	Less than 5 years	19	31.15%
	5–10 years	20	32.79%
	10–20 years	12	19.67%
	More than 20 years	10	16.39%
Education	Bachelor’s degree	10	16.39%
	Master’s degree	27	44.26%
	Doctoral degree	24	39.34%

presents the demographic characteristics of the participants. The participants from academia, including universities and research institutes, were the highest (60.66%), followed by private companies (22.95%), and government and public agencies (16.39%). The career period of 5 to 10 years had the highest percentage of 32.79%, followed by <5 years (31.15%), 10 to 20 years (19.67%), and >20 years (16.39%). The participants’ education level included a master’s degree (44.26%), followed by a doctoral degree (39.34%), and a bachelor’s degree (16.39%).

4.2. Relative Weight of Floor Plan Analysis Technology

In this study, the consistency rate of participants below the reference value of 0.1 was used to calculate relative weights. For each module, the relative importance of the criteria and factors were established (see

Table 6. Weight calculation of the assessment index.

Criteria	Importance (Rank)	Factors	Relative Importance (Rank)	Final Importance (Rank)
Completeness	0.2216 (3)	Technical development phase	0.3013 (3)	0.0668 (8)
		Completeness of technology	0.3577 (1)	0.0793 (5)
		Commercialization potential	0.3410 (2)	0.0756 (7)
Distinctiveness	0.3549 (1)	Novelty	0.1777 (3)	0.0631 (9)
		Originality	0.2222 (2)	0.0789 (6)
		Efficiency	0.6001 (1)	0.2130 (1)
Substitutivity	0.2461 (2)	Possibility of existence of similar technologies	0.6659 (1)	0.1639 (2)
		Possibility of new technologies to emerge	0.3341 (2)	0.0822 (4)
Supply and demand	0.1774 (4)	Demand from customers	0.8334 (1)	0.1478 (3)
		Supply from providers	0.1666 (2)	0.0296 (10)

). From the weight analysis of the criteria elements, distinctiveness (0.3549) scored the highest relative importance, followed by substitutivity (0.2461), completeness (0.2216), and supply and demand (0.1774).

Under the completeness criterion, completeness of technology (0.3577) is the most important factor, followed by commercialization potential (0.3410), and technical development phase (0.3013). Overall, the factors of completeness exhibited comparatively similar levels of relative importance. With respect to the distinctiveness criteria, the weights of the three factors exhibited relatively high differences. Efficiency had the highest importance, not only under the distinctiveness criteria (0.6001) but also among all factors (0.2130). Originality and novelty had relative index weights of 0.2222 and 0.1777, respectively. For the substitutivity criteria, the possibility of existence of similar technologies (0.6659) had approximately twice the relative importance of the possibility of new technologies emerging (0.3341). Finally, evaluators stated that the demand from customers (0.8334) was five times more important than supply from providers (0.1666).

Using the value of index weight of criteria and relative index weight among factors, the final importance was derived. Among all factors, efficiency from distinctiveness criteria had the highest importance (0.2130); the existence of the possibility of similar technologies (0.1639) from the substitutivity criteria and demand from customers (0.1478) from supply and demand criteria had similar weights and ranked second and third, respectively. The supply from providers (0.0269) from the supply and demand criteria had the lowest weight. Figure 5 shows the relative weights of the factors under each criterion in a radar diagram.

4.3. Application of Relative Weights for the Evaluation of Technology Value

From the AHP analysis, relative weights were derived to assess the technology value of automatic floor plan technology. In this section, a scoring model is suggested to evaluate both manual and automatic approaches. The second survey was conducted to score both methods by assessing the technology value for each criterion and factor based on a 5-point scale. Consequently, except in terms of “technical development phase” and “completeness of technology,” the automatic technology had a higher score than the manual edit. This is because the automatic floor plan analysis only extracts the basic geometry of indoor information, whereas manual edits can extract indoor information as desired. The limitation of automation technology is that only the information that is shown in the floor plan image can be extracted, whereas the manual edit can add additional information from the floor plan image to acquire indoor information such as wall types and materials. Distinctiveness criteria had the largest difference among all criteria with scores of the following sub-factors, novelty (1.76), efficiency (1.52), and originality (1.35) ranking with the highest difference between manual and automatic methods in that order. “Possibility of new technologies to emerge” had the lowest difference (0.20) between manual and automatic methods. Figure 6 shows the distinctive differences in the scoring of each criterion between the manual and automatic approaches. The detailed score result of manual and automatic floor plan analysis technology is described in

Table 7. Score result of manual and automatic floor plan analysis technology.

Criteria	Factor	Manual	Automatic	Difference	Absolute Difference (Ranking)
Completeness	Technical development phase	3.54	3.13	-0.41	0.41 (8)
	Completeness of technology	3.37	3.15	-0.22	0.22 (9)
	Commercialization potential	2.94	4.11	1.17	1.17 (4)
Distinctiveness	Novelty	2.22	3.98	1.76	1.76 (1)
	Originality	2.35	3.70	1.35	1.35 (3)
	Efficiency	2.50	4.02	1.52	1.52 (2)
Substitutivity	Possibility of existence of similar technologies	3.06	3.76	0.70	0.70 (5)
	Possibility of new technologies to emerge	3.35	3.56	0.20	0.20 (10)
Supply and demand	Demand from customers	3.22	3.93	0.70	0.70 (5)
	Supply from providers	2.65	3.17	0.52	0.52 (7)

.

The evaluation of creating indoor spatial information from floor plan images was derived by combining the score and relative weights derived from the AHP analysis.

Table 8. Technology value of manual and automatic floor plan analysis.

Criteria	Factor	Manual				Automatic
		Point	Weight	Final Value	Ratio	Point	Weight	Final Value	Ratio
Completeness (22)	Technical development phase (7)	3.54	0.708	4.96	71%	3.13	0.626	4.38	63%
	Completeness of technology (8)	3.37	0.674	5.39	67%	3.15	0.63	5.04	63%
	Commercialization potential (8)	2.94	0.588	4.70	59%	4.11	0.822	6.58	82%
					65%				70%
Distinctiveness (35)	Novelty (6)	2.22	0.444	2.66	44%	3.98	0.796	4.78	80%
	Originality (8)	2.35	0.47	3.76	47%	3.70	0.74	5.92	74%
	Efficiency (21)	2.50	0.5	10.50	50%	4.02	0.804	16.88	80%
					48%				79%
Substitutivity (25)	Possibility of existence of similar technologies (16)	3.06	0.612	9.79	61%	3.76	0.752	12.03	75%
	Possibility of new technologies to emerge (8)	3.35	0.67	5.36	67%	3.56	0.712	5.70	71%
					63%				74%
Supply and demand (18)	Demand from customers (15)	3.22	0.644	9.66	64%	3.93	0.786	11.79	79%
	Supply from providers (3)	2.65	0.53	1.59	53%	3.17	0.634	1.90	63%
					63%				76%
Total				58.38				75.00

presents the technology values of the manual and automatic approaches. The manual editing technology scored 58.38 out of 100 points, whereas automatic technology based on the deep learning algorithm scored 75 points. Efficiency has the highest importance among all factors, and received a higher score in the automatic approach. In the automatic approach, the score on the 5-point scale was 4.02, and the score for manual approach was 2.50. By applying relative weights, the automatic technology efficiency was 16.88 out of 21 (80%), whereas the manual approach exhibited 10.50 out of 21 (50%). The possibility of existence of similar technologies exhibited 12.03 out of 16 (75%) in the automatic approach, whereas the manual approach showed 9.79 out of 16 (61%). Except under commercialization potential, the manual approach had a higher value than the automatic approach. The biggest difference between the manual and automatic approaches was in novelty factor: manual having 2.66 out of 6 (44%) and automatic having 4.78 out of 6 (80%). Distinctiveness criteria generally had higher values in the automatic approach (79%) when compared with manual (48%).

5. Discussion and Conclusions

The main contribution of this study is to evaluate the list of criteria and factors for assessing the value of the floor plan analysis technology. Using the AHP method, the relative weights of qualitative criteria and factors could be quantified, and with this value, an assessment of the technology value of both manual and automatic approaches could be derived. This study developed a technology hierarchy assessment model and identified important criteria and their weights to evaluate the relative importance of the factors. Furthermore, this study contributes to extending the research on technology value evaluation by applying the AHP method, which is one of the most structured techniques for industrial and technical decision making.

The results suggest that distinctiveness is the predominant criterion of the floor plan analysis technology, followed by substitutivity, completeness, and supply and demand. Particularly, efficiency, possibility of existence of similar technologies, and customer demand have distinctively higher importance, indicating that they are key factors for the technology value of floor plan analysis. Based on the weights of the factors derived from AHP, this study performed a scoring checklist to provide comparative evaluation results between manual and automatic methods. Consequently, automatic extraction floor plan analysis technology rated 1.3 times higher than the manual editing approach.

In addition, this study highlights the practical efficiency of automating floor plan analysis. Beyond internal comparisons using detection rates in automatic floor plan analysis, this study enables quantitative identification of the strengths and weaknesses of automatic floor plan technology compared with the traditional method of manual editing technology. As a result, this study determined the effectiveness and usefulness of automatic floor plan analysis as a reasonable technology for acquiring indoor spatial information.

However, the limitation of this study is that it relies on weights and technical evaluation scores based on responses from only 61 experts in the field of spatial information. Additionally, this study focused on the EAIS floor plan, and therefore, it might not be directly applicable to other floor plans. However, by utilizing the derived criteria and factors from this study, it is possible to apply it universally in the field of floor plan analysis evaluation in future studies. The result herein suggests that overall, the automatic approach is more efficient technology than the manual approach. However, technology can be valuable or required regardless of efficiency. For example, there is a demand for more information or qualitative interpretation on the floor plan, which can be obtained from skilled technicians by manual digitization. The result showed that the technical development phase factor of completeness criteria had higher technical value in the manual approach rather than automatic approach, and it is needed to acknowledge the value of manual editing technology itself. This study attempted an untried assessment in existing studies and has implications for presenting further objective evaluation criteria and assessment of automatic floor plan technology. Within the framework of this study, future work on floor plan analysis can be systematically accomplished.