Recent Advances in Procedural Generation of Buildings: From Diversity to Integration

In this article, we survey the state-of-the-art in procedural generation of buildings (PGB). We analyzed the design choices, capabilities, and limitations of 71 PGB approaches. We structured our analysis based on a taxonomy of procedural content generation (PCG) that we adapted to PGB by adding relevant details. Overall, we identified 13 taxonomical aspects, which we organized in three groups: 1) generative capabilities, 2) accessibility aspects, and 3) technological aspects of the implementation. It is shown that the generation of different parts of buildings is easiest to accomplish with different PGB techniques. While some approaches built on other ones or even integrate two or more of them to arrive at solutions with a greater effective generative scope, all of them still exhibit rather basic limitations. For instance, the choice of room geometries might be limited to rectangular rooms, they might require numerous manual creation steps, or they might yield potentially invalid output. In addition, only few approaches implement fitting interiors and exteriors, which can be used as complete, traversable buildings in virtual environments. Hence, the combination and therefore integration of different approaches for different parts of buildings is a major concern of procedural building generators, rendering their compatibility a characteristic of central importance.


I. INTRODUCTION
P ROCEDURAL content generation (PCG) can be defined as the algorithmic creation of game content with limited or indirect user input [1].It has the potential to speed up development, decrease costs, and increase the scale of virtual worlds with a proper level of detail with given resource restrictions.It is applied in a wide range of application areas, e.g., for urban understanding [2] or cultural heritage reconstruction [3].PCG is used for numerous diverse purposes in game development: The content can range from 3-D objects to abstract puzzles [4] including levels, weapons, personalities, quests, etc. [5].It can even include PCG as a game artefact in itself, featuring in-game content creation [6], such as the generation of new levels during play.PCG in games has been successfully applied to numerous popular games, such as in Deep Rock Galactic (2020) for cavesystem generation [7] or in Valheim (2021) for biome-based maps [8].It is likely to grow in relevance both for assisting in the design and development of games as well as a game element by itself, e.g., for adapting games to particular demographics [9].There are several reasons for game developers to use procedural methods.Popular reasons have been, for instance, maintaining a low footprint on computational load by providing content on demand, saving time and effort as compared to manually creating content, and fostering the generation of novel content, also by overcoming our own imagination [10].Another reason is the replay value of games, experiencing the same games in a different way each time [11].
Buildings are an important category of assets to populate virtual worlds.The application of procedural generation of building (PGB) in practice seems to be rare when it comes to traversable buildings, but is applied more commonly for building exteriors, such as in star citizen [12].Another indicator for the rare occurrences of PGB in games is, that the success stories of ArcGIS CityEngine, a commercial tool supporting PGB, only contain one game use case in November 2022: a destroyed city in a student project [13].Depending on the use case, such as a specific type of game, buildings may only be part of the virtual stage designed to provide an atmospheric environment that a player is immersed into.In this case, of course, their exterior architecture is all that matters.Alternatively, buildings might themselves define individual environments that can be explored by the player.Their great diversity in functionality and design in combination with the need for potentially large numbers of buildings, e.g., in the context of open-world games, emphasize the need, and potential benefit of PGB.
This article provides the results of a structured analysis of PGB approaches for use in virtual, interactive 3-D environments.It presents the current state-of-the-art and takes steps to comprehensively and comparatively understand the individual approaches, best practices, and open challenges.Based on the gathered data, we recognize numerous highly specialized solutions and identify the need for their integration as an important next step in PGB, as hinted at by the title of this article.
The rest of this article is organized as follows.Section II details the method that was followed in conducting the survey and the analysis of the identified relevant works.Section III introduces a taxonomy for the different aspects of buildings as well as their procedural generation.The taxonomical aspects and groups of aspects are used in Section IV to present and classify the identified PGB approaches.In addition, the results are analyzed and discussed to yield insights about the development and challenges of PGB.Finally, Section V concludes this article.

II. METHODOLOGY
In order to achieve a comprehensive understanding of PGB approaches, best practices and open challenges, the following steps were taken.
First, we had to identify the various aspects according to which PGB approaches should be classified.To the best of our knowledge, there exists no specialized taxonomy for distinguishing between the design and capabilities of PGB systems so far.However, we decided to start this investigation based on a generic taxonomy of PCG by Togelius et al. [14].We adapted its distinct classifications with regard to validation and optimization of PCG approaches.
Next, we identified relevant works based on an extensive online search for scientific literature.We started with search terms derived from surveys we got and added possibly relevant additional terms to our list during the whole process.We dropped several other search terms when realizing that the search has converged (i.e., the same publications were found repeatedly) or no relevant publications at all.We consider the search terms, which lead to publications for our list core search terms of which we have the following."Procedural building generation," "procedural architecture," "computer generated architecture," "semantic procedural buildings,"procedural interior placement," "procedural content generation for buildings," "procedural stairs, "procedural roofs," "procedural interior design," and "procedural furnishings."Each core search term was used in each of the following search engines and publication repositories.SciTePress Digital Library [15], arXiv [16], MDPI [17], Dimensions [18], Google Scholar [19], IEEE Xplore [20], Sci-enceDirect [21], emerald insight [22], ACM Digital Library [23], and SpringerLink [24].Next, we identified patterns and alignments in publications we found on PGB.Based on these, we were able to propose an extended taxonomy for PGB, which considers 13 aspects in total.It is detailed in Section III.While working on the taxonomy, we realized that PGB has quickly evolved within the last several years, had received little attention before, and all state-of-the-art publications we found have been published since 2006.Even though we had not restricted our search in any way, the included publications range from 2006 to 2022 for our taxonomy-based analysis.A few important publications prior to 2006 deserve special credit as important baseline research, which is not part of the state-of-the-art and thus is handled separately at the beginning of Section IV.Despite of fulfilling our criteria for PGB systems, we also excluded several identified works.The first reason for this was missing novelty for PGB generators, e.g., by solely reporting about the application of an existing approach, such as the usage of the software ArcGIS CityEngine.The second reason is cases where some authors were involved in several works with largely overlapping contributions.We embraced opportunities to only refer to works that covered their contributions as a whole.
For the following step, 71 publications remained that we analyzed in accordance with the extended PGB taxonomy.Hence, we identified the necessary taxonomical aspects of the respective works and evaluated them in comparison to all the other works included in this study.This process of regression required numerous iterations to have the domains of the taxonomic aspects converge: the aspects were already fixed, but the possible set of values was adapted during this process.After each change in the possible values of one taxonomical aspect, all previously analyzed publications were analyzed again for the changed aspects.Since some aspects did not allow for quantifiable comparisons, we directly quoted the respective details in the tabular overview we composed (see Table I).In case we decided on quantifying one or the other aspect, even though the taxonomic alignment was not immediately obvious, we added according justifications in the collection of surveyed works.
Tutenel et al. stated that complete (virtual) buildings are those "consisting of not only a facade, but also interiors, stairs, furniture, etc." [25].Leblanc et al. [26] spoke of "coherent interior and exterior" and give examples by referring to staircases and furniture layouts.Yet, the authors do not explicitly state the requirements a complete building would have to fulfill.Surveying PCG, Smelik et al. [27] distinguished between the exterior and interior of buildings and consider floor plan generation and furniture layout solving belonging each to either category.
Having these rough schemata of virtual buildings in mind, we analyzed the selected PGB literature.We identified six recurring building constituents, which we consider necessary to reach a "complete building."Based on their descriptions, which can be found in Section III-A, one can determine which ones are addressed by a given PGB approach, and which ones are not (represented by opaque and transparent pictograms in Table I and Fig. 10, respectively).
A more detailed separation of the constituents is possible, e.g., by individually considering doors, windows, etc.We decided against it, since these construction elements are typically considered in the respective contextual generative tasks, such as the generation of floor plans or outer walls.Also, numerous optional elements of a building, such as chimneys could be addressed, but these are rare occurrences within the PGB literature and we focus on those components necessary for the generation of "simple buildings," analyzing the inclusion of the previously mentioned six building constituents.
The goal of this survey is to cover complete, simple, and common buildings for the game-domain.A game-focused, justified definition of these terms could be subject to a survey on its own and, to the best of our knowledge, is missing in public literature.We fixed our pragmatic understanding based on literature occurrences, especially in surveys from the game domain and PCG.In addition to our previously given explanations for simple and complete, we are targeting buildings commonly found in human settlements as "common buildings."As a result, complex dungeon and cave systems typically designed for riddles, traps, or battles which in addition are often generated with different requirements by different algorithmic approaches, such as cellular automata, are not subject to our analysis.
PGB is tightly linked to the procedural generation of cities.An often-cited survey work in this area from 2006 highlights that built environments can be an important factor for determining form, function and style of buildings, especially considering the available lot spaces and geometric features shared across a neighborhood [28].Yet, since the consideration of the spatial

TABLE I (CONTINUED)
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE I (CONTINUED)
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE I (CONTINUED)
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE I (CONTINUED)
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

TABLE I (CONTINUED)
and functional relationships between sets of buildings as well as architectural and cultural influences can be seen as parameters and restrictions for PGB given by a city generator, we limited this survey to building generation.We further do not consider aesthetics of any PGB approaches, including, for instance, different styles or visual attractiveness.Whereas such aesthetics are an important aspect of many game assets, they can hardly be captured by a literature-based survey and would require different research approaches, such as user polls.

III. TAXONOMY
As mentioned previously, Togelius et al. [14] introduced a taxonomy for PCG in general.They proposed the following aspects adapted for our taxonomy: automatic generation versus mixed authorship and constructive versus generate-and-test.Automatic generation for PCG means that generation is done automatically, whereas mixed authorship implies involvement of humans throughout the design process.Constructive PCG means that content is generated with the constraints of the overall artefact in mind, allowing to yield usable results in just one pass.On the opposite end, generate-and-test means that content is generated in a loop and evaluated until an artefact has been produced that satisfies all constraints at once.We used these two cited taxonomic aspects as our starting point.They describe how a concrete generation process is utilized to arrive at usable artefacts and, thus, are applicable to all sorts of generative approaches.At the same time, they are closely linked to the concrete use cases of PCG (see, for instance, the goals of PGB, as outlined in Section I).
Despite being important taxonomic aspects, the other taxonomic aspects of the general PCG taxonomy were not further explicitly considered in this survey.While they apply to PGB just as much as for any other PCG area, they only play a secondary role when comparing approaches with respect to their generative outcomes.In that sense, our extended PGB taxonomy can be understood as an extension with slight modifications, not a replacement of the taxonomy of Togelius' et al.The following exclusions and corresponding arguments are not about the general relevance of the taxonomic aspects.They are about their relevance for this survey and the included comparison of PGB generators.
Whether the content is generated during playtime (online) or not is highly dependent on hardware and use cases and none of the analyzed solutions includes online generation based on player behaviors (excludes online versus offline and generic versus adaptive).Whether a building is necessary or not for game progression is a design element and not directly depending on the generator (excludes necessary versus optional).Some publications did not address the presence or absence of stochastic elements without explaining enough details to reliably classify this aspect.In addition, this aspect can usually be switched with little effort: a stochastic generator can be made deterministic by fixing a seed for the underlying random number generator and the other way around inputs and aspects of a deterministic generator can be randomized (excludes stochastic versus deterministic).Degree and dimensions of control is partially dependent on the stochastic versus deterministic, which is not available.Some information about it are included in our aspects means of control and interaction required, though.
To specifically provide a taxonomy for PGB, we aimed for more details and structure by groups of aspects, i.e., specialized characteristics and generative capabilities, related to buildings.Without finding prior works to support this goal, we chose the generative capabilities of the PGB approaches as the primary taxonomic group.Generative capabilities target the different parts of a building and are described in detail in Section III-A.Next, we focus on the approaches' accessibility and usability in Section III-B.Finally, we investigate taxonomic aspects, which provide the basis for a technical comparison of the considered approaches (Section III-C).
Most of the aspects of the proposed PGB taxonomy are directly comparable, i.e., the domains are either binary or categorial.Yet, some also contain free text information to provide a better impression of the analyzed approaches in the overview.Note that several of the aspects can also be thought of as a continuum.They were grouped into categories and binaries for measurability.The presented form was chosen to make the survey more informative, thus it might not be the final and polished form of a taxonomy.We consider it as an elaborated and good basis for such a taxonomy and discuss measures to make a better taxonomy from the presented aspects after their explanation in the end of this section.

A. Generative Capabilities
The terminology we propose for the six generative capabilities below emerged from the goal to avoid ambiguities.Some approaches require user interaction.Such solutions were not excluded and instead the user interaction requirement is one aspect of this taxonomy (cf., Section III-B).For the generative capabilities this has the following effect: there is a fluent transition between design tools and procedural generators.For this taxonomy and the corresponding analysis, it was handled in the following way: manual generation steps are counted toward generative capabilities of the analyzed generators as long as they are integrated with PCG steps, i.e., are part of the same generation-process.We made this decision for two reasons: 1) Many state-of-the-art PGB generators we found during our survey include user interaction and 2) depending on the required details, it is likely that artists will play an important role in addition to PCG aligning generators with use cases and complementing them with their skills.Note that deterministic generation (from input data) is counted as PCG and full user control is permitted.
We consider this aspect fulfilled, if a PGB approach is capable of generating the outline and relative placement of more than one room for a single building.Note that it is possible to have a generator capable of generating floor plans but not outer walls, if the outer walls are a required input for the generator.
Outer walls ( )-Outer walls or facades, as in [32], [33], [34], connect the foundation of a building with its different levels.Based on our survey, the term "facade" is more frequently used than "outer walls."However, it also linked to the processes of generation, extraction, and UV-mapping of textures for outer walls, such as in [35], [36], [37].From a geometric perspective, facades can be generated based on data specifying the enclosing planes.As a common case, 2-D areas depicting the outer walls in floor plans are extruded into the third dimension, therefore generation of the outline of a building is counted as the generation of outer walls.
Texturing ( )-The procedural generation of textures is a research field on its own providing vital benefits for interactive entertainment and computer games [38].Input data, such as parameters, random values, or existing images, are processed and combined to generate new textures [39].For example, recent works used semantic descriptions [40] or real-world examples with convolutional neural networks [41] for texture generation.
Despite its general relevance, we have rarely found PGB approaches with other generative capabilities in combination with the generation of textures, besides the previously mentioned example on facades.Instead, existing textures are often applied during generation, such as in [42], [43], [44].
A PGB approach is considered capable of handling textures, if it generates building textures or applies given textures to at least one of the five building constituents corresponding to the other five generative capabilities.Note that pictures with textures are not taken into account if textures are not mentioned in the text, since textures may have been applied by hand for presentation or have been already applied, e.g., for placed furniture.
Roofs ( )-Roofs have been the sole generative targets of PGB approaches as in [45] or alongside outer wall generation, as demonstrated in [46].
We consider PGB approaches capable of handling roofs, which generate anything more elaborate than flat rooftops, i.e., created by means of planar triangulation.
Stairs ( )-The positioning of stairs in a multilevel building is a challenging task.Accessible areas, floors, and openings at different levels have to be carefully set and aligned, and the flight of stairs itself has to be generated accordingly [34].
A PGB approach is considered capable of handling stairs, if it can generate or select and place a flight of stairs to provide an effective functional geometry for transitioning between two or more floors of a building.
Furnishings ( )-As stated by all three basic building definitions quoted previously, furnishings are part of complete buildings (cf., Section II).Some PGB approaches make use of agents optimizing the placement by moving around, e.g., in [47].Others optimize global cost functions quantifying the quality of arrangements, as in [48].
We consider a PGB approach capable of handling furnishings, if it can place instances of at least one type of furniture.

B. Accessibility Aspects
Generative modeling is related to 3-D modeling and programming [49].Such software historically had issues with usability and productivity [50].The acceptance of procedural modeling has been persistently hindered by the lack of intuitive controls and disparity of isolated techniques that generate only specialized features [27].
To this end, we introduce the group of accessibility aspects comprising the following three aspects of PGB approaches that impact dependencies, applicability, and thus, accessibility and usability.
Means of control-PGB approaches are often difficult to control [51].They often require the definition or adaptation of nonintuitive rules and input parameters.This may result in steep learning curves for mastering the necessary, underlying technology [27].They often also rely on statistics or effects of structural emergence, both of which may result in effects that are difficult to anticipate.Many current procedural approaches utilize a special purpose language to encode the parameters and behaviors of the generative procedures [50].In contrast, a graphical user interface (GUI) can facilitate the work by visualizing the intricacies of a specific approach.For instance, it might highlight the configurable flow of data, provide instant feedback on possibly erroneous input data, or give real-time feedback on the effect of various introduced changes [52].
Considering these diverse challenges at the application level, we address the corresponding skill requirements of a PGB approach.To this end, we introduce the means of control, which covers a range of five values as follows.
1) GUI means that a GUI is provided for applying the respective PGB approach, e.g., working with a multiwindow editor, as in [53].2) Special purpose language means that users have to learn and use a special purpose language, such as the selectionexpression commands of the procedural modeling language SelEx, as in [54].3) Formal configuration means that users have to design formal rules, encoded, for instance, in logical expressions, or mathematical constraints, as in [55].4) Parametric configuration means that the core task of the user is to tweak parameters of the generator, e.g., setting parameters such as building width, length, shape, etc. for model instantiation, as in [34].5) Data provision means that the users have to generate or collect data, such as building images, existing floor plans, or sensor-based point clouds.For instance, [56] derives grammatical rules from sensor data to generate floor plans.Most approaches require several, maybe even all of these means of control.We made this choice based on the effects Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
regarding skill requirements and level of control over the final outcome.
For example, in [57], users define building profiles in an interactive editor, but can also set a few parameters, such as a width for repetitive elements.We considered interactive visual modeling to be more decisive for users tasks than setting simple parameters.We know that this cannot be fully objective for general publications, but consider it helpful, especially for comparison.
Interaction required-According to Smith [58], the interaction type is one of the mechanical aspects of PCG and can range from no human interaction (i.e., None), as seen, for instance in automatic furniture layout generation by means of a genetic algorithm [59], to human input to actively altering the generated content (i.e., direct manipulation), as in [53].
Although in her elaborations, Smith focused on gameplay, she pointed out that the classification of interaction is equally applicable to a potentially independent design process.While we express the type of interaction and control in the means of control previously, we now introduce an aspect that captures the need for interaction during the actual generative process.Configuration or programming tasks that take place before the generative process itself, as well as optional opportunities to interfere with contents or the process during generation are not taken into account.Considering any required interactions is all the more important due to the general need for automation as one of the main motivations for PCG-to overcome resource limitations, to create infinite worlds, or to increase replay value [9], [60], [61].With this aspect we show whether interaction is required or not as binary information.
Based on external data-By external data we refer to data that is provided to and processed by a PCG approach as, e.g., described in [27], [58].Providing some given data may simplify the overall process in terms of amount, adaptation, and diversity of the generation efforts.Yet, the availability of the respective datasets, in sufficient type, number, and quality, needs to be warranted in the first place [55].Some approaches might only require external data once, e.g., to train a Bayesian network one time and to use it repeatedly in subsequent generative steps [30].Other approaches require new data for each execution, for instance for creating architectural geometry from images, as in [62].Although there may be edge cases, we do not consider expert knowledge of system developers or users as external data.These deliberations result in three possible values as follows.
1) Execution means that external data are required for each run and thus for every generated artefact or set of artefacts.2) Initial means that external data are initially required for configuration or learning.3) None means that no external data are required at all.

C. Technological Aspects
In general, it is difficult to choose the right technology due to the increasing number of alternatives and complexity [63].When creating procedural generators, the central choice concerns the selection of data structures and technological approach, i.e., types of algorithms, such as formal grammar processing or procedural growth.Depending on the underlying representation, certain modeling operations may be difficult or impossible to implement [50].Addressing the crucial technological aspects of generators, this group of aspects contains information about the technical details of the evaluated PGB approaches.Our survey includes the following four aspects within this group.
Shape description-According to Havemann, none of the existing shape description methods is entirely satisfactory.He also roughly divides the methods in two classes.The first class follows the "list of primitives" approach to describe threedimensional objects and scenes.Compositions of elementary objects, such as points, triangles, cubes, spheres, and many others are used.The second class uses procedural shape representations including formal grammars, shape programming languages, physically based simulations, and others [64, p. 1].These classes can occur together: A formal grammar that places given primitives instead of generating everything from scratch combines elements of both classes.Considering these challenges in addition to the restricted scope of our analysis (focussing on PGB), our approach was to collect the methods used in the analyzed publications and manually group them by similarity.This allows us to provide detailed information about the shape description methods used for the PGB without having to describe all possibilities in general.
With this technological aspect we provide categorial information about the geometric representation handled by the system with six possible values as follows.
1) Points and vectors means that points and vectors are manipulated.For example, existing objects can be placed or polygons can be modified by procedural growth algorithms, as used in [34].2) Polygonal surfaces means that geometric shapes are processed as polygonal surfaces and altered by geometric operations, such as splitting or Boolean intersections.This was done with formal grammars using convex polyhedra to generate buildings [65].3) Parametric representations means that folded or encoded parametric content, such as geometric functions, is used for generation.For example, spline curve mesh generation can be used to construct complex roofs [66].4) Grid-cells/voxels means that occupation matrices are processed either in 2-D or 3-D, such as using grid cells for room growth, as in [31].5) Predefined building blocks means that prepared assets are arranged and properly combined.Van Aanholt et al. arranged tiles (e.g., stairs and wall fragments) by a solver according to their properties [55].6) Pixels/images means that images or pixels of images are processed for generation.For example, facade reconstruction can be done based on real world images [35].Note again that several of these values occur in many of the approaches and we picked the one seeming to be the most prominent one regarding the representation of the final geometry.For example, in [66], spline curves for geometry and component instantiation (based on the spline curves) are used.For the classification, parametric representations was chosen to be more important than component instantiation for the procedural generation, since the latter is more restrictive and dependent on the first one.We know that this cannot be fully objective for general publications, but consider it helpful, especially for comparison.
Data structures-The selection of data structures not only influences the computational efficiency, but also the ease of implementation by supporting or disallowing different basic operations.For example, tree structures grown through grammatical production can provide semantic, contextual information, such as groupings and compositions, and support attribute management by inheritance.In contrast, octrees are typically used to capture and describe spatial relationships, allowing for geometrical selections and operations.One of these operations is efficient solving of the interference problem, i.e., the detection of different objects occupying the same section in space [67].
By means of this aspect, we provide free text information about the central data structures of PGB approaches addressed in the corresponding publication.
Core technique-Different fundamental approaches of PCG determine many aspects of the generators' application as well as the result space.For procedural buildings, Schwarz and Müller [46] argued that grammar-based approaches are most common.Grammars can easily express hierarchical dependencies, which often occur in human-made architecture and foster repeated structures.In addition, grammars have several known challenges and limitations: Dependencies can be unclear and grammatical rules can be cumbersome to handle [68], (nonrepeating) details can be difficult to implement [66] and different derivation branches are hard to coordinate [54].Several other approaches also occur in PGB, such as (re-)construction based on images or point clouds, interactive sketching or arrangement, procedural growth algorithms, agent-based placements, and machine learning-based approaches.All these approaches have their own advantages and disadvantages, making the core technique a distinguishing technological aspect.While the type of application and also the user skill requirements are addressed by the accessibility aspect's means of control of our taxonomy, they are also affected by the core technique.As an example, a formal grammar requires the definition of production rules in an according syntax.If one tried to keep this chore from the user, the rules would have to be generated with different degrees of interactions required.
The core technique aspect captures categorial information about the core principle used in the PGB approach with four possible values as follows.
1) Substitution means that details of the buildings are defined by (hierarchical) substitutions.These are usually found in approaches based on L-systems (parallel rewriting systems motivated by cellular growth originating from 1968 [69]) or shape grammars (sequential rewriting systems using shapes in form of polygonal surfaces as geometric representations conceived by Stiny in 1971 [70]).2) Agents means that agents are used for generation.For example, agents can represent furniture optimizing arrangements as in [47].
3) Rules means that rules given by constraints, ontologies, etc. are the core concept of generation, such as rule-based layout solving [29].4) Machine learning means that the utilization of data by means of machine learning is the core concept of generation.For example, convolutional neural networks can be used for floor plan generation, as done in [71].Validation and optimization-This aspect provides categorial information about a PGB's method to find valid or high-quality solutions with three possible values as follows.
1) Constructive means that the system directly generates valid or high-quality solutions (with high probability), such as building blueprint generation in clear phases from start to end as in [72] 2) Automatically evaluated means that the system repeatedly generates solutions and assesses their fitness to find fitting ones.Yu et al. used a cost function to evaluate the fitness of samples from the search space for furniture arrangements [48].3) Designer evaluated means that users interactively have to care about the quality of the results, such as interactively drawing building profiles in an editor, as in [57].This aspect is taken from the literature in which it was introduced as one of several taxonomical aspects with constructive and generate-and-test as values [10], [14].We encountered several approaches, which used PCG techniques for the assistance of users in an interactive process.Using these approaches, the systems do not care for the quality of the results alone.Instead, they provide capabilities and delegate responsibility to their users, resulting in a foundational difference in terms of guiding the generation efforts.Thus, we split generate-and-test in the two values automatically evaluated and designer evaluated to address this important difference.

D. Meta Aspects
In addition to the taxonomical aspects, we provide one meta aspect, which we captured during our survey.We provide the code accessibility to ease getting in touch with generators and to estimate the necessary efforts for doing so.
Code accessibility-Although software is an integral part of modern research, its publication, acknowledgement, and citation are not common practice [73].Regarding the importance of source code, it has been claimed that all scripts and parameters have to be published to ensure scientific reproducibility [74].Even with published code of poor quality, other researchers are enabled to engage in corresponding research [75].This is the only aspect that is independent of the operating principles of the PGB approach and thus not a taxonomical aspect for PGB generators.As such, it is not used to compare the solutions directly, but gives indications about the efforts scientists and practitioners have to invest when starting to work with the approaches.It also indicates how precise the core of the approaches can be (re-)created compared to the version of the corresponding publication.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
This aspect provides categorial information with three possible values as follows.
1) Not accessible means that no code is provided.
2) Partial means that samples of (pseudo-)code are given as parts of the publication.3) Open source means that major parts of the source code are made available on the Internet.Claims with broken or dead links provided in the literature were not considered as open source.A few of the taxonomic aspects are not consistent in a way a robust taxonomy would require them, i.e., yielding the same results when different persons would use it for categorization.We included these aspects in the presented form for making the survey more informative.For making the taxonomy more robust and independent of this survey, the following steps could be taken.First, data structures, being a free text aspect, could be formalized for clear categories or left out.Second, the shape description and means of control could be left out or made a multiselect, providing separate binary information for each of the provided values.

IV. PROCEDURAL GENERATION OF BUILDINGS
In this section, the results of our analysis are presented.The underlying classification data for all graphics of this section are provided in Table I.Although our 71 samples are not sufficient to reveal reliable statistical insights, these overviews are considered helpful for an understanding of the domain in terms of relevant features, commonly used approaches and trends.We briefly discuss relevant publications prior to 2006 in the beginning of this section.
A cornerstone for PGB was published in 2003 [118].The authors observed the suitability of L-systems for other procedural content, such as plants and streets.Compared to these contents, they highlighted the different structure of buildings and proposed split grammars for procedural architecture.Split grammars are parametric shape grammars using rule type restrictions and controlled randomness.They are based on the work of Stiny et al., who introduced shape grammars a few decades earlier with a focus on visual arts and two-dimensional applications [70].
Later, grammar-based approaches were reported to be primarily used for PGB, hierarchically applying substitution rules [46], especially for building exteriors [26].Our data confirms this claim with substitution occurring 24 times (see Fig. 1).22 of these 24 occurrences of substitution appear in combination with outer walls, which is 22/45 ≈ 49 percent and 16 occur alongside of roofs, which is 16/33 ≈ 48 percent (cf., Fig. 2 for the occurrences of the generative capabilities).In contrast, ten occurrences appear alongside floor plans, which is 10/29 ≈ 34 percent and only one with furnishings (1/18 ≈ 6 percent), which in turn are often handled by constraints and fitness functions (cf., Fig. 7).In other words, different core techniques are usually used for the implementation of different generative capabilities.This indicates differences in the problem structure of the implementation of different generative capabilities.
Already more than a decade ago it was observed that most solutions for procedural buildings focus on the exteriors [48], [72], Fig. 2. Graphical overview of the generative capabilities: For each capability, the count of approaches implementing the corresponding building constituent is given.The count of each value is given as a number in the corresponding colored bar and the count of approaches not being able to generate the corresponding building constituent is given as a number in the gray bar.[119].This observation was reasserted in recent years [103] and is again affirmed by our data.This can be seen in Fig. 2, which shows the number of solutions implementing each generative capability.
Of the 18 approaches implementing furnishings, seven accompany outer walls and four accompany roofs (see Fig. 5).In contrast, outer walls and roofs appear together 30 times.We refer to this phenomenon as the "interior-exterior-gap."This can be seen from a different perspective when investigating the relative pairwise occurrences of the generative capabilities shown in Fig. 6.Some of the generative capabilities commonly occur together: Roofs are rarely implemented without the outer walls (91 percent) and stairs are rarely implemented without the outer walls (94 percent) and roofs (82 percent).The other way around, outer walls are rarely handled together with stairs (36 percent) and furnishings (16 percent).Generally, furnishings appear more rarely together with other generative capabilities.This can also be witnessed when investigating the more extensive PGB solutions: Only two of the 13 solutions with four or five generative capabilities implement furnishings (see Fig. 10 and Table I).With furnishings being the most detailed interior aspect of the six generative capabilities, this emphasizes the interiorexterior-gap.These observations also give a second indication for the differences in the problem structures of the different generative capabilities.
From a technological perspective (see Fig. 1), the problem domain of procedural buildings has a strong rule-based character according to the values of the core technique aspect: Rules occur 29 times and the special case of applying replacement rules by means of substitution occurs 24 times, totalling to 53 out of 71.This shows a distinguishing feature of buildings compared to other areas of PCG.A possible cause is that buildings are usually constructed by humans in a structured manner following rules and implications of the corresponding culture and environment, such as climatic conditions.Such rules are often explicitly expressed by construction planners and architects, and they may be formalized for automatic rule checking of building designs [120].This is also a possible cause for the rare occurrence of evolutionary algorithms and other approaches which behave well for complex challenges which are, on the contrary, typically hard to formalize.For the broader field of PCG, Togelius et al. conducted a survey and identified an overrepresentation of evolutionary approaches [10].
Machine learning-based core techniques follow with 16 occurrences showing a clear upward trend over time (see data provision in Fig. 8 and machine learning in Fig. 9).This aligns with the advances of machine learning and artificial intelligence in science and practical applications especially over the past decade.This was also noted by Summerville et al. in their survey on PCG via machine learning published in 2018, but the authors also emphasized that much work has to be done, especially since most efforts so far concerned 2-D levels [121].Machine learning still is an area of rapid evolution with many novel concepts currently being investigated and not all of them require (plenty of) data for application.An example for such a solution is the open-ended, novelty-guided evolution of Minecraft buildings, using autoencoders for 1-D representation of 3-D structures [122].This is based on deep learning novelty explorer (DeLeNoX), a system that autonomously creates artifacts in constrained spaces according to its own evolving interestingness criterion [123].
Regarding validation and optimization, the values are more balanced: All three values occur rather often with a minimum of 18 for automatically evaluated and a maximum of 31 for constructive methods, where the latter directly generates a (valid) result (see Fig. 1).The designer evaluated value often accompanies the GUI from the means of control and likewise interaction required (see Table I).Both these accessibility values, GUI and yes for the interaction requirement, are frequent values within the accessibility aspects (cf., Fig. 3).Considering the values designer evaluated for the aspect validation and optimisation and yes for the aspect interaction required it can be seen, that many current solutions do not implement or aim for fully automatic solutions (cf., Figs. 1 and 3).However, full automation is not always a goal.Some even consider the worst case for the level of control over a procedural approach to be limitation to the definition of the model and its parameters [27].Some of the approaches directly name artists as their target group and leverage procedural modeling as a means to augment the artists' skill sets.For example, a grammar-based approach can be combined with a visual workflow to enable artists to create buildings on a GUI level [53].
For practical application and research, code accessibility is important for the utilization of the corresponding approaches.Although it can foster future research or application, many authors may be interested in maintaining their edge in research experiments by not sharing their implementation work, future commercial usage or simply do not want to prepare their code for public presentation.Open source rarely occurs (3 out of 71), 29 publications do not provide any code and the majority shares parts of the code within the publications.
Besides the aforementioned machine learning trend, we could not identify other clear trends in the distributions of the occurrences over time, neither for the technological aspects nor for the accessibility aspects.In addition to the two printed trend diagrams in this article (cf., Figs. 8 and 9), all other binary and categorial aspects were investigated for trends without the identification of further peculiarities.

A. Challenges and Limitations
Challenges and limitations are important for choosing an approach for implementation and application.They can also be seen as open research questions, guiding the direction of future research.Many published approaches have limits in what they can produce or handle.For example, several approaches can only produce rectangular rooms, others are restricted to convex rooms [65], [124].More complex geometry can be generated by operations, such as Boolean subtraction, which extends the capabilities of the approach.This often requires handling of special cases in the underlying data structures to maintain the required invariant properties and may come with numerical challenges during implementation [112].
Furthermore, different solutions are suitable for different challenges: Grammar-based solutions, as stated in the literature and confirmed by our analysis (cf., substitution as the core technique Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply. in Fig. 1) are a very common approach for the PGB.However, they usually are not used for the implementation of each and every generative capability.In particular, furniture placements are typically done without the usage of grammars and use different rule systems or some kind of self-organisation, such as agents, whereby each agent represents a piece of furniture, optimizing local goals to find proper placements (see Table I).In contrast, self-organization and evolution are rarely used for the geometric generation of architecture.
Shape-based grammars as the most popular concrete approach for implementation also come with challenges when generating architecture, most notably reaching a realistic level of architectural complexity.This is often discussed, also as an open challenge from publications providing grammar-solutions themselves [46], [98].While performing well at producing repetitive structures and hierarchies, grammars struggle at the generation of complex details [82], [89].The complexity of grammars can also affect the users-they often have to learn and apply scripting in a grammar language [36], [37].One very important challenge for which no conclusive, scalable solution has been presented so far, is the selection problem: Hierarchical structures resulting from shape splitting operations are independent and unrelated, which renders it difficult to maintain relations between different subtrees during generation or manual postprocessing [44], [54].Approaches that address this challenge often include complex tree operations on the substitution tree.This has led to a search for extensions or different solutions: Jiang et al. replaced the traditional grammar approach with label-based selections.Therein, several hierarchies are maintained as views and a selection-based language allows for complex selection queries [54].
In general, we could not identify a single approach ready for generating complete buildings without further research and development efforts.

B. Generating Complete Buildings
Depending on the use case, different parts of buildings are of importance.A survey on PCG in 2017 lists entire accessible buildings, but states them to be only vaguely discussed [11].The freedom associated with open world games is perceived more intensely when the players encounter fewer barriers that prevent them from exploring surroundings [125].One of the most common types of barriers in urban settings are certainly buildings that the player cannot enter and that are only represented by facades [25], [106].For this reason, traversable buildings with interiors shift into focus [34].
Bidarra et al. compared manual modeling with PCG in terms of strength and weaknesses.Completeness was listed as a strength of manual modeling, but not of procedural modeling [126].Our results indicate that this also holds for the area of PGB (cf., Fig. 4).Numerous PGB approaches have one or the other generative capability.Similar approaches are typically pursued to implement similar generative capabilities.In particular, interior-related content generation is very different from generating exteriors, such as facades [27].We have also found indications for this based on the analyzed publications, discussed in the context of the interior-exterior-gap (see Section IV).Only a few approaches consider several or all aspects at the same time.Either such approaches are ensembles of different generators or the results are not convincing in terms of their restrictions or results.Two publications were found addressing all six building constituents [25], [84], however also exhibiting some limitations.In addition, several solutions exist for every building constituent, as can be seen in Fig. 2. Thus, it is theoretically possible to procedurally generate complete buildings   using the current state-of-the-art.But it is not easy to do, since in order to generate complete buildings different approaches have to be used together-and they offer hardly any infrastructure to support such a multifacetted engineering process.
At first glance, this renders the two publications covering all six generative capabilities all the more valuable.In 2010, Santos et al. proposed an ensemble approach based on external data and user interaction.Plans or photos are processed by automatic scale detection and are snapped to a spatial grid.The user can place staircases as floor transitions, draw walls with the cursor, define floor types and place assets like windows, doors and interior objects, etc. [84].In 2011, Tutenel et al. proposed a semantic approach to integrate different partial solutions by means of a semantic moderator.It requires the adaptation of partial solutions to work with the moderator concept.The semantic moderator is a central instance that coordinates the different approaches by means of a message bus: Communication, assessment, and combination of parts are handled by the central semantic moderator [25].Both solutions address the important question of how different approaches can work together-from a user experience and method integration perspective.Yet, they do not describe the interfaces between building constituents in a detailed and formalized way.This might explain why, so far, they have a comparably low influence on the other solutions we encountered: Santos et al. have not been cited and Tutenel et al. were cited twice within the set of analyzed publications.The average citation of all analyzed publications was 3.82 times, and the average citation of all analyzed publications up to 2015 was 6.2 times (cf., Fig. 10).
These observations suggest that a focus on the integration of different approaches currently is the most promising option for the generation of complete buildings.Consequently, we infer that compatibility, i.e., the capability of a generator to integrate with other approaches, is one of the most important characteristics of PGB approaches to contribute to the generation of complete buildings.
Compatibility can be achieved in different ways.One possibility is the adaption of a semantic moderator as proposed in [25], utilizing a central instance for integration of different approaches.Those approaches do not need to care (too much) about each other, but only implement interfaces for the moderator, which then handles the coordination of the generation and composition.
Alternatively, interfaces between different generative capabilities can be optimized for compatibility by the availability of standardized or flexible information (e.g., parameters for directions, sizes, etc. for staircases) or the absence of complex details in the interface areas, such as cornice at the interface between walls and roofs.
Another possibility is to generate building constituents in several steps starting with minimalistic generation, adding details in later steps considering the (intermediate) results of more than one generator to achieve integrated results.
Another option is the (compatibility for) deep integration between different approaches, e.g., by generating more than one building constituent together either with one algorithm or more than one optimized for their interworking.This approach most probably has the highest required effort, but also the potential for the best results since deep integration includes the possibility of complex details including two or more generative capabilities.
Each of these alternatives for increasing compatibility has different requirements for the corresponding interfaces.Having well-organized, standardized interfaces can help to keep the integration efforts manageable.It can affect the usage of elements such as data structures, defined formats, provided measures, or a given hierarchy organizing the assignment of details to Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.Fig. 10.Overview of the 71 analyzed publications with their implemented generative capabilities and mutual citations.For each publication, the number of citations within the set of the 71 publications is visualised by the radius of a circle which is calculated logarithmically from the number of citations.Each publication has a unique color for the circle and connections to all publications citing it.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
building elements.Also, generic constraints could be supported by such a standardized interface allowing for an integrated control of the interplay of different generators as well as the users' expectations.

C. Design Decisions and Tradeoffs
When selecting solution approaches or configuring given approaches, the capabilities and limitations of the resulting system are determined.This section discusses such design decisions and also tradeoffs coming along with them.
There are only a few PGB approaches implementing several or even all generative capabilities and most analyzed solutions have additional limitations, such as restricted shapes.Depending on the use case, one or several approaches have to be selected to fulfill the given requirements and design goals.Their selection not only affects the results directly, but also indirectly by determining the approaches' usability, the necessary efforts for application, their compatibility among each other and the interplay with other virtual assets from the environment of the buildings.Our classifications (see Table I) can support these decisions by matching given requirements to individual approaches.To summarize, the decision process can be driven by PGB approach classifications with use case-specific pros and cons and subsequent optimization of the faced tradeoffs.
In the following, we list design aspects we identified to be important and discuss their tradeoffs and effects, such as efforts and restrictions following certain decisions about selected technologies.Some of these directly come from the taxonomical aspects presented in Section III.Others are derived from the survey results.
Scope: The most obvious and nonetheless very important decision is about the scope of the solution, i.e., the decision about the generative capabilities and included details for generation.The number of different elements as well as the difference in the problem structures is likely to limit the solution quality or drastically increase the generators' complexity.In addition, scope complexity can increase when going into detail.For example, some of the implementations for furniture arrangements have rules for specific types of furniture.Additional types of furniture would either require additional implementation or even exclude the specific solutions we analyzed.
Compatibility: As stated in Section IV-B, compatibility may be one of the most important characteristics, at least when aiming for complete buildings.The strategy to ensure compatibility among complementary PGB approaches impacts the set of currently viable solutions to choose from.While some are easy to integrate, others come with intense challenges to do so.Those approaches with more flexible means of control and interaction required, are inherently easier to adapt.A rather challenging example nicely illustrating the issue would be a solver-based solution for outer walls and roofs.In addition to the given design constraints, it would have to provide and optimize additional interfaces to building interiors to be compatible to other solutions.
Interaction: Depending on the goal, PCG can be used to automatically create content or to augment the users' design capabilities.The involvement of users can allow for desirable, high-quality solutions.But if the users provide inconsistent input or if the generative capabilities of the given approach simply do not meet their expectations, the process might fail as well.User involvement can thus be seen as a stream of introduced constraints increasing the computational challenge.Alternatively, considering an illustrative example, the users' introduction of assets or process configurations may also accelerate the convergence of the search process or even extend the search space.
Data usage: The advances in data-based approaches, such as the availability of sensor-systems and successes in artificial intelligence have also yielded promising approaches for PGB (see trend discussion in Section IV).Incorporating (real world) data can lead to high-quality results and enable generators to take rules into account which humans have not even thought about, such as implicit aesthetics for furniture placements.On the other hand, and despite the increased availability of data in general, the availability of fitting data in adequate quantity and quality may still be a restriction and thus prevent such solutions from being used, especially if data is not only required for initial configuration or training, but also for each execution and thus for the generation of each artefact or set of artefacts.In addition, the fundamental challenge of uncertainty has to be considered for many data-based approaches.In particular, currently trending, induction-based machine learning models do not maintain ontological or logical knowledge.Instead, they just learn to reduce their error for approximation of likely results based on seen data.

V. CONCLUSION
In this article, we have presented an extensive survey on PGB.For categorization and comparison, we started with the creation of a procedural building taxonomy, which consists of 13 taxonomical aspects for procedural buildings in three groups: 1) generative capabilities (floor plans, outer walls, texturing, roofs, stairs, and furnishings), 2) accessibility aspects (means of control, interaction required, and based on external data), and 3) technological aspects (shape description, core data structures, core technique, and validation and optimization).In addition, the code accessibility is given as an additional meta aspect.These aspects were inferred from relevant literature and, in turn, were used to classify 71 state-of-the-art approaches.We presented the survey's results taxonomically and in terms of analytical figures.The interplay of several aspects, the effects and reasons of their combined occurrences as well as trends and best practices were discussed.
An important goal is the PGB of complete buildings.We provided evidence that relying on current solutions; it is possible to achieve this goal.However, our survey also suggests that it is not easy to do so, requiring potentially great efforts for the conceptual and programmatic integration of complementary approaches.Due to the different nature of the various involved subproblems, compatibility, i.e., the capability of an approach to integrate with other approaches, is an integral requirement and an important quality criterion.We emphasized this insight in our discussion on design decisions and tradeoffs when selecting PGB for concrete applications.

Fig. 1 .
Fig. 1.Graphical overview of the categorial values of the technological aspects.The count of each value is given as a number in the corresponding bar.

Fig. 3 .
Fig. 3. Graphical overview of the categorial and binary values of the accessibility aspects.The count of each value is given as a number in the corresponding bar.

Fig. 4 .
Fig. 4. Counts of approaches with certain numbers of generative capabilities.The counts are given as a number in the corresponding bar.

Fig. 5 .
Fig. 5. Occurrences of generative capabilities in absolute pairwise combinations in matrix representation.Each cell shows the number of solutions implementing both, the generative capability listed in the corresponding row and column.The matrix is symmetric by definition and redundant symmetric values are given for reading usability.

Fig. 6 .
Fig. 6.Occurrences of generative capabilities in relative pairwise combinations in matrix representation.The cells of the matrix show the percentage of bilateral occurrences relative to the single occurrences of the generative capabilities of the corresponding rows.For example, 71% of all solutions implementing stairs also implement floor plans.The percentages are rounded to integer values.

Fig. 7 .
Fig. 7. Relative distribution of core techniques for the generative capabilities.

Fig. 8 .
Fig. 8. Manifestations of the means of control with counts per year.Note that for the last year 2022, only publications before June 1, 2022 were considered.

Fig. 9 .
Fig. 9. Manifestations of the core technique with counts per year.Note that for the last year 2022, only publications before June 1, 2022 were considered.

TABLE I INVESTIGATED
SOLUTIONS FOR PROCEDURAL BUILDINGS AND THEIR TAXONOMY-BASED CLASSIFICATIONSAuthorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.