Automation of 3D average human body shape modeling using Rhino and Grasshopper Algorithm

The aim of this study is to develop an automated process for modeling average 3D human body according to body types using both NUBRS-based modeling software Rhinoceros 3D® (Rhino) and Grasshopper as an algorithm editor. First, we categorized men aged 36 to 55 years included in SizeUSA 3D data into the three body types (normal, overweight, and obese), and selected seven samples in each body type. To execute the automated process of generating an average 3D model of their lower bodies in a step-by-step manner, the following procedures were performed: (1) Determine the main reference lines on the 3D-scanned lower bodies, including six horizontal reference lines and six vertical reference lines; (2) Create horizontal and vertical line grids and intersection points (3) Generate an average 3D model in a position that corresponds to the average coordinates of the intersection points (vertex coordinates) of seven samples for each body type. A Grasshopper algorithm was formulated to automatically execute all procedures that had to be repeatedly performed. As a way to verify the average model’s size and shape, the girth measurements of the samples for each body type were averaged, and the results were compared with those of the 3D average body shape. It was found that the deviation was less than 1 cm, which indicates the validity of the 3D modeling approach applied in the present study. Each process was incorporated into commands available in the Rhino interface, and this automation allowed a number of 3D body shape modeling operations to be implemented in a significantly reduced time period.

. Statistics-based methods and 2D silhouette images-based methods have limitations in modeling 3D models since the human body involves a wide variety of 3D shape-related factors. The morphing techniques have been used the most in the following way. Polygons near the morphing points are morphed to form an average shape so that the empty space between the template shell and the reference shell can be filled. These techniques involve a problem, in that the ratio of the shape data reflected in each phase needs to be adjusted differently (Shin et al. 2011). Addressing the problems of morphing techniques requires developing a method that can average multiple entities at the same time instead of executing the same process through one-on-one matching. Also, it is necessary to develop a method to be able to generate 3D average models in a flexible manner and to keep errors consistently low whenever sample scan data are changed.
With respect to 3D human body modeling applications, polygon-based 3D modeling software has been widely used, but it is only able to reproduce curves composed of straight lines (Kwon et al. 2017). In fields where precision work is needed, such as geometric modeling and parametric design in architectural and engineering fields (Hsu et al. 2015), non-uniform rational B-splines (NUBRS)-based 3D modeling software is appropriate. NURBS conversion of human body has been discussed in several previous works (Dekker et al. 1999;Ju et al. 2000;Sul & Kang 2010;Wang et al. 2005). However, their modeling process was not effectively employed since each modeling step should be performed individually.
Therefore, the current study focused on the Rhinoceros 3D ® (commonly abbreviated to Rhino) software including the graphic algorithm editor, Grasshopper as a plug-in, which can provide the user with the ability to interact with the 3D system. The algorithm can be developed by arranging visual components, which correspond to commands used in Grasshopper. It can provide a convenient and visual approach to allow for continuous performance by connecting input and output parameters using "wire" components without writing lengthy programming language (Eltaweel & Su 2017). The advantage of the use of Grasshopper is that the results of the algorithm can be viewed directly in the 3D Rhino interface. Details of changes can be viewed on the Viewport of Rhino in real-time. There was no study to adopt this graphical user interface (GUI)-based surface interpolation using commercial software and its plugin for 3D human body modeling in the apparel field.
In this study, we proposed a whole process for 3D body scanning, from reference line designation to average 3D model generation using Grasshopper, and designed commands to execute each step of the process in the Rhino interface. We formulated the algorithm to automatically execute all procedures that had to be repeatedly performed. As a way to verify the validity of the generated average model, the measurements of each main body part of the average model were analyzed and compared with those of the sample scans for each body type.
A template-based morphing technique using 3D scan data has been widely used as a method. This method generates a new model by deforming a template model and scaling per its segments to create various body shapes (Baek and Lee 2012). This method was originally developed for modeling heads and faces (Decarlo et al, 1998;Blanz & Vetter 1999), and then it has been adopted to the whole body modeling. This technique calculates the position of morphing points on the two different models and attempts to morph polygons near the morphing points to form an average shape. Praun et al. (2001) established an n-way correspondence between arbitrary meshes of the same topological type with feature landmarks, but their method could not generate consistent results due to a number of holes on the body scans. Therefore, Carr et al. (2001) and Davis et al. (2002) recommended direct hole-free reconstruction methods, but the region between the legs was still challenging. Therefore, Allen et al. (2002) subdivided the template and re-parameterize the surface by sampling it along the template normal to construct a set of placement, and finally perform smooth filling in displacement space.
However, the morphing technique found in the previous works has an issue. The ratio of the shape data of the template shell to that of the reference shell needed to obtain an average shape is set to 50%. However, when 50% of each model's shape data is reflected, as above, the same ratio will be applied to the final model only if the number of sample models is 2 to the power of natural numbers. If not, there is a possibility that the characteristics of either one of the two models may be reflected more in the final result, and thus the ratio of shape data reflected in each phase needs to be adjusted differently (Shin et al. 2011).
The second modeling method that has been widely used is statistical methods. Blanz and Vetter (1999) collected 3D human face scan data and extract the variables determining the facial shape using Principal Component (PC) analysis. And they could generate 3D morphable face model. Then, Allen et al. (2003) applied this approach for the 3D human body modeling. They could create a morphable model by developing an objective function which is a weighted combination of three measures: proximity of transformed vertices to the range data, similarity between neighboring transformations, and proximity of sparse markers at corresponding locations on the template and target surface. But their method did not include a technique for generating a new model that satisfies the input constraints.
To overcome the limitation, Seo and Magnenat-Thalmann (2004) correlated the body sizes with the body shapes through a radial basis function (RBF) network, and they found the relationship between them. Then, they could develop a new model with userspecified body sizes by combining the example shapes in their database. Wang (2005) also proposed the similar way, but they first defined the feature points to construct a wireframe. By parameterizing the 3D body scan models, they described the whole body shapes of every model in their database with the patches interpolating the feature wireframe. Furthermore, in the model generation process, they adopted an optimization method to minimize the gap between the input body sizes and the actual body size of the resultant model. Through such an approach, they could establish a new framework for generating a feature-based mannequin model. Baek and Lee (2012) also employed a feature point-based shape analysis technique and statistically analyzed the relation between the body sizes and the body shapes. Furthermore, using an RBF network-based surface approximation method, they could generate a body shape surface in a rapid manner. Chu et al. (2010) extended the parametric modeling method by generating the regression function that relates the body sizes and the body shape models.
However, the statistical approach has been found to be less effective at reflecting any human body characteristics beyond the scope of the anthropometric data used to develop the corresponding model. Also, it has been found that it is difficult to predict and reproduce the resultant model on the 3D CAD application.
The third representative 3D modeling method is based on 2D silhouette images. The 3D body model can be constructed from a set of images obtained from multiple angles (Hilton et al. 2000;Kakadiaris & Metaxas 1998;Lee et al. 2000, Wang et al. 2003. As a recent work, Ji et al. (2019) used a Convolutional Neural Networks (CNN)-based regression network to design an approach where a 3D human body shape can be estimated and modeled using front-view and side-view 2D silhouette images of the human body. To improve the comprehensiveness of the human body shape, 3500 human body scan data were augmented through linear interpolation, and K-Means clustering was employed to create a certain number of samples. Furthermore, PC analysis was applied to perform sampling from the normal distribution. Although their method can reproduce a wide range of body shapes from silhouette images, because there is a large dataset available to train a dense CNN-based network, there are still some types of shapes that cannot be reproduced in the built body shape.
The issues that were found in the past studies make it necessary to develop a new method that can average multiple entities at the same time instead of executing the same process through one-on-one matching, and thus prevent errors that may cause shape modeling results to vary from one mesh model to another. Also, a new approach needs to be developed to reflect any human body characteristics beyond the scope of the anthropometric data used to develop the corresponding model.

Rhinoceros 3D ® software and Grasshopper plugin
Commercial software packages have generally been used for 3D modeling, but recently, Robert McNeel & Associates's Rhinoceros 3D ® (Robert 2019), a non-uniform rational B-splines (NUBRS)-based 3D modeling software package, has attracted significant attention from industry and research for its ability to reproduce complex geometric shapes in a flexible way (Shi & Yang 2013).
Polygon-based 3D modeling software programs are limited in their ability to reproduce curves smoothly, but their advanced rendering and shading functions can well reproduce 3D surfaces. Accordingly, they are widely used in fields such as computer graphics, advertising, animation making, and visual FX. However, in fields where precision work is needed because products are built in reality rather than in a virtual space, such as industrial design (e.g., aircraft and vessels), architectural design, and craft design (e.g., furniture and accessories), NUBRS-based 3D modeling software such as Rhino has been most widely used (Hsu et al. 2015;Kwon et al. 2017;Shi & Yang 2013).
The Rhinoceros 3D ® (commonly abbreviated to Rhino) software including the graphic algorithm editor Grasshopper as a plug-in works differently than existing text-based methods. It allows off-the-shelf compilers to serve as commands. More specifically, in Grasshopper, modeling is performed by arranging "components", which correspond to pre-defined commands (e.g., icons, connecting lines, and arrows) ( Fig. 1). When "wire" is connected between "components" that serve as input and output parameters, modeling can then be intuitively conducted in a continuous manner. Parameter values can be inputted through components or easily changed by dragging the mouse pointer. Details of changes can be viewed on the Viewport of Rhino in real-time (Hsu et al. 2015). In the architectural field, the most common software for parametric design is Grasshopper. Grasshopper can deal with many parameters simultaneously and provide fast results compared to other parameter software such as 3D Max (Eltaweel & Su 2017).
When Rhino and the Grasshopper editor are used, it is possible to view the detail of changes of the algorithm directly in the 3D Rhino interface. For that reason, Rhino and Grasshopper are currently widely used in various applications, such as architectural design or aircraft production. In the apparel field, these programs have been used only for 3D printing design research and accessory design, but not for human body shape modeling and clothing pattern design. There was no study to adopt this graphical user interface (GUI)-based surface interpolation using commercial software and its plugin for 3D human body modeling in the apparel field.

Objectives
The objective of this study is to design an entire process for 3D average human body model, from reference line designation to average 3D model generation using Grasshopper, a graphic algorithm edition, and to develop commands to execute each step of the process in a step-by-step manner in the Rhino interface. Then, the validity of the generated 3D average model was verified. The specific research objectives were as follows.
1. To categorize men aged 36 to 55 years included in SizeUSA 3D data into the three body types according to BMI, and to select samples to determine the average model. 2. To design algorithm for setting horizontal and vertical reference lines on the 3D scanned lower body shape, creating horizontal and vertical grids and intersection points, and generating average models for each group using Grasshopper in the Rhino interface. 3. To generate Rhino commands to allow the entire process to be automated. 4. To verify the validity of the generated average models by comparing their measurements with those of the sample models.
We expected that this algorithm can allow a number of 3D body shape modeling operations to be implemented in a significantly reduced time period. Also, the researchers in this study expect this approach to provide consistent results regardless of the number, position, and order of data sets used since the Rhino interface can create a 3D average model in a position that corresponds to the average vertex coordinates of activated mesh models.

Sample selection
From 1504 body scans of men aged 36-55 in the SizeUSA data, 1378 scans were extracted, excluding those missing height values needed for reference line designation and with poor scanning conditions. Among them, samples with BMIs from 18.5 to 24.9 were classified as the normal type, while those with a BMI of 25.0-29.9 and 30.0-39.9 were classified as the overweight type and the obese type, respectively (Nuttall, 2015). The number of samples for Group 1 (normal type), Group 2 (overweight type), and Group 3 (obese type) was found to be 379 (27.5%), 650 (47.2%), and 349 (25.3%), respectively. As shown in Table 1, seven scan data sets with the girth measurements of the main body parts (waist, hip, thigh, knee, and ankle) within the average range (mean ± SD) were selected for the samples in each group.

Reference line designation
Six horizontal reference lines and six vertical reference lines were designated on the lower body of the 3D scans, and the space between each reference line was divided into two to four parts by defining secondary reference lines ( Table 2). As primary horizontal reference lines, the waist line (W1), abdomen line (A1), hip line (H1), thigh line (T1), knee line (K1), and ankle line (A1) were designated. Among them, the waist line, abdomen line, and ankle line were determined by creating a horizontal plane in a position corresponding to the height of each body location available in the SizeUSA dataset. Regarding the thigh location, the SizeUSA dataset defines the position of the thigh as the height corresponding to the maximum girth of the leg, but it was found that the definition of that position varied from one scan to another in an unreliable manner. Accordingly, in this study, the height of the thigh was defined as 1 inch below the crotch available in the SizeUSA dataset.
With respect to the position of hips and ankles, we designed an algorithm to automatically spot these landmarks using Grasshopper instead of referencing the SizeUSA data. The hips were defined as the part most prominently protruding backward, while the ankle girth was defined as the circumference of the thinnest part of the leg.
As primary vertical reference lines, six lines, including the center front line (F1), center back line (B1), outseam line (S1), inseam line (I1), front crease line (FC1), back crease line (BC1), were designated ( Table 2). The center lines were drawn at the bisecting point of the waist width. From the side body, the outseam and inseam lines were created as follows: a plane that connected both the bisecting point of the waist depth and the bisection point of the ankle thickness was created, and this plane and the 3D body scan then meet in two lines. The one found on the outer side of the leg was defined as the outseam line, while the other on the inner side of the leg was defined as the inseam line.
The front crease line was defined as the line generated at the intersection of the 3D body scan with the vertical plane that passes through the front bisecting point between the inseam line and the outseam line at the thigh position. The back crease line was defined as the vertical line that connects the thigh and the bisecting point of the rearside circumference of the thigh girth. Then, the secondary horizontal and vertical reference lines were set by dividing the primary vertical lines into two to four parts.

Average model generation
To generate an average model, seven scans for each group were saved as mesh models in Rhino. Given that the vertices (grid intersection points) of each mesh model have their own coordinates in the 3D interface, 3D average models were generated in a position corresponding to the average vertex coordinates of the seven activated mesh models for each group. This method has the advantage that it could average multiple entities at the same time instead of executing the same process through one-on-one matching such as morphing technique.

Algorithm and commands for automatic creation of 3D average models
The entire process, from reference line designation to average model generation, was designed using the algorithm editor Grasshopper, and commands that could execute each step of the process in a step-by-step manner were generated in the Rhino interface. The automation procedures were as follows. 1. Generation of 3D body scan mesh models 2. Import of waist, abdomen, thigh, and ankle height measurements of each human body scan from the SizeUSA data 3. Automatic identification of hip and ankle positions 4. Designation of six horizontal reference lines, six vertical reference lines, and secondary reference lines on the lower body 5. Generation of an intersection grid and intersection points of horizontal and vertical reference lines, resulting in triangular mesh models 6. Generation of 3D average models in a position corresponding to the average intersection point (vertex) coordinates for sample models

Verification method of average models
In an attempt to verify the dimensional validity of the generated average models, the average waist, abdomen, and hip girth measurements of the seven samples in each group were compared with the girth measurements of the average model obtained as a result of 3D modeling. In addition, for morphological verification of the generated 3D average model, cross-sectional views of the waist, abdomen, and hip were extracted from both the seven sample scans of each group and the average model. Then, the cross-sectional views were overlapped with one another for comparison and analysis. Also, the distance from the center point of each cross-sectional view to each grid intersection point was measured and compared.

Results and Discussion
Grasshopper algorithm for triangle mesh models

Reference line designation
The algorithm was designed to import the scan data in the Rhino interface with the vertical line that passes through the bisection point of the waist width as the central axis. As input parameters to generate the primary horizontal reference lines, waist height, abdomen height, and knee height measurements from the SizeUSA were used, and the thigh height (1 inch subtracted from the crotch height) was added as an additional parameter. We developed an algorithm to save these parameters were saved in CSV file format. For the hip line and the ankle line, an algorithm was designed to automatically extract their positions. An algorithm for the hip line was designed, as follows: a vertical section was generated at the bisecting point between the outseam line (S1) and the inseam line (I1) at the back thigh (T1). And the space between waist line (A1) and the thigh line (T1) was divided into 20 parts, and the algorithm was designed to find the most protruding point shown in Fig. 2.
Then, an algorithm for the ankle line was designed as follows: the section with a height of 50 to 200 mm was divided into 30 segments, and among them, two parts with the smallest circumference were determined. Then, the space between the two parts was then again divided into 20 segments, and the part with the smallest circumference was determined (Fig. 2). The Grasshopper algorithm intended for extracting the ankle line position is shown in Fig. 3.

Triangle mesh model generation
According to the definition of reference lines, as shown in Table 2, an algorithm was designed to generate a grid on the right surface side of the 3D scan, and to connect each intersection point diagonally to generate a triangular mesh model (Fig. 5). As a result, complete mesh models were finally obtained, as shown in Fig. 4.

Algorithm integration
To automate the entire process, the whole algorithm was connected as shown in Fig. 5. A series of procedures for triangle mesh model generation were as follows: (1) the import the waist, abdomen, thigh, and ankle height measurements from the SizeUSA data; (2) automatic identification of hip and ankle points; (3) generation of an intersection grid and intersection points of horizontal and vertical reference lines; and (4) generation of triangle mesh models. The input and output parameters of each step were connected through components to allow for continuous operations.

3D average model generation in Rhino interface
To generate an average model, seven mesh model files for each group were activated at the same time, as shown in Fig. 6. Vertices (grid intersection points) of each mesh model have their own coordinates in the 3D interface. Accordingly, it was designed so that an average model (red-colored one in Fig. 3) could be generated in a position corresponding to the average vertex coordinates of each activated mesh model. This approach  has advantages, in that the number of mesh models to be activated is not limited, and modeling results are not affected by where mesh models are placed in the 3D interface. Also, this method can average multiple entities at the same time instead of executing the same process through one-on-one matching and can thus prevent errors that may cause shape modeling results to vary from one mesh model to another. This makes it possible to derive at least one desired result.
In the Toolbar of the Rhino interface (the red boxed region in Fig. 7), two command icons were created, which were designed to execute the Grasshopper algorithm. Among them, the MeshToBodyMesh was designed to open and arrange scan files and generate triangular meshes, including reference lines, while the MeanMesh command was to generate an average model out of the multiple mesh models. These new commands allowed for automation, to minimize the amount of manual work needed to process a number of scan data in a short period of time.
The left-side lower body of each generated average mesh model was additionally created using the mirror function, and both sides of each mesh model were then integrated to create a complete form of average mesh models (Fig. 8).  Lee and Song Fash Text (2021) 8:23 Dimensional and morphological verification of 3D average models In an attempt to verify the dimensional accuracy of the average model, the 3D average waist girth, abdominal girth, and hip girth measurements of each group's samples were compared with the corresponding girth measurements of the generated 3D average model (Table 3). The average waist, abdomen, and hip girth of the samples in Group 1 (Normal type) were 86.40 cm, 87.95 cm, and 95.06 cm, respectively. The deviation from the corresponding measurements of the generated 3D average model was not more than 0.50 cm. The average waist, abdomen, and hip girth of the samples in Group 2 (Overweight type) were 97.91 cm, 99.62 cm, and 103.69 cm, respectively. The deviation from the corresponding measurements of the generated 3D average model was not more than 0.70 cm. The average waist, abdomen, and hip girth of the   , 110.29 cm, and 113.04 cm, respectively. The deviation from the corresponding measurements of the generated 3D average model was not more than 0.8 cm.
The deviation in each body part's girth measurements was found to be larger in Group 2 (Overweight type) and Group 3 (Obese type) than in Group 1 (Normal type). This is attributed to the fact that the 3D average model was generated in a position corresponding to the average vertex coordinates of the samples in each grid, and thus the curves between each grid intersection points tended to change more severely in Group 2 and Group 3 than in Group 1. However, when the standard deviations (S.D) of the samples were compared with the difference between the sample's mean values (A) and the average model's value (B), it was found that all differences were within the S.D. This confirms that the final 3D average model properly represents the dimensions of each body part.
For morphological verification of the 3D average model, cross-sectional views of the waist, abdomen, and hip were extracted from both each group's sample scan data, and from the average model, as shown in Fig. 9a. Each body part's cross-sectional views were then overlapped with one another for comparison and analysis, as shown in Fig. 9c. As a result, it was confirmed that the samples and the final model provided similar results in terms of the shape of the entire curve and protrusions.
Based on the scan data, the distance from the center point of each cross-sectional view of the waist, abdomen, and hip to each grid intersection point was measured, as shown in Fig. 9b, and the results are presented in Table 4. In all groups, when the standard deviations (S.D) of the samples were compared with the difference between the sample's mean values (A) and the average model's value (B), it was found that all differences were within the S.D. This confirms that the final model properly represents the morphological characteristics of each body part of the samples.

Difference (B-A)
Waist level F1