Visualising higher-dimensional space-time and space-scale objects as projections to \(\mathbb{R}^3\)

Objects of more than three dimensions can be used to model geographic phenomena that occur in space, time and scale. For instance, a single 4D object can be used to represent the changes in a 3D object's shape across time or all its optimal representations at various levels of detail. In this paper, we look at how such higher-dimensional space-time and space-scale objects can be visualised as projections from \(\mathbb{R}^4\) to \(\mathbb{R}^3\). We present three projections that we believe are particularly intuitive for this purpose: (i) a simple `long axis' projection that puts 3D objects side by side; (ii) the well-known orthographic and perspective projections; and (iii) a projection to a 3-sphere (\(S^3\)) followed by a stereographic projection to \(\mathbb{R}^3\), which results in an inwards-outwards fourth axis. Our focus is in using these projections from \(\mathbb{R}^4\) to \(\mathbb{R}^3\), but they are formulated from \(\mathbb{R}^n\) to \(\mathbb{R}^{n-1}\) so as to be easily extensible and to incorporate other non-spatial characteristics. We present a prototype interactive visualiser that applies these projections from 4D to 3D in real-time using the programmable pipeline and compute shaders of the Metal graphics API. ABSTRACT 9 Objects of more than three dimensions can be used to model geographic phenomena that occur in space, time and scale. For instance, a single 4D object can be used to represent the changes in a 3D object’s shape across time or all its optimal representations at various levels of detail. In this paper, we look at how such higher-dimensional space-time and space-scale objects can be visualised as projections from R 4 to R 3 . We present three projections that we believe are particularly intuitive for this purpose: (i) a simple ‘long axis’ projection that puts 3D objects side by side; (ii) the well-known orthographic and perspective projections; and (iii) a projection to a 3-sphere ( S 3 ) followed by a stereographic projection to R 3 , which results in an inwards-outwards fourth axis. Our focus is in using these projections from R 4 to R 3 , but they are formulated from R n to R n − 1 so as to be easily extensible and to incorporate other non-spatial characteristics. We present a prototype interactive visualiser that applies these projections from 4D to 3D in real-time using the programmable pipeline and compute shaders of the Metal graphics API. BACKGROUND


22
Projecting the 3D nature of the world down to two dimensions is one of the most common problems at 23 the juncture of geographic information and computer graphics, whether as the map projections in both 24 paper and digital maps (Snyder, 1987;Grafarend and You, 2014) or as part of an interactive visualisation 25 of a 3D city model on a computer screen (Foley and Nielson, 1992; Shreiner et al., 2013). However, 26 geographic information is not inherently limited to objects of three dimensions. Non-spatial characteristics 27 such as time (Hägerstrand, 1970;Güting et al., 2000;Hornsby and Egenhofer, 2002;Kraak, 2003) and 28 scale (Meijers, 2011a) are often conceived and modelled as additional dimensions, and objects of three or 29 more dimensions can be used to model objects in 2D or 3D space that also have changing geometries  Objects of more than three dimensions can be however unintuitive (Noll, 1967;Frank, 2014), and 35 visualising them is a challenge. While some operations on a higher-dimensional object can be achieved by 36 running automated methods (e.g. certain validation tests or area/volume computations) or by visualising 37 only a chosen 2D or 3D subset (e.g. some of its bounding faces or a cross-section), sometimes there is 38 no substitute for being able to view a complete nD object-much like viewing floor or façade plans is This paper thus looks at a key aspect that allows higher-dimensional objects to be visualised inter-48 actively, namely how to project higher-dimensional objects down to fewer dimensions. While there is 49 previous research on the visualisation of higher-dimensional objects, we aim to do so in a manner that is 50 reasonably intuitive, implementable and fast. We therefore discuss some relevant practical concerns, such 51 as how to also display edges and vertices and how to use compute shaders to achieve good framerates in 52 practice.

53
In order to do this, we first briefly review the most well-known transformations (translation, rotation 54 and scale) and the cross-product in nD, which we use as fundamental operations in order to project 55 objects and to move around the viewer in an nD scene. Afterwards, we show how to apply three different 56 projections from R n to R n−1 and argue why we believe they are intuitive enough for real-world use. These 57 can be used to project objects from R 4 to R 3 , and if necessary, they can be used iteratively in order to 58 bring objects of any dimension down to 3D or 2D. We thus present: (i) a simple 'long axis' projection 59 that stretches objects along one custom axis while preserving all other coordinates, resulting in 3D objects 60 that are presented side by side; (ii) the orthographic and perspective projections, which are analogous to 61 those used from 3D to 2D; and (iii) an inwards/outwards projection to an (n − 1)-sphere followed by an 62 stereographic projection to R n−1 , which results in a new inwards-outwards axis. 63 We present a prototype that applies these projections from 4D to 3D and then applies a standard 64 perspective projection down to 2D. We also show that with the help of low-level graphics APIs, all the 65 required operations can be applied at interactive framerates for the 4D to 3D case. We finish with a 66 discussion of the advantages and disadvantages of this approach. As an alternative to the all these methods, it is possible to represent any number of parametrisable Nef (1988), and ordered topological models Brisson (1993); Lienhardt (1994). This is consistent with the 89 basic tenets of n-dimensional geometry (Descartes, 1637; Riemann, 1868) and topology (Poincaré, 1895), 90 which means that it is possible to apply a wide variety of computational geometry and topology methods 91 to these objects.

92
In a practical sense, 4D topological relationships between 4D objects provide insights that 3D which can be used to move and zoom around a scene composed of nD objects. In addition, the n-101 dimensional cross-product can be used to obtain a new vector that is orthogonal to a set of other n − 1 102 vectors in R n . We use these operations as a base for nD visualisation and are thus described briefly below.

103
The translation of a set of points in R n can be easily expressed as a sum with a vector t = [t 0 , . . . ,t n ], 104 or alternatively as a multiplication with a matrix using homogeneous coordinates 1 in an (n + 1) × (n + 1) 105 matrix, which is defined as: Scaling is similarly simple. Given a vector s = [s 0 , s 1 , . . . , s n ] that defines a scale factor per axis (which 107 in the simplest case can be the same for all axes), it is possible to define a matrix to scale an object as:

Computer Science
Rotation is somewhat more complex. Rotations in 3D are often conceptualised intuitively as rotations 109 around the x, y and z axes. However, this view of the matter is only valid in 3D. In higher dimensions, it 110 is necessary to consider instead rotations parallel to a given plane (Hollasch, 1991), such that a point that 111 is continuously rotated (without changing the rotation direction) will form a circle that is parallel to that 112 plane. This view is valid in 2D (where there is only one such plane), in 3D (where a plane is orthogonal to 113 the usually defined axis of rotation) and in any higher dimension. Incidentally, this shows that the degree 114 of rotational freedom in nD is given by the number of possible combinations of two axes (which define a 115 plane) on that dimension (Hanson, 1994), i.e. n 2 .

116
Thus, in a 4D coordinate system defined by the axes x, y, z and w, it is possible to define six 4D 117 rotation matrices, which correspond to the six rotational degrees of freedom in 4D (Hanson, 1994). These 118 respectively rotate points in R 4 parallel to the xy, xz, xw, yz, yw and zw planes: The n-dimensional cross-product is easy to understand by first considering the lower-dimensional 120 cases. In 2D, it is possible to obtain a normal vector to a 1D line as defined by two (different) points p 0 121 and p 1 , or equivalently a normal vector to a vector from p 0 to p 1 . In 3D, it is possible to obtain a normal 122 vector to a 2D plane as defined by three (non-collinear) points p 0 , p 1 and p 2 , or equivalently a normal 123 vector to a pair of vectors from p 0 to p 1 and from p 0 to p 2 . Similarly, in nD it is possible to obtain a 124 normal vector to a (n − 1)D subspace-probably easier to picture as an (n − 1)-simplex-as defined by n 125 linearly independent points p 0 , p 1 , . . . , p n−1 , or equivalently a normal vector to a set of n − 1 vectors from 126 p 0 to every other point (i.e. p 1 , p 2 , . . . , p n−1 ) (Massey, 1983;Elduque, 2004). 127 Hanson (1994) follows the latter explanation using a set of n − 1 vectors all starting from the first 128 point to give an intuitive definition of the n-dimensional cross-product. Assuming that a point p i in R n is 129 defined by a tuple of coordinates denoted as (p i 0 , p i 1 , . . . , p i n−1 ) and a unit vector along the i-th dimension 130 is denoted asx i , the n-dimensional cross-product N of a set of points p 0 , p 1 , . . . , p n−1 can be expressed 131 compactly as the cofactors of the last column in the following determinant: The components of the normal vector N are thus given by the minors of the unit vectorsx 0 ,x 1 , . . . ,x n−1 .  In a more concrete computer graphics context, already in the 1960s, Noll (1967) described a computer 144 implementations of the 4D to 3D perspective projection and its application in art (Noll, 1968).

146
4D objects that are rendered in real-time and use colour intensity to provide a visual cue for the 4D depth.

147
It is extended to n dimensions by Feiner and Beshers (1990).

148
Banks (1992) describes a system that manipulates surfaces in 4D space. It describes interaction 149 techniques and methods to deal with intersections, transparency and the silhouettes of every surface. 162 Similar to the methods described above, Hollasch (1991) gives a simple formulation to describe the 4D 163 to 3D projections, which is itself based on the 3D to 2D orthographic and perspective projection methods 164 described by Foley and Nielson (1992). This is the method that we extend to define n-dimensional 165 versions of these projections and is thus explained in greater detail below. The mathematical notation is 166 however changed slightly so as to have a cleaner extension to higher dimensions.  is also the depth of the point.
186 Figure 2. The geometry of a 4D perspective projection along the x axis for a point p. By analysing the depth along the depth axis given by e 3 , it is possible to see that the coordinates of the point along the x axis, given by e 0 , are scaled inwards in order to obtain e ′ 0 based on the viewing angle ϑ . Note thatx n−1 is an arbitrary viewing hyperplane and another value can be used just as well.

187
We present here three different projections from R n to R n−1 which can be applied iteratively to bring 188 objects of any dimension down to 3D for display. We three projections that are reasonably intuitive in 189 4D to 3D: a 'long axis' projection that puts 3D objects side by side, the orthographic and perspective 190 projections that work in the same way as their 3D to 2D analogues, and a projection to an (n − 1)-sphere 191 followed by a stereographic projection to R n−1 .

193
First we aim to replicate the idea behind the example previously shown in Fig. 1-a series of 3D objects   194 that are shown next to each other, seemingly projected separately with the correspondences across 195 scale or time shown as long edges (as in Fig. 1) or faces connecting the 3D objects. Edges would join 196 correspondences between vertices across the models, while faces would join correspondences between 197 elements of dimension up to one (e.g. a pair of edges, or an edge and a vertex). Since every 3D object is 198 apparently projected separately using a perspective projection to 2D, it is thus shown in the same intuitive 199 way in which a single 3D object is projected down to 2D. The result of this projection is shown in Fig. ?? 200 for the model previously shown in Fig. 1 and in Fig. 4 for a 4D model using 3D space with time.

201
Although to the best of our knowledge this projection does not have a well-known name, it is widely 202 used in explanations of 4D and nD geometry-especially when drawn by hand or when the intent is 203 to focus on the connectivity between different elements. For instance, it is usually used in the typical 204 explanation for how to construct a tesseract, i.e. a 4-cube or the 4D analogue of a 2D square or 3D cube, 205 which is based on drawing two cubes and connecting the corresponding vertices between the two (Fig. 5).  Figure 3. A model of a 4D house similar to the example shown previously in Fig. 1, here including also a window and a door that are collapsed to a vertex in the 3D object at the lower level of detail. (a) shows the two 3D objects positioned as in Fig. 1, (b) rotates these models 90 • so that the front of the house is on the right, and (c) orients the two 3D objects front to back. Many more interesting views are possible, but these show the correspondences particularly clearly. Unlike the other model, this one was generated with 4D coordinates and projected using our prototype that applies the projection described in this section.  Conceptually, describing this projection from n to n − 1 dimensions, which we hereafter refer to as Manuscript to be reviewed

Computer Science
Orthographic and perspective projections 218 Another reasonably intuitive pair of projections are the orthographic and perspective projections from 219 nD to (n − 1)D. These treat all axes similarly and thus make it more difficult to see the different (n − 1)-220 dimensional models along the n-th axis, but they result in models that are much less deformed. Also, 221 as shown in the 4D example in Fig. 6, it is easy to rotate models in such a way that the corresponding 222 features are easily seen. Based on the description of 4D-to-3D orthographic and perspective projection described from Hollasch

224
(1991), we here extend the method in order to describe the n-dimensional to (n − 1)-dimensional case, 225 changing some aspects to give a clearer geometric meaning for each vector.

226
Similarly, we start with a point f rom ∈ R n where the viewer is located, a point to ∈ R n that the viewer

Manuscript to be reviewed
Computer Science The vectorx n−1 is the first that needs to be computed and is oriented along the line from the viewer

263
A final projection possibility is to apply a stereographic projection from R n to R n−1 , which for us was 264 partly inspired by Jenn 3D 3 (Fig. 8). This program visualises polyhedra and polychora embedded in It is worth to note that the radius r of such a coordinate system is a measure of the depth with respect The stereographic projection from nD to (n − 1)D is particularly intuitive because it results in the n-th 278 axis being converted into an inwards-outwards axis. As shown in Fig. 9, when it is applied to scale, this 279 results in models that decrease or increase in detail as one moves inwards or outwards. The case with 280 time is similar: as one moves inwards/outwards, it is easy to see the state of a model at a time before/after.

282
We have implemented a small prototype for an interactive viewer of arbitrary 4D objects that performs the 283 three projections previously described. It was used to generate Figures 3, 6 and 9, which were obtained by 284 moving around the scene, zooming in/out and capturing screenshots using the software.

285
The prototype was implemented using part of the codebase of azul 5 and is written in a combination 286 of Swift 3 and C++11 using Metal-a low-level and low-overhead graphics API-under macOS 10.12 6 .

287
By using Metal, we are able to project and display objects with several thousand polygons with minimal 288 visual lag on a standard computer. Its source code is available under the GPLv3 licence at https: 289 //github.com/kenohori/azul4d. Unfortunately, this programming trick also means that extending the current prototype to dimensions 296 higher than four requires additional work and rather cumbersome programming. However, implementing 297 these operations in a dimension-independent way is rather not difficult outside in a more flexible pro-298 gramming environment. For instance, Fig. 10 shows how a double stereographic projection can be used 299 to reduce the dimensionality of an object from 5D to 3D. This figure was generated in a separate C++ 300 program which exports its results to an OBJ file. The models were afterwards rendered in Blender 7 .

301
In our prototype, we only consider the vertices, edges and faces of the 4D objects, as the higher-302 dimensional 3D and 4D primitives-whose 0D, 1D and 2D boundaries are however shown-would

307
The 4D models were manually constructed based on defining their vertices with 4D coordinates and 308 their faces as successions of vertices. In addition to the 4D house previously shown, we built a simpler 309 tesseract for testing. As built, the tesseract consists of 16 vertices and 24 vertices, while the 4D house 310 consists of 24 vertices and 43 faces. However, we used the face refining process described below to test 311 our prototype with models with up to a few thousand faces. Once created, the models were still displayed 312 and manipulated smoothly.

313
To start, we preprocess a 4D model by triangulating and possibly refining each face, which makes it

341
Visualising complete 4D and nD objects projected to 3D and displayed in 2D is often unintuitive, but it 342 enables analysing higher-dimensional objects in a thorough manner that cross-sections do not. The three 343 projections we have shown here are nevertheless reasonably intuitive due to their similarity to common 344 projections from 3D to 2D, the relatively small distortions in the models and the existence of a clear fourth 345 axis. They also have a dimension-independent formulation.

346
There are however many other types of interesting projections that can be defined in any dimension, 347 such as the equirectangular projection where evenly spaced angles along a rotation plane can be directly 348 converted into evenly spaced coordinates-in this case covering 180 • vertically and 360 • horizontally.

349
Extending such a projection to nD would result in an n-orthotope, such as a (filled) rectangle in 2D or a 350 cuboid (i.e. a box) in 3D.

351
By applying the projections shown in this paper to 4D objects depicting 3D objects that change in 352 time or scale, it is possible to see at once all correspondences between different elements of the 3D objects 353 and the topological relationships between them.

354
Compared to other 4D visualisation techniques, we opt for a rather minimal approach without lighting 355 and shading. In our application, we believe that this is optimal due to better performance and because it