http://orcid.org/0000-0002-9135-866X
Figures
Abstract
In this paper, a novel self-learning digital twin strategy is developed for fluid sloshing phenomena. This class of problems is of utmost importance for robotic manipulation of fluids, for instance, or, in general, in simulation-assisted decision making. The proposed method infers the (linear or non-linear) constitutive behavior of the fluid from video sequences of the sloshing phenomena. Real-time prediction of the fluid response is obtained from a reduced order model (ROM) constructed by means of thermodynamics-informed data-driven learning. From these data, we aim to predict the future response of a twin fluid reacting to the movement of the real container. The constructed system is able to perform accurate forecasts of its future reactions to the movements of the containers. The system is completed with augmented reality techniques, so as to enable comparisons among the predicted result with the actual response of the same liquid and to provide the user with insightful information about the physics taking place.
Citation: Moya B, Alfaro I, Gonzalez D, Chinesta F, Cueto E (2020) Physically sound, self-learning digital twins for sloshing fluids. PLoS ONE 15(6): e0234569. https://doi.org/10.1371/journal.pone.0234569
Editor: Jie Zhang, Newcastle University, UNITED KINGDOM
Received: April 16, 2020; Accepted: May 27, 2020; Published: June 16, 2020
Copyright: © 2020 Moya et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data have been uploaded to Github at https://github.com/beatrizmoya/sloshingfluids/.
Funding: Spanish Ministry of Economy and Competitiveness through Grant number DPI2017-85139-C2-1-R Regional 313 Government of Aragon and the European Social Fund, research group T88 ESI Group through the project UZ-2019-0060.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Digital twins are tools to fully address real-world phenomena and seamlessly connect reality with virtual predictions. The abundance of data inputs allows to relate physical objects or processes with the simulation to obtain online meaningful results. The digital twin of the object of interest offers, in the end, a cost-effective real-time solution for taking decisions analogous to those that we would take interacting with the real process or object.
These virtual replicas have already shown their usefulness through their implementation across many kinds of industries and branches of science. In the field of robotics, they are the means for scene understanding and interaction to connect robots with the actual reality [1] [2].
In this paper, we focus on the development of a self-learning digital twin to predict the sloshing phenomenon of a fluid so as to equip an online decision-making system. By self-learning digital twin we mean a system that, rather than performing data assimilation (to determine the viscosity of the fluid, for instance) it is able to construct a physically correct, data-driven replica of a previously unseen fluid, regardless of its constitutive behavior.
We employ computer vision techniques to obtain data of the movement of the glasses where the real fluid and the replica are contained. Next, we feed the simulator engine with that information. Finally, the output is represented within a real container, that may be the one containing the fluid or not. We let the simulation run in the background so as to interpret the physics taking place.
Machine learning is a powerful tool to learn the behavior and underlying dynamics of a liquid [3]. Notwithstanding their effectiveness, physical compliance is still a central issue. In [4], the authors proposed an alternative data-driven approach to model the physics behind the sloshing dynamics as a regression guided by thermodynamic criteria.
Here, we substantially extend the algorithm so as to be able to construct replicas of a wide range of fluids for building a self-learning digital twin. We aim to prove that our method is also able to learn patterns of materials with non-linear behavior, with synthetic or real information from the scene.
Related work
Fluid dynamics understanding based on machine learning is a rapidly growing field. It provides a deep insight into the physical complexity of flows and behaviors for their control and manipulation. Yunzhu Li et al. [3] study fluid simulation learnt from particle interaction. They developed the so-called dynamic particle interaction networks (DPI-Nets) in order to learn hierarchical interaction of the particles in which the objects of study have been discretized. They capture non linear behaviors in deformable and fluid bodies for control purposes. Schenck and Fox [2] also employed convolutional networks to learn the physics of pouring liquids. Noteworthy, these predictions have no guarantee to comply with the laws of Navier-Stokes, for instance. The same problem arises in the work by Kennedy et al. [5], where approximate particle methods are employed to simulate pouring. In general, in the field of robotics, existing approaches employ approximate methods to overcome the very stringent real-time constraints imposed by closed-loop fluid simulation. This need for very fast simulation results has prevented existing methods from accurately satisfying basic conservation laws such as Navier-Stokes of the laws of thermodynamics. The present work overcomes these limitations and guarantees the satisfaction of the laws of thermodynamics while avoiding the need of solving Navier-Stokes equations.
Very few references analyze the possibility of applying the concept of digital twin to fluid mechanics. Among them, we cite the application to drilling fluids in the field of geology, see [6], or an approach for the oil and gas industry [7].
Brunton et al. [8] make a review of these recent advances in machine learning applied to fluid mechanics and dynamics. It stands out the need for these systems to overcome challenges such as generalization and interpretation. Particularly, there is a growing interest on machine learning strategies that satisfy well-known principles of physics and do not construct black-box models whose application to previously unseen data does not guarantee to comply with basic principles such as energy conservation or equilibrium.
In order to overcome these limitations, the machine learning community has recently taken a step forward physically-informed learning [9], whose application field ranges from neural networks [10] [11] [3] [12] to linear regression learning [13] [14]. This new perspective of machine learning aims to guide the learning process with criteria that fulfills the laws of physics. As a result, we achieve a higher degree of accuracy, as well as generality and interpretability. In turn, those models still entail sometimes a high computational cost difficult to surmount. Under this rationale, real-time constraints have not been—to the best of our knowledge—attained.
Real-time digital twins offer applications for continuously changing, monitored systems [15]. For this purpose, time constraints must be satisfied. Reduced order models supply digital twins with tools to face such disadvantage [16] [17] [18] [19]. Kapteyn et al. [20], for instance, work on physically-constrained digital twins developed in a reduced order space for aircraft replanning. The model for each topology is defined in a reduced basis, giving support for immediate decision making.
Twins continuously incorporate features to improve interpretation and application. Augmented reality (AR), for instance, enables the contextualization of the twin in the physical world. Consequently, interaction between reality and the replica, as well as user control and interpretation, are straightforward. AR tools have also been successfully implemented for non-rigid representation, such as aerodynamics [21], or deformable objects [16]. On the other and, existing digital twins for sloshing problems revealed limitations, such time computing and localization in the scene, that inhibit their implementation [22].
Method
In this work we suggest a data-driven twin which performs a thermodynamically admissible emulation of sloshing dynamics. To this end, the method consists of the merge of three stages: data assimilation, interpretation and representation. The final goal is to predict the next state of a fluid, which could be unknown, from interpretable data of the scene. Some of the fluids employed to learn the dynamics show viscoelastic properties. In addition, the twin needs to operate at a frequency of 30 Hz or faster (the one at which standard cameras operate). The result is an AR digital twin convergent and synchronized with reality for optimal control and decision making.
Integration from data without the need to solve Navier-Stokes
Machine learning abstracts insights about the physics from given data. A priori, they are unknown, but the algorithm finally learns the appropriate correlations to fit an accurate model.
Our digital twin is modeled as a time-evolution problem, expressed as a function of a set of variables with D representing the full dimensionality of the problem. These variables describe the energy and entropy evolution of the fluid. If we know them at discrete time steps, we will try to find patterns so as to predict future states of the liquid. With this integration scheme, given new real or synthetic measurements of an unseen trajectory, the sloshing dynamics will be accurately reconstructed in real time.
This approach employs the so-called General Equation for Non-Equilibrium Reversible-Irreversible Coupling, GENERIC [23]. This methodology has been employed satisfactorily in several works for learning models [24] [25] [4], as well as corrections to existing models [26], from data.
GENERIC arises as a general thermodynamic framework to describe mesoscopic dynamics. Actually, it can be seen as the formulation that describes the process of obtention of less detailed models, reduced from models involving more details (more degrees of freedom). The phase portrait of a reduced model is seen under this theory as a pattern in the phase portrait of the detailed model [27]. As a result, it expresses the dynamics in terms of the mesoscopic variables z, that lead the evolution of energy and entropy of a system as:
(1)
where L and M are the poisson and friction matrices, and ∇E and ∇S the gradients of energy and entropy, respectively. We distinguish two main parts in this expression, one related to the conservative part of the dynamics (L∇E), and another representing the posible dissipation of the system (M∇S).
We also ensure thermodynamic consistency by imposing the so-called degeneracy conditions:
(2)
that establish that the energy functional does not contribute to dissipation mechanisms, while entropy does not contribute to reversible mechanics [28].
We guarantee both impositions by forcing L to be the classical symplectic, skew symmetric, matrix of Hamiltonian mechanics, and M symmetric, positive semi definite. With the former restriction we ensure energy conservation:
(3)
and with the latter, entropy generation
(4)
We assume that the sloshing dynamics of each fluid evolve on a finite-dimensional, smooth and real manifold , reconstructed from synthetic data. In this approach, the discretized, pseudo-experimental measurements of the state variables allow to build an also discretized expression of GENERIC:
(5)
from which we can also learn new trajectories. Here, L(zn+1), DE(zn+1), M(zn+1) and DS(zn+1) represent the discretized approaches to their original counterparts. Note also that ∇E and ∇S can be approximated, in the finite element style, by piece-wise polynomials so that:
with A and B two matrices of shape function derivatives. This makes L, M, A and B the objective of the following (piece-wise) linear regression procedure:
(6)
Nonetheless, the optimization and calculations are still computationally demanding. In order to achieve real time performance for our digital twin, we make use of model order reduction techniques to find a reduced order manifold , where d ≪ D. Fig 1 sketches our approach. On this reduced manifold, we will preserve the important features of the dynamics expressed in a new system of latent variables.
We extract synthetic data as vectors in from computational simulations. These data are possibly noisy, see the zoomed detail. An example sloshing phenomenon is represented as the red line in phase space. Given the inherent high-dimensionality of the manifold, nonlinear dimensionality reduction techniques will also be applied. This will project the data to a lower dimensionality embedding space in
, with d ≪ D. We develop structure-preserving integration schemes to integrate the evolution of the system in this low-order manifold. These are then mapped back to the physical space in
so as to obtain meaningful results.
In [4], the authors showed the efectiveness of non-linear model order reduction techniques (namely, LLE [29] and TDA [30]) for this problem. Following these results, we have employed k-PCA (kernel Principal Component Analysis) [31] to distill the reduced order manifold of each fluid.
In our problem, we have a data matrix Z. Each column zi, i = 1, …, n is a snapshot, a vector of state variables that represent the state of the fluid at a specific time instant. For each fluid, we have a total of n snapshots.
Given a matrix Z snapshots, we compute the product S = ZT Z to obtain the matrix of pairwise scalar products. The key hypothesis of k-PCA is that the projection of the points to a new space , where Q is the new dimension, probably higher than the current space, can result to be linearly separable. We apply PCA in
. As a result, we obtain the most relevant nonlinear principal components of Z and thus a projection to a much lower dimensional manifold. With this method we successfully embedded then into a three dimensional system
.
The discretized GENERIC formulation is also consistent in the low dimensional manifold we have built [32]. L and M are squared matrices whose shape we frequently know from the description of the problem we model—there is a vast literature in the field [33] [34] [25] [28] [23]—. Nevertheless, they cannot be projected to the non-linear low dimensional manifold where the database has been projected, where we risk to loss the rich thermodynamic structure induced by Eqs (3) and (4). As a result, they are also the objective of the regression procedure in the reduced manifold.
Experimental and pseudo-experimental data
GENERIC learns the sloshing dynamics of the liquids from the measurements of the state variables at discrete time steps. We have obtained pseudo-experimental data from simulations performed by employing Smoothed Particle Hydrodynamics (SPH, [35]) for training and test. Once it reaches the test phase, the twin learns previously unseen trajectories from experimental data.
Here, we generalize the scope of our first work [4] to identify the dynamics of a wider database. It has been specifically constructed with pseudo-experimental results of water, glycerine, butter, honey, mayonnaise and chocolate. Our intention is to cover different viscosities and densities, as well as behaviors. The different fluid constitutive models are sketched in Fig 2. Chocolate, mayonnaise and blood are considered shear thinning [36] [37] [38] [39]. We have selected the Herschel-Bulkley model [40] to reproduce their rheology. It follows the next expression:
in which τ is the shear stress, k the consistency index,
the shear rate, n the flow index, and τ0 the yield stress.
In the case of Newtonian fluids, their properties are constant over time and show a linear response. Their flow index n is then set to 1, and their yield stress to τ0 = 0. In contrast, shear thinning fluids start flowing when the stimulus is greater than the yield stress τ0 > 0. For these fluids, n > 1. In this work, fluids whose behavior can be assimilated to shear thinning have been considered. Fluids that incorporate some kind of plastic behavior need for a special treatment and have not been considered yet.
We have simulated four different sloshing trajectories for each liquid assuming Smooth Particle Hydrodynamics discretization [35], according to the parameters in Table 1. This approach provides discrete data of the properties of each particle. Data have been extracted from snapshots taken every 0.005 seconds to capture the important features of the dynamics and avoid the effect of overfitting as well.
When learning Newtonian fluids, we only need the position rj, velocity vj and internal energy Ej of each particle at discrete time instants. However, non-Newtonian fluids are not fully described with these state variables [41]. Some of the liquids we attempt to describe here have viscoelastic properties, and this behavior is captured by including the stress tensor τj—or a related magnitude—of each particle in the set of state variables. The fluid is then characterized at each time by a vector of these particle variables, such as
for every n > 1 equally spaced time steps in the interval (0, T] of each simulation. The fluid volume has been discretized into M = 2134 particles. If we have 13 degrees of freedom per particle, the full dimensionality of each snapshot vector is D = 27742.
Once the integrator has been built, we learn an unseen trajectory from a new vector of state variables, which establishes the initial conditions to emulate the sloshing dynamics. When running the digital twin, this information comes from the scene.
We make use of computer vision techniques to track the movement of the glass. For these purpose, we have added texture to it to track its features. See the details in the Appendix. This information has to be converted to an interpretable input for the integrator. The simulations of the database have been obtained by defining different input velocities of the glass to trigger the sloshing. Therefore, we relate the initial state of the virtual fluid with the tracked velocity of the container as an interpolation. With this information, we can perform the computation of the next state of the liquid.
There are some features of the algorithm that also require data acquisition of the free surface. Albeit the use of sensors is widely spread, image analysis is an attractive option for experimental data acquisition. The use of RGB Depth cameras is common used for this purpose [42] [43]. However, due to its lack of texture, it is a difficult task that could need to be supported with CNN for accurate tracking [44].
We followed an approach similar to the one employed in [45]. We analyze the color gradient of the binarized frame (Fig 3). There is a noticeable change between the free surface and the background. Therefore, the points in the boundary where there is a color gradient are considered free surface. These points are stored, tracked, and augmented in the image for user interaction and verification (Fig 4).
Original frame (top) versus binarized one (bottom). The picture is transformed firstly to gray scale for gentle binarization. Noise is also filtered to detect a smooth surface, which is highlighted in red in the top figure.
The points selected to belong to the free surface are highlighted in red over the original frame for verification.
Fluid clustering
The digital twin should be able to mimic the behavior of any type of fluid, not necessarily in the database. Then, in future applications, it must perceive and recognize the fluid as well as explain what it is watching in understandable terms for a human. This is called interpretable machine learning [46].
We expect to classify the fluid with only a few observations and, knowing which fluid it is—or interpolating between the closest neighbors in the phase space—, make the calculations in the appropriate reduced order manifold.
The high dimensionality of the problem would make unfeasible to perform the classification as fast as the application requires. We address this problem by applying model order reduction techniques. This allows us to perform the classification in a lower-dimensional manifold. Dimensionality reduction is a common preprocessing step for classification tasks. This process entails a noticeable time reduction in the classification. In addition, if the dataset has a lower dimensional structure, we avoid the noise of the large dimensional set, what results in an improvement of the accuracy.
We choose again k-PCA to perform the reduction of our problem [31]. k-PCA enables us to project the data onto a low dimensional manifold. In our case, and despite the smooth decay in the eigenvalues, see Fig 5, some 3 dimensions have shown to provide with good clustering results. The different types of fluids remain clustered in the new projection, as can be seen in Fig 6. k-PCA is able to unveil the features that make the behavior of each fluid unique with respect to the others. We expect this fact to be advantageous for the classification process.
We distinguish three modes that stand out with regard to the others. This fact justifies the reduction to the embedded manifold. As a result, we aim to provide with a more manageable and efficient system for classification.
In our case, we employ random forests [47] for classifying our dataset. This technique consists of constructing several decision trees in different subspaces of the training set to generalize the classification and, as a result, avoid the overfitting that usually appears in single regression tree techniques.
We also apply k-fold cross validation as part of the learning process. The method suggests to split the database into two parts, one assigned for training and the rest saved for testing. This process is repeated iteratively k loops, changing the distribution data of training and test sets, to improve the fitting.
We need enough data to recognize the underlying trend, but we also need to leave sufficient for testing so as to avoid high variance error. According to this criteria, we establish a relation of 80% of snapshots for training, and 20% for test, in our algorithm. We trained the model following a cross validation scheme in k = 5 iterations.
Overall, the results obtained from the classification algorithm showed a good performance. High accuracy was achieved analyzing both global results as well as the error obtained for each fluid individually (Table 2). With this result, we consider the model valid for our problem, as well as for decision making applications.
Augmented reality
The prediction performed by the integration scheme is presented to the user through augmented reality (AR). It provides a user friendly interface from control and understanding. This technology is usually employed for rigid representation, i.e. the virtual object does not interact with any real stimulus. Our twin is deformable, and interacts actively with the scene.
We employ the tracking information previously obtained for precise placing of the augmented liquid. In consequence, it continuously updates and shows real time connection and interaction with the glass.
AR representation shows the position of the particles and the free surface. In addition, it can augment the representation of the liquid showing additional information, such as the velocity field.
Results and discussion
We have tested the online performance of the twin to evaluate the implementation of the algorithm in conjunction with the computer vision techniques that we employ for data extraction. We obtained positive results in the merging of both elements. Trajectory generation and augmented representation is done online, and coupled with the video. As a result, we perform real time calculation and representation.
The replica of the liquid has been also compared with a glass filled with a liquid of the same type. Both containers are subjected to the same forces. Qualitatively, liquids are synchronized. Nevertheless, the movement of the digital twin seems to be a bit more amplified.
While previous approaches in the field (notably [44] [2] [5]) report qualitative performance measurements only, we have also tried to provide with quantifiable results to perform accurately the experimental validation. We quantify the error in the reproduction of the free surface reconstruction, defined as the integral of the differences between the heights regarding a medium line, see Fig 7:
(7)
with R and V representing the real (physical) height and the virtual one, respectively, see.
Free surface is defined as a function of its height at different points. These heights are compared in a same snapshot to evaluate the reconstruction error.
The resulting errors are shown in Table 3. Absolute value measurements are provided at h(0) and h(l) in mm for comparison. The error is also expressed in mm. They remain adequate according to the state-of-the-art in computer vision applications. The error grows with higher amplitudes of slosh, see Fig 8. Some sources of error could root in the approximation performed for velocity estimation. Remember that the pseudo experimental data with which we have built the model came also from a numerical approximation, SPH. Nevertheless, the resemblance is sufficient for learning a model, as well as corrections that will improve its performance.
The free surface reconstruction has been evaluated to compute the error.
Snapshot number refer to the ones shown in Fig 8.
Conclussions and recommendations
In this paper we have presented and described a digital twin able to learn the sloshing dynamics occurring within a glass. We have shown that it satisfactorily reproduces the dynamics that a real stimulus would cause. The merge of online and realtime data acquisition, calculation and result representation has enabled realistic interaction between the two mediums.
The digital twin connects with the scene trough computer vision techniques based on feature extraction to obtain the velocity of the container. This is the input of the simulations that we have performed to obtain synthetic data with which the model has been built. Therefore, we establish a straight relationship between scene data and interpretable inputs for the twin.
Realtime performance has been achieved through the use of model order reduction techniques. k-PCA finds a space of 3-4 dimensions where the dynamics are embedded to perform the calculation with minimal loss of information. We have also proved the efficacy of GENERIC to learn more complex behaviors, such as viscoelasticity, widening the options that the twin offers.
While it has been difficult, in general, to obtain fully meaningful and quantitative comparisons with existing methods—that, in addition, focus on the pouring process, while we are interested in the sloshing phenomenon—, our method guarantees by construction the fulfillment of the laws of thermodynamics while bypassing the integration of Navier-Stokes equations. This has shown to provide very accurate results.
Undoubtedly, perception performance is a field of deep interest. New techniques constantly appear for achieving new learning challenges. Given these tools, and the results obtained from the classification training, new capabilities can be added to the twin. We expect to feed the algorithm with data of the free surface of a real fluid so as to distinguish the liquid properties of the liquid perceived. Also, the model could be corrected in case the liquid is unknown to go a step further and transform the model into a hybrid twin.
Appendix—Stereo camera fundamentals
The camera employed in our application is a Zed Mini model from Stereo Labs (https://www.stereolabs.com/zed-mini/). This camera incorporates a stereo system and an Inertial Measurement Unit, IMU. The camera is able to instantaneously provide the user with its intrinsic and extrinsic parameters, see Fig 9. This fact helps to speed up the computation of the inputs and outputs of our digital twin, as well as its augmented reality reconstruction.
The camera moves freely, and its movement is related to the origin position through the extrinsic parameters. In computer vision, at least two images are required for the 3D reconstruction of a point. The camera performs continuous triangulations so as to export the depth of each pixel relating the 2D matches detected among right and left lenses.
The relationship between the 3D world and a 2D image, and viceversa, is build from the so-called intrinsic and extrinsic parameters. Intrinsic parameters relate the 2D position of a point, in pixel coordinates, with its 3D position with respect to the camera. Those parameters are the focal length of right and left cameras, and pixel spacing sx and sy. They form the calibration matrix K.
On the other hand, extrinsic parameters are those that represent the camera’s rotation (R) and translation (t) with regard to a reference coordinate system. With all this information, a point can be mapped from the real world to the pixel coordinates, and viceversa.
Having prior knowledge of the camera’s calibration and position, the 3D estimation of every point is performed through triangulation [48]. As mentioned before, the camera outputs the intrinsic and extrinsic parameters at each frame. As a result, we know the projection of the 2D features in the 3D real world system to perform any operation in the algorithm implemented.
Feature extraction
Transparent objects, such us those made of glass, have always entailed an extreme difficulty for feature extraction algorithms due to their lack of texture. Only recently techniques based on deep convolutional neural networks, CNN, have been developed to overcome this problem [49] [50].
In contrast, we aim to simplify the process for this application. Instead of using fiducial markers [22] or object alignment, we have added texture to the glass. Little points were painted in the glass to create points of interest that the feature detector could select. From the detection of these features, or points of interest, we localize the center of masses of the glass projected to its bottom surface.
We assume that we do not have prior knowledge of the position of the glass. We apply the Shi-Tomasi algorithm [51] for feature extraction in the area where the glass is expected to be. It finds the strongest, and more stable, features to track along the video sequence. The camera straightly provides the 3D position of the selected points of interest. With those points, we compute the center of masses of the glass projected to the bottom of the container. By tracking that point, we obtain information of the position and velocity of the glass.
Acknowledgments
This work has been partially supported by the Spanish Ministry of Economy and Competitiveness through Grant number DPI2017-85139-C2-1-R and by the Regional Government of Aragon and the European Social Fund, research group T88. The support of ESI Group through the project UZ-2019-0060 is also gratefully acknowledged.
References
- 1.
Pan Z, Park C, Manocha D. Robot motion planning for pouring liquids. In: Twenty-Sixth International Conference on Automated Planning and Scheduling; 2016.
- 2. Schenck C, Fox D. Perceiving and reasoning about liquids using fully convolutional networks. The International Journal of Robotics Research. 2018;37(4-5):452–471.
- 3.
Li Y, Wu J, Tedrake R, Tenenbaum JB, Torralba A. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. arXiv preprint arXiv:181001566. 2018;.
- 4. Moya B, Gonzalez D, Alfaro I, Chinesta F, Cueto E. Learning slosh dynamics by means of data. Computational Mechanics. 2019;64(2):511–523.
- 5. Kennedy M, Schmeckpeper K, Thakur D, Jiang C, Kumar V, Daniilidis K. Autonomous Precision Pouring From Unknown Containers. IEEE Robotics and Automation Letters. 2019;4(3):2317–2324.
- 6.
Shirangi MG, Ettehadi R, Aragall R, Furlong E, May R, Dahl T, et al. Digital Twins for Drilling Fluids: Advances and Opportunities. In: IADC/SPE International Drilling Conference and Exhibition. Society of Petroleum Engineers; 2020.
- 7. Cameron DB, Waaler A, Komulainen TM. Oil and Gas digital twins after twenty years. How can they be made sustainable, maintainable and useful? 2018;.
- 8. Brunton SL, Noack BR, Koumoutsakos P. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics. 2020;52:477–508.
- 9.
Raissi M, Perdikaris P, Karniadakis GE. Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations. arXiv preprint arXiv:171110561. 2017;.
- 10.
Lee K, Carlberg K. Deep Conservation: A latent dynamics model for exact satisfaction of physical conservation laws. arXiv preprint arXiv:190909754. 2019;.
- 11.
Qian E, Kramer B, Peherstorfer B, Willcox K. Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems. arXiv preprint arXiv:191208177. 2019;.
- 12. Brunton SL, Proctor JL, Kutz JN. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences. 2016;113(15):3932–3937.
- 13. De Vuyst F, Villon P. Identification of nonlinear dynamical system equations using dynamic mode decomposition under invariant quantity constraints. Comptes Rendus Mécanique. 2019;347(11):882–890.
- 14.
Wu K, Qin T, Xiu D. Structure-preserving Method for Reconstructing Unknown Hamiltonian Systems from Trajectory Data. arXiv preprint arXiv:190510396. 2019;.
- 15. El Saddik A. Digital twins: The convergence of multimedia technologies. IEEE MultiMedia. 2018;25(2):87–92.
- 16. Badías A, Alfaro I, González D, Chinesta F, Cueto E. Reduced order modeling for physically-based augmented reality. Computer Methods in Applied Mechanics and Engineering. 2018;341:53–70.
- 17. Aversano G, Bellemans A, Li Z, Coussement A, Gicquel O, Parente A. Application of reduced-order models based on PCA & Kriging for the development of digital twins of reacting flow applications. Computers & chemical engineering. 2019;121:422–441.
- 18.
Hartmann D, Herz M, Wever U. Model order reduction a key technology for digital twins. In: Reduced-Order Modeling (ROM) for Simulation and Optimization. Springer; 2018. p. 167–179.
- 19.
Tezzele M, Demo N, Mola A, Rozza G. An integrated data-driven computational pipeline with model order reduction for industrial and applied mathematics. arXiv preprint arXiv:181012364. 2018;.
- 20.
Kapteyn MG, Willcox K, Knezevic DJ. Toward predictive digital twins via component-based reduced-order models and interpretable machine learning. In: AIAA Scitech 2020 Forum; 2020. p. 0418.
- 21. Badías A, Curtit S, González D, Alfaro I, Chinesta F, Cueto E. An augmented reality platform for interactive aerodynamic design and analysis. International Journal for Numerical Methods in Engineering. 2019;120(1):125–138.
- 22.
Fujisawa M, Kato H. Interactive Fluid Simulation Using Augmented Reality Interface. In: International Conference on Virtual and Mixed Reality. Springer; 2009. p. 431–438.
- 23. Grmela M, Öttinger HC. Dynamics and thermodynamics of complex fluids. I. Development of a general formalism. Physical Review E. 1997;56(6):6620.
- 24. González D, Chinesta F, Cueto E. Thermodynamically consistent data-driven computational mechanics. Continuum Mechanics and Thermodynamics. 2019;31(1):239–253.
- 25. Ghnatios C, Alfaro I, González D, Chinesta F, Cueto E. Data-Driven GENERIC Modeling of Poroviscoelastic Materials. Entropy. 2019;21(12):1165.
- 26.
González D, Chinesta F, Cueto E. Learning corrections for hyperelastic models from data. 2019;.
- 27.
Grmela M, Klika V, Pavelka M. Gradient and GENERIC evolution towards reduced dynamics. arXiv preprint arXiv:191207693. 2019;.
- 28. Mielke A. Formulation of thermoelastic dissipative material behavior using GENERIC. Continuum Mechanics and Thermodynamics. 2011;23(3):233–256.
- 29. Roweis ST, Saul LK. Nonlinear dimensionality reduction by locally linear embedding. science. 2000;290(5500):2323–2326. pmid:11125150
- 30. Wasserman L. Topological data analysis. Annual Review of Statistics and Its Application. 2018;5:501–532.
- 31. Schölkopf B, Smola A, Müller KR. Nonlinear component analysis as a kernel eigenvalue problem. Neural computation. 1998;10(5):1299–1319.
- 32. Öttinger HC. Preservation of thermodynamic structure in model reduction. Physical Review E. 2015;91(3):032147.
- 33. Romero I. Thermodynamically consistent time-stepping algorithms for non-linear thermomechanical systems. International Journal for Numerical Methods in Engineering. 2009;79(6):706–732.
- 34. Portillo D, García Orden J, Romero I. Energy–entropy–momentum integration schemes for general discrete non-smooth dissipative problems in thermomechanics. International Journal for Numerical Methods in Engineering. 2017;112(7):776–802.
- 35. Monaghan JJ. Smoothed particle hydrodynamics. Annual review of astronomy and astrophysics. 1992;30(1):543–574.
- 36. Izidoro D, Sierakowski MR, Waszczynskyj N, Haminiuk CW, de Paula Scheer A. Sensory evaluation and rheological behavior of commercial mayonnaise. International journal of food engineering. 2007;3(1).
- 37. Kumbár V, Nedomová Š, Ondruš#x00ED;ková S, Polcar A. Rheological behaviour of chocolate at different temperatures. Potravinarstvo. 2018;12(1).
- 38.
Elert G. The Physics Hypertextbook. https://physicsinfo. 2020;.
- 39.
Smith M. Approximate viscosities of some common liquids. https://wwwmichael-smith-engineerscouk/mse/uploads/resources/useful-info/General-Info/Viscosities-of-Common-Liquidspdf. 2020;.
- 40. Herschel WH, Bulkley R. Konsistenzmessungen von gummi-benzollösungen. Kolloid-Zeitschrift. 1926;39(4):291–300.
- 41. Español P, Serrano M, Öttinger HC. Thermodynamically admissible form for discrete hydrodynamics. Physical review letters. 1999;83(22):4542.
- 42.
Do C, Schubert T, Burgard W. A probabilistic approach to liquid level detection in cups using an RGB-D camera. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE; 2016. p. 2075–2080.
- 43. Tosun U, Aghazadeh R, Sert C, Özer MB. Tracking free surface and estimating sloshing force using image processing. Experimental Thermal and Fluid Science. 2017;88:423–433.
- 44.
Schenck C, Fox D. Detection and tracking of liquids with fully convolutional networks. arXiv preprint arXiv:160606266. 2016.
- 45.
Eppel S. Tracing liquid level and material boundaries in transparent vessels using the graph cut computer vision approach. arXiv preprint arXiv:160200177. 2016;.
- 46.
Doshi-Velez F, Kim B. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:170208608. 2017;.
- 47. Breiman L. Random forests. Machine learning. 2001;45(1):5–32.
- 48.
Hartley R, Zisserman A. Multiple view geometry in computer vision. Cambridge university press; 2003.
- 49.
Khaing MP, Masayuki M. Transparent Object Detection Using Convolutional Neural Network. In: International Conference on Big Data Analysis and Deep Learning Applications. Springer; 2018. p. 86–93.
- 50.
Sajjan SS, Moore M, Pan M, Nagaraja G, Lee J, Zeng A, et al. ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation. arXiv preprint arXiv:191002550. 2019;.
- 51.
Shi J, et al. Good features to track. In: 1994 Proceedings of IEEE conference on computer vision and pattern recognition. IEEE; 1994. p. 593–600.