Toward trustworthy medical device in silico clinical trials: a hierarchical framework for establishing credibility and strategies for overcoming key challenges

Computational models of patients and medical devices can be combined to perform an in silico clinical trial (ISCT) to investigate questions related to device safety and/or effectiveness across the total product life cycle. ISCTs can potentially accelerate product development by more quickly informing device design and testing or they could be used to refine, reduce, or in some cases to completely replace human subjects in a clinical trial. There are numerous potential benefits of ISCTs. An important caveat, however, is that an ISCT is a virtual representation of the real world that has to be shown to be credible before being relied upon to make decisions that have the potential to cause patient harm. There are many challenges to establishing ISCT credibility. ISCTs can integrate many different submodels that potentially use different modeling types (e.g., physics-based, data-driven, rule-based) that necessitate different strategies and approaches for generating credibility evidence. ISCT submodels can include those for the medical device, the patient, the interaction of the device and patient, generating virtual patients, clinical decision making and simulating an intervention (e.g., device implantation), and translating acute physics-based simulation outputs to health-related clinical outcomes (e.g., device safety and/or effectiveness endpoints). Establishing the credibility of each ISCT submodel is challenging, but is nonetheless important because inaccurate output from a single submodel could potentially compromise the credibility of the entire ISCT. The objective of this study is to begin addressing some of these challenges and to identify general strategies for establishing ISCT credibility. Most notably, we propose a hierarchical approach for assessing the credibility of an ISCT that involves systematically gathering credibility evidence for each ISCT submodel in isolation before demonstrating credibility of the full ISCT. Also, following FDA Guidance for assessing computational model credibility, we provide suggestions for ways to clearly describe each of the ISCT submodels and the full ISCT, discuss considerations for performing an ISCT model risk assessment, identify common challenges to demonstrating ISCT credibility, and present strategies for addressing these challenges using our proposed hierarchical approach. Finally, in the Appendix we illustrate the many concepts described here using a hypothetical ISCT example.

proposed hierarchical ISCT credibility assessment framework.The inclusion of this example should not be interpreted as an indication of FDA acceptance of ISCTs for mitral valve edge-to-edge repair technologies, or that the methods, results, or justifications would be sufficient for these devices within a regulatory context.

A1. Description of Computational Model
A hypothetical ISCT is performed for a generic transcatheter edge-to-edge mitral valve repair device using a combination of (i) physics-based modeling to simulate device and patient structural mechanics, dynamics, and hemodynamics and (ii) statistical and data-driven modeling to generate a virtual synthetic cohort as inputs to the physics-based patient models.Following the proposed framework (Figure 2 of the main document), the ISCT is organized into six submodels: (i) device model, (ii) patient model, (iii) coupled device-patient model, (iv) virtual patient cohort model, (v) clinician model, and (vi) clinical outcome mapping model (Figure A1).Each of these models are addressed in examples that follow.

A1.1. Device Model
The device model is a physics-based, finite element (FE) rigid-body dynamics model implemented in Abaqus/Explicit (7).In brief, the model solves the three-dimensional Newton-Euler equations to simulate the motion of the components of a generic mitral valve clipping device (Figure A2).The delivery system is not considered.The notional clipping device is symmetrical and consists of two pairs of rotating bodies: two primary "arms" and two "grippers" (Figure A2-a).The arms and grippers are both 9 mm long and 4 mm wide, and they have thicknesses of 1.5 mm and 0.5 mm, respectively.The edges of the device are rounded to mitigate numerical contact singularities that could occur when incorporating the model into coupled device-patient simulations (Figure A2-a).Stop plates are also implemented to limit leaflet insertion (Figure A2-a).A rotational boundary condition about a single axis of rotation is used to constrain the motion of the clip components (Figure A2b-d).In addition to the constrained rotational axis, the overall translation and rotation of the clip is controlled using a single rigid body reference node with six degrees of freedom (black dot in Figure A2-a).The device is discretized using rigid-body elements and assigned a uniform density.

Limitations and Gaps:
• The device model only considers rigid body motion.The neglect of local device stresses and strains limits the ability of the model to predict device mechanics, and it introduces approximations in the prediction of device performance.• The model is highly idealized and does not consider components characteristic of a real implantable edge-to-edge repair device including pins, fixation barbs, and polymeric elements.

A1.2. Patient Model
The baseline patient model is a multi-physics model performed using a co-simulation between Abaqus/Explicit and Dymola (8) (Figure A3).In brief, Abaqus is used to solve the three-dimensional timeresolved Cauchy momentum equations and to simulate valve solid mechanics (Figure A3-a).Dymola, a zero-dimensional system-based lumped-parameter tool, is used to estimate a critical boundary condition for the Abaqus simulations, the transmitral pressure difference (Figure A3-b).The two solvers are coupled to simulate valve motion during a cardiac cycle (Figure A3-c).The baseline patient model includes input parameters that are specified to create individual patient models (Table A1).Variable input parameters include constitutive model inputs, initial and boundary condition quantities, and key geometric measurements (Table A1).By modifying these parameters, the model can be adapted to either approximate a specific real patient or to generate fully synthetic virtual patients.

Mitral Valve Structural Model:
The structural model consists of both mitral valve leaflets and idealized chordae (Figure A3-a).Although papillary muscles are not modelled, the coordinates of the two papillary-chordae attachment locations are defined (Figure A3-a and Table A1).Leaflets are modelled with a uniform thickness of approximately 2mm and are discretized using linear, incompatible mode, hexahedral brick elements.An anisotropic Holzapfel-Ogden constitutive law is used to simulate leaflet tissue.Baseline material constants are calibrated using biaxial human leaflet tensile data from the literature.Chordae are modelled with a uniform cross-sectional area and are discretized using truss elements.The material response of chordae is modelled using a neo-Hookean hyperelastic constitutive law.Chordae insertion points on the leaflets are uniformly distributed along the bottom surface with a uniform density based on observations in literature.No branching of the chordae between the attachment and insertion points is considered.Chordae start at each insertion point and are attached to the nearest papillary-chordae attachment point.
The papillary-chordae attachment locations and the annular edges of the leaflets are prescribed as boundary conditions, both of which can be fixed or time-varying (Table A1).General surface-to-surface frictionless contact is defined to account for collisions between opposing leaflets and among nodes within the same leaflet (i.e., self-contact).
Circulatory System Lumped-Parameter Model: Pressure boundary conditions on the valve are predicted using a closed-loop lumped-parameter model (Figure A3-b).The model simulates the bulk effects of blood flow within the heart and to and from the circulatory system.The model is comprised of capacitors with active time-varying capacitance to simulate the compliance of ventricular and atrial chambers.Systemic vein, systemic artery, pulmonary vein, and pulmonary artery chambers are simulated as passive capacitors.Resistances are assigned to model flow through the aortic and pulmonary valves and between adjacent chambers.Blood flow initiates in response to the initial pressure difference assigned to the chambers.Flow between the ventricles and atria through the mitral and tricuspid valves is modeled using a Bernoulli-like equation that captures the effect of blood inertance, blood resistance, and time-varying valve orifice area.

Co-simulation Model:
Co-simulation between the three-dimensional structural and zero-dimensional lumped-parameter models is performed by coupling the two solutions at a user-defined time interval (Figure A3-c).At each time step, a custom post-processing routine calculates the time-dependent anatomic orifice area of the leaflets given by the structural model and communicates the orifice area to the lumped parameter model (Figure A3-c).The lumped parameter model then calculates the instantaneous left ventricular and atrial pressures, and the pressure difference is communicated back to the structural model (Figure A3-c).The pressure difference is imposed as a spatially uniform pressure boundary condition on the ventricular side of the mitral valve leaflets.
The coupled mitral valve and circulatory system models are used to predict several acute quantities of interest (Table A1).The models are also used to estimate a categorical mitral valve regurgitation severity grade based on model predictions of regurgitant volume (Table A1) using a suggested threshold from clinical guidelines (9).

Simulating the Pre-Operative Patient State:
The pre-operative state of a patient is simulated using the following steps starting at end diastole: 1. Initial chamber pressures are specified (Table A1) 2. The nodes at the annulus and the papillary muscle attachment points are fixed and chordae lengths are adjusted at diastole (lengthened or shortened) by performing an analysis to achieve an approximated target systolic state where most chordae become straight and are under tension.Subsequently, any resulting residual strain in the chordae and leaflets caused by these adjustments are set to zero to approximate the nearly strain-free state at 70% diastole.
3. The co-simulation begins during which pressure is calculated and applied to the leaflets as described above and the annular and attachment node locations that were fixed in step 2 are prescribed to be time-varying 4. Cardiac cycles are simulated until reaching a periodic steady-state response

Limitations and Gaps:
• The model only considers mitral valve structural mechanics and simplified circulatory system fluid dynamics.Electrophysiology is not included.The model also neglects any tissue remodeling that may occur in response to changes in patient mechanics or hemodynamics.• Valve mechanics are complex and rely critically on three-dimensional fluid-structure interactions (10,11).The current model only simulates fluid mechanics using a lumped-parameter approach.The ability of this approach to precisely predict leaflet motion, coaptation behavior, and regurgitation metrics may be limited.• Accurate mitral valve simulation likewise relies critically on accurate modeling of the geometry of the patient's valve and associated tissues (12).As expressed in literature, "no clinical imaging modality is capable of visualizing the complete mitral valvular and sub-valvular anatomy with adequate spatial or temporal resolution for accurate computational modeling" (13).Here, the valve and cardiac anatomy are highly idealized, and chordae tendinea geometry, density, stiffness, and attachment locations are approximated.• Several key model input parameters such as valve material properties are approximated based on nominal values from literature.Accurately measuring these parameters for patient-specific models or generating representative parameter sets for synthetic patients remains a significant challenge.• Model outputs are limited to physics-based predictions (Table A1).

A1.3. Coupled Device-Patient Model
The coupled device-patient model combines the device and patient models incorporating the discretized valve and clip into the same co-simulation.Specifically, hard frictionless contact is initially prescribed between the device stop plates and the leaflets, and a penalty contact condition is prescribed between the device arms and grippers and the leaflets.The combined model is used to simulate clip deployment (Figure A4).First, the clip reference node is translated and rotated (Table A2) such that the leaflets are inserted into the clip arms (Figure A4-a).Second, the grippers are rotated (Table A2) toward the arms to fully grasp the leaflets (Figure A4-b).The arms are then rotated inward (Table A2) to close the clip (Figure A4-c).The rotational magnitudes of the grippers and arms are chosen carefully to ensure adequate contact occurs without excessive leaflet compression and element distortion.Frictionless contact conditions are then converted to bonded (no-slip) to simulate the influence of migration mitigating features of a real device.Finally, boundary conditions on the rigid body reference node are removed, and the clip is allowed to move in six degrees of freedom throughout the co-simulation of the cardiac cycle (Figure A4-d).

Table A2:
Coupled device-patient model inputs and outputs, in addition to the inputs and outputs listed in Table A1.

Inputs Outputs Clip deployment translation and rotation
Clipping forces and moments prior to clip release Rotations of clip grippers and arms Clipping length, symmetry, and grip pressure Post-clipping regurgitant orifice location(s)

Limitations and Gaps:
• The coupled model inherits all limitations of the individual device and patient models (see above).• The chosen contact models idealize interactions between the device and the patient as purely frictionless or bonded.Real contact interactions depend on local surface topology, lubrication, and other aspects not modelled herein.The neglect of barb fixation features in the device model also limits the realism of the contact interactions.• The model neglects any acute biological effects (e.g., damage) or longer-term remodeling that may occur in the valve in response to device placement.

A1.4. Virtual Patient Cohort Model
The objective of the virtual patient cohort model is to generate a collection of synthetic patients that are representative of real patients that would be enrolled in a real human clinical trial.Note the generation of realistic pathological patients is not trivial, as the influence of each individual input variable on valve performance may be nonlinear, interactions among input variables are complex, and correlations among input variables are generally unknown.Accordingly, generation of the virtual synthetic patient cohort involves several steps.The approach used here focuses on generating synthetic pathological patients and relies on a combination of physics-based and surrogate modeling to reduce computational costs (Figure A5).First, real patient measurements from literature are used to generate probability distributions for key input parameters needed by the baseline patient model (Table A1).The probability distributions are then sampled to generate an initial virtual patient cohort (Figure A5-i

Limitations and Gaps:
• The proposed filtering process excludes healthy synthetic patients based on predictions of physics-based quantities of interest.However, there is no guarantee that the remaining synthetic pathological patients are representative of real pathological patients "in a precisely defined way" (14).For example, correlations among input parameters are neglected during synthetic patient generation.Accounting for such correlations may be critical to the generation of realistic synthetic patients (15).

A1.5. Clinician Model
The clinician model combines real clinician input and a quantitative decision tree to define clipping boundary conditions for each virtual patient in the ISCT.The model is semi-automated and follows relevant consensus recommendations from clinical literature.
The clinician model proceeds as follows: • A real clinician examines a set of predictions from the pre-operative patient model and determines where to place a clip along the mitral valve line of coaptation • An automated procedure is used to combine the device, patient, and coupled device-patient models and to execute the clipping procedure for a first clip • Based on key predictions from the post-operative physics-based model, a second clip may be placed.In brief, if the predicted transmitral pressure gradient and mitral regurgitation grade fall within predefined ranges, the treatment is complete.If instead the values are outside the target post-treatment ranges, the first clip is repositioned, a second clip is placed, or both.

Limitations and Gaps:
• As implemented, the clinician model does not capture the diversity of all possible treatments, the likely intraprocedural variability among clinicians, or other human factors aspects of the treatment.
• The model uses a single clip size.Real devices feature several sizes to accommodate patient variability.• The model ensures at least one leaflet is completely seated into the clip.This is not guaranteed during a real clinical procedure.• The model only places the clip at pre-defined locations between 30% to 70% along the line of coaptation at 10% intervals.Real procedures, however, include a continuum of possible clip placement locations.• The model does not allow for the same degree of iterative adjustment as in a real intervention.

A1.6. Clinical Outcome Mapping Model
Currently, there is not a widely accepted correlation between acute physics-based quantities of interest and clinical outcomes for edge-to-edge mitral valve repair devices.Accordingly, to predict clinical outcomes using an ISCT, a correlation would first need to be established and then validated.
At a high level, historical data show reducing mitral valve regurgitation via edge-to-edge repair improves heart function and reduces mortality and re-hospitalization in some patients.A mapping function to correlate changes in one or more acute post-procedural clinical measurements (potential quantities of interest) with patient outcomes could be proposed based on clinical studies in the literature.Possible physics-based quantities that may correlate with clinical outcomes include: • diastolic mitral valve pressure gradient (16) • residual mitral regurgitation (6,17) • pulmonary venous waveforms (18) • ratio of left ventricle end-diastolic volume to effective regurgitant orifice area (proportionate or disproportionate mitral regurgitation) (19) • decreased left atrial pressure (20) For the hypothetical ISCT, we propose beginning with the identified correlation between post-operative mean left atrial pressure and mortality identified by El Shaer et al. (20).In brief, the investigators found the patient cohort with post-operative left atrial pressure less than 22 mmHg had higher survival rates compared to the cohort with higher post-operative left atrial pressure (hazard ratio=1.71[95% CI, 1.10-2.70]P=0.02; see Figure 1 in El Shaer et al. (20) and accompanying text).

Limitations and Gaps:
• As mentioned, there is currently not a widely accepted correlation between acute physics-based quantities and clinical outcomes for edge-to-edge mitral valve repair.The proposed mapping model relies on a correlation recently observed from a clinical study in the literature (20).However, the observation of a correlation in one patient cohort is not guaranteed to be predictive for all patients.The predictive capability of the observed correlation and the associated mapping model should ideally be supported by follow-on independent validation.• In the study by El Shaer et al. (20), mortality still occurred in the low post-procedural left atrial pressure patient cohort (approximately 25% compared to 40% in the high left atrial pressure patient cohort).Thus, mapping models should consider the strength of underlying correlations and include estimates of the uncertainty in any outcome predictions.

A1.7. Full ISCT
Following the proposed framework (Figure 3 in the main document), the full ISCT combines all of the previously described submodels to predict patient outcomes (Figure A6).Specifically, the clinical treatment and virtual cohort models drive physics-based simulations of device placement in virtual patients.Physics-based simulations then predict the post-operative mean left atrial pressure.The clinical outcome mapping model then predicts patient outcomes for the virtual cohort based on the acute, physics-based predictions of post-operative mean left atrial pressure.Finally, the ISCT results are used to inform revisions to the baseline enrollment criteria to prospectively exclude patients who are less likely to respond favorably to the intervention.

A2. Model Credibility Assessment A2.1. Summary of Hierarchical Credibility Assessment Approach
The specific activities that are needed to demonstrate credibility of the hypothetical ISCT will depend on the model risk assessment.For most applications that have non-negligible model risk, it is important to demonstrate the credibility of each of the ISCT submodels in some way to ensure that the full ISCT produces the "right answer for the right reasons" (21,22).This hypothetical ISCT example follows the proposed hierarchical credibility assessment approach described in the main body of this study.First, the ISCT question of interest and context of use are defined.A model risk assessment is then performed to evaluate the overall ISCT model risk.Given our estimated model risk, we then present credibility evidence.Importantly, note that we do not attempt to provide examples for all of the possible credibility evidence.Rather, we provide several examples of the types of evidence that could be collected for a subset of the ISCT submodels that are highlighted in Table A3

A2.2. Question of Interest
In this hypothetical example, an enrichment strategy is undertaken in which an ISCT is performed for a generic edge-to-edge mitral valve repair device, and the results are combined with other information to justify enrollment criteria for a proposed clinical trial in an investigational device exemption (IDE) application.

Question of Interest:
What enrollment criteria should be used for the clinical trial of the generic edge-toedge mitral valve repair device to prospectively exclude patients who are less likely to respond favorably to the intervention?

Context of Use: In this hypothetical enrichment example, an ISCT is performed to provide supporting evidence to establish enrollment criteria for a clinical trial of a generic edge-to-edge mitral valve repair device. Recent evidence shows that mean left atrial pressure measured immediately following edge-toedge mitral valve repair is an independent predictor of improved patient survival (20). To address the question of interest, baseline enrollment criteria are first defined based on prior data. An ISCT is then performed to predict acute post-operative mean left atrial pressure in a virtual patient cohort, and a mapping function converts mean left atrial pressure to predicted mortality (Figure A7). The ISCT results inform revisions to the baseline enrollment criteria to prospectively exclude patients who are less likely to respond favorably to the intervention. In this hypothetical example, the ISCT only informs enrollment criteria refinement; it does not influence any other aspects of the clinical trial.
In the event that the real human clinical trial successfully demonstrates that the device is safe and effective, the device would be indicated for the same patient population studied in the trial.

A2.4. Model Risk Assessment A2.4.1. Model Influence
In addition to ISCT evidence, other information will be used to define the trial enrollment criteria, including pre-clinical experiences in animal studies, enrollment criteria for clinical trials performed for other similar devices, and clinical judgement.The overall strategy consists of defining baseline enrollment criteria based on these other sources of information and then using the results of the ISCT to refine the criteria to prospectively exclude patients who are less likely to respond favorably to the intervention.Given this overall strategy, the ISCT evidence has a moderate influence on the decision, but it is weighted less heavily than the other supporting evidence.From the example gradation of possible model influence scenarios for an ISCT in Table 4 of the main document, this corresponds to a model influence of Low-Medium.
Please note that an assessment of Low-Medium model influence does not imply that the ISCT evidence is unimportant.Rather, the ISCT evidence may contribute to strengthening the hypothetical investigational device exemption (IDE) submission toward a favorable benefit-to-risk determination.

A2.4.2. Decision Consequence
For this hypothetical example, the decision tree flowchart shown in Figure 4 of the main document is used to show how the ISCT results inform the selection of enrollment criteria for the proposed real human clinical trial and the potential consequences.As illustrated, potential patient harm could result from an incorrect decision concerning the question of interest, which is: "What enrollment criteria should be used for the clinical trial of the generic edge-to-edge mitral valve repair device to prospectively exclude patients who are less likely to respond favorably to the intervention?" To assess the decision consequence, we consider what patient harm could stem from a bad choice of the enrollment criteria for the proposed clinical trial.In this case, the potential hazardous situation is that of selecting a patient population that will not benefit from the mitral valve repair device, but will potentially suffer procedure-or device-related adverse events that may include ( 23 Following ISO 14971:2019 (24), to evaluate decision consequence we consider the potential severity and probability of occurrence of harm for such potential adverse events.Using the qualitative levels listed in Table 5 of the main document, the potential severity of harm is estimated to be "Critical" given that many of the foregoing adverse events are potentially life threatening (e.g., device embolization).To estimate the probability of occurrence of harm, we use the device-related complication rate of 3.4% that was reported for a similar marketed device by Stone et al. (17).From Table 6 of the main document, this corresponds to an estimated probability of occurrence level of "Probable."Combining the severity and probability of occurrence estimates, using the 5×5 matrix in Figure A8, we assess that the overall decision consequence is Medium-High.

A2.4.3. Model Risk
To summarize, the model influence and decision consequence for a hypothetical ISCT of a generic edgeto-edge mitral valve repair device were estimated to be:

Model Influence: Low-Medium Decision Consequence: Medium-High
To assess the model risk, we combine these estimates in Figure A9.Here we see that the model risk is estimated to be Medium.

A2.5.2. Patient Model
Evidence Category: ② -Model calibration evidence Summary of Credibility Activity: Patient model parameters are calibrated and the ability of the calibrated model to predict clinical measurements for a single patient with secondary mitral valve regurgitation is assessed.

Calibration approach:
The patient model as described in Section A1.2 is used for the calibration simulations, with the exception that the leaflet geometries are generated based on patient-specific cardiac computed tomography data.In brief, computed tomography images at diastole are manually segmented using iTK-Snap and then processed in 3DEXPERIENCE as follows: triangulated surface meshes corresponding to the ventricular side of the mitral valve leaflets are extracted, three-dimensional leaflet volumes are created by extruding the surface mesh uniformly in the normal (atrial) direction, and finite element volume meshes are generated.The locations of the nodes on the leaflet annulus and those that correspond to the attachment points of the chordae to the papillary muscles are also extracted from the clinical images at end systole and end diastole, and time-varying displacement boundary conditions are specified to control their positions during simulated cardiac cycles.
To perform the calibration, a preliminary sensitivity study of the lumped parameter hemodynamic model is first used to identify the ten input parameters (out of 92 in total) with the greatest influence on clinical quantities of interest.The identified influential parameters are then calibrated in the patient model co-simulation using an automated multi-objective optimization routine to minimize the difference between the simulation predictions and clinical measurements listed in Table A4.During this step, the initial chordae configuration is estimated based on patient images and anatomical studies from the literature.Fixed ranges are also specified for the ten calibrated parameters to limit the search space to physiological ranges based on reference values from literature.Finally, after the automated calibration procedure, ad-hoc manual tuning of the chordae configuration and leaflet geometries is performed to minimize the error between the predicted and segmented shapes of the leaflets during systole.

Data (clinical):
Comparator data for model calibration includes pre-operative transesophageal echocardiography, transthoracic echocardiography, computed tomography (191 frames with voxel size 0.488 x 0.488 x 1 mm for each of 10 cardiac time points), and catheterization measurements.All data were obtained under Institutional Review Board (IRB) approval from Boston Children's Hospital and Washington University following patient consent.

Final calibration assessment:
After calibrating the model, the final patient model response and the clinical calibration data are compared quantitatively and qualitatively.Quantitative outputs that are compared are either direct outputs of the patient model or are calculated based on one or more direct outputs.Two categorical outputs are also compared: (i) mitral regurgitation grade, estimated based on the predicted regurgitant volume (9), and (ii) stenosis, evaluated based on mitral valve area at diastole (32).For numerical quantities of interest, a percent difference is calculated using the clinical data as the referent.
Summary of Results: Following calibration, predicted leaflet shapes in diastole and systole qualitatively match those observed from clinical transesophageal echocardiography imaging (Figure A10).Predicted and observed categorical measurements agree (Table A4), and percent differences in predicted quantities of interest range from -19% to +16% (Table A4).Limitations and Gaps: • The calibration activity relies on data for a single patient with secondary mitral valve regurgitation.Some model parameters may be universally applicable such that calibration on a limited patient data set can be justified.However, in practice, robustly establishing input parameters via calibration will likely involve considering multiple patients spanning the patient population, or possibly several separate calibration activities to establish appropriate input parameters for specific patient sub-populations.• Following calibration, final percent difference magnitudes for predicted quantities of interest remain as large as approximately 20%.This indicates not all parameters could be simultaneously calibrated to closely match clinical measurements.Potential reasons for the large percent differences include limitations in the ability of the model to resolve all relevant physics (i.e., model form error), incorrect estimation of physiological limits for the calibrated parameters, limitations of the calibration procedure itself to identify a global optimum parameter set within the multivariate search space, or inappropriate values used for the fixed (less influential) parameters of the model.• The calibration activity does not consider uncertainty in the calibration data.Clinical measurements considered herein rely on imaging data with limited spatial resolution and potential errors in temporal gating.Variability in measurements across cardiac cycles is also not estimated or considered.Thus, errors in clinical calibration measurements may bias the calibrated parameters and reduce the accuracy of the model for predictions away from calibration conditions.• Even in ideal scenarios using extremely precise calibration measurements and robust multivariate nonlinear optimization approaches, calibration does not guarantee the predictive accuracy of a computational model away from calibration conditions.Independent validation activities covering a range of patients representative of the target population would strengthen the overall model credibility evidence and help to confirm calibrated parameters are appropriate.5)).Using these revised credibility factors, goals for the number and range of patients to include in the validation activity could be established based on the model risk, and later reviewed as part of prospective and post-study adequacy assessments (1).• Using the previously calibrated input parameters for the patient model (Section A2.5.2), the coupled patient-device model predictions agree with clinical measurements within approximately 10% for all but one quantity of interest, mitral valve diastolic pressure gradient.
For the latter quantity, model predictions are 40% higher than the comparator measurements.For a real ISCT application, credibility factors from ASME V&V 40-2018 could again be useful to systematically assess the validation evidence (5).For example, goals for sub-factor 'agreement of output comparison' could be prospectively established and later assessed following the validation exercise.If the goal is not met -for example, if 40% error in mitral valve diastolic pressure gradient is not acceptable -then the model may need to be revised, the validation measurement accuracy improved, or the overall model context of use modified to account for limitations in model accuracy (1,2).• As in Section A2.5.2, no uncertainty quantification is performed for input parameters, model predictions, or clinical measurements.All inputs are fixed, and the validation exercise relies on a single deterministic simulation.Credibility factors from ASME V&V 40-2018 could once again be adapted (as in ( 5)) to establish goals for and assess the adequacy of uncertainty quantification activities performed to support the coupled device-patient model validation.

Figure A1 :
Figure A1: Submodels of the hypothetical ISCT for a generic edge-to-edge mitral valve repair device.

Figure A3 :
Figure A3: The baseline patient model includes (a) a structural Abaqus finite element model, (b) a Dymola lumped-parameter model, and (c) a co-simulation model that combines the structural and lumped-parameter models to predict the time-varying anatomic orifice area of the mitral valve.

Figure A4 :
Figure A4: The coupled device-patient model illustrating the four steps used to (a) position, (b-c) deploy, and (d) release the clip.
). Patient model finite element simulations are performed using this initial cohort to train a surrogate model for the prediction of primary quantities of interest associated with valve pathology (FigureA5-ii).Additional samples are then drawn from the input probability distributions to create a larger virtual cohort (FigureA5-iii), and the surrogate model is used to predict quantities of interest for each synthetic patient.Note that, at this stage, the synthetic cohort potentially includes both realistic and unrealistic pathological and healthy patients.To exclude healthy patients, the cohort is filtered based on predefined criteria (FigureA5-iv).Finally, a clustering analysis is performed to extract representative pathological patients (FigureA5-v) for use in the full ISCT.

Figure A5 :
Figure A5: Schematic illustrating an approach for generating a virtual patient cohort with target characteristics.(i) An initial synthetic patient cohort is generated by sampling probability distributions for key model input parameters.(ii) The initial cohort is used to train a surrogate model for predicting quantities of interest related to mitral valve regurgitation.(iii) A second, larger synthetic patient cohort is generated, and the surrogate model is used to predict the severity of regurgitation for each synthetic patient.(iv) The second, larger cohort is filtered to include only pathological patients based on predefined criteria.(v) A clustering analysis is performed to extract representative pathological patients for use in the full ISCT.

Figure A6 :
Figure A6: Workflow for the hypothetical ISCT combining submodels from Figure A1.

Figure A7 :
Figure A7: Graphical summary of the context of use for the hypothetical ISCT of an edge-to-edge mitral valve repair device.
): • Single leaflet device attachment • Device embolization • Endocarditis requiring surgery • Mitral valve stenosis • Myocardial perforation • Iatrogenic atrial septal defect • Need for valve replacement rather than repair due in part to procedure-or device-related complications

Figure A8 :
Figure A8: Assessment of decision consequence for the hypothetical ISCT of an edge-to-edge mitral valve repair device.

Figure A9 :
Figure A9: Assessment of model risk for the hypothetical ISCT of an edge-to-edge mitral valve repair device.

Figure A10 :
Figure A10: Qualitative comparison of three-dimensional transesophageal echocardiography (top) and patientspecific computational simulation (bottom) showing the mitral valve in diastole (left) and systole (right).The diastolic state from the clinical imaging data and the model demonstrates adequate leaflet opening with unobstructed inflow.In systole, echocardiography demonstrates a central to posterior coaptation defect as a source of regurgitation.The model predicts a corresponding central to posterior coaptation defect consistent with a common pseudocleft defect in the posterior leaflet as an etiology of the regurgitation in this patient.

Figure A12 :
Figure A12: Distributions of regurgitant volume (RVOL) for the initial virtual patient cohort.

Figure A13 :
Figure A13: Representative samples for each mitral valve classification type.Such representative samples are reviewed by a real clinician to assess whether the valve geometries, predicted leaflet behaviors, and predicted regurgitation grades are clinically realistic.

Table A1 :
Key Patient Model Inputs and Outputs

Table A3 :
(2)n doing so, we also do not present credibility factors for the types of evidence or evaluate the adequacy of such evidence.ASME V&V 40-2018 provides credibility factors for verification, validation, and applicability of traditional credibility evidence collected through bench testing.In practice, credibility factors facilitate the systematic review and documentation of the strengths, limitations, and gaps associated with credibility evidence.Although we do not address credibility factors here, we do summarize limitations and gaps for Evidence categories from Table1of the main document that are most relevant for establishing the credibility of the various ISCT submodels and the full ISCT (gray).Evidence categories that are partially addressed with the hypothetical ISCT of an edge-to-edge mitral valve repair device are highlighted in orange.This table does not imply that all of the listed activities should be performed to establish ISCT credibility.The specific activities that are needed to demonstrate credibility will generally depend on the model risk assessment.†Codeverification includes both software quality assurance (SQA) and numerical code verification (NCV)(2).Dark gray indicates both SQA and NCV are applicable.Light gray indicates only SQA applies.
each credibility example.In future work, a critical exercise for ISCTs will be defining new credibility factors for non-traditional evidence sources.The FDA CM&S Credibility Guidance (1) provides some recommendations and examples for defining such credibility factors.Recent work by Pathmanathan et al. (3), Bischoff et al. (4), and Galappaththige et al. (5) provides additional recommendations and examples.The types of credibility evidence described here are provided as possible examples.Any data or comparisons that are presented are purely hypothetical and do not represent an accepted approach for establishing ISCT credibility.*

Table A4 :
Quantitative comparison of patient data measured clinically and that predicted computationally after final parameter calibration.LV: left ventricle; LA: left atrium; EF: ejection fraction; ED: end-diastolic; ES: end-systolic; TTE: transthoracic echocardiography; CT: computed tomography

Table A5 :
Quantitative comparison of patient data measured clinically and that predicted computationally.LV: left ventricle; LA: left atrium; EF: ejection fraction; TTE: transthoracic echocardiography; Cath: catheter; TEE: transesophageal echocardiography The validation activity relies on post-operative comparator data from a single patient that would likely be inadequate to support model credibility in practice.For a real scenario, the adequacy of the validation evidence could be systematically assessed by adapting ASME V&V 40-2018 validation credibility factors for patient-specific modeling as proposed in Galappaththige et al. (5).For example, ASME V&V 40-2018 credibility sub-factors 'quantity of test samples' and 'range of characteristics of test samples' could be revised to 'number of validation subjects' and 'range and characteristics of validation subjects,' respectively (see Table 5 in (

Table A7 :
Classifications for the virtual patients in the initial virtual patient cohort.