Including Calorimeter Test Beams in Geant-val—The Physics Validation Testing Suite of Geant4

Abstract

The Geant4 simulation toolkit is currently adopted by many particle physics experiments, including those at the Large Hadron Collider and the ones proposed for future lepton and hadron colliders. In the present era of precision tests for the Standard Model and increasingly detailed detectors proposed for the future colliders scenario, Geant4 plays a key role. It is required to remain a reliable and stable toolkit for detector simulations and at the same time undergo major improvements in both physics accuracy and computational performance. Calorimeter beam tests involve various particles at different energy scales and represent ideal benchmarks for the physics modeling and assessment of Monte Carlo tools for radiation–matter simulation. We present the first results of a broad validation campaign on test beam data targeting data deployment and preservation with geant-val, the Geant4 validation and testing suite. We investigate the Geant4 capability to model the calorimeter response, energy fluctuations, and shower shapes using data from the ATLAS hadronic end-cap calorimeter and the CALICE silicon-tungsten calorimeter. The evolution over the recent years of the recommended set of physics processes for high-energy physics applications is outlined and compared to alternative models for hadronic interactions.

1. The Geant4 Toolkit

The Geant4 simulation toolkit [1,2,3] is a general purpose Monte Carlo (MC) code for radiation–matter interaction simulation. It consists of nearly two million lines of code written in object-oriented C++ that have been developed over three decades by an international collaboration of physicists, computer scientists, mathematicians, and engineers. It currently supports both high-energy and low-energy particle physics experiments, neutrino experiments, and detector design studies for post-Large-Hadron-Collider (LHC) experiments as well as medical, space, and atmosphere applications.

One of the key goals of Geant4 is to provide the main LHC experiments with a reliable and stable MC tool to simulate the response of complex detectors to the passage of the large variety of particles produced during beam collisions. The most challenging part of these simulations, both in terms of simulation speed and physics accuracy, comes from the particles multiplication mechanisms responsible for the creation of showers in calorimeters. From the computing budget point of view, it requires tracking thousands of particles within each shower, with the actual number heavily depending on the production cuts applied. The MC simulation currently accounts for the largest contribution to the computing time of big experiments, with this contribution being dominated by the simulation of the calorimetric component. For instance, it amounted to $38\%$ of the entire computing time in the case of the ATLAS experiment in 2018 [4]. From the physics modeling point of view, showers in calorimeters involve several particle types, processes, and energy scales.

Hadronic showers are a good example of the complexity involved. Their first stage is governed by the occurrence of the first inelastic nuclear interaction of the primary projectile, while most of the calorimeter signal is carried by relatively low-energy charged particles arising from nuclear breakups. These particles are a mixture of purely electromagnetic and hadronic interacting particles. Therefore, for a reliable description of both the shower shape fluctuations and the calorimeter response, the MC engine must provide excellent representations of hadronic processes happening at high-energy scales as well as electromagnetic and hadronic low-energy ones.

A smart solution is offered by Geant4 via the Physics Lists (PLs). PLs are easy-to-use descriptions of consistent sets of particles and processes to be simulated. Usually, the more physically accurate a PL is, the larger the computing time needed for a given event to be simulated. Each PL represents a meeting point between physical accuracy and computational cost; it is the responsibility of the user to pick the most suitable one for their application. Within Geant4, each user is allowed to create their own PL; however, it is worth noting that the four largest LHC experiments, for their Run2 simulations (2015–2018), recently adopted the Geant4-recommended PL for the high-energy-physics application, FTFP_BERT, eventually with mild variants. The same choice is foreseen for the upcoming LHC Run3 simulations and will likely apply to the High-Luminosity-LHC simulations as well.

The Standard Model tests envisaged for future LHC runs will require outstanding descriptions of all the physics processes involved in calorimeters in order to limit the systematic errors driven by simulation as much as possible. At the same time, calorimeters envisaged for future colliders will improve the current shower descriptions both in terms of energy resolution and sampling granularity. The prototypes under construction offer a unique chance for superior Geant4 validation, which will open the possibility to provide a realistic description of complex conceptual detector designs, thus helping to save money and time. The Geant4 Collaboration recently started a validation campaign on calorimeter beam tests in close collaboration with ATLAS and CALICE Calorimetry Groups; extensions to other groups are under investigation. Each validation study targets its inclusion into geant-val, the Geant4 validation and testing suite, which is outlined in Section 2. The main Geant4 validation results related to the ATLAS hadronic end-cap calorimeter (HEC) and the CALICE silicon-tungsten (SiW) calorimeter are described, together with the detector features, in Section 3 and Section 4, respectively. Conclusions are drawn in Section 5.

2. The Geant-val Project

Geant4 validation on beam tests is usually performed via regression testing and PL comparison. Regression testing consists of running the same simulation with different software versions while comparing the results with experimental data, thus finding indications of any temporal evolution of a single PL. PL comparison, on the other hand, exploits a fixed software version and compares the results for different PLs, thus providing indications of which model is more accurate with respect to experimental data. On top of that, the user might want to investigate the dependence of results on other parameters, e.g., the production cuts or the signal integration time. Large validation campaigns typically require the same MC data production and analysis to be performed over tens or hundreds of different combinations of beam particle type, beam particle energy, detector description, and physics list. They stand among the most time-consuming tasks that the Geant4 Collaboration undertakes. Data preservation and deployment to the entire Geant4 Community is another major task, as each validation test should be updatable and distributable. To facilitate these tasks, the geant-val team developed a validation and testing suite [5] to support both the validator and the end-user. For the benefit of the end-user, geant-val offers a web interface (https://geant-val.cern.ch/, (accessed on 29 August 2022)) that makes it possible to fetch data in the form of static images for every PL and software version desired. For the benefit of the validator, geant-val offers a Python tool, the mc-config-generator, to encapsulate simulation job metadata (software version, compiler, physics list, primary particles, etc.) in the form of a JSON file. Geant-val also provides a uniform way of preparing and running the jobs in parallel on common batch systems, thus providing a consistent way of executing all the combinations at once. Currently, the geant-val database hosts results of about forty validation tests from different Geant4 domains. In addition to data visual inspection, geant-val performs ${\chi }^2$ and Kolmogorov–Smirnov statistical tests for results comparison.

3. Geant4 Validation on ATLAS HEC Test Beam Data

The ATLAS HEC [6] is a sampling calorimeter that exploits liquid argon (LAr) gaps interspersed between parallel copper plates. Within the ATLAS Detector, it is devoted to (almost) fully absorb hadrons in the pseudorapidity range of $1.5<\left|\eta \right|<3.2$. The HEC adopts a wedge-shaped design with 32 azimuthal modules replicated around the beam axis; longitudinally, it is divided into two main wheels (HEC1 and HEC2). The absorber layers are 2.5 and 5.0 cm thick, respectively, for the HEC1 and the HEC2. Annular spacers are used to define an 8.5 mm-thick region for LAr gaps. The total thickness amounts to ≃9.7 ${\lambda }_{int}$ (≃103 $X_0$). Each wedge module is read out by 88 channels, with the readout scheme being optimized in order to easily reveal the $\eta$ coordinate of the impinging particles in the ATLAS experiment. The transverse size of the HEC readout cells is $\Delta \eta \times \Delta \varphi =0.1\times 2\pi /64$ in the region $\left|\eta \right|<2.5$, and $0.2\times 2\pi /32$ for larger values of pseudorapidity. Longitudinally, modules are readout by 4 layers. Production modules were exposed to the CERN-SPS particle beam during the construction stage in 2000 and 2001. Secondary and tertiary beams from the H6 (beam) line were used to steer electrons, muons, and hadrons in the energy range of $6\le E_{Beam}\le 200$ GeV on the front face of three $\varphi$-modules positioned inside a cryostat; see Figure 1 (left). For several years, related results [7,8] have represented a test bed for the ATLAS simulation framework [9,10,11]. In 2022, a summary of selected test-beam results was made public by the ATLAS Liquid Argon HEC Collaboration [12]. In the following, experimental results are extracted from the 2022 reference where details of the test-beam setup as well as of the treatment of the uncertainties are given.

Recently, in a close collaboration between the ATLAS HEC Group and the Geant4 Hadronic Working Group, the test beam simulation was refactored in a standalone Geant4-based code and included in geant-val for the benefit of data preservation and distribution. The outcome of this activity represents a valuable test for hadronic interaction models and will be exploited in future Geant4 validation studies and parameter tuning. Figure 1 (right) shows the simulated test beam geometry. Simulated results in the following are obtained with the Geant4 standalone simulation.

Monte Carlo-to-Data Comparison

The Geant4 standalone simulation takes into account realistic materials and geometry descriptions of the calorimeter modules and the cryostat. For the sake of Geant4 validation, the impact of the beam-line auxiliary detectors is considered marginal and not reproduced in the simulation.

The seed for detector response simulation is provided by Geant4 in the form of ionizing energy deposition for every charged particle hit in the LAr gaps. The HEC signal is induced by the free electric charged particles on capacitively coupled copper boards that are immersed in between the LAr gaps. To model the ion recombination mechanism, an attenuation law for the ionizing energy deposition was included in the simulation, with the actual attenuation dependence on specific energy losses being parameterized with the same functional form used by the Birks Law to describe light emission in organic scintillators.

The gauge for energy reconstruction is offered by the sampling fraction estimated with $e^{-}$ beams. According to Geant4 simulations, it amounts to $4.5\%$ in the first wheel (HEC1) and is constant within $0.1\%$ in the $20\le E_{Beam}\le 150$ GeV energy range. Due to the different thickness of copper plates in the two wheels, the sampling fraction was divided by a factor of 2 when the energy depositions in the second wheel (HEC2) were calibrated. A signal integration time of 75 ns was also considered in the simulation, with the actual readout cut adjusted for the four longitudinal layers mimicking the test beam readout electronics performance. Signal integration over calorimetric cells depends on the nature of the impinging particle. Electron energies are reconstructed from the cumulated signal over the seven cells with the highest average signals; cell selection does not depend on the $e^{-}$ energy and is kept identical for every event. Pion energies are reconstructed from the cumulated signals in the cells with a visible energy deposition greater than 2.1 MeV (corresponding to an integrated charge of 15 nA according to the simulation of the readout chain). This procedure leads to a selection of ≃50 cells estimated with 180 GeV ${\pi }^{-}$ events; this cell selection is kept fixed for every event regardless of the energy scale.

No sources of systematic uncertainties, as for instance the ones arising from a non-pure beam composition, are included in the Monte Carlo. The stochastic uncertainties for MC results are within $0.1\%$ of the corresponding value. The $\pi /e$ ratio, i.e., the ratio of the response to the charged pions and electrons, is directly estimated as the ratio of the average ${\pi }^{-}$ reconstructed energy calibrated at the electromagnetic scale, divided by the beam energy. Figure 2 (left) shows the Geant4 FTFP_BERT PL prediction for recent releases (2017–2020) and compares it to experimental data. We observe a systematic increase over the years in the $\pi /e$ value as described by Geant4. The best Monte Carlo-to-data agreement is offered by the 10.4.p01 version, while the 10.7.p01 one predicts hadronic response values that are $2\%$ higher than the experimental measurements. This consideration is valid for the energy range of $20\le E_{Beam}\le 120$ GeV, while at 150 GeV, the Monte Carlo estimation lies within the experimental uncertainties for every release.

At the time of writing, Geant4 10.7.p01 is the latest tested version. Figure 2 (right) shows the prediction for this software release, with four different PLs: FTFP_BERT, FTFP_BERT_ATL, QGSP_BERT, and FTFP_INCLXX. The FTFP_BERT_ATL PL adopts the Bertini intra-nuclear cascade model [14] and the Fritiof string model [15], with the two models overlapping in the range of 9–12 GeV. This is the only difference in comparison to FTFP_BERT, for which the overlap region is 3–6 GeV. This larger use of the Bertini model is responsible for the lowering of the calorimeter response to hadrons, and it leads to a better agreement with data. Currently, the ATLAS experiment adopts the FTFP_BERT_ATL PL. The INCL intra-nuclear cascade model is used in the (experimental) FTFP_INCLXX PL, which uses an overlapping range of 15–20 GeV for the transition with the Fritiof model; the overall prediction is 5–6% higher than the experimental reference, almost independently of the energy scale. The QGSP_BERT PL corresponds to the FTFP_BERT at low energies and introduces the QGSP string model within the overlapping range of 12–25 GeV, resulting in a hadronic response about 3–4% higher than the experimental data.

Important information also comes from the hadronic response fluctuations. They are measured as the $\sigma /E$ value, with $\sigma$ and E having been extracted from a Gaussian fit to the ${\pi }^{-}$ energy distributions calibrated at the electromagnetic scale. Figure 3 (left) illustrates the FTFP_BERT estimation for four recent releases and compares them to the experimental reference. We observe a reduction of the hadronic response fluctuations when switching from Geant4 10.4.p01 to 10.5.p01. This discrepancy amounts to ≃20% regardless of the energy scale. Such a result is a great example of the importance of regular Geant4 validation on experimental data through realistic simulations. Foreseeing such a big change through code examination would be impossible, while simplified simulation tests would spot the difference without indicating whether it had improved the Monte Carlo-to-data agreement. Figure 3 (right) shows the 10.7.p01 comparison of the previously described PLs, indicating that the minimal fluctuations of energy response correspond to those of the FTFP_INCLXX PL.

The path of calorimetry to future lepton colliders is leading to detectors with higher granularity. Such detectors will allow for detailed shower shape reconstruction to be used for particle identification purposes in combination with the tracking information. It is therefore of paramount importance for Geant4 to provide an accurate description of shower shapes. The ATLAS HEC is a good benchmark for longitudinal shower shapes in copper-based sampling calorimeters. The shower profile is extracted from the fraction of the measured energy deposited in each layer, $F_i=E_i/E_{sum}$, with $E_{sum}$ being the total measured energy, while $E_i$ is the energy measured in the layer i. The mean of the profile ($L_0$) is a direct measurement of the shower barycenter longitudinal position. The $L_0$ evolution with the beam energy ($E_{Beam}$) is shown in Figure 4. Figure 4 (left) shows the FTFP_BERT evolution for the $L_0$ values in the energy range of $20\le E_{Beam}\le 200$ GeV. It indicates the constant shortening of the barycenter longitudinal position from 2017 to 2020; currently the barycenter position can be constrained with a sub-percent precision for every energy point. Figure 4 (right) compares the same variable for several PLs of the 10.7.p01 Geant4 release, with the longitudinal barycenter position being 5% higher for the QGSP_BERT description with respect to the other PLs and the experimental data.

Another key aspect is the Geant4 capability to reconstruct the hadronic shower length. An indirect measurement comes from the RMS (${\sigma }_L$) of the longitudinal energy profile introduced above (the longer the shower, the higher the RMS value). Figure 5 (left) shows the FTFP_BERT evolution of the ${\sigma }_L$ measurement and compares it to the experimental reference in the energy range of $20\le E_{Beam}\le 200$ GeV. The FTFP_BERT PL recently evolved towards shorter ${\pi }^{-}$ showers in the copper-based calorimeter, finding a recent Monte Carlo-to-data agreement of ≃2%. A similar agreement for the 10.7.p01 version is provided by the FTFP_BERT_ATL and the FTFP_INCLXX PLs, while according to the QGSP_BERT PL, the hadronic showers ${\sigma }_L$ is on average $\simeq 2\%$ higher, see Figure 5 (right).

4. Geant4 Validation on CALICE SiW Test Beam Data

The CALICE SiW detector prototype [16] is a sampling calorimeter made of alternating layers of silicon and tungsten. Each of the 30 silicon layers has an active area of $18\times 18$ cm${}^2$ segmented into a $3\times 3$ matrix of Si-wafers. Each wafer consists of $6\times 6$ pixels, for a total of 9720 active elements. The pixel dimension is $1\times 1$ cm${}^2$. The first ten Si-layers (1–10) are interspersed between 1.4 mm-thick W-slabs. The absorber layer thickness changes to $2.8$ mm for the following ten layers (11–20) and to 4.2 mm for the remaining layers (21–30). The effective length amounts to 24 $X_0$ (≃1 ${\lambda }_{int}$), therefore more than half of the hadrons traversing it would undergo a nuclear interaction. Figure 6 (left) is a schematic reconstruction of the prototype. The prototype was tested at the Fermilab Test Beam Facility in 2008. Runs with ${\pi }^{-}$ mesons in the energy range of 2–10 GeV were used to study the properties of the first stage of hadronic showers in a W-based calorimeter. The results were published in 2015 [17]. In 2022, the Geant4 Collaboration developed a Geant4-standalone simulation code, with the aim of ensuring regular Geant4 validation on these data as well as deployment to geant-val. Figure 6 (right) shows a simulated ${\pi }^{-}$ interacting with the SiW prototype via a hadronic inelastic process. For clarity, only the first ten layers are displayed. Hits are marked with yellow dots, which correspond to tracks interacting in the SiW calorimeter and in the hadronic calorimeter placed downstream. The hadronic calorimeter is not shown, and no information from the latter is used in the analysis. The subsequent experimental results come from [17]. Beam purity studies, contamination removal, and data corrections together with the systematic and stochastic error treatment are discussed at length in [17] and thus are not repeated here. The following Monte Carlo results were obtained with the Geant4-standalone simulation. They correspond to pure ${\pi }^{-}$ beams. No systematic error is considered in the simulation, and stochastic uncertainties are within $1\%$ of the corresponding value.

Monte Carlo-to-Data Comparison

The calorimeter signals are calibrated with ${\mu }^{-}$ beams whose energy loss in an active Si-pixel defines the energy unit MIP. In the following, an energy threshold for a pixel selection of 0.6 MIP is considered, and only events with at least 25 fired pixels are retained. Pion events undergoing a nuclear inelastic reaction are tagged according to two selecting cuts:

• They correspond to events that have three consecutive layers with a measured energy ($E_i$) greater than 8 MIP. The first of the consecutive layers is considered as the one closer to the point where the nuclear breakup occurred.
• Alternatively, they are selected as the events with a relative increase in the layer energy above a certain threshold $F_{cut}$:

  $$\frac{E_i+E_{i+1}}{E_{i-1}+E_{i-2}}>F_{cut}\mathrm{and}\frac{E_{i+1}+E_{i+2}}{E_{i-1}+E_{i-2}}>F_{cut}$$

  (1)

  with $F_{cut}=6$.

To reduce the $e^{-}$ contamination in ${\pi }^{-}$ beams, events with an interacting layer number $\le 7$ are neglected. Figure 7 shows the longitudinal energy profiles in MIP units for ${\pi }^{-}$ showers in the energy range of 2–10 GeV. The first layer corresponds to the identified interaction layer, so the x-axis represents the shower depth in layers. As most of the hadronic shower extends beyond the detector, the average value in a given bin is determined by considering only the events that contribute energy to the corresponding layer. To take into account the sampling fraction decrease with the layer number, pseudolayers are introduced. As explained in [17], the first ten layers correspond to the first ten pseudolayers, while each layer with a number within the ranges of 11–20 and 21–30 is assigned to two and three pseudolayers, respectively. Experimental data are compared to the Geant4 FTFP_BERT prediction for releases from 2017 to 2020. We observed a constant improvement in the Monte Carlo-to-data agreement over the years in the energy range of 6–10 GeV. For 10 GeV ${\pi }^{-}$ events simulated with the 10.7.p03 PL, a residual tension between data and simulation has been observed to affect the longitudinal shower maximum; see Figure 7 (bottom-left). The same is not true for the description provided by the (experimental) FTFP_INCLXX PL; see Figure 7 (bottom-right).

Another important source of information comes from the longitudinal hit distribution for ${\pi }^{-}$ showers. Hits are individual pixels retained after the cleaning cuts. As stated before, the average value in a given bin is determined by considering only events which contribute in the corresponding layer and the first layer corresponds to the identified interaction layer. To compare longitudinal distribution shapes, they have been normalized to unity; see Figure 8. Experimental data are compared to the FTFP_BERT prediction for Geant4 releases from 2017 to 2020. Simulated results are stable over the years and indicate a better capability to reproduce longitudinal hit distributions in the 6–10 GeV energy scale with respect to the 2–4 GeV one. It corresponds to a higher precision in the hit distribution description for the Fritiof string model as compared to the Bertini intra-nuclear cascade one. Similar results are obtained with the FTFP_INCLXX PL and are available on the geant-val website.

5. Conclusions

The Geant4 simulation toolkit is widely adopted by nuclear and particle physics experiments for calorimetric simulation. The success of the Geant4 project stands on the continuous validation of its physics models as well as on the R&D targeting faster and better computing solutions. In the last years, most of the hadronic model development has been based on thin target experiments, with typical absorber dimensions of a few millimeters. This allowed for the precise modeling of nuclear interactions at fixed energy and projectile type. The possibility to achieve wider, more general validation is offered by test beam results on calorimeters, in which several processes and different particle types with wide energy ranges are involved. Such validation tests require strong collaborations between Geant4 developers and experimental groups in order to design accurate and realistic simulations. In this respect, both calorimeters from the past and prototypes designed for future experiments are equally suitable.

We showed an example from the first case, the ATLAS HEC beam test, and the latter case, the CALICE SiW calorimeter beam test. The HEC results clearly indicate a trend in the simulated response to pions in copper-based calorimeters towards higher values. They also provide a good example of the solution adopted by the ATLAS experiment, i.e., the FTFP_BERT_ATL PL, in which the Monte Carlo-to-data agreement in the calorimeter response simulation is largely improved with respect to the FTFP_BERT PL. Results from the HEC also show the great capability of Geant4 to reproduce the correct hadronic shower shape in terms of shower length and barycenter location in calorimeters with the typical segmentation of a few longitudinal layers as the ones currently adopted by the main LHC experiments.

The CALICE SiW calorimeter stands among the prototypes with the highest granularity envisaged for the post-LHC era. When tested with ${\pi }^{-}$ showers, it provides extremely valuable information on the first stage of the hadronic shower development. Its results on longitudinal distributions have been compared with recent versions of the FTFP_BERT PL, observing a good improvement in the MC-to-data comparison for the visible energy depositions following a nuclear breakup in the 2–10 GeV energy range. It also shows how the INCL model provides a good alternative to the FTFP_BERT PL for the highest energy considered in that test beam.

We strongly believe the validation of Geant4 to be a collective effort shared among diverse experiments, and the most valuable drivers of physics model changes to be the ones arising from different sources of inputs. In this respect, the geant-val project represents the best effort to preserve experimental inputs, compare simulated results, and distribute the information to the broadest community possible. On top of that, geant-val offers solutions for standardized job preparation and submission on batch systems that help the Geant4 validators to perform extremely time- and CPU-consuming validation campaigns.