Simulation framework

In the Simulation Models Work Package (WP4), the main objective is to establish a unified framework that integrates the efforts from WP1, WP2, and WP3. This unified framework aims to create a standardized simulation tool for Euclid's spectroscopic datasets.

Challenges in Simulating Euclid NISP data

The Simulation Models Work Package (WP4) faces several challenges in its endeavor to create a unified and standardized simulation tool for Euclid's spectroscopic datasets. One significant challenge lies in establishing a common formalism that seamlessly integrates diverse existing models from different sources, including those within the Euclid consortium and new models developed within the project. Ensuring compatibility and coherence among these varied models is a complex task, demanding meticulous attention to detail and comprehensive understanding of each model's intricacies.

Moreover, ensuring the coexistence of multiple models within the simulation framework poses a challenge. This coexistence is essential for generating diverse and realistic datasets. Managing the interactions and dependencies among these models while maintaining the integrity of the simulations demands sophisticated coordination and meticulous validation processes.

Finally, balancing computational efficiency with accuracy in reproducing instrument response and source features is crucial, making the selection of appropriate machine learning techniques a critical decision.

Our Approach

This work package will first provide a common model framework to ensure the convergence of the three other work packages to coherent a simulation tool. Beyond the simulation tool the goal of this work package is to provide a common framework for the WP1, WP2, and WP3 to facilitate the model implementation and its diffusion to the community.

The first step of this work will be to agree on a common formalism for the definition of the different models and that would allow both to express the models developed in the framework of this project, and the models currently used in the Euclid consortium. Based on this formalism, we will develop a common framework that will share the implementation of the different operations related to the formalism, the interfaces between the models, as well as the management of the inputs / outputs, while leaving the implementation of the models free. We also will develop common tools to control the errors of the models and compute standardized output products that facilitate model comparison and validation.

A prototype was already implemented in python, in the framework of the ground test campaign and the processing of the preliminary model of the spectral dispersion for simulation applications. The interface is presented in the form of singleton with non-mutable attribute to control its usages. In this preliminary version, the code allows you to load the optical model and to use several validation functions provided to compare the model implementation with the data.

Another important aspect of the architecture we are developing is to allow the coexistence of several models in the goal to propose an original way to generate large simulation of the spectroscopic datasets, with special emphasis on slit-less spectroscopy, by splitting the source sample in several sub-sample, target sources, bright non-target sources and the faint non-target sources which have very different constrains in term of model accuracy and significance of certain features of the models. The target sources will be generated using the most accurate, and the most time-consuming models developed within the work package above to ensure the reliability and the traceability of the physical features we are interested in. The faint non-target sources that may represent up to 90% of the total simulated sample behave like a structured background and could be generated to be statistically representative with a generative adversarial network (GAN) or other ML generative methods that would be chosen for their computing efficiency and their accuracy in reproducing instrument response and the sources features. The bright non-target sources may need a different treatment because their effect is local unlike the faint non-target sources and unlike to target sources, their high flux may make significant some negligible effects like ghost, stray-light and persistence.