Prompt Products
This article describes how the Roman Space Telescope mission plans to produce prompt data products throughout the lifetime of the mission. It contains information on which products are considered prompt, their expected timelines for accessibility via MAST, and considerations regarding files that are planned to be overwritten.
Introduction
The Roman Space Telescope (Roman) Wide Field Instrument (WFI) data products are classified in different levels, according to their processing and calibration stages (for more information, please see the WFI Data Format and Data Levels and Product articles). In this article, we will focus on the Level 0 (L0; raw, packetized data), Level 1 (L1; uncalibrated ramp cubes), Level 2 (L2; calibrated rate images), Level 3 (L3; co-added, re-pixelated images), and Level 4 (L4; extracted, high-level products, e.g., catalogs and 1-D spectra). As soon as the raw, packetized data are received from the telescope, they are converted to L1 products and data processing begins. Products created immediately after the observations are executed are called prompt products, as opposed to data release (DR) products created as part of Data Releases. In contrast to DR products, prompt products are not versioned in the MAST archive and will be overwritten with each subsequent data reprocessing. Furthermore, selected prompt products are superseded by DR products and will be removed from the archive after a designated period of time. Note that some high-level data product types are not generated as part of prompt data processing. In most cases, L1 and L2 data products are considered prompt products, while L3 and L4 products may be either prompt or DR products. A summary table of the different prompt data products can be seen in the Table Summary of Prompt Data Products.
Prompt Product Creation
L1 Products
Prompt processing occurs as the data arrive at the ground system after the observations are executed. L1 products are created by the Science Data Formatting (SDF) pipeline within 24 hours of receipt of the L0 packets and ancillary data required to generate the products. During nominal operations, L0 products are not expected be routinely reprocessed; however, reprocessing of L1 products may occur as necessary. L1 includes science data products, as well as guide window and fine attitude correction estimate (FACE) data products. Unless the L1 science data are reprocessed from L0 after definitive ephemeris information becomes available, the metadata of the L1 science data products will contain planned ephemeris information. During the processing from L1 to L2, a supplemental file containing refined pointing and definitive ephemeris information (when available) is created and archived with the L1 science data products.
L2 Products
L1 files are processed through romancal to create calibrated L2 rate images. Prompt L2 products will typically be available within 48 hours after all the packets and the ancillary data necessary to create them have been received. The L1 products will be processed using the best available calibration reference files and pipeline version at the time each L2 is created. Therefore, L2 prompt products might not have uniform calibrations nor be of the best possible quality. During DR reprocessing, the Calibration Reference Data System (CRDS) context and romancal pipeline version will be fixed to create uniformly processed DR L2 products (see the Data Releases article for more information). The L2 products created during DR reprocessing will overwrite the existing L2 products. An L2 product will remain available in the archive unless it is overwritten by a subsequent reprocessing of the parent L1 product. For example, a High-Latitude Time-Domain Survey (HLTDS) prompt L2 product in the archive will be overwritten during a HLTDS DR; however, reprocessing for a Galactic Bulge Time-Domain Survey (GBTDS) DR will not overwrite any HLTDS L2 products.
As L2 products are overwritten with each reprocessing, only the most recently reprocessed version is available in the archive. This is in contrast to DR L3 and L4 products, that are versioned and archived for each DR. For example, if there are two DRs for a survey, each release would contain versioned L3 and L4 products, but only a single version of L2 products will be available corresponding to the reprocessing that was done for the second release.
L3 Products
Prompt L3 products for WFI imaging mode data from the HLTDS, High-Latitude Wide-Area Survey (HLWAS), and General Astrophysics Survey (GAS) programs are created at the visit level (see APT Program Elements for more information). For each visit, a rectified, co-added mosaic will be created using multiple prompt L2 products as the inputs, and is referred to as a prompt VISIT L3 product. Prompt VISIT L3 products will be created with romancal and will be resampled using the native pixel scale onto the Skymap Tessellation. Prompt VISIT L3 products will typically be available in the archive no more than 5 days after the execution of the last observation required to make them. Prompt L3 products for the GBTDS will be generated by the Science Support Center (SSC) using the microlensing science pipeline.
L4 Imaging Products
Prompt L4 products for WFI imaging mode data from the HLTDS, HLWAS, and GAS programs are created from both prompt L2 and prompt L3 inputs. These prompt L4 products are created by romancal , and consist of a single-band source catalog and a segmentation map. The prompt L4 products created from prompt L2 inputs are thus associated with individual exposures, while the prompt L4 products created from prompt VISIT L3 products are associated with each visit. Prompt L4 imaging products for HLTDS, HLWAS, and GAS imaging data will typically be available in the archive no more than 7 days after the execution of the last observation required to make them. Prompt L4 products for the GBTDS will be generated by the SSC using the microlensing science pipeline.
Prompt L4 products for the WFI spectroscopic mode for all surveys will be generated by the SSC using the grism and prism data processing pipelines.
More information on the microlensing science pipeline and the grism and prism data processing pipelines will be added in future RDox releases.
Please note that all prompt L4 products will be stored only as files in the archive and will not be added to the MAST database catalog.
Summary Table
Table Summary of Prompt Data Products
| Data Processing Levels1 | Prompt Product Description | Availability on MAST Archive | Notes |
|---|---|---|---|
| Level 0 | Raw, packetized data from the telescope. | Not accessible to general users. | |
| Level 1 | Uncalibrated detector ramps. | Within 24 hours of downlinked data being received at the SOC. | Not planned to be included in regular reprocessing associated with data releases2. If re-created, existing L1 files will be overwritten. |
| Level 2 | Calibrated detector rate images. | Within 48 hours of downlinked data being received at the SOC. | Planned reprocessing during data releases will over-write existing L2 files2. |
| Level 3 | Re-pixelated data including, e.g., co-additions and mosaics from single visits. | Within 5 days of the execution of the last observation required to make this product. | DR L3 products will supersede prompt products. Prompt L3 products will be removed from the archive after a standard period of time (TBD). |
| Level 4 | Information extracted from pixel data including, e.g., source catalogs, segmentation maps, and 1-D spectra. | Within 7 days of the execution of the last observation required to make this product. | L4 prompt products associated to the L2 will be overwritten when the L2s are re-obtained. L4s associated to L3s will remain in the archive until they both are removed after a period of time (TBD). |
1 See Data Levels and Products for more information.
2 The Roman mission anticipates that early data reprocessing may deviate from the planned reprocessing associated with DRs. The Roman Science Operations Center (SOC) will determine if any calibration or data updates are substantial enough to require regenerating L1 and L2 prompt products.
Example science cases for prompt data products
Prompt data products are intended to provide quick access to calibrated Roman observations immediately after they are executed. Some examples of how these products could be used include:
Image differencing of L2 products covering the same region of the sky to investigate source variability.
Examining extended sources in multiple L2 images or near the edges of WFI detectors while awaiting L3 prompt product processing.
Using L3 prompt products for data quality inspection and early science investigations that utilize native sampling.
Cross-referencing L4 prompt product catalogs with known databases and existing catalogs to quickly compare Roman population statistics with existing observations.
For additional questions not answered in this article, please contact the Roman Help Desk.
References
- Ferguson, H. C. et al. "Roman Reprocessing and Data Release Strategy" Nancy Grace Roman Space Telescope Technical Report, Roman-STScI-000650, SE-01, September, 2024.