CRDS for Reference Files
The Calibration Reference Data System (CRDS) facilitates access to calibration reference and parameter files for use in the Roman STScI Data Pipelines. The Roman exposure-level pipeline utilizes the reference and parameter files stored in CRDS to process the Level 1 uncalibrated data into Level 2 calibrated rate images. In addition, simulation tools such as Roman I-Sim make use of reference files in CRDS.
Selecting Reference Files from CRDS
Most Level 1 (L1) to Level 2 (L2) data processing steps rely on calibration reference files that contain information necessary to remove instrumental effects. The Wide Field Instrument (WFI) reference files will be improved over time as additional data are acquired. The reference files for the WFI imaging mode are created, tested, and validated by the WFI Reference File Pipeline developed by the Science Operation Center (SOC) at STScI, and are available via the Roman Calibration Reference Data System (CRDS).
There are specific keywords used to match the data being processed in the exposure-level pipeline to the appropriate and most up-to-date reference file to be used in a particular calibration step. The USEAFTER keyword is utilized for all reference files in CRDS and represents the date after which the reference file is applicable, with CRDS matching science data to the nearest preceding USEAFTER date. For instance, if the observation date of the data being processed is 2027-01-01, and there are dark reference files with USEAFTER dates of 2026-12-01 and 2027-01-07, the first file would be selected. Beyond the USEAFTER date, there are additional matching criteria for each reference file, which are listed in the table below.
Additional information on the format and contents of each reference file may be found in the romancal readthedocs documentation on calibration reference files.
Table of Reference Files on CRDS
Reference File Type | SOC Use | CRDS Matching Criteria (in addition to WFI detector) | Pipeline Steps |
---|---|---|---|
DARK | romancal Roman I-Sim | INSTRUMENT.DETECTOR EXPOSURE.TYPE EXPOSURE.MA_TABLE_NUMBER | dark current subtraction |
DISTORTION | romancal Roman I-Sim | INSTRUMENT.OPTICAL_ELEMENT | assign wcs |
FLAT | romancal Roman I-Sim | INSTRUMENT.OPTICAL_ELEMENT | flat field correction |
GAIN | romancal Roman I-Sim | jump detection, ramp fitting | |
INVERSE LINEARITY |
Roman I-Sim
| ||
INTER-PIXEL CAPACITANCE | Roman I-Sim | INSTRUMENT.OPTICAL_ELEMENT | |
LINEARITY | romancal Roman I-Sim | linearity correction | |
MASK | romancal Roman I-Sim | data quality initialization | |
READ NOISE | romancal Roman I-Sim | EXPOSURE.TYPE | jump detection, ramp fitting |
REFERENCE PIXEL | romancal Roman I-Sim | reference pixel correction | |
SATURATION | romancal Roman I-Sim | saturation detection | |
AREA | |||
PHOTOM | romancal Roman I-Sim | photometric calibration |
The Area reference file is intended for use by Roman users conducting analysis on images in the distorted frame.
CRDS in Roman Pipelines
The exposure-level pipeline, which calibrates data from L1 to L2, automatically retrieves the appropriate reference files from the Roman CRDS operations server. The versions of the reference files used by the exposure-level pipeline are dictated by the CRDS context file, also known as a "pipeline mapping" (PMAP file). This context file lists for each reference file type the "reference mapping" (RMAP) file to be used. The RMAP files contain the mapping of a specific reference file to the selection criteria. The name of the context file used in calibrated WFI data products is also stored in the ASDF metadata. The names of the reference files used to process the data are stored in a dictionary in the metadata of the L2 Advanced Scientific Data Format (ASDF) files and are also recorded in logs.
Additional information for utilizing this system can be found in the CRDS documentation. Command line tools and python packages allow users to examine the reference files and explore the data of a specific reference type.
For additional questions not answered in this article, please contact the Roman Help Desk at STScI.