Data Levels and Products
Wide Field Instrument (WFI) science data products are described by data levels that indicate the level of calibration and types of products. This article offers a comprehensive overview the composition of WFI data products across each data level.
Overview of Data Levels
Roman WFI science data products will be available to users via the Roman Archive (see Accessing WFI Data for more information). Science data products from the Roman WFI instrument are stored in Advanced Scientific Data Format (ASDF). The WFI files are described by five data levels 1 – 5 (often abbreviated L1, L2, etc.). Note that a sixth level (L0) refers to the raw, packetized data received from the WFI instrument; however, L0 data are mission internal and not publicly accessible. Level 1 – 4 products are generated by the Roman science centers, while L5 products are contributed by the community.
Changes to the technical details and other specifications presented here are anticipated as part of the development of the Roman data management system. In addition, some details may be omitted while topics are in active development; information on these topics will be added in future RDox releases.
Table with High-Level Summary of WFI Data Products
Data Processing Levels1 | File Suffix | Description |
---|---|---|
Level 0 | TBD | Raw, packetized data from the telescope. |
Level 1 |
| Uncalibrated detector ramps. |
Level 2 |
| Calibrated detector rate images. |
Level 3 | TBD | Re-pixellated data including, e.g., co-additions and mosaics. |
Level 4 | TBD | Information extracted from pixel data including, e.g., source catalogs, 1-D spectra, and light curves. |
Level 5 | TBD |
1 All L1 – 5 products are accessible to Roman Archive users, but L0 data are restricted. See Accessing WFI Data for more information.
WFI File Naming Conventions
WFI file names consist of a root name and a suffix that denotes the data product type (see Overview of Data Levels above). Root names are a combination of several types of information (e.g., observing program and instrument information), and the letter "r" is always prefixed to WFI file root names to indicate that the data products are from Roman. Note that higher data levels (L3 and L4) as described above may contain one or more data products with the same root name but differing suffixes.
L1 and L2 File Names
Both L1 and L2 WFI data products share common root names with the differing suffixes _uncal
(L1) and _cal
(L2). The root names of L1 and L2 files are also sometimes called the observation identifier (or "Observation ID") and consist of the following components (file metadata keywords for each component are shown in parentheses):
Table of L1 and L2 Root Name Components
Component | Format | Elements |
---|---|---|
Visit Identifier (visit_id) | PPPPPCCAAASSSOOOVVV |
|
Visit File Statement (visit_file_statement) | ggsaa |
|
Exposure Identifier (exposure_id) | eeee |
|
1The value of the sequence identifier "s
" will be 1 if the WFI is the prime instrument or > 1 if the WFI is operated in parallel with the Coronagraph.
The root name components are separate by an underscore such that the final root name is of the format 'PPPPPCCAAASSSOOOVVV_ggsaa_eeee'
. Using the examples in the table above, the resulting root name of an L1 and L2 observation would be 'r0012301008002013005_01101_0005
'. For L1 and L2 files, the root name is followed by the WFI detector number in the format WFINN, where NN is a zero-padded integer between 01 and 18 (e.g., WFI06
). Finally, the detector number is followed by the data product suffix and file extension. As an example, an L2 data product may have a complete file name like '
r0012301008002013005_01101_0005_WFI06_cal'
, followed by the file extension.
Guide Window File Names
Guide window files contain the same root name components as the L1 and L2 files described above, with an additional component that denotes the guide star acquisition number. A guide window file root name may be represented as 'PPPPPCCAAASSSOOOVVV_ggsaa_eeee_Q'
, where 'Q'
is the guide star acquisition number and can have values in the range of 1–9 (inclusive). Using the previous L2 file name example, a complete guide window file name may be 'r0012301008002013005_01101_0005_1_WFI06_uncal'
followed by the file extension. Note that the guide window files are always archived as L1 data products, thus the suffix will always be _uncal
.
Detailed Descriptions of WFI Data Products
In addition to the descriptions below, the schema detailing the contents of the WFI science files may be found in the Roman Attribute Dictionary (RAD) repository on GitHub.
Level 1 - Uncalibrated Data
Science Ramps
WFI L1 files are reconstructed from the packetized L0 data. During this reconstruction, the data are reoriented from the detector to the science coordinate frame (see Coordinate Systems article for more information on the WFI coordinate frames), and basic metadata are populated. The L1 file contains the science data as a three-dimensional data cube that represents a single, uncalibrated ramp exposure. Each detector is mapped to an individual L1 file; therefore an exposure of the WFI instrument produces 18 L1 files (one per detector). The science data cube in the L1 file is shaped as an array with (N resultants, 4096 rows, 4096 columns). The number of resultants (N) along the z-axis of the cube is dictated by the settings of the multi-accumulation table used for the exposure. The L1 file also contains an additional data cube shaped as an array of (N resultants, 4096 rows, 128 columns) corresponding to samples of the 33rd amplifier virtual reference pixels (see Description of WFI for more information).
Table of L1 Science Data Specifications
Array | Description | Units | Type | Dimensions |
---|---|---|---|---|
data | Science data, including the border reference pixels. | Digital Number (DN) | uint16 | (N resultants, 4096 rows, 4096 columns) |
amp33 | Amp 33 reference pixel data. | DN | uint16 | (N resultants, 4096 rows, 128 columns) |
Guide Window Data
Guide window data, composed of high-frequency readouts of small windows on the WFI detectors, are used onboard for target acquisition and fine attitude control adjustments during exposures. The guide window data are stored as L1 files separate, and are not processed to higher data levels.
Table of L1 Guide Window Specifications
Array | Description | Units | Type | Dimensions |
---|---|---|---|---|
signal_frames | Reconstituted and oriented signal frames. | DN | uint16 | (I frames, J combined resultants, K reads, Y rows, X columns) |
pedestal_frames | Reconstituted and oriented pedestal frame GW images. | DN | uint16 | (I frames, J combined resultants, K reads, Y rows, X columns) |
amp33 | Amp 33 reference pixel data. | DN | uint16 | (I frames, J combined resultants, K reads, Y rows, X columns) |
Level 2 - Calibrated Exposures
The L2 WFI data are calibrated rate images in instrumental units of DN per second. L2 files are generated from L1 inputs using the Exposure Pipeline in romancal , which applies detector-level calibrations (e.g., bad pixel masking, classic non-linearity correction, dark current subtraction), aligns the image to Gaia astrometry, and populates additional metadata such as conversion information to physical surface brightness units (see Roman STScI Data Pipelines for more information). However, note that WFI L2 spectroscopic data are not flat-fielded, nor their metadata contain photometric calibration information. A wavelength-dependent flat field and flux calibration are applied during the 1-D spectral extraction and calibration in the Science Support Center (SSC) spectroscopic pipeline. As with L1 science data files, the L2 files contain one detector per file, therefore a complete WFI exposure consists of 18 L2 files.
In the L2 files, the science data arrays have dimensions of (4088 rows, 4088 columns) after the removal of the 4-pixel-wide border of reference pixels present in L1 files (refer to the Description of WFI for additional information on reference pixels). During processing, the reference pixel information from the L1 files is copied into new arrays in the L2 files. This data facilitates user access to the reference pixel values used to compute the reference pixel correction applied by the Exposure Level Pipeline.
Data quality information is added to the L2 products by the Exposure Level Pipeline. The data quality array results from the bitwise addition of data quality bit flags. In this context, the value of a pixel is the unique sum of powers of two, where each power corresponds to a specific data quality flag. Further details on data quality flags will be provided in future updates.
Table of L2 Science Data Specifications
Array | Description | Units | Type | Dimensions |
---|---|---|---|---|
data | Science data, excluding border reference pixels. | DN / sec | float32 | (4088 rows, 4088 columns) |
err | The total error array. | DN / sec | float32 | (4088 rows, 4088 columns) |
dq | The data quality array. | N/A | unit32 | (4088 rows, 4088 columns) |
var_flat | The variance array associated with the flat field. | DN2 / sec2 | float32 | (4088 rows, 4088 columns) |
var_poisson | The variance array associated with the Poisson noise. | DN2 / sec2 | float32 | (4088 rows, 4088 columns) |
var_rnoise | The variance array associated with the read noise. | DN2 / sec2 | float32 | (4088 rows, 4088 columns) |
amp33 | Amp 33 reference pixel data. | DN | uint16 | (4096 rows, 128 columns) |
border_ref_pix_left | Original border reference pixels (left). | DN | uint16 | (4096 rows, 4 columns) |
border_ref_pix_right | Original border reference pixels (right). | DN | uint16 | (4096 rows, 4 columns) |
border_ref_pix_top | Original border reference pixels, (top). | DN | uint16 | (4 rows, 4096 columns) |
border_ref_pix_bottom | Original border reference pixels, (bottom). | DN | uint16 | (4 rows, 4096 columns) |
dq_border_ref_pix_left | Data quality for border reference pixels (left). | N/A | uint32 | (N resultants, 4096 rows, 4 columns) |
dq_border_ref_pix_right | Data quality for border reference pixels (right). | N/A | uint32 | (N resultants, 4096 rows, 4 columns) |
dq_border_ref_pix_top | Data quality for border reference pixels (top). | N/A | uint32 | (N resultants, 4 rows, 4096 columns) |
dq_border_ref_pix_bottom | Data quality for border reference pixels (bottom). | N/A | uint32 | (N resultants, 4 rows, 4096 columns) |
Level 3 - Mosaics
L3 products are the co-additions or mosaics of L2 files. A single L3 product may be based on the input of one or more L2 products. During L3 product generation, the data are corrected for geometric distortion and are converted from instrumental units to physical surface brightness units of MegaJanskys per steradian (MJy / sr). Data quality information from the L2 file(s) is used to screen out various undesirable detector effects from the final L3 product. The size and shape of the L3 products depends on the settings used to create the final product. For more information on the L3 science data pipeline, see Roman STScI Data Pipelines.
Information regarding specific L3 products produced by the Roman science centers will be added in a future RDox release.
Level 4 - Extracted Data
L4 products contain information that has been extracted from pixelated L2 and L3 data. These products may include source catalogs, 1-D spectra, and light curves.
Information regarding specific L4 products produced by the Roman science centers will be added in a future RDox release.
Level 5 - User Contributed Products
L5 data products are any products created by Roman users and made available to the community via the Roman Archive. As these products are heterogeneous in nature, documentation on collections of L5 products will be made available via MAST. See Accessing WFI Data for more information on L5 products.
For additional questions not answered in this article, please contact the Roman Help Desk at STScI.