Mosaic Level Pipeline

This article contains a user-oriented overview of the mosaic level pipeline, one of the Roman pipelines developed at STScI. It explains how mosaics are created with the drizzle algorithm, the steps available, and provides a basic level of support for those who want to process Roman data from Level 2 to Level 3.



Introduction to the Mosaic Pipeline 

The mosaic pipeline is responsible for combining individual detector exposures into mosaics. In this context, mosaic indicates a combination or stack of multiple exposures into a larger or deeper image. The mosaic pipeline combines groups of Roman Wide Field Instrument (WFI) images with an implementation of the drizzle algorithm (Fruchter & Hook 2002). Drizzle was originally developed to combine dithered Hubble Space Telescope (HST) imaging data as a way to remove the effects of undersampling while also enabling the removal of image artifacts, cosmic rays, and other types of spurious detections. The drizzle algorithm combines and distributes the flux from multiple input pixels onto a new pixel grid, which is typically of a different size. A larger area on the sky, enabled by large dither offsets between individual exposures, or it could represent a finer pixel grid achieved through subpixel dithers that oversample the pixels. Please refer to the DrizzlePac software documentation for more information.

For the JWST mission, STScI developed a code called resample which is based on the drizzle algorithm. Please see the JWST pipeline readthedocs for code documentation or the JDox article for an introduction to the package. The resample code has been repurposed for Roman data and is one of the modules in the mosaic pipeline.

The mosaic pipeline takes Roman Level 2 (L2) data products, individual calibrated exposures, and processes them into Level 3 (L3) mosaic images. Please review the Roman Data Levels and Products article for a detailed explanation of the Roman naming conventions for data levels and the associated products created at each level.

While users can define a mosaic that suits their science goals, the automatic pipeline products that are stored in the Roman Archive will utilize a sky tessellation to limit individual file size while enabling the creation of mosaics across large, contiguous regions of the sky. More information the Roman tessellation is available in the RDox Tessellation article.

Please note that in previous developmental versions, the mosaic pipeline was called the  High Level pipeline. 

WFI Prism and Grism Data

The mosaic pipeline is NOT applied to prism and grism data. The Science Support Center (SSC) at IPAC is responsible for the pipeline development and processing of higher level prism and grism data products.

Flux 

Flux is the initial step in the STScI Roman mosaic pipeline. This step scales the science data, error, and variance arrays by a flux scale factor that converts the input L2 pixel units from digital numbers per second (DN/s) into megaJanskys per steradian (MJy/sr). Users can find the scale factor in the metadata under the name meta.photometry.conversion_megajanskys which is different for each of the 18 detectors in the WFI. Please visit the WFI Detectors article to learn more about the properties of the WFI detectors. Additionally, the formulas used by the Flux step for this conversion are available in the code developers documentation. The key point is that the data and error arrays are multiplied by the flux scale factor, while the variance arrays are multiplied by the square of the flux scale factor. 

This step runs from an association file, similarly to the mosaic pipeline, and neither requires nor accepts other user-supplied arguments. 

SkyMatch

SkyMatch is the next step in the STScI Roman mosaic pipeline. This step measures the sky level in the input images and allows for the manipulation of backgrounds in the output mosaics. Note that the term "sky" in this context refers to any  non-source background signal, which may include actual sky emission as well as thermal emission from the spacecraft. SkyMatch offers multiple types of sky measurements, two of which are more commonly used: a local sky measurement that calculates the background value of each image separately, and a matching measurement that calculates the offsets between overlapping regions of two or more images. SkyMatch can also subtract a local background or apply the calculated offset to groups of images so that the backgrounds are consistent.  

A complete list of all arguments accepted by this step is available on readthedocs. The three most commonly used are provided in the table below for convenience. They include:

Table of Common SkyMatch Step Arguments 

ArgumentTypeDefaultOptionsExplanation
skymethodstringMatch

Local, Global, Match, Global+Match

Specifies the algorithm SkyMatch uses to calculate the sky background. Local computes sky background values of each input image or group of images. Global computes a common sky value for all input images and groups of images. Match computes differences in sky values between images and/or groups in (pair-wise) common sky regions. Global+Match first uses the Match method to equalize sky values between images and then finds a minimum "global" sky value amongst all input images.

match_downbooleanTrueTrue, FalseApplies an offset based on the measured differences between multiple images so that all images will have the same level of sky background after processing.
dqbitsstring~DO_NOT_USE+NON_SCIENCEA table with Roman data quality (DQ) values is provided on readthedocs.

Indicates the DQ bits that should be used when calculating the sky background. The default instructs SkyMatch to NOT use pixels flagged with the DO_NOT_USE and NON_SCIENCE values.

 Lastly, additional information can be found in the code developer documentation on readthedocs.

Outlier Detection 

The next step in the mosaic pipeline is the outlier detection step. The availability of multiple, overlapping images allows for the identification of pixels impacted by cosmic rays and other image artifacts, such as bad pixels. The larger amount of data allows for the detection of outliers that might not have been identified in the ramp fitting step in the exposure level pipeline. The outlier detection code was originally developed for JWST, and it has been adapted for Roman data. A description of the algorithm is available on readthedocs. 

This step offers an extensive number of arguments to customize the outlier detection. A complete list of arguments is available on the Argument page of the outlier detection readthedocs. 


Please note that outlier detection can falsely identify moving objects and/or sources with high variability as outliers and flag them, which remove these sources from subsequent processing steps. The input arguments of the outlier detection step needs to be customized based on the science goals. 


The most commonly used argument for this step is the  weight_type , which specifies how the data should be weighted to create a median image for outlier detection. The  weight_type has the available options of: 

  • exptime (default)
  • ivm
  • None

The default value  exptime uses the exposure time to weight the pixels.  ivm is an acronym that stands for inverse-variance map, which can be supplied to the pipeline by the user, or can be created on the fly by the pipeline. Lastly, None means that no weight image is used, and a mask will be generated based on the DQ arrays of the input calibrated rate (L2) images. More information about the types of weights can be found on readthedocs, while details on how weights are used in HST AstroDrizzle processing can be found in the section titled "A Note about Photometry and Weights in AstroDrizzle" in the DrizzlePac Handbook. 

Pixels identified as outliers in this step are flagged and excluded from further processing. 

Resample

The actual image combination happens in the resample step of the mosaic pipeline, which combines groups of WFI images with an implementation of the Drizzle algorithm. There is an extensive list of parameters that can be supplied by the user. The automated L3 outputs produced by the mosaic pipeline will use the parameters defined by the CRDS reference file which, as of the publication of this article, is still under development. When the reference file does not override a given argument, resample uses the default value for that argument. A table of the most commonly used arguments is provided below with guidance on how they might be modified by users. 

Table of Common Resample Step Arguments 

ArgumentTypeDefaultOptionsExplanation
pixfracfloat1.00.0 to 1.0

The scale fraction by which the original pixel is reduced in size before being transposed to the output image. The default of 1.0 indicates that no reduction should take place, but a value of 0.5 halves the pixel size. 

pixel_scale_ratiofloat1.00.0 to 1.0

The ratio of the pixel scale of the output image compared to the original pixel size. For the default value of 1.0, the pixel scale of the output image remains the native WFI pixel scale. A value of 0.5 indicates that for every pixel in the input image, four output pixels exist. This is often used to reverse the effects of undersampled observations, assuming that the exposures depth allows to recover the information.

weight_typestringivmivm, exptime, none

The ivm option uses the VAR_RDNOISE array to generate an inverse-variance map. If no VAR_RDNOISE array exists, then all pixels are given equal weighting. The exptime weights the pixels by the exposure time values of each image. 

good_bitsstring~DO_NOT_USE+NON_SCIENCEA table with Roman DQ values is provided on readthedocs.This string indicates which DQ bits are considered good when processing the images. For the default value, all of the pixels that are not flagged by DO_NOT_USE or NON_SCIENCE are considered "good". The tilde at the beginning indicates values that should not be used as good bits. 

The output image of resample corresponds to the L3 data product. 

Finally, there are several utilities written to work with with the resulting L3 data products. These include the ability to:

  • build a model of the weighing used by resample;
  • create a pixel grid map that demonstrates the transformation between the input and output world coordinate system (WCS);
  • identify which images contributed flux to each pixel in the output mosaic based on the context image;
  • and construct a function that translates the input image coordinates into the output image coordinates. 

A description of all these convenience utilities can be found on readthedocs. 

Source Catalog

This step, added to the Roman mosaic pipeline, creates a source catalog. It is performed automatically and does not accept any user inputs. For users not interested in the source catalog and L4 products, this module can be turned off as described in the mosaic pipeline instructions below. 

In Development

The Source Catalog step is still in development and will be documented fully in the future.

How to Run the Mosaic Pipeline 

To start using the mosaic pipeline, users need:

  • an environment that contains the mosaic pipeline, 
  • WFI calibrated (L2) images, and 
  • an association file that contains a properly formatted list of the WFI calibrated (L2) images. 

The environment can be on a local system or on the Roman's science platform. Detailed installation instructions can be found on the romancal readthedocs installation page. romancal is the package name for the Roman pipelines developed at STScI. The mosaic pipeline runs on WFI calibrated (L2) images, and they must be listed in an association file. In the examples below, the association file name is the one created in the Associations article example. 

The command line format is currently recommended, but other methods of calling the pipeline are described in the readthedocs, with supporting details about running the individual pipeline steps. 

Run via the Command Line

The command line call to run  romancal is strun, and the mosaic pipeline can be called by the class romancal.pipeline.MosaicPipeline or the alias roman_mos. The alias is used below. 

Basic Mosaic Pipeline Command
strun roman_mos r00000-o9999_RDoxExampleName_ATYPE_001_asn.json

Run with User-Supplied Arguments

We add user supplied arguments, one for the overall pipeline and one that is used exclusively by the  resample step. 

Mosaic Pipeline with User-Supplied Arguments
strun roman_mos r00000-o9999_RDoxExampleName_ATYPE_001_asn.json --output_file=RDox_Example_Output --steps.resample.weight_type=exptime

This command produces an output mosaic image that combines the four example images in the ASN file and names it RDox_Example_Output_i2d.asdf . It uses  exptime for the  weight_type when performing the resample step.

Run with Specific Steps Turned Off

To turn off a specific step, such as the Source Catalog step, use the format

Mosaic Pipeline with Source Catalog Turned Off
strun --disable-crds-steppar roman_mos r00000-o9999_RDoxExampleName_ATYPE_001_asn.json --steps.sourcecatalog.skip=True

The --disable-crds-steppar command prevents the mosaic pipeline from connecting to CRDS and using the step parameter reference files hosted there, which is a helpful command for users who plan to provide their own arguments. The --steps.sourcecatalog.skip=True skips the source catalog step.


The STScI Roman pipelines, including the mosaic pipeline, are developed and maintained by the Roman Science Operation Center at STScI. The code is publicly available on GitHub. 



For additional questions not answered in this article, please contact the Roman Help Desk at STScI.



References



Latest Update

Publication

 

Initial publication of the article.