Computational Benchmarks
This page provides representative benchmarking examples for common scientific workflows on the Roman Research Nexus. It summarizes measured CPU and memory usage for well-defined processing tasks to help users understand the computational requirements of different scientific analyses. For guidance on using these benchmarks to estimate compute, storage, and credit consumption for complete scientific workflows, see Estimating Resource Usage and Costs.
The benchmarks are intended as guidelines rather than fixed estimates. Each result should be interpreted in the context of its specific workload and assumptions. Reproducible benchmarking scripts are provided to allow users to rerun the examples and adapt them to their scientific needs, building on existing Roman Research Nexus tutorials and workflows.
Benchmark Results Summary
The Table of Summary of Benchmark Results summarizes the computing resources measured for each benchmark, including wall-clock time (execution time), CPU usage, memory requirements, and the minimum server configuration needed to execute the workflow. All benchmarks were executed using the general-purpose server configurations described in the Compute Resources documentation. CPU hours are normalized per detector, mosaic, object, or batch, as indicated.
These results provide a starting point for estimating the resources required for similar scientific analyses. Users should consider differences in data volume, processing parameters, and workflow complexity when extrapolating these measurements to their own science cases.
The scripts used to generate these results will be available on the Roman Research Nexus in the upcoming December 2026 update, and can be rerun or modified to explore different scientific workloads.
The benchmarking scripts are also available in the roman_notebooks GitHub repository. These scripts were developed against the software stack planned for the December 2026 Roman Research Nexus update. Benchmarking code is organized into individual Python scripts corresponding to the operations listed in the Operation column of the table below. Within each script, functions named caseX (where X corresponds to the Case number in the table) implement the benchmark scenarios.
Some cases evaluate scalability by executing the same functionality with different input sizes or parameters, while others benchmark distinct workflows, such as source detection or point-spread function (PSF) fitting. The code was largely adapted from Roman Research Nexus tutorial Jupyter notebooks.
Interpreting the results: CPU hours represent the total computational effort required by a benchmark and should not be interpreted as the elapsed execution time. Different algorithms make use of CPU resources in different ways, depending on factors such as vectorization and parallelization. As a result, wall-clock runtime and Nexus credit consumption depend on the server configuration, degree of parallel execution, and overall workflow characteristics, including resource contention from concurrently running tasks.
The values reported in the table below are averages derived from multiple benchmark executions.
Summary of Benchmark Results
Operation | Case | Server Size† | Wall-clock Time (seconds) | CPU Hours (normalized) | RAM‡ (GB) |
|---|---|---|---|---|---|
| RomanCal Exposure Pipeline (L1 → L2) | Case 1: Full processing, 5 science resultants | Small | 135.55 | 0.036 | 2.91 |
Case 2: No source products, 5 science resultants | Small | 59.20 | 0.015 | 2.85 | |
Case 3: No source products, 15 science resultants | Small | 306.90 | 0.084 | 5.40 | |
Case 1: 4-point dither, full processing | Small | 150.35 | 0.032 | 5.47 | |
Case 2: 4-point dither, no catalog | Small | 298.09 | 0.074 | 5.54 | |
Case 3: Full WFI FOV mosaic (18 detectors) | Medium | 957.27 | 0.214 | 34.21 | |
Case 1: L1 simulation (1,000 galaxies; 5 resultants) | Small | 400.07 | 0.111 / detector | 6.94 | |
Case 2: L2 simulation (1,000 galaxies; 5 resultants) | Small | 431.10 | 0.118 / detector | 8.24 | |
Case 3: L1 simulation (1,000 stars; 5 resultants) | Small | 178.91 | 0.049 / detector | 6.94 | |
Case 4: L2 simulation (1,000 stars); 5 resultants | Small | 196.00 | 0.054 / detector | 8.25 | |
Case 5: L1 simulation (500 galaxies + 500 stars; 5 resultants) | Small | 271.15 | 0.075 / detector | 6.94 | |
Case 6: L2 simulation (500 galaxies + 500 stars; 5 resultants) | Small | 299.43 | 0.082 / detector | 8.23 | |
Case 7: L1 simulation (10,000 stars; 5 resultants) | Small | 134.89 | 0.040 / detector | 6.92 | |
Case 8: L1 simulation (1,000 stars; 15 resultants) | Small | 489.56 | 0.138 / detector | 10.37 | |
Case 1: Source Detection - 10,000 stars | Small | 2.78 | 7.66x10-4 / 10,000 stars | 0.45 | |
Case 2: Source Detection - 1,000 stars | Small | 2.84 | 7.77x10-4 / 1,000 stars | 0.45 | |
Case 3: Aperture Photometry - 10,000 stars | Small | 0.96 | 2.65x10-4 / 10,000 stars | 0.20 | |
Case 4: Aperture Photometry - 1,000 stars | Small | 0.21 | 5.69x10-5 / 1,000 stars | 0.16 | |
Case 5: PSF Photometry - 10,000 stars | Small | 25.34 | 7.00x10-3 / 10,000 stars | 0.00 | |
Case 6: PSF Photometry - 1,000 stars | Small | 2.61 | 7.31x10-4 / 1,000 stars | 0.00 | |
Case 1: Simple moments | Small | 4x10-4 | 1.74x10-7 / galaxy | 0.00 | |
Case 2: Sérsic profile fitting | Small | 2.5 | 7.09x10-4 / galaxy | 0.02 | |
Case 1: Cutout generation (100×100 pixels) | Small | 5.11 | 9.97x10-4 / 100 cutouts | 0.17 | |
Case 2: Cutout generation (10×10 pixels) | Small | 5.20 | 1.02x10-3 / 100 cutouts | 0.17 | |
Case 1: 100 SNR estimates | Small | 64.44 | 0.015 / 100 estimates | 0.30 | |
Case 2: 100 limiting magnitude estimates | Small | 1,191.12 | 0.192 / 100 estimates | 3.17 | |
Case 3: 100 exposure time estimates | Small | 373.50 | 0.086 / 100 estimates | 1.59 | |
Case 1: ZTF × Pan-STARRS (~10,000 sources) | Small | 18.25 | 3.47x10-3 / 10,000 sources matched | 1.29 |
† Server sizes listed reflect the smallest configuration on which each benchmark case could be executed. Larger servers may reduce wall-clock time, but total CPU hours typically decrease only for inherently multi-threaded tasks (e.g., RomanCal and Roman I-sim), not for primarily single-threaded or user-parallelized analyses.
‡ RAM use is measured as the increase in RAM during the operation from the baseline at the start of the execution to the peak used by the process.
Benchmark Case Definition
This section describes the benchmark cases used to estimate CPU and memory requirements for common scientific workflows on the Roman Research Nexus. Each case specifies the processing configuration, input data, and relevant assumptions needed to interpret the results and apply them to other scientific workloads.
RomanCal Exposure-Level Pipeline: L1 → L2
These benchmarks execute the RomanCal Exposure Pipeline (L1 → L2) on a single WFI detector. Processing is performed entirely in memory, without file I/O.
Case 1: Single-detector calibration, full processing, 5 science resultants
Executes the complete exposure calibration pipeline, including source detection and source catalog generation.
- Input file contains 5 science resultants in the ramp cube, corresponding to approximately 66 seconds of exposure time.
Input scene contains approximately 1,500 uniformly distributed sources (stars and galaxies).
Processing requirements depend on source density.
Case 2: Single-detector calibration, no catalog products, 5 science resultants
Executes the exposure calibration pipeline with source detection and catalog generation excluded.
- Input file contains 5 science resultants in the ramp cube, corresponding to approximately 66 seconds of exposure time.
Processing requirements are independent of source density because the detection and catalog steps are omitted.
Case 3: Single-detector calibration, no catalog products, 15 science resultants
- Executes the exposure calibration pipeline with source detection and catalog generation excluded.
- Processing requirements are independent of source density because the detection and catalog steps are omitted.
- Input file contains 15 science resultants in the ramp cube, corresponding to approximately 294 seconds of exposure time.
RomanCal Mosaic Level Pipeline: L2 → L3
These benchmarks execute the RomanCal Mosaic-Level Pipeline (L2 → L3) to combine exposure-level images into mosaics. All three cases include file I/O via streaming files from AWS S3.
Case 1: Single-detector 4-point gap-filling dither, full processing
Combines four exposure-level images from a [4-point gap-filling dither pattern] into a single mosaic.
Includes source detection and source catalog generation.
Input scene contains approximately 4,500 sources, with galaxies uniformly distributed and stars clustered to simulate a star cluster.
Case 2: Single-detector 4-point gap-filling dither, no catalog products
Combines four exposure-level images from the same 4-point gap-filling dither pattern into a single mosaic.
Excludes source detection and source catalog generation.
Processing requirements are independent of source density because the detection and catalog steps are omitted.
Case 3: Full WFI field-of-view mosaic, no source products
Combines all 18 WFI detector images from a single exposure into one mosaic.
Excludes source detection and source catalog generation.
Processing requirements are independent of source density because the detection and catalog steps are omitted.
Requires a Medium server (64 GB RAM) or larger to accommodate the memory footprint.
Roman I-Sim Simulations
These benchmarks use Roman I-Sim to simulate WFI imaging data for a single detector. All cases include PSF generation using STPSF, with sources uniformly distributed across a circular region contained within a single WFI detector footprint. The cases differ in the output data level (uncalibrated L1 or calibrated L2) and the input source catalog composition.
Case 1: Simulation of uncalibrated data (L1), galaxies
Simulates uncalibrated imaging data (L1) with 1,000 galaxies.
- Uses a MultiAccum table composed of 5 science resultants and uses the F106 imaging filter.
Case 2: Simulation of exposure-level calibrated product (L2), galaxies
Simulates an exposure-level calibrated product (L2) with 1,000 galaxies.
- Uses a MultiAccum table composed of 5 science resultants and uses the F106 imaging filter.
Case 3: Simulation of uncalibrated data (L1), stars
Simulates uncalibrated imaging data (L1) with 1,000 stars.
- Uses a MultiAccum table composed of 5 science resultants and uses the F106 imaging filter.
Case 4: Simulation of exposure-level calibrated product (L2), stars
Simulates an exposure-level calibrated product (L2) with 1,000 stars.
- Uses a MultiAccum table composed of 5 science resultants and uses the F106 imaging filter.
Case 5: Simulation of uncalibrated data (L1), mixed scene
Simulates uncalibrated imaging data (L1) with 500 galaxies and 500 stars.
- Uses a MultiAccum table composed of 5 science resultants and uses the F106 imaging filter.
Case 6: Simulation of exposure-level calibrated product (L2), mixed scene
Simulates an exposure-level calibrated product (L2) with 500 galaxies and 500 stars.
- Uses a MultiAccum table composed of 5 science resultants and uses the F106 imaging filter.
Case 7: Simulation of uncalibrated data (L1), 10,000 stars
Simulates an exposure-level calibrated product (L2) with 10,000 stars taking advantage of the fastpointsources setting.
- Uses a MultiAccum table composed of 5 science resultants and uses the F106 imaging filter.
- Fastpointsources is currently only available for simulations containing more than 1,000 stars.
Case 8: Simulation of exposure-level calibrated product (L2), stars with a long ramp
Simulates uncalibrated imaging data (L1) with 1,000 stars.
- Uses a MultiAccum table composed of 15 science resultants and uses the F106 imaging filter.
Photometry
These benchmarks perform various source detection and photometry tasks using:
Two stellar populations are considered to evaluate resource usage for different catalog sizes.
Case 1: Source detection – large stellar catalog
Performs source detection and aperture photometry on an image containing 10,000 stars.
Approximately 95% of sources are recovered.
Case 2: Source detection – large stellar catalog
Performs source detection and aperture photometry on an image containing 1,000 stars.
All injected sources are recovered, with some spurious detections.
Case 3: Forced aperture photometry – large stellar catalog
- Performs forced aperture photometry on an image containing 10,000 stars with an accompanying catalog of known star positions.
Case 4: Forced aperture photometry – moderate stellar catalog
- Performs forced aperture photometry on an image containing 1,000 stars with an accompanying catalog of known star positions.
Case 5: Forced PSF photometry – large stellar catalog
- Performs forced PSF photometry on an image containing 10,000 stars with an accompanying catalog of known star positions.
- The PSF is determined from STPSF, and the benchmark does not include PSF grid construction.
Case 6: Forced PSF photometry – large stellar catalog
- Performs forced PSF photometry on an image containing 10,000 stars with an accompanying catalog of known star positions.
- The PSF is determined from STPSF, and the benchmark does not include PSF grid construction.
Galaxy Shape Measurements
These benchmarks evaluate two approaches to measuring galaxy shapes, ranging from simple moment-based estimates to more computationally intensive model fitting.
Case 1: Simple moment-based shape measurements
Uses a lightweight per-object estimator to compute position angle and major-to-minor axis ratio.
Case 2: Sérsic profile fitting
Fits a Sérsic surface-brightness model to each individual galaxy.
Computes the full covariance matrix of the fitted parameters.
Astrocut
These benchmarks use AstroCut to generate batches of 100 image cutouts, comparing two cutout sizes.
Case 1: Cutout generation (100 × 100 pixels)
Generates 100 cutouts, each measuring 100 × 100 pixels.
Case 2: Cutout generation (10 × 10 pixels)
Generates 100 cutouts, each measuring 10 × 10 pixels.
Note: A dedicated cutout service based on Astrocut is currently under development. The service will provide optimized, parallelized cutout generation. The benchmarks presented here reflect standalone Astrocut execution and are not representative of the expected performance or resource consumption of the new service. Additional information will be provided as the service becomes available.
Exposure Time Calculations (Pandeia)
These benchmarks use Pandeia to evaluate three types of exposure time calculations. Each case consists of 100 estimates.
Case 1: Signal-to-noise ratio estimates
Computes the SNR for a source of a given magnitude and exposure configuration.
Case 2: Limiting magnitude estimates
Computes the 5σ point-source limiting magnitude for a given exposure configuration.
Case 3: Exposure time estimates
Computes the exposure time required to reach a target SNR
Catalog Cross-matching
Case 1: ZTF × Pan-STARRS cross-match
This benchmark cross-matches approximately 10,000 ZTF sources against the Pan-STARRS catalog using a parallelized implementation distributed across multiple CPU cores.
Input data:
ZTF catalog retrieved from IRSA's public S3 bucket.
Pan-STARRS catalog retrieved from the STScI Open Data S3 bucket.
The ZTF sample is preselected through a cone search to approximately 10,000 sources.
Execution assumptions:
Catalogs are streamed directly into memory, without intermediate disk I/O during matching.
Reported CPU usage includes only the cross-matching computation, excluding the initial catalog query.
How To Interpret the Benchmarks
The benchmark results provide representative measurements of the computing resources required for specific scientific workflows. When using these results to estimate resource needs, users should consider the following:
CPU hours vs. wall-clock time: CPU hours measure the total computational effort required by a task, while wall-clock time represents the elapsed time needed to complete it. Parallel processing can reduce wall-clock time without necessarily reducing total CPU usage. However, using larger or multiple servers may increase the credit consumption rate. Consequently, faster execution does not necessarily translate into lower total credit costs.
Memory requirements: The minimum server configuration reported for each benchmark reflects the resources needed to execute that particular workload. Larger datasets or more complex analyses may require additional memory.
Credit consumption: Nexus credits are charged based on the compute resources allocated and their usage duration. CPU hours cannot be converted directly into credits without considering the server configuration and runtime. See the Compute Resources documentation for available server configurations and their associated credit consumption rates.
Scaling to larger workloads: Resource requirements may not scale linearly with data volume or processing complexity. Users should consider the assumptions of each benchmark when extrapolating to their own analyses.
Users are encouraged to rerun and adapt the provided benchmarking scripts to obtain estimates more representative of their specific scientific workflows. For worked examples showing how to translate benchmark measurements into estimates of compute and storage requirements, account for repeated analyses, and calculate Nexus credit consumption, see Estimating Resource Usage and Costs.
For additional questions not answered in this article, please contact the Roman Help Desk.
