EXESReduction

class sofia_redux.pipeline.sofia.exes_reduction.EXESReduction

Bases: Reduction

EXES reduction steps.

Primary image reduction algorithms are defined in the EXES package (sofia_redux.instruments.exes). Spectroscopy-related algorithms are pulled from the sofia_redux.spectroscopy package, and some utilities and display tools come from the sofia_redux.toolkit, sofia_redux.calibration, and sofia_redux.visualization packages.

This reduction object defines a recipe for data reduction and a method for each pipeline step that calls the appropriate algorithm from its source packages.

Attributes:

prodtype_mapdict

Maps the pipeline step to a product type, to assign to the PRODTYPE header key. Keys are pipeline step function names.

prodnamesdict

3-letter file type code, to assign to the output of a pipeline step. Keys are the product types (as defined in prodtype_map).

step_mapdict

Inverse of the prodtype_map, for looking up pipeline step names from product types. Keys are the product types.

prodtypeslist

List of product types, corresponding to the currently loaded recipe. This list is populated whenever the recipe attribute is set.

default_recipelist

Processing recipe used for standard science processing.

default_prodtype_maplist

Product type map used for standard science processing.

default_prodnameslist

Product names used for standard science processing.

default_step_maplist

Step name map used for standard science processing.

sky_spectrumbool

Flag to indicate that data should be reduced as a sky spectrum extraction, instead of standard science processing.

sky_prodtype_maplist

Alternate product types to use for sky spectrum extraction.

sky_prodnameslist

Alternate product names to use for sky spectrum extraction.

sky_step_maplist

Alternate step name map to use for sky spectrum extraction.

wcs_keyslist

List of header keywords used for tracking and propagating the spectral world coordinate system.

spec1d_prodtypelist

List of product types currently and historically used to designate 1D spectral products, used for display purposes and backwards-compatible handling.

Initialize the reduction object.

Methods Summary

clean_badpix()	Clean or flag bad pixels.
coadd_pairs()	Coadd nod pairs.
coadd_readouts()	Coadd raw readouts.
combine_spectra()	Combine spectra.
concatenate_filenum(filenum)	Concatenate file numbers, first-last.
convert_units()	Convert calibrated units to flux in Jy/pixel.
correct_calibration()	Correct calibration for blackbody variation by wavenumber.
debounce()	Correct for optical shifts (bounces).
despike()	Flag temporal outliers (spikes).
extract_spectra()	Extract 1D spectra from apertures.
flat_correct()	Calibrate and flat correct science data.
get_filename(header[, filenum, prodtype, update])	Create an output filename from an input header.
get_filenum(filename)	Get a file number from an EXES file name.
load(data)	Load input data to make it available to reduction steps.
load_data()	Load FITS data into memory and standardize headers.
load_fits()	Load FITS data into the input attribute.
locate_apertures()	Automatically find aperture centers.
make_flat()	Make a processed flat file.
make_profiles()	Produce spatial profile fits from rectified images.
merge_orders()	Merge all spectral orders.
refine_wavecal()	Refine wavecal by setting a new central wavenumber.
register_viewers()	Return a new QADViewer, ProfileViewer, and SpectralViewer.
set_apertures()	Set aperture radii.
set_display_data([raw, filenames])	Store display data for viewer.
specmap()	Generate a quick-look spectral plot.
subtract_background()	Subtract background along columns.
subtract_nods()	Subtract B nods from A nods for background correction.
undistort()	Correct for optical distortion.
update_output(hdul, filenum, prodtype)	Update output FITS file after a pipeline step.
update_sofia_keys(header)	Update required SOFIA header keywords.
write_output(hdul, outname)	Write an output FITS file to disk.

Methods Documentation

clean_badpix()

Clean or flag bad pixels.

Calls sofia_redux.instruments.exes.clean.

coadd_pairs()

Coadd nod pairs.

Calls sofia_redux.instruments.exes.coadd to mean combine input frames in each file.

Optionally, all files may be combined into a single file to be coadded together. Aternatively, all files may be split into separate files, one per frame, so that no frames are coadded.

Prior to the coadd, it is possible to apply a small spatial shift to align frames (exes.spatial_shift). It is also possible to subtract the mean value at each column to correct residual background levels, if INSTMODE = NOD_ON_SLIT.

To inspect the effect of these options, it is possible to save intermediate pre-coadd files to disk. If saved, they will have PRODTYPE = coadd_input and file code COI.

coadd_readouts()

Coadd raw readouts.

Calls sofia_redux.instruments.exes.derasterize for raster flats and darks and sofia_redux.instruments.exes.readraw for all other data.

Optionally, if the ‘fix_row_gains’ parameter is set, exes.correct_row_gains is called after readouts are coadded.

combine_spectra()

Combine spectra.

Calls sofia_redux.toolkit.image.combine.combine_images for coaddition. The combination method may be configured in parameters.

static concatenate_filenum(filenum)

Concatenate file numbers, first-last.

File numbers are sorted alphanumerically.

Parameters:

filenumstr or list

File numbers to concatenate.

Returns:

str

Concatenated file number.

convert_units()

Convert calibrated units to flux in Jy/pixel.

Uses the OMEGAP keyword to correct for the solid angle per pixel.

correct_calibration()

Correct calibration for blackbody variation by wavenumber.

The flat_correct step calibrates the spectral flux to physical units based on the blackbody function at the central wavenumber in the flat. This step corrects the calibration for the variation of the blackbody with respect to wavenumber values at each pixel in the spectral image.

Calls sofia_redux.instruments.exes.makeflat.bnu on wavenumber values in the WAVECAL extension to determine a correction image for the FLUX. This correction image is directly multiplied into the FLUX data array.

debounce()

Correct for optical shifts (bounces).

Calls sofia_redux.instruments.exes.debounce.

despike()

Flag temporal outliers (spikes).

Calls sofia_redux.instruments.exes.despike on each input file.

Typically, A beams within are compared separately from B beams, but if desired, the beam designation can be ignored and all frames compared together.

It is also possible to combine all input files before running despike. In this case, all input will be treated as a single file in all subsequent steps.

extract_spectra()

Extract 1D spectra from apertures.

Calls sofia_redux.spectroscopy.extspec.

flat_correct()

Calibrate and flat correct science data.

Input data must have FLAT extensions attached.

Calls sofia_redux.instruments.exes.calibrate.

get_filename(header, filenum=None, prodtype=None, update=True)

Create an output filename from an input header.

Parameters:

headerastropy.io.fits.Header

Header to create filename from.

filenumstr or list, optional

List of file numbers to concatenate for filename.

prodtypestr, optional

Three letter product type designator.

updatebool, optional

If set, the FILENAME key will be added or updated in the header.

Returns:

filenamestr

The output name.

static get_filenum(filename)

Get a file number from an EXES file name.

Formats expected are for raw EXES files (e.g. sirius.sci.10001.fits) or for intermediate processed files (e.g. F0001_EX_SPE_0101123_EXEELONEXEECHL_COA_10001-10002.fits. In either case, the field just before the .fits is expected to contain the file number.

If the input file references a range of files, e.g. ‘10001-10002’, the first and last file numbers are returned in a list. Otherwise, a single integer is returned. If neither pattern is found, ‘UNKNOWN’ is returned.

Parameters:

filenamestr

FITS file name, raw or intermediate.

Returns:

filenumstr or list of str

Integer file numbers referenced in the input file name.

load(data)

Load input data to make it available to reduction steps.

The process is:

• Call the parent load method to initialize data reduction variables.

• Use the first loaded FITS header that is not a flat or dark file to set the observation configuration for the reduction set.

• Use the product type in the base header to determine the data processing recipe.

• Load parameters for all steps.

• Load the data immediately if it is a 1D spectrum that needs visualization only; otherwise, just load the raw headers and defer loading the data from the FITS files.

After this step, the input attribute is populated as required for the first pipeline step in the recipe.

Parameters:

datalist of str or str

Input file paths to load.

load_data()

Load FITS data into memory and standardize headers.

Calls sofia_redux.instruments.exes.readhdr to standardize headers and load expected default values.

If the sky_spec parameter is set, product types and names are updated for sky spectrum extraction.

load_fits()

Load FITS data into the input attribute.

locate_apertures()

Automatically find aperture centers.

Calls sofia_redux.spectroscopy.findapertures.

make_flat()

Make a processed flat file.

If the input data is already a processed flat, it is passed through without modification. Otherwise, exes.makeflat is called to process the flat file.

If multiple unprocessed flat files are provided, they are mean-combined before processing. If multiple processed flat files are provided, only the first one is used.

The output data from this step includes only the science files with new flat extensions attached. Any flat or dark files are dropped from further processing.

make_profiles()

Produce spatial profile fits from rectified images.

The rectified images and profiles are stored in self.input for continued processing.

Calls sofia_redux.spectroscopy.mkspatprof and sofia_redux.spectroscopy.rectify.

merge_orders()

Merge all spectral orders.

Calls sofia_redux.spectroscopy.mergespec to merge the 1D spectra and sofia_redux.toolkit.image.coadd to combine the 2D spectral images.

refine_wavecal()

Refine wavecal by setting a new central wavenumber.

User input should identify the order and pixel position of a known spectral feature as well as the calibrated wavenumber for that feature. This information is used to derive a new central wavenumber.

Calls sofia_redux.instruments.exes.wavecal to update the wavelength calibration from the central wavenumber.

register_viewers()

Return a new QADViewer, ProfileViewer, and SpectralViewer.

set_apertures()

Set aperture radii.

Calls sofia_redux.spectroscopy.getapertures and sofia_redux.spectroscopy.mkapmask.

set_display_data(raw=False, filenames=None)

Store display data for viewer.

Parameters:

rawbool

If True, display data is taken from self.rawfiles. If False, display data is taken from self.input.

filenameslist of str, optional

If provided and raw is False, file names will be passed to the viewer instead of self.input.

specmap()

Generate a quick-look spectral plot.

Calls sofia_redux.visualization.quicklook.make_spectral_plot.

The output from this step is identical to the input, so is not saved. As a side effect, a PNG file is saved to disk to the same base name as the input file, with a ‘.png’ extension.

subtract_background()

Subtract background along columns.

Calls sofia_redux.spectroscopy.extspec.col_subbg.

subtract_nods()

Subtract B nods from A nods for background correction.

Calls sofia_redux.instruments.exes.diff_arr to do the array subtraction.

If desired, for INSTMODE = NOD_OFF_SLIT, exes.cirrus may be additionally called to correct for residual background.

undistort()

Correct for optical distortion.

Calls sofia_redux.instruments.exes.tort to do the distortion correction and sofia_redux.instruments.exes.wavecal to generate wavenumber and spatial calibration images.

After this step, all spectral images are rotated as needed to align the spectral axis with the x-axis.

update_output(hdul, filenum, prodtype)

Update output FITS file after a pipeline step.

Sets the PRODTYPE key, adds a HISTORY message with the pipeline step name, and updates the FILENAME key with a new filename, appropriate to the prodtype. The new filename is returned from the function.

Parameters:

hdulastropy.io.fits.HDUList

Output FITS HDUList.

filenumstr or list

File number(s) to use in the filename.

prodtypestr

Product type for the completed step.

Returns:

str

File base name for the output product.

update_sofia_keys(header)

Update required SOFIA header keywords.

Keywords added or updated are:

• PROCSTAT: set to LEVEL_2

• ASSC_AOR: copied from AOR_ID

• ASSC_OBS: copied from OBS_ID

• ASSC_MSN: copied from MISSN-ID

• OBS_ID: prepended with ‘P_’

• PIPELINE: set to pipe_name

• PIPEVERS: set to pipe_version

Parameters:

headerastropy.io.fits.Header

Header to update.

write_output(hdul, outname)

Write an output FITS file to disk.

Outname is joined to self.output_directory, before writing.

Parameters:

hdulastropy.io.fits.HDUList

FITS HDUList to write.

outnamestr

File basename to write.

Returns:

str

Full path to the output file.