StepMkflat

class sofia_redux.instruments.hawc.steps.stepmkflat.StepMkflat[source]

Bases: StepMOParent, StepLoadAux

Create a flat file from internal calibrator data.

This step combines the internal calibrator observations (INTCALs) taken adjacent to a science observation, then multiplies by a master flat (skycal) to generate an observation flat (OFT) file for each input group of INTCALs. The skycal file is generally a flat generated from a Chop-Scan observation of a bright source, divided by its own associated INTCAL. The INTCAL files are used to correct for temporal variations in detector responsivity, caused by changes in bias, background loading, and/or ADR control temperature; the skycal corrects for more static detector pixel response variations.

Variances from the INTCAL files are also propagated into the output flat files.

The input for this step is multiple demodulated files from internal calibrator observations (CALMODE = ‘INT_CAL’ files), demodulated using the ‘mode_intcal’ pipeline mode. Thus mode must set the following parameters for StepDemodulate:

  • l0method = RE

  • chopavg = True

  • phasefile = 0.0

The output for this step is a normalized INTCAL, one per input file, before combination and multiplication by the skycal. As a side effect, the observation flat files are also written to disk with product identifier ‘OFT’, in a folder designated by the ‘flatoutfolder’ parameter. OFT files have R ARRAY GAIN, T ARRAY GAIN, R ARRAY GAIN VAR, T ARRAY GAIN VAR, R BAD PIXEL MASK, and T BAD PIXEL MASK image extensions.

The mode_intcal pipeline ending in StepMkFlat should be run before the regular chop/nod pipeline to make the observation flats. Then, the regular pipeline can be run, making sure that StepFlat is configured to look in flatoutfolder for flatfiles, with the appropriate fitkeys settings to make sure the correct flat is paired with the observation.

Methods Summary

run()

Run the data reduction algorithm.

setup()

Set parameters and metadata for the pipeline step.

Methods Documentation

run()[source]

Run the data reduction algorithm.

Because this step is multi-in, multi-out (MIMO), self.datain must be a list of DataFits objects. The output is also a list of DataFits objects, stored in self.dataout.

The process is:

  1. Read data from the INTCAL files.

  2. For each file, flag outliers and normalize by the median signal in the R and T arrays.

  3. Identify groups of data by matching across the ‘groupkey’ parameter.

  4. Identify and read the data from an auxiliary skycal file.

  5. Mean-combine the normalized gain for each image in group.

  6. Multiply the mean INTCAL images by the skycal master flat.

  7. Save resulting OFT files to disk, in ‘flatoutfolder’.

setup()[source]

Set parameters and metadata for the pipeline step.

Output files have PRODTYPE = ‘mkflat’, and are named with the step abbreviation ‘DCL’.

Parameters defined for this step are:

flatoutfolderstr

Path for the folder to write flat files to. May be relative or absolute. Set to an empty string to write to the same folder as the input file.

groupkeystr

Header keyword that must match across INTCAL files in order to group them together. Typically set to FILEGPID.

skip_startint

Chops to exclude from the beginning of the file.

skip_endint

Chops to exclude from the end of the file.

bad_deadfloat

Raw data threshold for dead pixels.

bad_rampingfloat

Raw data threshold for ramping pixels.

normstdfloat

Threshold to exclude pixels with high standard deviation.

ynormlowlimlist of float

Threshold to exclude pixels with low normalized signal, given for [R0, R1, T0].

ynormhighlimlist of float

Threshold to exclude pixels with high normalized signal, given for [R0, R1, T0].

ttorfloat

Scale factor for the T array to the R array.

scalfilestr

Filename for auxiliary file(s). Can contain * and ? wildcards to match multiple files to be selected using fitkeys.

bkupscalstr

Back up filename for auxiliary file(s). Can contain * and ? wildcards to match multiple files to be selected using fitkeys.

scalfitkeyslist of str

List of header keys that need to match the scal data file. These are only used if multiple files match the input INTCAL file.

daterangefloat

If DATE-OBS is in scalfitkeys, files are matched within this many days.