Reduction

class sofia_redux.pipeline.reduction.Reduction

Bases: object

Reduction steps and data.

Attributes:

namestr

Name of the reduction type.

instrumentstr

Name of the instrument supported by this reduction object.

modestr

Name of the instrument mode supported by this object.

pipe_namestr

Name of the data reduction software package.

pipe_versionstr

Pipeline version.

param_filestr

Input parameter file, if provided.

parametersParameters

A Redux Parameters object. This will hold all current values of reduction parameters, to be provided to the steps when they are run.

recipelist of str

List of processing steps to run. Normally, the step names are method names implemented in the current reduction class.

processing_stepsdict

Dictionary associating pipeline step names with more descriptive display names.

step_indexint

The current step in the recipe, to be run next.

errorlist of str

A list of error messaged generated by reduction steps, for reporting back to the user.

data_keyslist of str

A list of keywords that describe the input data, for display to the user. ‘File Name’ and ‘File Size’ keywords have general definitions applicable to most input types. Other keywords are assumed to be readable from the data structure’s .header attribute, as data.header[key].

data_idOrderedDict

Holds the association between data_keys and values for the currently loaded data.

allow_undobool

Flag to indicate whether an ‘undo’ operation should be allowed in GUI context. The undo operation is based on a serialization of this object: if the current object cannot be serialized, or if it would be too memory-expensive to do so, this flag should be set to False.

raw_fileslist of str

Input file paths.

out_fileslist of str

Output file paths, for files generated by the current reduction.

inputlist

Input data for the next processing step. List members may be any type understood by the processing step.

display_data: dict

Data to be displayed by any viewers associated with the current reduction. Keys should be the names of the viewers (Viewer.name); values may be any data type understood by the viewer.

output_directorystr

Directory path to which any output files generated should be saved.

Initialize the reduction.

Attributes Summary

description

str: Description of the reduction (name, instrument, and mode).

Methods Summary

cleanup()	Perform any necessary cleanup tasks before shutting down.
edit_parameters(key[, step_index, value, ...])	Edit the parameters for a reduction step.
get_key_value(header, key)	Retrieve a value from a metadata header.
get_parameter_set([step_index])	Get the current parameter set for a step.
load(data)	Load input data to make it usable to reduction steps.
load_data_id()	Load the data description.
load_fits_files(data)	Load input FITS files into memory.
load_parameters()	Set the parameter list for the current reduction recipe.
log_input()	Log input data at INFO level.
log_reduce([end])	Log the pipeline name and version.
log_step(step_name[, params])	Log the reduction step name, time, and parameters.
read_data_key(key[, fname, header])	Read a data keyword from a metadata header.
record_outfile(outfile)	Store an output file name.
reduce()	Reduce the current data.
register_viewers()	Instantiate viewers appropriate to the reduction.
set_parameter_set(param[, step_index])	Set a parameter set for a reduction step.
step([alt_method])	Run a reduction step.
update_parameters()	Update parameter values for the current reduction.

Attributes Documentation

description

str: Description of the reduction (name, instrument, and mode).

Methods Documentation

cleanup()

Perform any necessary cleanup tasks before shutting down.

edit_parameters(key, step_index=None, value=None, options=None, option_index=None, hidden=False)

Edit the parameters for a reduction step.

Parameters:

keystr

Parameter key name.

step_indexint, optional

Index of the reduction step in the reduction recipe. If not provided, the current step_index will be used.

valuestr, float, int, bool, or list; optional

Parameter value.

optionslist; optional

Enumerated parameter value options.

option_indexint, optional

Index of the selected option, for enumerated values.

get_key_value(header, key)

Retrieve a value from a metadata header.

Parameters:

headerdict-like

Metadata header. May be any type for which the data is accessible as header[key].

keystr

Header keyword.

Returns:

str

String representation of the header value, or ‘UNKNOWN’ if the keyword was not found or could not be converted to a string.

get_parameter_set(step_index=None)

Get the current parameter set for a step.

Parameters:

step_indexint, optional

Index of the step in the reduction recipe. If not provided, the current step_index will be used.

Returns:

ParameterSet

The parameters for the specified step.

load(data)

Load input data to make it usable to reduction steps.

This function only stores the data as is in the raw_files attribute. This function should be overridden by subclasses if more complex behavior is needed, e.g. to call the load_fits_files method, or set the display data variables.

Parameters:

datalist of str or str

Input data file names to be loaded.

load_data_id()

Load the data description.

Calls read_data_key for all keys in the data_keys attribute on the currently loaded data. This method populates the data_id attribute.

load_fits_files(data)

Load input FITS files into memory.

Input data files are stored as astropy.io.fits.HDUList objects in the input attribute. Any input files that cannot be read as FITS files will trigger a log warning, but will otherwise be ignored.

Parameters:

datalist of str

Input FITS file paths.

load_parameters()

Set the parameter list for the current reduction recipe.

For each reduction step, this method calls Parameters.add_current_parameters to set default values, then checks for a method corresponding to the step name in the Parameters object. If such a method is defined, it is called. This allows custom logic related to parameter setting to be defined for each reduction step.

If the param_file attribute is not null, loaded parameters are overridden with any values defined in that file (via the Parameters.from_config method).

log_input()

Log input data at INFO level.

log_reduce(end=False)

Log the pipeline name and version.

This method is called at the beginning and end of the reduce function to describe the reduction that is about to run. It produces log messages at the INFO level.

Parameters:

endbool

If True, log a finishing message for the end of the reduction. Otherwise, log the start up message.

log_step(step_name, params=None)

Log the reduction step name, time, and parameters.

This method is called at the beginning of the step method. Messages are logged at INFO level. Hidden parameters are not logged.

Parameters:

step_namestr

Name of the reduction step.

paramsParameterSet

Parameters for the reduction step.

read_data_key(key, fname=None, header=None)

Read a data keyword from a metadata header.

Parameters:

keystr

Data keyword, not case sensitive. ‘File Name’ and ‘File Size’ have special values; all other keywords are retrieved from the header.

fnamestr, optional

Name of the file being described. Its basename is returned if key is ‘File Name’.

headerdict-like

Metadata header.

Returns:

str

A string representation of the metadata value.

record_outfile(outfile)

Store an output file name.

Output file names are recorded in the out_files attribute, which is used to generate output manifests. The output file name is also logged at the INFO level.

Parameters:

outfilestr

Path to the output file.

reduce()

Reduce the current data.

All steps in the current reduction recipe is run. This function assumes that no steps have been run yet.

register_viewers()

Instantiate viewers appropriate to the reduction.

This method instantiates and returns any Viewer objects that may be used to display data from this reduction. Data for the viewers is stored in the display_data attribute, but no reference to the viewers is stored in the reduction object itself. Viewers should be controlled through an Interface object.

In this implementation, the default Viewer is returned, which only prints a message at DEBUG level. This method should be overridden if more complex viewers are required.

Returns:

list of Viewer

All viewers supported by this reduction object.

set_parameter_set(param, step_index=None)

Set a parameter set for a reduction step.

This function is intended for restoring parameters from a previously saved ParameterSet object. Use edit_parameters for an interface that does not require a ParameterSet object as input.

Parameters:

paramParameterSet

A full set of parameters for the reduction step.

step_indexint, optional

The reduction step index in the current reduction recipe. If not provided, the current step_index will be used.

step(alt_method=None)

Run a reduction step.

This method calls the reduction step specified in the recipe attribute, at the current step_index. If the step index is out of range for the recipe, this method will just return.

Normally, reduction steps are defined as methods in the reduction object. Sometimes, a reduction package may have its own calling methods, rather than offering modular reduction steps. In this case, the alt_method parameter should be used to specify a reduction method that calls the external interface for each pipeline step, marshalling and passing data to the interface as required.

Parameters:

alt_methodstr, optional

If provided, this method will be run instead of the method named by the reduction step name. This is used in cases where the reduction step is not implemented or described in the reduction object itself, but in an external package.

Returns:

str

An error message if the reduction step produced any errors; an empty string otherwise.

Raises:

RuntimeError

If there is no input to process, and self.check_input is True.

update_parameters()

Update parameter values for the current reduction.

For each reduction step, this method checks for a method corresponding to each step name in the Parameters object. If such a method is defined, it is called.

If the param_file attribute is not null, loaded parameters are overridden with any values defined in that file (via the Parameters.from_config method).

Unlike the load_parameters method, any previously edited parameters not affected by either the step methods or the input file retain their edited values. If parameters have not yet been loaded, this function has no effect.