FlaggedArray

class sofia_redux.scan.flags.flagged_array.FlaggedArray(data=None, blanking_value=None, dtype=None, shape=None)

Bases: FlaggedData

Creates a flagged array instance.

The flagged array provides a wrapper around a numpy array allowing for each element to have an associated flag value. There are also a number of arithmetic operators provided that automatically discount certain flags and invalid values from any processing.

Parameters:

datanumpy.ndarray, optional

Data to initialize the flagged array with. If supplied, sets the shape of the array. Note that the data type will be set to that defined by the dtype parameter.

blanking_valueint or float or bool or complex, optional

The blanking value defines invalid values in the data array. This is the equivalent of defining a NaN value.

dtypetype, optional

The data type of the data array.

shapetuple (int), optional

The shape of the data array. This will only be relevant if data is not defined.

Attributes Summary

blanking_value	Return the blanking value for the array.
data	Return the data array.
data_range	Get the data range.
nan_blanking	Return whether the blanking value is NaN.
ndim	Return the number of dimensions in the map image data.
shape	Return the shape of the data image array.
size	Return the size of the data image array.
valid	Return a boolean mask array of valid data elements.
valid_data	Return valid data.

Methods Summary

abs_sum()	Return the sum of the absolute data values.
add(values[, indices, factor])	Add a value to the data array.
argmax()	Return the index of the maximum value.
argmaxdev()	Return the index of maximum (absolute) deviation from zero.
argmin()	Return the index of the minimum value.
check_equal_contents(other)	Check the data contents of this data are equal with another.
clear([indices])	Clear flags and set data to zero.
copy()	Return a copy of the FlaggedArray.
count_flags([flag])	Count the number of flagged elements.
count_points()	Return the number of valid points.
count_valid_points()	Return the number of valid data points in the array.
crop(ranges)	Crop the array to the required dimensions.
default_blanking_value()	Return a default blanking value type for the current data type.
delete_indices(indices_or_mask)	Completely deletes data elements.
despike(threshold[, noise_weight])	Discard spikes whose significance is above a given threshold.
destroy()	Destroy the image data.
direct_resample_from(image, to_indices)	Resample an image onto the current flagged array.
discard([indices])	Set the flags for discarded indices to DISCARD and data to zero.
discard_flag(flag[, criterion])	Clear all data flagged with the given flag.
discard_min_neighbors(min_neighbors)	Discard points with insufficient neighbors.
discard_range(discard_range)	Discard data values in a certain range.
fast_smooth(beam_map, steps[, ...])	Smooth using the fast method.
fill(value[, indices])	Fill the data array with a given value.
find_fixed_indices(fixed_indices[, cull])	Returns the actual indices given fixed indices.
get_cropped(ranges)	Return a copy of the cropped Flagged Array.
get_fast_smoothed(beam_map, steps[, ...])	Return smoothed values using the fast method.
get_index_range()	Return the index ranges of valid points.
get_indices(indices)	Return selected data for given indices.
get_info()	Return a string descriptors of the array.
get_neighbor_kernel()	Return a neighbor kernel.
get_neighbors()	Return the number of valid neighbors for each point, including itself.
get_refined_peak_index(peak_index)	Get the peak index given a local peak.
get_size_string()	Return the shape of the data array as a string.
get_smoothed(beam_map[, reference_index, ...])	Return smoothed data and weights.
get_valid_smoothed(beam_map[, ...])	Return smoothed data and weights, where invalid entries are zeroed.
index_of_max([sign, data])	Return the maximum value and index of maximum value.
insert_blanks(insert_indices)	Inserts blank frame data.
is_valid()	Return a boolean mask array of valid data elements.
kernel_resample_from(image, kernel, to_indices)	Resample an image onto the current flagged array via a kernel.
level([robust, weights])	Remove the mean or median value from the data.
max()	Return the maximum value of the valid data.
mean([weights])	Return the mean of the data.
median([weights])	Return the median of the data.
mem_correct(model, noise, lg_multiplier)	Apply a maximum entropy correction given a model.
merge(data)	Add additional data onto the end of this data.
min()	Return the minimum value of the valid data.
paste(source)	Paste data from another FitsData onto the data array.
resample_from(image, to_indices[, kernel, ...])	Resample an image onto given indices of this map.
restrict_range(keep_range)	Discard data values outside of a certain range.
rms([robust])	Return the RMS of the data.
scale(factor[, indices])	Scale the data by a given factor.
select(fraction)	Return the value representative of the fraction of the sorted data.
set_blanking_level(value)	Set the blanking level for the map data.
set_data(data[, change_type])	Set the data of the flagged array.
set_data_shape(shape)	Set the shape of the data array.
set_validating_flags(flag)	Set the validating flags (flags that are considered invalid).
smooth(beam_map[, reference_index, weights])	Smooth the data gy a given kernel.
subtract(value[, indices, factor])	Subtract a value or FlaggedData from the data array.
sum()	Return the sum of the data.
validate([validator])	Discard all invalid data.
value_at(index[, degree, reduce_degrees])	Return the data value at a given index.
variance([robust])	Return the variance of the data.

Attributes Documentation

blanking_value

Return the blanking value for the array.

The blanking value is that which defines an invalid value.

Returns:

blanking_valueint or float or None

data

Return the data array.

Returns:

datanp.ndarray or None

data_range

Get the data range.

Returns:

Range

nan_blanking

Return whether the blanking value is NaN.

Returns:

bool

ndim

Return the number of dimensions in the map image data.

Returns:

int

shape

Return the shape of the data image array.

Returns:

tuple (int)

size

Return the size of the data image array.

Returns:

int

valid

Return a boolean mask array of valid data elements.

Valid elements are neither NaN, set to the blanking value, or flagged as the validating_flags.

Returns:

numpy.ndarray (bool)

A boolean mask where True indicates a valid element.

valid_data

Return valid data.

Returns:

valid_datanumpy.ndarray (self.dtype)

The valid data of shape (n_valid,).

Methods Documentation

abs_sum()

Return the sum of the absolute data values.

Returns:

float

add(values, indices=None, factor=None)

Add a value to the data array.

Parameters:

valuesint or float or FlaggedArray

The value to add.

indicesnumpy.ndarray (int or bool), optional

The indices on self.data for which to add value. If a boolean mask is supplied, this assumes value is the same shape as self.data and indicates valid elements to add.

factorint or float, optional

An optional factor to scale the data by.

Returns:

None

argmax()

Return the index of the maximum value.

Returns:

int or tuple (int)

argmaxdev()

Return the index of maximum (absolute) deviation from zero.

Returns:

int or tuple (int)

argmin()

Return the index of the minimum value.

Returns:

int or tuple (int)

check_equal_contents(other)

Check the data contents of this data are equal with another.

Parameters:

otherFitsData

Returns:

equalbool

clear(indices=None)

Clear flags and set data to zero.

Parameters:

indicestuple (numpy.ndarray (int)) or numpy.ndarray (bool), optional

The indices to discard. Either supplied as a boolean mask of shape (self.data.shape).

Returns:

None

copy()

Return a copy of the FlaggedArray.

Returns:

FlaggedArray

count_flags(flag=None)

Count the number of flagged elements.

Parameters:

flagstr or int or enum.Enum

The flag to count.

Returns:

int

count_points()

Return the number of valid points.

Returns:

int

count_valid_points()

Return the number of valid data points in the array.

Returns:

int

crop(ranges)

Crop the array to the required dimensions.

Parameters:

rangesnumpy.ndarray (int,)

The ranges to set crop the data to. Should be of shape (n_dimensions, 2) where ranges[0, 0] would give the minimum crop limit for the first dimension and ranges[0, 1] would give the maximum crop limit for the first dimension. In this case, the ‘first’ dimension is in numpy format. i.e., (y, x) for a 2-D array. Also note that the upper crop limit is inclusive so a range of (0, 3) includes indices [0, 1, 2, 3].

Returns:

None

default_blanking_value()

Return a default blanking value type for the current data type.

Returns:

valuefloat or int or bool or complex

The default blanking value.

delete_indices(indices_or_mask)

Completely deletes data elements.

Actual indices should be passed in. To delete based on fixed index values, please convert first using find_fixed_indices.

Parameters:

indices_or_maskint or list or numpy.ndarray of (bool or int)

The indices to delete, or a boolean mask where True marks an element for deletion.

Returns:

None

despike(threshold, noise_weight=None)

Discard spikes whose significance is above a given threshold.

Parameters:

thresholdfloat

noise_weightnumpy.ndarray (float), optional

Optional noise weights.

Returns:

None

destroy()

Destroy the image data.

Returns:

None

direct_resample_from(image, to_indices)

Resample an image onto the current flagged array.

Parameters:

imageFlaggedArray or numpy.ndarray (float)

to_indicesnumpy.ndarray (float or int)

An array of shape (n_dimensions, self.shape). Indices should be supplied using numpy ordering (z, y, x,…), not (x, y, z…) FITS style.

Returns:

None

discard(indices=None)

Set the flags for discarded indices to DISCARD and data to zero.

Parameters:

indicestuple (numpy.ndarray (int)) or numpy.ndarray (bool), optional

The indices to discard. Either supplied as a boolean mask of shape (self.data.shape).

Returns

——-

None

discard_flag(flag, criterion=None)

Clear all data flagged with the given flag.

All data matching the given flag and criteria will be cleared according to FlaggedArray.clear() method. This generally should result in all matching elements having their flag and data values set to zero.

Parameters:

flagint or str or ChannelFlagTypes

The flag to discard_flag.

criterionstr, optional

One of {‘DISCARD_ANY’, ‘DISCARD_ALL’, ‘DISCARD_MATCH’, ‘KEEP_ANY’, ‘KEEP_ALL’, ‘KEEP_MATCH’}. *_ANY refers to any flag that is not zero (unflagged). *_ALL refers to any flag that contains flag, and *_MATCH refers to any flag that exactly matches flag. The default (None), uses DISCARD_ANY if flag is None, and DISCARD_ALL otherwise.

Returns:

None

discard_min_neighbors(min_neighbors)

Discard points with insufficient neighbors.

Parameters:

min_neighborsint

Returns:

None

discard_range(discard_range)

Discard data values in a certain range.

Parameters:

discard_rangeRange

Returns:

None

fast_smooth(beam_map, steps, reference_index=None, weights=None)

Smooth using the fast method.

Parameters:

beam_mapnumpy.ndarray (float)

The kernel to convolve with.

stepsnumpy.ndarray (int)

The size of the steps in each dimension.

reference_indexnumpy.ndarray (float)

The reference index of the beam map center. The default is (beam_map.shape - 1) / 2.0.

weightsnumpy.ndarray (float)

Weights the same shape as beam map.

Returns:

None

fill(value, indices=None)

Fill the data array with a given value.

Parameters:

valueint or float

The value to fill.

indicestuple (numpy.ndarray (int)) or numpy.ndarray (bool), optional

The indices to discard. Either supplied as a boolean mask of shape (self.data.shape).

Returns:

None

find_fixed_indices(fixed_indices, cull=True)

Returns the actual indices given fixed indices.

The fixed indices are those that are initially loaded. Returned indices are their locations in the data arrays.

Parameters:

fixed_indicesint or np.ndarray (int)

The fixed indices.

cullbool, optional

If True, do not include fixed indices not found in the result. If False, missing indices will be replaced by -1.

Returns:

indicesnumpy.ndarray (int) or tuple (numpy.ndarray (int))

The indices of fixed_indices in the data arrays. A tuple will be returned, in the case where we are examining more than one dimension.

get_cropped(ranges)

Return a copy of the cropped Flagged Array.

Parameters:

rangesnumpy.ndarray (int,)

The ranges to set crop the data to. Should be of shape (n_dimensions, 2) where ranges[0, 0] would give the minimum crop limit for the first dimension and ranges[0, 1] would give the maximum crop limit for the first dimension. In this case, the ‘first’ dimension is in numpy format. i.e., (y, x) for a 2-D array. Also note that the upper crop limit is not inclusive so a range of (0, 3) includes indices [0, 1, 2] but not 3.

Returns:

FlaggedArray

get_fast_smoothed(beam_map, steps, reference_index=None, weights=None, get_weights=False)

Return smoothed values using the fast method.

Parameters:

beam_mapnumpy.ndarray (float)

The kernel to convolve with.

stepsnumpy.ndarray (int)

The size of the steps in each dimension.

reference_indexnumpy.ndarray (float)

The reference index of the beam map center. The default is (beam_map.shape - 1) / 2.0.

weightsnumpy.ndarray (float)

Weights the same shape as beam map.

get_weightsbool, optional

If True, calculate the smoothed weights in addition to the smoothed data.

Returns:

smoothed_data, [smoothed_weights]numpy.ndarray, [numpy.ndarray]

The smoothed data and weights. Will only return smoothed data if get_weights is False.

get_index_range()

Return the index ranges of valid points.

Returns:

rangesnumpy.ndarray (int)

A range for each dimension or shape (n_dimensions, 2) giving the minimum and maximum range in each dimension. Note that this is numpy dimension ordering (y-range = ranges[0], x-range = ranges[1]). Also note that the upper range is returned such that the real upper index is included in any slice operation. i.e., max = real max index + 1.

get_indices(indices)

Return selected data for given indices.

Parameters:

indiceslist or int or numpy.ndarray (bool or int)

The indices to extract.

Returns:

FlaggedData

get_info()

Return a string descriptors of the array.

Returns:

list of str

get_neighbor_kernel()

Return a neighbor kernel.

The neighbor kernel contains the inverse square distance to the center pixel. The center pixel will always be set to zero. E.g., in 2D:

kernel = [[0.5, 1. , 0.5],

[1. , 0. , 1. ], [0.5, 1. , 0.5]]

Returns:

numpy.ndarray (float)

get_neighbors()

Return the number of valid neighbors for each point, including itself.

Partial implementation of GetNeighborValidator.

Returns:

numpy.ndarray (int)

get_refined_peak_index(peak_index)

Get the peak index given a local peak.

Basically performs a quadratic fit on local neighborhood to determine the maximum.

Parameters:

peak_indextuple (int)

The peak index of the data array in (y, x) numpy format.

Returns:

refined_peak_indexnumpy.ndarray (float)

get_size_string()

Return the shape of the data array as a string.

Returns:

str

get_smoothed(beam_map, reference_index=None, weights=None)

Return smoothed data and weights.

Parameters:

beam_mapnumpy.ndarray (float)

reference_indexnumpy.ndarray (float or int), optional

weightsnumpy.ndarray (float), optional

Returns:

smoothed_data, smoothed_weightsnumpy.ndarray, numpy.ndarray

get_valid_smoothed(beam_map, reference_index=None, weights=None, get_weights=False)

Return smoothed data and weights, where invalid entries are zeroed.

Parameters:

beam_mapnumpy.ndarray (float)

The kernel to convolve with.

reference_indexnumpy.ndarray (float)

The reference index of the beam map center. The default is (beam_map.shape - 1) / 2.0.

weightsnumpy.ndarray (float)

Weights the same shape as beam map.

get_weightsbool, optional

If True, calculate the smoothed weights in addition to the smoothed data.

Returns:

smoothed_data, [smoothed_weights]numpy.ndarray, [numpy.ndarray]

The smoothed data and weights. Will only return smoothed data if get_weights is False.

index_of_max(sign=1, data=None)

Return the maximum value and index of maximum value.

Parameters:

signint or float, optional

If positive, find the maximum value in the array. If negative, find the minimum value in the array. If zero, find the maximum magnitude in the array.

datanumpy.ndarray (float), optional

The data array to examine. Default is self.data.

Returns:

maximum_value, maximum_indexfloat, int

insert_blanks(insert_indices)

Inserts blank frame data.

Actual indices should be passed in. To delete based on fixed index values, please convert first using find_fixed_indices.

Blank data are set to 0 in whatever unit is applicable.

Parameters:

insert_indicesint or list or numpy.ndarray of (bool or int)

The index locations to insert.

Returns:

None

is_valid()

Return a boolean mask array of valid data elements.

Valid elements are neither NaN, set to the blanking value, or flagged as the validating_flags.

Returns:

numpy.ndarray (bool)

A boolean mask where True indicates a valid element.

kernel_resample_from(image, kernel, to_indices, kernel_reference_index=None, weights=None)

Resample an image onto the current flagged array via a kernel.

Parameters:

imageFlaggedArray or numpy.ndarray (float)

kernelnumpy.ndarray (float)

to_indicesnumpy.ndarray (float or int)

An array of shape (n_dimensions, self.shape or self.size)

kernel_reference_indexnumpy.ndarray (int or float)

The reference index of the kernel defining center of the convolution operation. The default is (kernel.shape - 1) / 2.

weightsnumpy.ndarray (float or int)

The data weights for resampling. Should be the same shape as image.

Returns:

None

level(robust=True, weights=None)

Remove the mean or median value from the data.

Parameters:

robustbool, optional

If True, remove the median value. Otherwise, remove the mean value.

weightsnumpy.ndarray (float), optional

If provided, perform a weighted mean. Must be an array of shape (self.shape).

Returns:

None

max()

Return the maximum value of the valid data.

Returns:

float or int

mean(weights=None)

Return the mean of the data.

Parameters:

weightsnumpy.ndarray (float), optional

If provided, perform a weighted mean. Must be an array of shape (self.shape). If not provided, the weight assigned to each datum is 1.

Returns:

mean, mean_weight2-tuple (float)

median(weights=None)

Return the median of the data.

Parameters:

weightsnumpy.ndarray (float), optional

If provided, perform a weighted mean. Must be an array of shape (self.shape).

Returns:

median, median_weight2-tuple (float)

mem_correct(model, noise, lg_multiplier)

Apply a maximum entropy correction given a model.

Parameters:

modelnumpy.ndarray or None

The model from which to base MEM correction. Should be of shape (self.shape).

noisenumpy.ndarray

The associated noise values. Should be of shape (self.shape).

lg_multiplierfloat

The Lagrange multiplier (lambda) for the MEM correction.

Returns:

None

merge(data)

Add additional data onto the end of this data.

Parameters:

dataFlaggedData

Returns:

None

min()

Return the minimum value of the valid data.

Returns:

float or int

paste(source)

Paste data from another FitsData onto the data array.

Parameters:

sourceFlaggedArray

Returns:

None

resample_from(image, to_indices, kernel=None, kernel_reference_index=None, weights=None)

Resample an image onto given indices of this map.

Parameters:

imageFlaggedArray or numpy.ndarray (float)

The image to resample.

to_indicesnumpy.ndarray (float or int)

An array of shape (n_dimensions, self.shape or self.size) specifying which image pixels belong to the resampled map. I.e., if this were a 2-D array and the pixel at (x, y) = (2, 2) corresponds to the image at pixel (3.3, 4.4) then to_indices[:, 2, 2] = [4.4, 3.3] (reversed because numpy).

kernelnumpy.ndarray (float), optional

The kernel used to perform the resampling. If supplied, the result will be smoothed accordingly.

kernel_reference_indexnumpy.ndarray (int), optional

If a kernel is supplied, specifies the center pixel of the kernel to be used during kernel convolution. The default is (kernel.shape - 1) / 2.

weightsnumpy.ndarray (int or float), optional

An optional weighting array used for kernel convolution.

Returns:

None

restrict_range(keep_range)

Discard data values outside of a certain range.

Parameters:

keep_rangeRange

Returns:

None

rms(robust=True)

Return the RMS of the data.

Parameters:

robustbool, optional

If True, use the robust (median) method to determine the RMS. Otherwise, use the mean.

Returns:

float

scale(factor, indices=None)

Scale the data by a given factor.

Parameters:

factorint or float

The factor to scale by.

indicestuple (numpy.ndarray (int)) or numpy.ndarray (bool), optional

The indices to discard. Either supplied as a boolean mask of shape (self.data.shape).

Returns:

None

select(fraction)

Return the value representative of the fraction of the sorted data.

Parameters:

fractionfloat

The fraction of the sorted data array to return. Must be between 0 and 1.

Returns:

valueint or float

set_blanking_level(value)

Set the blanking level for the map data.

The blanking level is the value for all null values. If a new blanking level is set, and the data exists, all previously set blanked values will be set to the new level.

Parameters:

valuefloat or int or None

Returns:

None

set_data(data, change_type=False)

Set the data of the flagged array.

All flags are set to zero.

Parameters:

datanumpy.ndarray or FlaggedArray

change_typebool, optional

If True, change the data type to that of the data.

Returns:

None

set_data_shape(shape)

Set the shape of the data array.

Parameters:

shapetuple (int)

Returns:

None

set_validating_flags(flag)

Set the validating flags (flags that are considered invalid).

Parameters:

flagint or str of enum.Enum

Returns:

None

smooth(beam_map, reference_index=None, weights=None)

Smooth the data gy a given kernel.

Parameters:

beam_mapnumpy.ndarray (float)

reference_indexnumpy.ndarray (float or int), optional

weightsnumpy.ndarray (float), optional

Returns:

None

subtract(value, indices=None, factor=None)

Subtract a value or FlaggedData from the data array.

Parameters:

valueint or float or FlaggedArray

The value to add.

indicestuple (numpy.ndarray (int)) or numpy.ndarray (bool), optional

The indices to discard. Either supplied as a boolean mask of shape (self.data.shape).

factorint or float, optional

An optional factor to scale the data by.

Returns:

None

sum()

Return the sum of the data.

Returns:

float

validate(validator=None)

Discard all invalid data.

Parameters:

validatorValidator, optional

An optional object or function that can take FlaggedArray as an argument and perform the validation.

Returns:

None

value_at(index, degree=3, reduce_degrees=False)

Return the data value at a given index.

Parameters:

indexnumpy.ndarray (int or float)

An array of shape (n_dimensions,). Should be supplied in (y, x) order (numpy).

degreeint

The spline degree to fit.

reduce_degreesbool, optional

If True, allow the spline fit to reduce the number of degrees in cases where there are not enough points available to perform the spline fit of degree. If False, a ValueError will be raised if such a fit fails.

Returns:

float

variance(robust=True)

Return the variance of the data.

The variance is given as:

var = func(d[valid]^2)

where d is the data and func is median(x)/0.454937 if robust is True and mean(x) otherwise.

Parameters:

robustbool, optional

If True, use the robust (median) method to determine the variance. Otherwise, use the mean.

Returns:

float