hats.pixel_math.sparse_histogram

hats.pixel_math.sparse_histogram#

Sparse 1-D histogram of healpix pixel counts.

Classes#

`SparseHistogram`	Wrapper around a naive sparse array, that is just non-zero indexes and counts.
`HistogramAggregator`	Utility for aggregating sparse histograms.

Functions#

supplemental_count_histogram(mapped_pixels, ...)

Specialized method for getting a histogram of some supplemental count,

Module Contents#

class SparseHistogram(indexes, counts, order)[source]#

Wrapper around a naive sparse array, that is just non-zero indexes and counts.

e.g. for a dense 1-d numpy histogram of order 0, you might see:

[0, 4, 0, 0, 0, 0, 0, 0, 9, 0, 0]

There are only elements at [1, 8], and they have respective values [4, 9]. You would create the sparse histogram like:

SparseHistogram([1, 8], [4, 9], 0)

indexes[source]#

counts[source]#

order[source]#

to_array()[source]#

Convert the sparse array to a dense numpy array.

Returns:

np.ndarray: dense 1-d numpy array.

to_file(file_name)[source]#

Persist the sparse array to disk.

NB: this saves as a sparse array, and so will likely have lower space requirements than saving the corresponding dense 1-d numpy array.

Parameters:

file_namepath-like: intended file to save to

to_dense_file(file_name)[source]#

Persist the DENSE array to disk as a numpy array.

Parameters:

file_namepath-like: intended file to save to

classmethod from_file(file_name)[source]#

Read sparse histogram from a file.

Parameters:

file_namepath-like: intended file to save read from

Returns:

SparseHistogram: new sparse histogram

__eq__(value)[source]#

__str__()[source]#

class HistogramAggregator(order)[source]#

Utility for aggregating sparse histograms.

order[source]#

full_histogram[source]#

add(other)[source]#

Add in another sparse histogram, updating this wrapper’s array.

Parameters:

otherSparseHistogram: the wrapper containing the addend

to_sparse()[source]#: Return a SparseHistogram, based on non-zero values in this aggregation.

supplemental_count_histogram(mapped_pixels, supplemental_count, highest_order)[source]#

Specialized method for getting a histogram of some supplemental count, collating according to the pixels in the first argument.

Typically used during import, when you wish to partition according to some supplemental data, such as in-memory size, or length of a nested column.

Parameters:

mapped_pixelsarray_like of int: 1-D array of healpix pixel IDs. Values will be aggregated by pixel to produce the row-count histogram.
supplemental_countNone or array_like of int: Optional 1-D array of supplemental counts (for example per-row memory sizes or nested-column lengths). If None, no supplemental histogram will be produced and the returned second element will be None.
highest_orderint: Healpix order used for the histograms.

Returns:

tuple: (row_count_histo, supplemental_count_histo) where both elements are SparseHistogram. row_count_histo contains counts of rows per pixel. supplemental_count_histo contains the sum of the supplemental counts per pixel, or None if supplemental_count was None.