hats.catalog

hats.catalog#

Catalog data wrappers

Submodules#

Classes#

`AssociationCatalog`	A HATS Catalog for enabling fast joins between two HATS catalogs
`Catalog`	A HATS Catalog with data stored in a HEALPix Hive partitioned structure
`CatalogCollection`	A collection of HATS Catalog with data stored in a HEALPix Hive partitioned structure
`CatalogExtension`	An extension of a HATS Catalog, holding additional columns for its rows
`CatalogType`	Enum for possible types of catalog
`CollectionProperties`	Container class for catalog metadata
`Dataset`	A base HATS dataset that contains a properties file and the data contained in parquet files
`ExtensionProperties`	Container class for the metadata of a catalog extension.
`HatsProperties`	Container class for catalog metadata
`TableProperties`	Container class for catalog metadata
`IndexCatalog`	An index into HATS Catalog for enabling fast lookups on non-spatial values.
`MapCatalog`	A HATS table to represent non-point-source data in a continuous map.
`MarginCatalog`	A HATS Catalog used to contain the 'margin' of another HATS catalog.
`PartitionInfo`	Container class for per-partition info.

Package Contents#

class AssociationCatalog(catalog_info: hats.catalog.dataset.table_properties.TableProperties, pixels: hats.catalog.partition_info.PartitionInfo | hats.pixel_tree.pixel_tree.PixelTree | list[hats.pixel_math.HealpixPixel], catalog_path: str | pathlib.Path | upath.UPath | None = None, moc: mocpy.MOC | None = None, schema: pyarrow.Schema | None = None, snapshot: hats.catalog.catalog_snapshot.CatalogSnapshot | None = None, generate_snapshot: bool = False)[source]#

Bases: hats.catalog.healpix_dataset.healpix_dataset.HealpixDataset

A HATS Catalog for enabling fast joins between two HATS catalogs

class Catalog(catalog_info: hats.catalog.dataset.table_properties.TableProperties, pixels: hats.catalog.partition_info.PartitionInfo | hats.pixel_tree.pixel_tree.PixelTree | list[hats.pixel_math.HealpixPixel], catalog_path: str | pathlib.Path | upath.UPath | None = None, moc: mocpy.MOC | None = None, schema: pyarrow.Schema | None = None, snapshot: hats.catalog.catalog_snapshot.CatalogSnapshot | None = None, generate_snapshot: bool = False)[source]#

Bases: hats.catalog.healpix_dataset.healpix_dataset.HealpixDataset

A HATS Catalog with data stored in a HEALPix Hive partitioned structure

Catalogs of this type are partitioned spatially, contain partition_info metadata specifying the pixels in Catalog, and on disk conform to the parquet partitioning structure Norder=/Dir=/Npix=.parquet

generate_negative_tree_pixels() → list[hats.pixel_math.HealpixPixel][source]#

Get the leaf nodes at each healpix order that have zero catalog data.

For example, if an example catalog only had data points in pixel 0 at order 0, then this method would return order 0’s pixels 1 through 11. Used for getting full coverage on margin caches.

Returns:

list[HealpixPixel]: List of HealpixPixels representing the ‘negative tree’ for the catalog.

class CatalogCollection(collection_path: upath.UPath, collection_properties: hats.catalog.dataset.collection_properties.CollectionProperties, main_catalog: hats.catalog.catalog.Catalog, storage_options: dict | None = None)[source]#

A collection of HATS Catalog with data stored in a HEALPix Hive partitioned structure

Catalogs of this type are described by a collection.properties file which specifies the underlying main catalog, margin catalog and index catalog paths. These catalogs are stored at the root of the collection, each in its separate directory:

catalog_collection/
├── main_catalog/
├── margin_catalog/
├── index_catalog/
├── extension_catalog.properties
├── collection.properties

Margin and index catalogs are optional but there could also be multiple of them. The catalogs used by default are specified in the collection.properties file in the default_margin and default_index keywords. Extensions are optional too, and each is described by an <extension>.properties file, listed by alias in the all_extensions keyword. That file usually sits at the root of the collection, but it may live elsewhere, e.g. remotely, and be listed by its absolute path.

collection_path#

collection_properties#

main_catalog#

storage_options = None#

property main_catalog_dir: upath.UPath#: Path to the main catalog directory

property all_margins: list[str] | None#: The list of margin catalog names in the collection

property default_margin: str | None#: The name of the default margin

property default_margin_catalog_dir: upath.UPath | None#: Path to the default margin catalog directory

property all_extensions: dict[str, str] | None#: The mapping of extensions in the collection

get_extension_path(extension_alias: str) → upath.UPath[source]#: Path to the properties file of an extension in the collection

property all_indexes: dict[str, str] | None#: The mapping of indexes in the collection

property default_index_field: str | None#: The name of the default index field

property default_index_catalog_dir: upath.UPath | None#: Path to the default index catalog directory

get_index_dir_for_field(field_name: str | None = None) → upath.UPath | None[source]#: Path to the field’s index catalog directory

get_healpix_pixels() → list[hats.pixel_math.HealpixPixel][source]#: The list of HEALPix pixels of the main catalog

plot_pixels(**kwargs)[source]#

Create a visual map of the pixel density of the collection’s main catalog.

Parameters:

**kwargs: Additional args to pass to hats.inspection.visualize_catalog.plot_healpix_map

plot_density(**kwargs)[source]#

Create a visual map of the density of input points of the collection’s main catalog.

Parameters:

**kwargs: Additional args to pass to hats.inspection.visualize_catalog.plot_density

Raises:

ValueError: if the main catalog is not on disk.

plot_moc(**kwargs)[source]#

Create a visual map of the coverage of the collection’s main catalog.

Parameters:

**kwargs: Additional args to pass to hats.inspection.visualize_catalog.plot_moc

get_margin_thresholds() → dict[str, float][source]#

Get the margin thresholds for all margin catalogs in the collection.

Returns:

dict[str, float]: A dictionary mapping margin catalog names to their threshold values.

Raises:

ValueError: if a catalog listed as a margin catalog is not configured as a margin catalog.

classmethod resolve_inner_path(collection_path, path, storage_options: dict | None = None) → upath.UPath[source]#

Convenience method to find the path of a related catalog, either relative or absolute.

Parameters:

collection_path: UPath: fully-specified path to the collection root
path: UPath | Path | str: location of the related catalog
storage_options: dict: dictionary of storage options for the connection to the related catalog

Returns:

UPath: fully-specified path to the related catalog.

class CatalogExtension(config_path: upath.UPath, extension_info: hats.catalog.dataset.extension_properties.ExtensionProperties, catalog: hats.catalog.catalog.Catalog | hats.catalog.catalog_collection.CatalogCollection, storage_options: dict | None = None)[source]#

An extension of a HATS Catalog, holding additional columns for its rows

Extensions are described by an <extension>.properties file, which references the primary catalog, points to the extension data, and names the columns to join them on. The extension data is a HATS catalog of its own (or a collection, when it has margins), which is valid and can be read on its own.

The reference to the extension data is resolved against the directory holding the <extension>.properties file, unless it is absolute, so the file can point to data hosted elsewhere. The file sits at the root of a collection, and names that collection as its primary, unless it gives the primary by absolute path (e.g. a remote one):

catalog_collection/
├── main_catalog/
├── extension_catalog.properties
├── collection.properties

config_path#

extension_info#

storage_options = None#

catalog#

property join_catalog_dir: upath.UPath#: Path to the extension data

class CatalogType[source]#

Bases: str, enum.Enum

Enum for possible types of catalog

OBJECT = 'object'#

SOURCE = 'source'#

ASSOCIATION = 'association'#

INDEX = 'index'#

MARGIN = 'margin'#

MAP = 'map'#

EXTENSION = 'extension'#

classmethod all_types()[source]#: Fetch a list of all catalog types

class CollectionProperties(/, **data: Any)[source]#

Bases: hats.catalog.dataset.hats_properties.HatsProperties

Container class for catalog metadata

name: str#

hats_primary_table_url: str#: Reference to object catalog. Relevant for nested, margin, association, and index.

all_margins: Annotated[list[str] | None, Field(default=None), PlainSerializer(HatsProperties.serialize_list_as_space_delimited_list), BeforeValidator(HatsProperties.space_delimited_list)]#

default_margin: str | None#

all_indexes: Annotated[dict[str, str] | None, Field(default=None), PlainSerializer(HatsProperties.serialize_dict_as_space_delimited_list)]#

default_index: str | None#

all_extensions: Annotated[dict[str, str] | None, Field(default=None), PlainSerializer(HatsProperties.serialize_dict_as_space_delimited_list)]#: Extensions of this collection, each holding a set of additional columns, and described by an <extension>.properties file. Maps the alias of each extension to the path to that file, relative to the collection root or absolute.

classmethod space_delimited_dict(str_value: str, info: pydantic.ValidationInfo) → dict[str, str][source]#

Convert a space-delimited list string of key-value pairs into a python dict of strings.

Parameters:

str_value: str: a space-delimited list string
info: ValidationInfo: information about the field being validated

Returns:

dict[str, str]: a python dict of strings

Raises:

ValueError: if the string list has an odd number of elements (and so is not pairs of name and path)

check_default_margin_exists() → Self[source]#: Check that the default margin is in the list of all margins.

check_default_index_exists() → Self[source]#: Check that the default index is in the list of all indexes.

check_extension_paths() → Self[source]#: Check that each extension is listed by the path to its properties file.

classmethod read_from_dir(catalog_dir: str | pathlib.Path | upath.UPath) → Self[source]#

Read field values from a java-style properties file.

Parameters:

catalog_dir: str | Path | UPath: base directory of catalog.

Returns:

CollectionProperties: new object from the contents of a collection.properties file in the directory.

to_properties_file(catalog_dir: str | pathlib.Path | upath.UPath)[source]#

Write fields to a java-style properties file.

Parameters:

catalog_dir: str | Path | UPath: base directory of catalog.

class Dataset(catalog_info: hats.catalog.dataset.table_properties.TableProperties, catalog_path: str | pathlib.Path | upath.UPath | None = None, schema: pyarrow.Schema | None = None, snapshot: hats.catalog.catalog_snapshot.CatalogSnapshot | None = None, generate_snapshot: bool = False)[source]#

A base HATS dataset that contains a properties file and the data contained in parquet files

catalog_info#

catalog_name#

catalog_path = None#

catalog_base_dir = None#

schema = None#

snapshot = None#

property original_schema: pyarrow.Schema | None#: The original on-disk schema, before any column selection.

property on_disk: bool#: Is the catalog stored on disk?

property unmodified: bool#: Has the catalog been modified from its original on disk state?

aggregate_column_statistics(*, exclude_hats_columns: bool = True, exclude_columns: list[str] = None, include_columns: list[str] = None, warn_on_modified_catalog: bool = True)[source]#

Read footer statistics in parquet metadata, and report on global min/max values.

Parameters:

exclude_hats_columnsbool: exclude HATS spatial and partitioning fields from the statistics. Defaults to True.
exclude_columnslist[str]: additional columns to exclude from the statistics.
include_columnslist[str]: if specified, only return statistics for the column names provided. Defaults to None, and returns all non-hats columns.
warn_on_modified_catalogbool: if True, this method will warn if the catalog has been modified from its original on-disk state. Defaults to True.

Returns:

Dataframe: aggregated statistics.

per_pixel_statistics(*, exclude_hats_columns: bool = True, exclude_columns: list[str] | None = None, include_columns: list[str] | None = None, only_numeric_columns: bool = False, include_stats: list[str] | None = None, multi_index=False, per_row_group: bool = False, warn_on_modified_catalog: bool = True)[source]#: Read footer statistics in parquet metadata, and report on statistics about each pixel partition.

per_partition_statistics(*, exclude_hats_columns: bool = True, exclude_columns: list[str] = None, include_columns: list[str] = None, only_numeric_columns: bool = False, include_stats: list[str] = None, multi_index=False, per_row_group: bool = False, warn_on_modified_catalog: bool = True)[source]#

Read footer statistics in parquet metadata, and report on statistics about each pixel partition.

Parameters:

exclude_hats_columnsbool: exclude HATS spatial and partitioning fields from the statistics. Defaults to True.
exclude_columnslist[str]: additional columns to exclude from the statistics.
include_columnslist[str]: if specified, only return statistics for the column names provided. Defaults to None, and returns all non-hats columns.
include_statslist[str]: if specified, only return the kinds of values from list (min_value, max_value, null_count, row_count). Defaults to None, and returns all values.
multi_indexbool: should the returned frame be created with a multi-index, first on pixel, then on column name? Default is False, and instead indexes on pixel, with separate columns per-data-column and stat value combination. (Default value = False)
warn_on_modified_catalogbool: if True, this method will warn if the catalog has been modified from its original on-disk state. Defaults to True.

Returns:

Dataframe: all statistics.

class ExtensionProperties(/, **data: Any)[source]#

Bases: hats.catalog.dataset.hats_properties.HatsProperties

Container class for the metadata of a catalog extension.

An extension, described by an <extension>.properties file, holds additional columns for the rows of a primary catalog, and is joined to it on a pair of key columns.

The extension data is referenced by path. A relative path is resolved against the directory holding the <extension>.properties file, and an absolute path, local or remote, is used as it is, so the data does not need to be co-located with the file. The file sits at the root of a collection, and names it as the primary, unless the primary is given by absolute path, local or remote.

name: str#: Name of the extension, which names its <extension>.properties file.

catalog_type: Literal[hats.catalog.catalog_type.CatalogType.EXTENSION]#

primary_catalog: str#: Primary catalog that the extension is joined to, or the collection holding it. Either the name of the collection holding the <extension>.properties file, or an absolute path, local or remote.

primary_column: str#: Column name in the primary catalog to join on.

join_catalog: str#: Path to the extension data, a HATS catalog or a collection.

join_column: str#: Column name in the extension data that matches primary_column.

extension_columns: Annotated[list[str] | None, Field(default=None, alias='hats_ext_cols'), PlainSerializer(HatsProperties.serialize_list_as_space_delimited_list), BeforeValidator(HatsProperties.space_delimited_list)]#: The list of columns provided by the extension.

extension_join_style: Literal['left', 'inner'] | None#: The type of join to use when joining the extension to its primary catalog.

shares_primary_coordinates: bool | None#: Whether the extension holds the primary catalog’s RA and Dec columns, with their values unchanged.

extension_product_type: str | None#: Modality of the data that the extension stores.

to_properties_file(catalog_dir: str | pathlib.Path | upath.UPath)[source]#

Write fields to a java-style <extension>.properties file, named after the extension.

Parameters:

catalog_dir: str | Path | UPath: directory to write the file to.

class HatsProperties(/, **data: Any)[source]#

Bases: pydantic.BaseModel

Container class for catalog metadata

model_config#: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

classmethod space_delimited_list(str_value: str) → list[str][source]#

Convert a space-delimited list string into a python list of strings.

Parameters:

str_value: str: a space-delimited list string

Returns:

list[str]: a python list of strings

classmethod space_delimited_int_list(str_value: str | list[int]) → list[int][source]#

Convert a space-delimited list string into a python list of integers.

Parameters:

str_valuestr | list[int]: string representation of a list of integers, delimited by space, comma, or semicolon, or a list of integers.

Returns:

list[int]: a python list of integers

Raises:

ValueError: if any non-digit characters are encountered

classmethod serialize_list_as_space_delimited_list(str_list: Iterable[str]) → str[source]#

Convert a python list of strings into a space-delimited string.

Parameters:

str_list: Iterable[str]: a python list of strings

Returns:

str: a space-delimited string

classmethod serialize_dict_as_space_delimited_list(str_dict: dict[str, str]) → str[source]#

Convert a python list of strings into a space-delimited string.

Parameters:

str_dict: dict[str, str]: a python dict of strings

Returns:

str: a space-delimited string

explicit_dict(by_alias=False, exclude_none=True)[source]#

Create a dict, based on fields that have been explicitly set, and are not “extra” keys.

Parameters:

by_aliasbool: (Default value = False)
exclude_nonebool: (Default value = True)

Returns:

dict: all keys that are attributes of this class and not “extra”.

provenance_dict(by_alias=False, exclude_none=True)[source]#

Create a dict, based on known provenance fields, that have been explicitly set.

Parameters:

by_aliasbool: (Default value = False)
exclude_nonebool: (Default value = True)

Returns:

dict: all keys that are attributes of this class and in a list of known provenance fields.

extra_dict(by_alias=False, exclude_none=True)[source]#

Create a dict, based on fields that are “extra” keys.

Parameters:

by_aliasbool: (Default value = False)
exclude_nonebool: (Default value = True)

Returns:

dict: all keys that are not attributes of this class, e.g. “extra”.

__repr__()[source]#

__str__()[source]#: Friendly string representation based on named fields.

classmethod read_from_file(file_path: upath.UPath) → Self[source]#

Read field values from a java-style properties file.

Parameters:

file_path: UPath: path to properties file.

Returns:

CollectionProperties: new object from the contents of a collection.properties file in the directory.

to_properties_file_path(file_path: upath.UPath, **kwargs)[source]#

Write fields to a java-style properties file.

Parameters:

file_path: UPath: path to properties file.

static new_provenance_dict(path: str | pathlib.Path | upath.UPath | None = None, builder: str | None = None, **kwargs) → dict[source]#

Constructs the provenance properties for a HATS catalog.

Parameters:

path: str | Path | UPath | None: The path to the catalog directory.
builderstr | None: The name and version of the tool that created the catalog.
**kwargs: Additional properties to include/override in the dictionary.

Returns:

dict: A dictionary with properties for the HATS catalog.

class TableProperties(/, **data: Any)[source]#

Bases: hats.catalog.dataset.hats_properties.HatsProperties

Container class for catalog metadata

catalog_name: str#

catalog_type: hats.catalog.catalog_type.CatalogType#

total_rows: int | None#

ra_column: str | None#

dec_column: str | None#

default_columns: Annotated[list[str] | None, Field(default=None, alias='hats_cols_default'), PlainSerializer(HatsProperties.serialize_list_as_space_delimited_list), BeforeValidator(HatsProperties.space_delimited_list)]#: Which columns should be read from parquet files, when user doesn’t otherwise specify.

healpix_column: str | None#: Column name that provides a spatial index of healpix values at some fixed, high order. A typical value would be _healpix_29, but can vary.

healpix_order: int | None#: For the spatial index of healpix values in hats_col_healpix what is the fixed, high order. A typicaly value would be 29, but can vary.

primary_catalog: str | None#: Reference to object catalog. Relevant for nested, margin, association, and index.

margin_threshold: float | None#: Threshold of the pixel boundary, expressed in arcseconds.

primary_column: str | None#: Column name in the primary (left) side of join.

primary_column_association: str | None#: Column name in the association table that matches the primary (left) side of join.

join_catalog: str | None#: Catalog name for the joining (right) side of association.

join_column: str | None#: Column name in the joining (right) side of join.

join_column_association: str | None#: Column name in the association table that matches the joining (right) side of join.

assn_max_separation: float | None#: The maximum separation between two points in an association catalog, expressed in arcseconds.

contains_leaf_files: bool | None#: Whether or not the association catalog contains leaf parquet files.

indexing_column: str | None#: Column that we provide an index over.

extra_columns: Annotated[list[str] | None, Field(default=None, alias='hats_index_extra_column'), PlainSerializer(HatsProperties.serialize_list_as_space_delimited_list), BeforeValidator(HatsProperties.space_delimited_list)]#: Any additional payload columns included in index.

npix_suffix: str#: Suffix of the Npix partitions. In the standard HATS directory structure, this is '.parquet' because there is a single file in each Npix partition and it is named like 'Npix=313.parquet'. Other valid directory structures include those with the same single file per partition but which use a different suffix (e.g., 'npix_suffix' = '.parq' or '.snappy.parquet'), and also those in which the Npix partitions are actually directories containing 1+ files underneath (and then 'npix_suffix' = '/').

skymap_order: int | None#: Nested Order of the healpix skymap stored in the default skymap.fits.

skymap_alt_orders: Annotated[list[int] | None, Field(default=None, alias='hats_skymap_alt_orders'), PlainSerializer(HatsProperties.serialize_list_as_space_delimited_list), BeforeValidator(HatsProperties.space_delimited_int_list)]#: Nested Order (K) of the healpix skymaps stored in altnernative skymap.K.fits.

hats_max_rows: int | None#: Maximum number of rows in any partition of the catalog.

hats_max_bytes: int | None#: Maximum number of bytes in any partition of the catalog.

hats_estsize: int | None#: Estimated size of the catalog on disk, in kilobytes.

moc_sky_fraction: float | None#

check_required() → Self[source]#: Check that type-specific fields are appropriate, and required fields are set.

copy_and_update(**kwargs)[source]#

Create a validated copy of these table properties, updating the fields provided in kwargs.

Parameters:

**kwargs: values to update

Returns:

TableProperties: new instance of properties object

classmethod read_from_dir(catalog_dir: str | pathlib.Path | upath.UPath) → Self[source]#

Read field values from a java-style properties file.

Parameters:

catalog_dir: str | Path | UPath: path to a catalog directory.

Returns:

TableProperties: object created from the contents of a hats.properties file in the given directory

Raises:

FileNotFoundError: if there is no properties or hats.properties file in the directory

to_properties_file(catalog_dir: str | pathlib.Path | upath.UPath)[source]#

Write fields to a java-style properties file.

Parameters:

catalog_dir: str | Path | UPath: directory to write the file

class IndexCatalog(catalog_info: hats.catalog.dataset.table_properties.TableProperties, catalog_path: str | pathlib.Path | upath.UPath | None = None, schema: pyarrow.Schema | None = None, snapshot: hats.catalog.catalog_snapshot.CatalogSnapshot | None = None, generate_snapshot: bool = False)[source]#

Bases: hats.catalog.dataset.Dataset

An index into HATS Catalog for enabling fast lookups on non-spatial values.

Note that this is not a true “HATS Catalog”, as it is not partitioned spatially.

loc_partitions(ids) → list[hats.pixel_math.HealpixPixel][source]#

Find the set of partitions in the primary catalog for the ids provided.

Parameters:

ids: primary catalog for the ids

Returns:

list[HealpixPixel]: partitions of leaf parquet files in the primary catalog that may contain rows for the id values

class MapCatalog(catalog_info: hats.catalog.dataset.table_properties.TableProperties, pixels: hats.catalog.partition_info.PartitionInfo | hats.pixel_tree.pixel_tree.PixelTree | list[hats.pixel_math.HealpixPixel], catalog_path: str | pathlib.Path | upath.UPath | None = None, moc: mocpy.MOC | None = None, schema: pyarrow.Schema | None = None, snapshot: hats.catalog.catalog_snapshot.CatalogSnapshot | None = None, generate_snapshot: bool = False)[source]#

Bases: hats.catalog.healpix_dataset.healpix_dataset.HealpixDataset

A HATS table to represent non-point-source data in a continuous map.

class MarginCatalog(catalog_info: hats.catalog.dataset.table_properties.TableProperties, pixels: hats.catalog.partition_info.PartitionInfo | hats.pixel_tree.pixel_tree.PixelTree | list[hats.pixel_math.HealpixPixel], catalog_path: str | pathlib.Path | upath.UPath | None = None, moc: mocpy.MOC | None = None, schema: pyarrow.Schema | None = None, snapshot: hats.catalog.catalog_snapshot.CatalogSnapshot | None = None, generate_snapshot: bool = False)[source]#

Bases: hats.catalog.healpix_dataset.healpix_dataset.HealpixDataset

A HATS Catalog used to contain the ‘margin’ of another HATS catalog.

Catalogs of this type are used alongside a primary catalog, and contains the margin points for each HEALPix pixel - any points that are within a certain distance from the HEALPix pixel boundary. This is used to ensure spatial operations such as crossmatching can be performed efficiently while maintaining accuracy.

filter_by_moc(moc: mocpy.MOC) → Self[source]#

Filter the pixels in the margin catalog to only include the margin pixels that overlap with the moc

For the case of margin pixels, this includes any pixels whose margin areas may overlap with the moc. This is not always done with a high accuracy, but always includes any pixels that will overlap, and may include extra partitions that do not.

Parameters:

mocmocpy.MOC: the moc to filter by

Returns:

MarginCatalog: A new margin catalog with only the pixels that overlap or that have margin area that overlap with the moc. Note that we reset the total_rows to None, as updating would require a scan over the new pixel sizes.

class PartitionInfo(pixel_list: list[hats.pixel_math.healpix_pixel.HealpixPixel], catalog_base_dir: str = None)[source]#

Container class for per-partition info.

METADATA_ORDER_COLUMN_NAME = 'Norder'#

METADATA_PIXEL_COLUMN_NAME = 'Npix'#

pixel_list = []#

catalog_base_dir = None#

get_healpix_pixels() → list[hats.pixel_math.healpix_pixel.HealpixPixel][source]#

Get healpix pixel objects for all pixels represented as partitions.

Returns:

list[HealpixPixel]: List of HealpixPixel

get_highest_order() → int[source]#

Get the highest healpix order for the dataset.

Returns:

int: int representing highest order.

__len__()[source]#

The number of partitions.

Returns:

int: The number of partition pixels.

Write all partition data to CSV file.

If no paths are provided, the catalog base directory from the read_from_dir call is used.

Parameters:

partition_info_filestr | Path | UPath | None: path to where the partition_info.csv file will be written.
catalog_pathstr | Path | UPath | None: base directory for a catalog where the partition_info.csv file will be written.

Raises:

ValueError: if no path is provided, and could not be inferred.

classmethod read_from_dir(catalog_base_dir: str | pathlib.Path | upath.UPath | None) → PartitionInfo[source]#

Read partition info from a file within a hats directory.

This will look for a partition_info.csv file, and if not found, the partition info will be computed from the individual catalog files. Computing from catalog files will be slower: in internal testing, it took about half a second to compute from a catalog with ~40k partitions, versus a few milliseconds to read from the CSV file.

Parameters:

catalog_base_dirstr | Path | UPath | None: Path to the root directory of the catalog

Returns:

PartitionInfo: A PartitionInfo object with the data from the file

classmethod read_from_file(metadata_file: str | pathlib.Path | upath.UPath) → PartitionInfo[source]#

Read partition info from a _metadata file to create an object

Parameters:

metadata_filestr | Path | UPath: path to the _metadata file

Returns:

PartitionInfo: A PartitionInfo object with the data from the file

classmethod read_from_csv(partition_info_file: str | pathlib.Path | upath.UPath) → PartitionInfo[source]#

Read partition info from a partition_info.csv file to create an object

Parameters:

partition_info_filestr | Path | UPath: path to the partition_info.csv file

Returns:

PartitionInfo: A PartitionInfo object with the data from the file

as_dataframe()[source]#

Construct a pandas dataframe for the partition info pixels.

Returns:

pd.DataFrame: Pandas Dataframe with order, directory, and pixel info.

classmethod from_healpix(healpix_pixels: list[hats.pixel_math.healpix_pixel.HealpixPixel]) → PartitionInfo[source]#

Create a partition info object from a list of constituent healpix pixels.

Parameters:

healpix_pixels: list[HealpixPixel]: a list of constituent healpix pixels

Returns:

PartitionInfo: A PartitionInfo object with the same healpix pixels

calculate_fractional_coverage()[source]#: Calculate what fraction of the sky is covered by partition tiles.