hats.io.paths

hats.io.paths#

Methods for creating partitioned data paths

Attributes#

PARTITION_ORDER

PARTITION_DIR

PARTITION_PIXEL

MARGIN_ORDER

MARGIN_DIR

MARGIN_PIXEL

HIVE_COLUMNS

DATASET_DIR

PARTITION_INFO_FILENAME

PARTITION_JOIN_INFO_FILENAME

PARQUET_METADATA_FILENAME

PARQUET_COMMON_METADATA_FILENAME

POINT_MAP_FILENAME

Functions#

pixel_directory(→ upath.UPath)

Create path pointer for a pixel directory. This will not create the directory.

get_healpix_from_path(...)

Find the pixel_order and pixel_number from a string like the

dict_to_query_urlparams(→ str)

Converts a dictionary to a url query parameter string

pixel_catalog_file(→ upath.UPath)

Create path pointer for a pixel catalog file. This will not create the directory

get_partition_info_pointer(→ upath.UPath)

Get file pointer to partition_info.csv metadata file

get_common_metadata_pointer(→ upath.UPath)

Get file pointer to _common_metadata parquet metadata file

get_parquet_metadata_pointer(→ upath.UPath)

Get file pointer to _metadata parquet metadata file

get_point_map_file_pointer(→ upath.UPath)

Get file pointer to point_map.fits FITS image file.

get_partition_join_info_pointer(→ upath.UPath)

Get file pointer to partition_join_info.csv association metadata file

Module Contents#

PARTITION_ORDER = 'Norder'[source]#
PARTITION_DIR = 'Dir'[source]#
PARTITION_PIXEL = 'Npix'[source]#
MARGIN_ORDER = 'margin_Norder'[source]#
MARGIN_DIR = 'margin_Dir'[source]#
MARGIN_PIXEL = 'margin_Npix'[source]#
HIVE_COLUMNS[source]#
DATASET_DIR = 'dataset'[source]#
PARTITION_INFO_FILENAME = 'partition_info.csv'[source]#
PARTITION_JOIN_INFO_FILENAME = 'partition_join_info.csv'[source]#
PARQUET_METADATA_FILENAME = '_metadata'[source]#
PARQUET_COMMON_METADATA_FILENAME = '_common_metadata'[source]#
POINT_MAP_FILENAME = 'point_map.fits'[source]#
pixel_directory(catalog_base_dir: str | pathlib.Path | upath.UPath | None, pixel_order: int, pixel_number: int | None = None, directory_number: int | None = None) → upath.UPath[source]#

Create path pointer for a pixel directory. This will not create the directory.

One of pixel_number or directory_number is required. The directory name will take the HiPS standard form of:

<catalog_base_dir>/dataset/Norder=<pixel_order>/Dir=<directory number>

Where the directory number is calculated using integer division as:

(pixel_number/10000)*10000
Parameters:

  • catalog_base_dir (UPath) – base directory of the catalog (includes catalog name)

  • pixel_order (int) – the healpix order of the pixel

  • directory_number (int) – directory number

  • pixel_number (int) – the healpix pixel

Returns:

UPath directory name

get_healpix_from_path(path: str) → hats.pixel_math.healpix_pixel.HealpixPixel[source]#

Find the pixel_order and pixel_number from a string like the following:

Norder=<pixel_order>/Dir=<directory number>/Npix=<pixel_number>.parquet

NB: This expects the format generated by the pixel_catalog_file method

Parameters:

path (str) – path to parse

Returns:

Constructed HealpixPixel object representing the pixel in the path. INVALID_PIXEL if the path doesn’t match the expected pattern for any reason.

dict_to_query_urlparams(query_params: dict | None = None) → str[source]#

Converts a dictionary to a url query parameter string

Parameters:

query_params (dict) – dictionary of query parameters.

Returns:

query parameter string to append to a url

pixel_catalog_file(catalog_base_dir: str | pathlib.Path | upath.UPath | None, pixel: hats.pixel_math.healpix_pixel.HealpixPixel, query_params: dict | None = None, npix_suffix: str = '.parquet') → upath.UPath[source]#

Create path pointer for a pixel catalog file. This will not create the directory or file.

The catalog file name will take the HiPS standard form of:

<catalog_base_dir>/Norder=<pixel_order>/Dir=<directory number>/Npix=<pixel_number>.parquet

Where the directory number is calculated using integer division as:

(pixel_number/10000)*10000
Parameters:

  • catalog_base_dir (UPath) – base directory of the catalog (includes catalog name)

  • pixel (HealpixPixel) – the healpix pixel to create path to

  • query_params (dict) – Params to append to URL. Ex: {‘cols’: [‘ra’, ‘dec’], ‘fltrs’: [‘r>=10’, ‘g<18’]}

Returns:

string catalog file name

get_partition_info_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to partition_info.csv metadata file

Parameters:

catalog_base_dir – pointer to base catalog directory

Returns:

File Pointer to the catalog’s partition_info.csv file

get_common_metadata_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to _common_metadata parquet metadata file

Parameters:

catalog_base_dir – pointer to base catalog directory

Returns:

File Pointer to the catalog’s _common_metadata file

get_parquet_metadata_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to _metadata parquet metadata file

Parameters:

catalog_base_dir – pointer to base catalog directory

Returns:

File Pointer to the catalog’s _metadata file

get_point_map_file_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to point_map.fits FITS image file.

Parameters:

catalog_base_dir – pointer to base catalog directory

Returns:

File Pointer to the catalog’s point_map.fits FITS image file.

get_partition_join_info_pointer(catalog_base_dir: str | pathlib.Path | upath.UPath) → upath.UPath[source]#

Get file pointer to partition_join_info.csv association metadata file

Parameters:

catalog_base_dir – pointer to base catalog directory

Returns:

File Pointer to the catalog’s partition_join_info.csv association metadata file

Contents