Skip to content

ENH: Use a neutral format to have lossless interface with scipp, Astropy, Xarray #58217

@loco-philippe

Description

@loco-philippe

Feature Type

  • Adding new functionality to pandas

  • Changing existing functionality in pandas

  • Removing existing functionality in pandas

Problem Description

Each tool has a specific structure for processing multidimensional data with the following consequences:

  • interfaces dedicated to each tool,
  • partially processed data,
  • no unified representation of data structures

Feature Description

The proposed format (see jupyter notebook, github repository, PyPI package ) is based on the following principles:

  • neutral format available for tabular or multidimensional tools (e.g. Numpy, pandas, xarray, scipp, astropy),
  • taking into account a wide variety of data types as defined in NTV format,
  • high interoperability: reversible (lossless round-trip) interface with tabular or multidimensional tools,
  • reversible and compact JSON format,
  • Ease of sharing and exchanging multidimensional and tabular data,

Alternative Solutions

A first tool exists (NTV-pandas) but only deals with the JSON interface and the analysis of tabular structures. It would therefore be extended to multidimensional structures.

Additional Context

numpy/numpy#12481 (comment)
astropy/astropy#16286
pydata/xarray#8927
scipp/scipp#3422

Metadata

Metadata

Assignees

No one assigned

    Labels

    EnhancementIO DataIO issues that don't fit into a more specific label

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions