File formats¶

otswap reads and writes tables in two formats, chosen by the file’s extension: .fits, .fit and .fits.gz are FITS, anything else is ASCII. The C++ functions are in the namespace otswap::io, the Python ones in the module otswap.io; both run the same code, so a file written from Python is the same, byte for byte, as the one the C++ library writes from the same values.

Reading¶

otswap::io::read() and otswap.io.read() read selected columns of a table, as double, and others exactly as 64-bit integers.

FITS: the table in HDU 2; columns by name, case insensitively, scalar; an integer column must have an integer type.
ASCII: a column given by digits alone is a 0-based index; any other name is looked up in the last line starting with ### before the first data row, where otswap writes the names. Lines that are empty or start with the comment character (# by default) are skipped; fields are separated by the delimiter, blanks or tabs. An integer field is an optional - followed by digits.
NaN and infinities are returned as stored, in both formats, so a file written with NaN rows reads back with them.

The other inputs:

A mask: a full-sky HEALPix map, from a FITS file (otswap::Mask, otswap.Mask), in RING or NESTED order, or from an array in memory (otswap.Mask.from_array()). A pixel is observed when its value is greater than 0.
A bias table: b(z) at redshift nodes, read by otswap::io::readBiasTable() and otswap.io.read_bias_table(): FITS from the columns REDSHIFT and BIAS, ASCII from the first two columns.
A distance table: computed from a flat cosmology, or built from the caller’s own columns (otswap.DistanceTable.from_table()), for example read with read.

The two layouts¶

The layouts of every table otswap writes, with otswap::io::write(), otswap.io.write() or one of the writers below:

group File formats

The layouts of the tables io::write and the writers produce.

ASCII:

## PRODUCT = displacements / what the file holds
# tracX   comoving position of the tracer [Mpc/h]
# ...
#
###   tracX   tracY   tracZ ...
269.123 ...

one “## NAME = value / comment” line per keyword, when keywords are given;
one “# name description [unit]” line per column with a description or a unit, the names padded to a common width, and a line holding “#” alone after the keyword and description lines, when there are any;
the “###” line, with the column names;
one line per row, values separated by single spaces: ‘D’ with WriteOptions::precision significant digits, a NaN as nan whatever its sign, ‘J’ and ‘K’ as integers.

Every header line starts with ‘#’, so io::read, numpy.loadtxt and most table readers skip them.

FITS: a new file, its table in HDU 2 as a binary table of 1D, 1J and 1K columns; descriptions in TCOMMn, units in TUNITn, keywords in the header of HDU 2.

The writers of a Result, a RealSpaceCatalog and an MpsProfile write one row per input object, in input order, tracers left out of the reconstruction included, with NaN where nothing was computed, so a row’s index is the object’s index in the input. Angles are in degrees, right ascension in [0, 360). Each column has a description and a unit, and the header carries keywords: PRODUCT, what the file holds; OTSWAPV, the library version; GEOMETRY, box or lightcone; NOBJECTS; and those of the product, listed with each writer.

The writers¶

Each product has its writer, the same in both languages. The groups of columns and the keywords are listed with each.

C++	Python	Writes
`io::writeDisplacements`	`io.write_displacements`	the mean displacement of each tracer
`io::writeDisplacementField`	`io.write_displacement_field`	every realization, losslessly
`io::writeRealSpaceCatalog`	`io.write_real_space_catalog`	a catalogue moved to real space
`io::writeMpsProfile`	`io.write_mps_profile`	a mean particle separation profile

In Python the groups are strings, the C++ names in snake case (DisplacementGroup::TracerSky is "tracer_sky").

Displacements¶

void otswap::io::writeDisplacements(const std::string &file, const Result &result, const std::vector<DisplacementGroup> &groups = {})¶

Write the mean displacement field of a result, one row per tracer.

The default groups, with the columns of CosmoBolognaLib’s displacement file first, in its order, and its random columns named lagr:

box: Tracer, Lagrangian, Displacement, ValidRealizations (10 columns);
lightcone: TracerSky, LagrangianSky, Tracer, Lagrangian, Displacement, ValidRealizations, Selection (18 columns).

‘D’ values are written with 9 significant digits in ASCII. Keywords: NREC, SEED (the seed used), CONVERG; MPS for a box; ZCUTMIN and ZCUTMAX for the finite bounds of a redshift cut, MASK = 1 when a mask selected the objects, FILTNSID and MAXPIXCR when a mask filter was applied.

Parameters:

file – the file to write; its extension sets the format.
result – a box or lightcone result.
groups – the groups to write, in this order; empty for the default of the result’s geometry.

Throws:

Error – if a group is repeated or does not apply to the geometry, if a field a group reads does not have the size of its layout, or as io::write.

enum class otswap::io::DisplacementGroup¶

The column groups of writeDisplacements.

Values:

enumerator Index¶: index (K): the row of the tracer in the input, from 0

enumerator TracerSky¶: tracRA, tracDec (deg), tracRed: the observed tracer; lightcone only

enumerator LagrangianSky¶: lagrRA, lagrDec (deg), lagrRed: Result::lagrangianSky; lightcone only

enumerator Tracer¶: tracX, tracY, tracZ (Mpc/h): Result::tracers

enumerator Lagrangian¶: lagrX, lagrY, lagrZ (Mpc/h): Result::lagrangian

enumerator Displacement¶: displX, displY, displZ (Mpc/h): Result::meanDisplacement

enumerator ValidRealizations¶: nValidRec (J): Result::validRealizations

enumerator Selection¶: outsideRedshiftCut, outsideMask (J, 0 or 1); lightcone only

Displacement field¶

void otswap::io::writeDisplacementField(const std::string &file, const Result &result)¶

Write every realization of a result, losslessly: one row per realization and tracer, ordered [realization][object], the order of Result::displacement.

Columns: realization (J, from 0), index (K), tracX..tracZ (Result::tracers, repeated in every realization), lagrX..lagrZ (Result::matchedRandom), displX..displZ (Result::displacement), valid (J), outsideRedshiftCut, outsideMask (J, repeated). Keywords as writeDisplacements.

‘D’ values are written with 17 significant digits in ASCII, and as they are in FITS, so every double reads back identical, NaN rows as NaN, in both formats. A Result filled from such a file, with recomputeMeans, has the reconstruction’s meanDisplacement, validRealizations and lagrangian bit for bit, and its lagrangianSky given the distance table. displacement also equals lagr - trac bit for bit, as the reconstruction computes it.

Throws:: Error – if a field read does not have the size of its layout, or as io::write.

Real-space catalogue¶

void otswap::io::writeRealSpaceCatalog(const std::string &file, const RealSpaceCatalog &catalog, const std::vector<CatalogGroup> &groups = {})¶

Write a catalogue moved to real space, one row per tracer.

The default groups, CosmoBolognaLib’s reconstructed catalogue first:

lightcone: Sky, Cartesian, ValidRealizations, Neighbours, Status (10 columns);
box: Cartesian, ValidRealizations, Neighbours, Status (7 columns).

‘D’ values are written with 9 significant digits in ASCII. Keywords: SIGMA, WEIGHTED (0 or 1); NEXTRAP for a lightcone; AXIS, ZBOX and BIAS for a box.

Parameters:

file – the file to write; its extension sets the format.
catalog – a box or lightcone catalogue.
groups – the groups to write, in this order; empty for the default of the catalogue’s geometry.

Throws:

Error – as writeDisplacements.

enum class otswap::io::CatalogGroup¶

The column groups of writeRealSpaceCatalog.

Values:

enumerator Index¶: index (K): the row of the tracer in the input, from 0

enumerator Sky¶: tracRA, tracDec (deg), tracRed: RealSpaceCatalog::sky; lightcone only

enumerator Cartesian¶: tracX, tracY, tracZ (Mpc/h): RealSpaceCatalog::cartesian

enumerator ValidRealizations¶: nValidRec (J)

enumerator Neighbours¶: nNeighbours, nRealizationsAveraged (J)

enumerator Shift¶: shift (Mpc/h), rsdFactor: RealSpaceCatalog::shift, factor

enumerator Status¶: status (J): RealSpaceCatalog::status

Mean particle separation profile¶

void otswap::io::writeMpsProfile(const std::string &file, const MpsProfile &profile)¶

Write a mean particle separation profile: redshift (the bin centre), MPS (Mpc/h), nTracers (J), one row per bin. Keywords: ZMIN, ZMAX, MPSREPR (the representative value). MpsProfile::at gives the profile anywhere.

Throws:: Error – if the profile is empty or malformed, or as io::write.