YODA data format

Introduction

The YODA data format will be an extension of Hendrik's "make_plot" format, i.e. a flat file with column delimited data. We want to store enough (meta)data for titles, data series names, etc., but also to retain human read/write-ability. This last point (as well as some shortcomings in the data that can be stored) means that we are not enamoured with the AIDA XML-based data format.

General properties

• Columns of data are delimited in all cases by an arbitrary number of consecutive whitespace characters. This is to avoid any dependence on distinctions between invisible characters (see Makefile and Python syntax for examples of the sort of mess that can cause).
• Each data object is wrapped with type-specific BEGIN...END lines, to trigger the processing of the contained data in the correct way.
• The hash sign (#) is to be interpreted as a comment character in the manner of Python. Need some way to declare that a comment is actually a header?
• An arbitrary number of headers may be present on each object. We recommend that these headers are escaped by a prefix # character, for compatibility with gnuplot
• For compatibility with gnuplot, the first 6 ??? data columns of records must be compatible with the gnuplot "plot with xyerrors" format, namely:

  x, y, xlow, xhigh, ylow, yhigh
  

HISTOGRAM

The Histogram type is special: it is not a YODA type, but the data format will define it for backwards compatibility with Hendrik's make_plot data files. It should be pre-deprecated.

DETAILS

COUNTER

A Counter class is not yet confirmed for inclusion in YODA, but seems useful (and is usually overlooked)

sumw, sumw2

(only one line)

DBN1D

An extension of Counter which is used for internally managing the x data of Histo1D bins and the content of Profile1D bins. Not yet confirmed for public use.

xmean, xerr, sumw, sumw2, sumwx, sumwx2

(only one line)

BAR1D

Bar plots are like histograms, but the x axis is discrete. Hence, there is no meaning to the x-spread data, and y-heights should be displayed without a width scaling. Not yet confirmed for use in YODA, but the confusion over histogram height scaling (and the mess of HepData? records) suggests that we could do with an explicit type for this. How to deal with integral distributions? Convert from Histo1D -> ???

barname, <Counter>

(multiple lines - one per bar)

HISTO1D

The main data type.

xcenter, ycenter, xlow, xhigh, ylow, yhigh, sumw, sumw2, sumwx, sumwx2

(multiple lines - one per bin)

PROFILE1D

xcenter, ycenter, xlow, xhigh, ylow, yhigh, sumw, sumw2, sumwx, sumwx2, sumwy, sumwy2

(multiple lines - one per bin)

SCATTER1D

Unconfirmed type.

x, xlow, xhigh, weight

(multiple lines - one per point)

SCATTER2D

Unconfirmed type.

x, y, xlow, xhigh, ylow, yhigh, weight

(multiple lines - one per point)