1.2.3.1. Rules for Structuring Information in NeXus Files

All NeXus files contain one or many groups of type NXentry at root level. Many files contain only one NXentry group, then the name is entry. The NXentry level of hierarchy is there to support the storage of multiple related experiments in one file. Or to allow the NeXus file to serve as a container for storing a whole scientific workflow from data acquisition to publication ready data. Also, NXentry class groups can contain raw data or processed data. For files with more than one NXentry group, since HDF requires that no two items at the same level in an HDF file may have the same name, the NeXus fashion is to assign names with an incrementing index appended, such as entry1, entry2, entry3, etc.

In order to illustrate what is written in the text, example hierarchies like the one in figure Raw Data are provided.

Content of a Raw Data NXentry Group

An example raw data hierarchy is shown in figure Raw Data (only showing the relevant parts of the data hierarchy). In the example shown, the data field in the NXdata group is linked to the 2-D detector data (a 512x512 array of 32-bit integers). The attribute signal = data on the NXdata group marks this field as the default plottable data of the data:NXdata group. The NXdata group attribute axes = . . declares that both dimensions of the data field do not have associated dimension scales (plotting routines should use integer scaling for each axis). Note that [,] represents a 2D array.

NeXus Raw Data Hierarchy

 1    entry:NXentry
 2        @default = data
 3        instrument:NXinstrument
 4            source:NXsource
 5            ....
 6            detector:NXdetector
 7                data:NX_INT32[512,512]
 8        sample:NXsample
 9        control:NXmonitor
10        data:NXdata
11            @signal = data
12            @axes = [".", "."]
13            data --> /entry/instrument/detector/data

An NXentry describing raw data contains at least a NXsample, one NXmonitor, one NXdata and a NXinstrument group. It is good practice to use the names sample for the NXsample group, control for the NXmonitor group holding the experiment controlling monitor and instrument for the NXinstrument group. The NXinstrument group contains further groups describing the individual components of the instrument as appropriate.

The NXdata group contains links to all those data items in the NXentry hierarchy which are required to put up a default plot of the data. As an example consider a SAXS instrument with a 2D detector. The NXdata will then hold a link to the detector image. If there is only one NXdata group, it is good practice to name it data. Otherwise, the name of the detector bank represented is a good selection.

Content of a processed data NXentry group

Processed data, see figure Processed Data, in this context means the results of a data reduction or data analysis program. Note that [] represents a 1D array.

NeXus Processed Data Hierarchy

 1    entry:NXentry
 2        @default = data
 3        reduction:NXprocess
 4            program_name = "pyDataProc2010"
 5            version = "1.0a"
 6            input:NXparameters
 7                filename = "sn2013287.nxs"
 8        sample:NXsample
 9        data:NXdata
10            @signal = data
11            @axes = "."
12            data

NeXus stores such data in a simplified NXentry structure. A processed data NXentry has at minimum a NXsample, a NXdata and a NXprocess group. Again the preferred name for the NXsample group is sample. In the case of processed data, the NXdata group holds the result of the processing together with the associated axis data. The NXprocess group holds the name and version of the program used for this processing step and further NXparameters groups. These groups ought to contain the parameters used for this data processing step in suitable detail so that the processing step can be reproduced.

Optionally a processed data NXentry can hold a NXinstrument group with further groups holding relevant information about the instrument. The preferred name is again instrument. Whereas for a raw data file, NeXus strives to capture as much data as possible, a NXinstrument group for processed data may contain a much-reduced subset.

NXsubentry or Multi-Method Data

Especially at synchrotron facilities, there are experiments which perform several different methods on the sample at the same time. For example, combine a powder diffraction experiment with XAS. This may happen in the same scan, so the data needs to be grouped together. A suitable NXentry would need to adhere to two different application definitions. This leads to name clashes which cannot be resolved easily. In order to solve this issue, the following scheme was implemented in NeXus:

  • The complete beamline (all data) is stored in an appropriate hierarchy in an NXentry.

  • The NXentry group contains further NXsubentry groups, one for each method.

  • Each NXsubentry group is constructed like a NXentry group. It contains links to all those data items required to fulfill the application definition for the particular. method it represents.

  • The name of the application definition is stored in the definition field of the NXsubentry group

  • Each NXsubentry group contains a NXdata group describing the default plottable data for that experimental method. To satisfy the NeXus requirement of finding the default plottable data from a NXentry group, the NXdata group from one of these NXsubentry groups (the fluoresence data) was linked.

See figure NeXus Multi Method Hierarchy for an example hierarchy. Note that [,] represents a 2D array.

NeXus Multi Method Hierarchy

 1    entry:NXentry
 2        @default = data
 3        user:NXuser
 4        sample:NXsample
 5        instrument:NXinstument
 6            SASdet:NXdetector
 7                data:[,]
 8            fluordet:NXdetector
 9                data:[,]
10            large_area:NXdetector
11                data:[,]
12        SAS:NXsubentry
13            definition = "NXsas"
14            instrument:NXinstrument
15                detector:NXdetector
16                    data --> /entry/instrument/SASdet/data
17            data:NXdata
18                data --> /entry/instrument/SASdet/data
19        Fluo:NXsubentry
20            definition = "NXfluo"
21            instrument:NXinstrument
22                detector --> /entry/instrument/fluordet/data
23                detector2 --> /entry/instrument/large_area/data
24            data:NXdata
25                @signal = detector
26                @axes = [".", "."]
27                detector --> /entry/instrument/fluordet/data
28        data:NXdata --> /entry/Fluo/data

Rules for Special Cases

Scans

Scans are difficult to capture because they have great variety. Basically, any variable can be scanned. Such behaviour cannot be captured in application definitions. Therefore NeXus solves this difficulty with a set of rules. In this section, NP is used as a symbol for the number of scan points.

  • The scan dimension NP is always the first dimension of any multi-dimensional dataset. The reason for this is that HDF allows the first dimension of a dataset to be unlimited. Which means, that data can be appended to the dataset during the scan.

  • All data is stored as arrays of dimensions NP, original dimensions of the data at the appropriate position in the NXentry hierarchy.

  • The NXdata group has to contain links to all variables varied during the scan and the detector data. Thus the NXdata group mimics the usual tabular representation of a scan.

  • The NXdata group has attributes to enable the default plotting, as described in the section titled NXdata Facilitates Automatic Plotting.

Simple scan

Examples may be in order here. Let us start with a simple case, the sample is rotated around its rotation axis and data is collected in a single point detector. See figure Simple Scan for an overview. Then we have:

  • A dataset at NXentry/NXinstrument/NXdetector/data of length NP containing the count data.

  • A dataset at NXentry/NXsample/rotation_angle of length NP containing the positions of rotation_angle at the various steps of the scan.

  • NXdata contains links to:

    • NXentry/NXinstrument/NXdetector/data

    • NXentry/NXsample/rotation_angle

  • All other fields have their normal dimensions.

NeXus Simple Scan Example

 1    entry:NXentry
 2        @default = data
 3        instrument:NXinstrument
 4            detector:NXdetector
 5                data[NP]
 6        sample:NXsample
 7            rotation_angle[NP]
 8        control:NXmonitor
 9            data[NP]
10        data:NXdata
11            @signal = "data"
12            @axes = "rotation_angle"
13            @rotation_angle_indices = 0
14            data --> /entry/instrument/detector/data
15            rotation_angle --> /entry/sample/rotation_angle

Simple scan with area detector

The next example is the same scan but with an area detector with xsize times ysize pixels. The only thing which changes is that /NXentry/NXinstrument/NXdetector/data will have the dimensions NP, xsize, ysize. See figure Simple Scan with Area Detector for an overview.

NeXus Simple Scan Example with Area Detector

 1    entry:NXentry
 2        instrument:NXinstrument
 3            detector:NXdetector
 4                data:[NP,xsize,ysize]
 5        sample:NXsample
 6            rotation_angle[NP]
 7        control:NXmonitor
 8            data[NP]
 9        data:NXdata
10            @signal = "data"
11            @axes = ["rotation_angle", ".", "."]
12            @rotation_angle_indices = 0
13            data --> /entry/instrument/detector/data
14            rotation_angle --> /entry/sample/rotation_angle

The NXdata group attribute axes = rotation_angle . . declares that only the first dimension of the plottable data has a dimension scale (by name, rotation_angle). The other two dimensions have no associated dimension scales and should be plotted against integer bin numbers.

Complex hkl scan

The next example involves a complex movement along the \(h\) axis in reciprocal space which requires mutiple motors of a four-circle diffractometer to be varied during the scan. We then have:

  • A dataset at NXentry/NXinstrument/NXdetector/data of length NP containing the count data.

  • A dataset at NXentry/NXinstrument/NXdetector/polar_angle of length NP containing the positions of the detector’s polar_angle at the various steps of the scan.

  • A dataset at NXentry/NXsample/rotation_angle of length NP containing the positions of rotation_angle at the various steps of the scan.

  • A dataset at NXentry/NXsample/chi of length NP containing the positions of chi at the various steps of the scan.

  • A dataset at NXentry/NXsample/phi of length NP containing the positions of phi at the various steps of the scan.

  • A dataset at NXentry/NXsample/h of length NP containing the positions of the reciprocal coordinate \(h\) at the various steps of the scan.

  • A dataset at NXentry/NXsample/k of length NP containing the positions of the reciprocal coordinate \(k\) at the various steps of the scan.

  • A dataset at NXentry/NXsample/l of length NP containing the positions of the reciprocal coordinate \(l\) at the various steps of the scan.

  • NXdata contains links to:

    • NXentry/NXinstrument/NXdetector/data

    • NXentry/NXinstrument/NXdetector/polar_angle

    • NXentry/NXsample/rotation_angle

    • NXentry/NXsample/chi

    • NXentry/NXsample/phi

    • NXentry/NXsample/h

    • NXentry/NXsample/k

    • NXentry/NXsample/l

    The NXdata also contains appropriate attributes as described in Associating plottable data using attributes applied to the NXdata group.

  • All other fields have their normal dimensions.

NeXus Complex hkl Scan

 1    entry:NXentry
 2        @default = data
 3        instrument:NXinstrument
 4            detector:NXdetector
 5                data[NP]
 6                polar_angle[NP]
 7                name
 8        sample:NXsample
 9            name
10            rotation_angle[NP]
11            chi[NP]
12            phi[NP]
13            h[NP]
14            k[NP]
15            l[NP]
16        control:NXmonitor
17            data[NP]
18        data:NXdata
19            @signal = data
20            @axes = "h"
21            @h_indices = 0
22            @k_indices = 0
23            @l_indices = 0
24            @chi_indices = 0
25            @phi_indices = 0
26            @polar_angle_indices = 0
27            @rotation_angle_indices = 0
28            data --> /entry/instrument/detector/data
29            rotation_angle --> /entry/sample/rotation_angle
30            chi --> /entry/sample/chi
31            phi --> /entry/sample/phi
32            polar_angle --> /entry/instrument/detector/polar_angle
33            h --> /entry/sample/h
34            k --> /entry/sample/k
35            l --> /entry/sample/l

Multi-parameter scan: XAS

Data can be stored almost anywhere in the NeXus tree. While the previous examples showed data arrays in either NXdetector or NXsample, this example demonstrates that data can be stored in other places. Links are used to reference the data.

The example is for X-ray Absorption Spectroscopy (XAS) data where the monochromator energy is step-scanned and counts are read back from detectors before (I0) and after (I) the sample. These energy scans are repeated at a sequence of sample temperatures to map out, for example, a phase transition. While it is customary in XAS to plot log(I0/I), we show them separately here in two different NXdata groups to demonstrate that such things are possible. Note that the length of the 1-D energy array is NE while the length of the 1-D temperature array is NT

NeXus Multi-parameter scan: XAS

 1    entry:NXentry
 2        @default = "I_data"
 3        instrument:NXinstrument
 4            I:NXdetector
 5                data:NX_NUMBER[NE,NT]
 6                energy --> /entry/monochromator/energy
 7                temperature --> /entry/sample/temperature
 8            I0:NXdetector
 9                data:NX_NUMBER[NE,NT]
10                energy --> /entry/monochromator/energy
11                temperature --> /entry/sample/temperature
12        sample:NXsample
13            temperature:NX_NUMBER[NT]
14        monochromator:NXmonochromator
15            energy:NX_NUMBER[NE]
16        I_data:NXdata
17            @signal = "data"
18            @axes = ["energy", "temperature"]
19            @energy_indices = 0
20            @temperature_indices = 0
21            data --> /entry/instrument/I/data
22            energy --> /entry/monochromator/energy
23            temperature --> /entry/sample/temperature
24        I0_data:NXdata
25            @signal = data
26            @axes = ["energy", "temperature"]
27            @energy_indices = 0
28            @temperature_indices = 0
29            data --> /entry/instrument/I00/data
30            energy --> /entry/monochromator/energy
31            temperature --> /entry/sample/temperature

Rastering

Rastering is the process of making experiments at various locations in the sample volume. Again, rasterisation experiments can be variable. Some people even raster on spirals! Rasterisation experiments are treated the same way as described above for scans. Just replace NP with P, the number of raster points.

Special rules apply if a rasterisation happens on a regular grid of size xraster, yraster. Then the variables varied in the rasterisation will be of dimensions xraster, yraster and the detector data of dimensions xraster, yraster, (orginal dimensions) of the detector. For example, an area detector of size xsize, ysize then it is stored with dimensions xraster, yraster, xsize, ysize.

Warning

Be warned: if you use the 2D rasterisation method with xraster, yraster you may end up with invalid data if the scan is aborted prematurely. This cannot happen if the first method is used.

Streaming Data Acquisition And Logging

More and more data is collected in streaming mode. This means that time stamped data is logged for one or more inputs, possibly together with detector data. Another use case is the logging of parameters, for example temperature, while a long running data collection is in progress. NeXus covers this case too. There is one simple rule for structuring such files:

Just use the standard NeXus raw data file structure, but replace the corresponding data object with an NXlog or NXevent_data structure of the same name.

For example, consider your instrument is streaming detector images against a magnetic_field on the sample. In this case both NXsample/magnetic_field and NXdetector/data would become NXlog structures instead of simple arrays i.e. the NXlog structure will have the same name as the NeXus field involved.

NXcollection

On demand from the community, NeXus introduced a more informal method of storing information in a NeXus file. This is the NXcollection class which can appear anywhere underneath NXentry. NXcollection is a container for holding other data. The foreseen use is to document collections of similar data which do not otherwise fit easily into the NXinstrument or NXsample hierarchy, such as the intent to record all motor positions on a synchrotron beamline. Thus, NXcollection serves as a quick point of access to data for an instrument scientist or another expert. NXcollection is also a feature for those who are too lazy to build up the complete NeXus hierarchy. An example usage case is documented in figure NXcollection example.

NXcollection Example

 1    entry:NXentry
 2        positioners:NXcollection
 3            mxx:NXpositioner
 4            mzz:NXpositioner
 5            sgu:NXpositioner
 6            ttv:NXpositioner
 7            hugo:NXpositioner
 8            ....
 9        scalars:NXcollection
10            title (dataset)
11            lieselotte (dataset)
12            ...
13        detectors:NXcollection
14            Pilatus:NXdata
15            MXX-45:NXdata
16            ....