Imaging: Data Format and Location

Raw files

Raw, unreduced, data files are in /ukirtdata/raw/uist/YYYYMMDD (where YYYYMMDD is the numeric UT date) or $ORAC_DATA_IN. The files are stored as starlink HDS containers (a file with multiple data arrays). Each file is equivalent to 1 observation, and as such contains a header component and 1 or more (actually NINT) integrations. Each integration is stored as an NDF component (single data array) of the HDS file. The raw filenames are uYYYYMMDD_NNNNN.sdf where NNNNN is the observation number, padded with leading zeros when necessary.

Reduced files

Reduced data files are in /ukirtdata/reduced/uist/YYYYMMDD (where YYYYMMDD is the numeric UT date) or $ORAC_DATA_OUT. The filename structure is: (PREFIX)(UTDATE)_(FRAME NUMBER)[_(EXTENSION)].sdf, where (THIS) is always there and [THIS] is optional. (PREFIX) is the letter 'u' if the file contains data from a single observation. It is 'gu' if the file contains data from a number of observations - ie a group. The pipeline adds the individual frames into the group as they are processed. The group number is usually the frame number of the first frame in the group. Reduced files have NDF (single data array) format.

_(EXTENSION) is used by individual primitives (think of a primitive as a single step within a recipe) for their output files. The pipeline keeps track of passing these files between primitives; useful ones are left on the disk at the end so you can look at intermediate data products if you wish. For example, u20000410_00123_ff.sdf would be data from a single observation, number 123, that has been flat fielded. Commonly used extensions are listed in the table below, a complete list of extensions is available.