Spectroscopy: Data Format and Location

Wavelength scale

The astute observer may notice that the dispersion axis on the raw frames is flipped, i.e. wavelength INCREASES to the LEFT. To deal with this, the pipeline DR updates the CDELT1 header early in the reduction process. The DISPERN header (the dispersion in um/pixel) remains positive, but the CDELT1 header (the wavelength co-ordinate increment, also in um/pixel) becomes negative, although it retains the same absolute value as DISPERN.

Note that, at present, orac-dr does not accurately wavelength calibrate UIST long-slit spectroscopy data (IFU data are properly calibrated). Only an estimated wavelength scale is attached to the data.

Slit orientation on the sky

A slit position angle anywhere between 90 degrees and -90 degrees may be selected in the OT. All angles are measured east-of-north. An angle of -90 degrees is recommended for point sources; this will put EAST to the TOP of the array. In the table below we list the orientation of the slit on the sky for specific position angles.

Raw files

Raw, unreduced, data files are in /ukirtdata/raw/uist/YYYYMMDD (where YYYYMMDD is the numeric UT date) or $ORAC_DATA_IN. The files are stored as starlink HDS containers (a file with multiple data arrays). Each file is equivalent to 1 observation, and as such contains a header component and 1 or more (actually NINT) integrations. Each integration is stored as an NDF component (single data array) of the HDS file. The raw filenames are uYYYYMMDD_NNNNN.sdf where NNNNN is the observation number, padded with leading zeros when necessary. Note: you may have a hard time working with this data format unless you have access to starlink and ORAC-DR at your home institute (see below).

Reduced single frame files

Reduced data files are in /ukirtdata/reduced/uist/YYYYMMDD (where YYYYMMDD is the numeric UT date) or $ORAC_DATA_OUT. The filename structure is: (PREFIX)(UTDATE)_(FRAME NUMBER)_[EXTENSION].sdf, where (THIS) is always there and [THIS] is optional. (PREFIX) is the letter u if the file contains data from a single observation. It is gu if the file contains data from a number of observations - i.e. a group (see below). _(EXTENSION) is used by individual primitives for their output files; think of a primitive as a single step within a recipe. The pipeline keeps track of passing these files between primitives; useful ones are left on the disk at the end so you can look at intermediate data products if you wish.

For example, u20000410_00123_ff.sdf would be data from a single observation, number 123, that has had all the reduction steps up to and including flat fielding applied to it (see, e.g., the table below). Reduced files can have either HDS (multiple data arrays) or NDF (single data array) format as appropriate. Extensions for single frames are listed in the table below.

Reduced group files

The pipeline adds the individual frames into a "group" file as they are processed. The group number is usually the frame number of the first frame in the group. Extensions for group frames are listed in the table below. Note that some of these data are 2-D spectral images, and some are extracted 1-D spectra; the former will be displayed in gaia and the latter in a Kapview plotting window.

So which data products should I work with?

Once you have transferred the raw data to your home institute, you can run ORAC-DR locally to produce reduced and semi-reduced data products, and to get a first-look at the data.

For post-reduction with Starlink software or IRAF you probably only need the individual _wce spectral images; from these you can do your own "object-sky" differencing and subsequent coaddition of the data to produce a "group" spectral image (that contains all of the data on any one target). Alternatively, if the weather was stable and all data are usable, you may be able to simply work with the reduced group image produced by ORAC-DR - this is the group file in the above table with no extension. Either way, the arc spectra (also stored as _wce files) may then be used to accurately calibrate the dispersion axis of the 2-D image or of extracted spectra. An extracted spectrum from the "group" spectral image of your standard star can then be used to flux-calibrate and correct for telluric absorption lines.

Arguably, the "most useful" data products are highlighted in red, though this is really up to you...

Finally, note that each time you run ORAC-DR a "hidden" log file is written called .oracdr-number.log. If you type ls -a you'll see these. Have a look at one of them; it'll list everything the DR did during a particular session. This could be useful for following DR steps or identifying some of the more obscure files.

NOTE: At present, long-slit spectroscopy data only have an ESTIMATED wavelength scale (IFU data are the exception; these have a proper wavelength calibration).

Converting to fits

Non-Starlink users may convert their data to fits format with the Starlink routines:

     > convert
     > ndf2fits "*" "*"

     > ndf2fits "*" "*|.fit|.fits|"