The Basics
The software and file format is based on the Starlink software
collection, and as a result they are straightforward to embed within
scripts. The latest release can be downloaded from here.
The Data
JCMT heterodyne data obtained with ACSIS are available for download from the JCMT Science Archive at the CADC. These files have dimensions of frequency, position and time. They are a time stamped stream of data (spectra) for each detector.
At the telescope, ORAC-DR takes this file and reassembles it into a data cube with (RA, Dec, frequency) coordinates. As explained below in the cookbook, this can also be done manually using the makecube command in SMURF.
[Note that in the past, the ACSIS gridder task used to produce cubes, creating file names with an 'ac' prefix, but this task is now obsolete and users are advised not to use these data cubes.]
At the JCMT the raw data are written to /jcmtdata/raw/acsis/spectra/date
where data is yyyymmdd, with each scan written to a separate subdirectory (according to scannumber).
Data reduced by the ORAC-DR pipeline are written to /jcmtdata/reduced/acsis/date
Observers who are reducing their data further may find that the diskspace at /home/jcmt* is limited. They can use
/export/data/visitors for larger amounts of data.
The
software
Most of the routines and commands that will be used can all be found
within the KAPPA
and SMURF
packages (an on-line manual is in preparation), while visualisation and analysis will be through the GAIA (for cubes) and
SPLAT (for single spectra) programs
(although routines found in KAPPA
may also be used). The DATACUBE
package is a set of helper shell scripts which utilise KAPPA
routines
to visualise cubes.
KAPPA and other commands can be entered on the command line, or alternatively combined in
CSH scripts.
To access the packages you need to make sure that you have sourced the
Starlink logins. Usually this involves putting the following lines in
your .login
and .cshrc
files, although they can also be executed from the command line.
> source /star/etc/login
> source /star/etc/cshrc
This assumes that the Starlink software has been installed in the /star
directory. To make the software routines within the KAPPA
and SMURF
packages
available, just type their names into the command line:
> kappa
KAPPA commands are now available -- (Version 1.6-5)
Type kaphelp for help on KAPPA commands.
Type 'showme sun95' to browse the hypertext documentation.
NOTE, several applications have had major changes made to their
parameter lists. See the 'Release Notes' section of SUN/95 for
details.
> smurf
SMURF commands are now available -- (Version 0.2.2)
Type smurfhelp for help on SMURF commands.
Type 'showme sunXXX' to browse the hypertext documentation.
NOTE, several applications have had major changes made to their
parameter lists. See the 'Release Notes' section of SUN/XXX for
details.
You will note that to get help on these packages and to view all the
routines available within them, type either kaphelp or smurfhelp on
the command line. To get help on an individual routine, follow the help
comand with the name of the routine. For example, the following gives
help on the KAPPA
display
command which is used to plot images,
> kaphelp display
DISPLAY
Displays a one- or two-dimensional NDF.
Usage:
display in [comp] clear [device] mode [centre] [xmagn] [ymagn]
[out] { low=? high=?
{ percentiles=?
{ sigmas=?
mode
Description:
This application displays a one- or two-dimensional NDF as an
image on the current image-display device. The minimum and
maximum data values to be displayed can be selected in several...
<...snipped by editor here...>
When you first run up the software (see above) you can see that the
complete documentation for KAPPA
can be obtained by typing,
> showme sun95
This load up the KAPPA
manual on your web browser and is definitely recommended as a bookmark
as it provides a very detailed description of the software and its
functionality (including the parameter system, the graphics database,
coordinate systems, the style
parameters, etc.). Alternatively, if you do not know the name of the
package you can use the findme
command,
> findme gaia
A list of documents which contain keyword are displayed in your web
browser.
As
both of these commands search through your local documents (i.e. not on
the internet) they assume that you have the full Starlink collection
installed on your system, including the documentation. This also gives
you the flexibility of working offline (if you're travelling, for
example).
Sometimes, if one killed a
KAPPA
command with ctrl-C, commands will not work again. Then functionality can be restored by removing the agi*.sdf file in the home directory.
Alternatively, one can try to delete the corresponding sdf file in the
adam/ directory.
Other
ways to get help
Asking for prompts
There are other ways to get help on how to best use the software
routines. One of the most useful tips is to use the prompt
parameter on the command line for any Starlink program. This makes the
routine prompt for every parameter that it has control over. If you
were to run the command without the prompt
parameter, options which are considered to be defaults will not be
prompted for and their default values passed to the software. Using the
prompt
parameter gives you a feel for the versatility of the routine you are
using and also what parameters you can control by overiding the
defaults. Entering a \
at any prompt will then accept all subsequent default options.
Once you get used to the software, you will find that certain requested
parameters will become defaults for you. For instance, plotting between
certain percentile ranges in the display
command. To accept all defaults at the command line without further
prompting type \\
or accept
at the end of the command. The following two commands are identical:
> display in=temp \\
> display in=temp accept
Asking for ?s
To get help on individual parameters within routines you can enter a ? at the
prompt. The following example uses a combination of the prompt and ? methods for
getting help.
> display prompt
IN - NDF to be displayed /@temp/ > a20070128_00027_00_0001
COMP - Component to display /'Data'/ >
AXES - Are annotated axes to be drawn? /TRUE/ > ?
DISPLAY
Parameters
AXES
AXES = _LOGICAL (Read)
TRUE if labelled and annotated axes are to be drawn around the
image. These display co-ordinates in the current co-ordinate
Frame of the supplied NDF, and may be changed using application
WCSFRAME (see also parameter USEAXIS). The width of the
margins left for the annotation may be controlled using
parameter MARGIN. The appearance of the axes (colours, founts,
etc.) can be controlled using the STYLE parameter.
[current value]
AXES - Are annotated axes to be drawn? /TRUE/ >
To get help on the command as well as the parameter then a double
question-mark, ??,
can be entered as a response. This would be equivalent to searching for
help on the routine via the kaphelp
command, for instance.
Reset
the defaults
We have seen that using accept
on the command line will accept all the defaults in the parameter
system, using reset
will have the effect of resetting those defaults. For instance, if you
had previously entered the command...
> display in=temp device=xw mode=per accept
...which you then follow up with...
> display in=temp reset
...you will then be prompted for device (it has
no default value), and will be prompted for the plot-scaling mode which has scale as a
default in the parameter system (over-riding the previous invocation of
percentiles).
The
file format
The data structure
To look at the data structure you can use the HDSTRACE
command:
> hdstrace a20070128_00027_00_0001.sdf
A20070128_00027 <NDF>
DATA_ARRAY <ARRAY> {structure}
DATA(8192,16,32) <_REAL> *,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,*,
... 6.60745,-12.58472,2.012694,8.57546
ORIGIN(3) <_INTEGER> 1,1,1
HISTORY <HISTORY> {structure}
CREATED <_CHAR*24> '2007-JAN-27 23:16:37.927'
CURRENT_RECORD <_INTEGER> 2
RECORDS(10) <HIST_REC> {array of structures}
<...snipped by editor here...>
This shows the structure of a time series cube, including the pixel
origin of the data structure (lots of other detail has been omitted for
the sake of brevity!). To show all the lines and information that is available in each extension then include nlines=all on the command line with the hdstrace command (but look out with large files such as the ACSIS cubes!).
To show e.g. all receiver temperature values type
> hdstrace a20070128_00027_00_0001.sdf.more.ACSIS.TRX nlines=all
The hdstrace command is a stand-alone Starlink command. It is not part of KAPPA and so will not appear under kaphelp.
Help on this command can be obtained by typing
> showme sun102
The FITS header
The
FITS header contains important information relevant to the original
file and how it was observed and recorded (e.g. telescope, position,
instrumentation, exposure, etc). It is not continually updated. To look
at the FITS header you can use the fitslist
command:
> fitslist a20070128_00027_00_0001.sdf
TELESCOP= 'JCMT ' / Name of Telescope
ORIGIN = 'Joint Astronomy Centre, Hilo'/ Origin of file
---- x,y,z triplet for JCMT relative to centre of earth ----
ALT-OBS = 4111.0 / [m] Height of observatory above sea level
LAT-OBS = 19.82583335521 / [deg] Latitude of Observatory
LONG-OBS= -155.4797222301 / [deg] East longitude of observatory
OBSGEO-X= -5464594.335493 / [m]
OBSGEO-Y= -2492695.151639 / [m]
OBSGEO-Z= 2150964.058506 / [m]
ETAL = 0.850099503994 / Telescope efficiency
<...snipped by editor here...>
There are approximately 200 lines of FITS header information available
to the user!
The properties of the data structure
To look at a useful summary of the attributes and properties (e.g. the
name of the
file, the dimensions of the array, the length of each axis, the WCS
names of the axes and their extent, file extensions) of the data
structure can be obtained using the ndftrace
command:
> ndftrace a20070128_00027_00_0001.sdf
NDF structure /Users/acc/Data/JCMT/HARP/a20070128_00027_00_0001:
Label: TA* corrected antenna temperature
Units: K
Shape:
No. of dimensions: 3
Dimension size(s): 8192 x 16 x 32
Pixel bounds : 1:8192, 1:16, 1:32
Total pixels : 4194304
Data Component:
Type : _REAL
Storage form: SIMPLE
Bad pixels may be present
World Co-ordinate Systems:
Number of co-ordinate Frames: 4
Current co-ordinate Frame (Frame 4):
Frame title : "3-d compound coordinate system"
Domain : DSBSPECTRUM-SPACEINDEX-TIME
First pixel centre : -108.3581, 1, 2007-01-28 08:37:40
Axis 1:
Label: Radio velocity (USB)
Units: km/s
Axis 2:
Label: Receptor Number
Units: pixel
Axis 3:
Label: Date/Time
Units: d
Extensions:
ACSIS <ACSIS_COMP>
JCMTSTATE <RTS_ARR>
FITS <_CHAR*80>
History Component:
Created : 2007 Jan 27 23:16:37
No. records: 2
Last update: 2007 Jan 27 23:16:39 (ACSIS-DA (V0.5-1))
Update mode: NORMAL
Using the command ndftrace
<filename> fullframe
reveals even more
information!
Accessing and amending the WCS component of the data
There are commands which take advantage of the world coordinate system
of the image. In particular, the wcsattrib
command allows you to manage the coordinate frames. For now we leave
discussion of this and other WCS commands to the Advanced Methods
section of the
document.
The
adam
directory
If you've used Starlink software in the past you will have noticed that
an adam
directory is now present in your /home space.
Have a look in there and you will find .sdf
files, all named after the starlink commands that you have been using.
This is where the parameters that were used to execute the command last
are stored. To see which parameters were used, type the hdstrace
command on any of those files. For instance, to see which parameters
were used the last time the display
command was entered, you can type the following from anywhere
on your system:
> hdstrace ~/adam/display.sdf
DISPLAY <STRUC>
ADAM_DYNDEF <DEFAULTS> {structure}
{structure is empty}
IN <ADAM_PARNAME> {structure}
NAMEPTR <_CHAR*132> 'temp'
COMP <_CHAR*132> 'Data'
AXES <_LOGICAL> TRUE
BORDER <_LOGICAL> FALSE
SQRPIX <_LOGICAL> TRUE
KEY <_LOGICAL> FALSE
DEVICE <ADAM_PARNAME> {structure}
NAMEPTR <_CHAR*132> 'xw'
XMAGN <_REAL> 1
CLEAR <_LOGICAL> TRUE
FILL <_LOGICAL> FALSE
BADCOL <_CHAR*132> 'MIN'
SCALE <_LOGICAL> TRUE
MODE <_CHAR*132> 'per'
LOW <_DOUBLE> 0.0040830573998392
HIGH <_DOUBLE> 0.047642957419157
SCALOW <_DOUBLE> 2.8295915126801
SCAHIGH <_DOUBLE> 19.572355270386
PERCENTILES(2) <_REAL> 10,90
NUMBIN <_INTEGER> 2048
STYLE <ADAM_PARNAME> {structure}
NAMEPTR <_CHAR*132> '^~/style.def'
SIGMAS(2) <_REAL> -3,2
MARGIN <_REAL> 0.1
LUT <ADAM_PARNAME> {structure}
NAMEPTR <_CHAR*132> '~/lutgaia'
NN <_LOGICAL> FALSE
KEYPOS <_REAL> 0.05
NDF
sections
It is possible to provide just a section of an image or cube on which
to perform
operations. An NDF section is defined by specifying the bounds of the
portion of the NDF to be processed immediately following the name of
the file. You can do this with any Starlink command and anywhere where
you would normally enter the filename, i.e. either on the command line
or in response to a prompt. For instance, the following two commands
are identical,
> stats 'temp(1:200,1:200,-50:50)'
> stats
NDF - Data structure to analyse /@temp/ > temp(1:200,1:200,-50:50)
The start and end of the range is separated by a colon, while the axes
are separated by a comma. Note that the use of quotes on the command
line is necessary, whereas they are not in response to a prompt.
What
have you done to my data?
The hislist
command in
KAPPA
will give you a history of all the commands that have been
executed on a file, including all the parameters that were passed to
those commands. This gives a very useful record of what has happened to
your data. The following shows an example,
> hislist test
History listing for NDF structure /Users/acc/Data/JCMT/HARP/test:
History structure created 2007 Jan 12 19:30:29.060
1: 2007 Jan 12 19:30:29.060 - ACSIS-DA (V0.4-3)
Initial writing of raw data.
2: 2007 Jan 12 20:01:24.579 - ACSIS-DA (V0.4-3)
Finalise headers and make ICD compliant.
3: 2007 Feb 26 08:32:33.418 - COMPAVE (KAPPA 1.6-5)
Parameters: ALIGN='ORIGIN' COMPRESS=[3,1,1]
IN=@a20070113_00022_00_0001.sdf OUT=@test PRESERVE=FALSE TITLE=!
TRIM=TRUE WLIM=0.3
Software: /star/bin/kappa/compave
Wildcards
Most Starlink routines will take wildcards which allows you to process
multiple data sets. For instance,
> stats "tempcube*.sdf"
will calculate statistics for files in the present directory which
satisfy the wildcard. Using a wildcard on the command line requires
that it is escaped to avoid the shell interpreting the * symbol, hence
the use of the quotes in this example. See the KAPPA
manual
(SUN95) for a more detailed description and other options for
wildcards.
Making
postscript files
To make a postscript file that can be used for publishing, you need to
include device=epsfcol_p
for example either in the command line or as a response to the device
parameter (other choices include pscol_p, epsf_l, etc).
For example,
> display image.sdf device=epsf_p \\
The p
and l
in this device naming scheme refer to whether the output is in portrait
or landscape mode. For colour you need to include col as shown
and if you need
encapsulated postscript files you to specify epsf.
Each time a display command is used, whether for images, line graphics
or defining windows in the output plot device, a pgplot.ps file
is created. Existing files will be overwritten so you have to be
careful to rename your pgplot.ps
files if you want to keep them and merge them into new files (for
instance, for viewing contour plots over an image). To merge your
several postscript files (renamed to pgplot.ps.n
where n=1,2,3...):
> /star/bin/psmerge -e pgplot.ps.* > out.eps
The -e
switch will write
the output file as encapsulated postscript which is good for embedding
figures
into LaTeX documents (works best if the input files are also eps).
<Back to
Contents>
|
