Joint Astronomy Centre
Google
Show document only
JAC Home
JCMT
UKIRT
Contact info
JAC Divisions
OMP
Outreach
Seminars
Staff-only Wiki
Weather
Web Cameras
____________________

Observing at UKIRT
Service Observing
UKIDSS Survey Operations
Target of Opportunity
Calibration & Utilities
UKIRT Archive
Public wiki
Accessing Flexed Data
Accessing UKIDSS Data
Reduction Cookbooks
Telescope
Site Quality
Instruments
Newsletter/Publications
UKIRT Faults
JAC Safety Manual
WFCAM daytime checks

WFCAM daytime checks from Hilo

Before starting any daytime checks, make sure that you contact the UKIRT daywork leader first to get clearance to run WFCAM. You will also need to get clearance to run the TCS in simulation mode if you are doing the full check and nighttime darks as you will need the TCS running to get all the necessary headers in the data files. Note that if ETS are running the TCS you will not be able to run TCS_SIM without causing the TCS running at the summit to crash. The darks sequence will also require that the dome lights are off and the dome is closed.

Full daytime check with nighttime darks
Full daytime check without nighttime darks
Regular engineering tests
Potential problems and fixes
preliminary new version (20080416/jk):

Full daytime check with nighttime darks (needs TCS running)


  1. log in to ohi as observer with ssh observer@ohi
  2. launch WFCAM engineering interface with wfcamControlEng
  3. start low level camera software for all cameras [0a]
  4. run simulated tcs [1a]; wait for startup messages and launched windows
  5. run ocs_up [2a]; wait for startup messages and launched windows
  6. add wfcam instrument software to ocs [3]; confirm on sequence console
  7. enable arrays for all four cameras [4]; confirm on sequence console
  8. send WFCAM flush and readnoise measurements from query tool to ocs queue; start ocs queue
  9. start wfcamReduce; confirm data directory presence and reduction started
  10. start wfcamNoise; confirm nominal readnoise
  11. start ompobslog; confirm data present and add readnoise results shift comment
  12. send WFCAM darks from query tool to ocs queue; start ocs queue
  13. wait for dark observations to finish; wait for dark reductions to finish
  14. stop and exit wfcamReduce pipelines; run wfcamReduceNuke
  15. disable arrays for all four cameras [5]; confirm disabled in log
  16. remove wfcam instrument software from ocs [6]
  17. type exit and ctrl-c in and close tcs_sim_cli window
  18. run ocs_down [7]
  19. run tcs_nuke [8a]
  20. stop low level camera software for all cameras [9]; confirm stopped and ignore noticeboard in log
  21. exit wfcamControlEng
  22. log out of ohi
  23. send ukirt_ops wfcam status e-mail report with instrument status, darks status, and final frame

still needed to be done separately (for the time being, still working on directory mount check script):
  1. ssh kauwa, ompobslog; confirm data present
  2. ssh hokulea, ompobslog; confirm data present

new wfcamControlEng features:
  1. most routinely necessary features now incorporated
  2. standard and engineering WFCAM software control both included
  3. standard and engineering OCS/TCS software both included
  4. various data tool access now included
  5. ukirtmon functionality added in case of zero countdown
  6. tcs_sim now runs on ohi
  7. ocs_down and tcs_nuke must both be run if one is run due to software thread issues
 
  canonical old version:

Full daytime check with nighttime darks (needs TCS running)

  1. Log into Hokulea as "telescope" (using "ssh -Y telescope@hokulea") and type tcs_sim. Wait 30 seconds for the TCS software to run up. Running the TCS will ensure that the data files contain full headers, necessary for the database.
  2. Log into Ohi as "observer" (using "ssh -Y observer@ohi") and type "wfcamControl". This will launch the WFCAM control GUI. Run up the low level software from the GUI by hitting the "start" button and check that all four cameras are running by clicking on "status". Ignore any beeps you get when hitting buttons on the GUI, this behaviour is quite normal.
  3. On Ohi, type "ocs_up" and wait for all the OCS screens to launch.
  4. Back on the WFCAM control GUI, hit the "add inst" button, wait for the sequence console to become "live", and then hit the "enable" button to activate the arrays.
  5. Send the "WFCAM flush" sequence to the queue and start the queue, or send directly via "send for engineering" and start the sequence in the sequence console. Follow this with the "WFCAM readnoise measurement" sequence.
  6. While the flush and read noise data are being taken, log onto Kauwa as "observer" (using "ssh -Y observer@kauwa") and type wfcamObs. From this GUI, hit the wfcamReduce button to start the DRs on all four WFDR machines. Check that the DRs can see the raw data and the read noise level as reported by the DRs (you can use the "wfcamNoise" button on wfcamObs to see the read noise results as well).
  7. Run the  the "WFCAM darks" sequence (takes about 20 minutes) and make sure the DRs reduce the data.
  8. Assuming all is well, quickly run ompobslog on both Kauwa and Hokulea and ensure they see the data. Enter the results of the read noise test as a shift log comment.
  9. Disable the arrays from the WFCAM control GUI (NB. please make sure that you get the response in the logging pane of the GUI that the cameras are powered off).
  10. Type "ocs_down" on Ohi, then "drama_nuke".
  11. Run down the WFCAM low level software, check the status to make sure the software has run down successfully, exit wfcamControl and log out. You can ignore any messages in the wfcamControl logging pane that say "Failed to unlink noticeboard file".
  12. Run down the TCS by typing "exit" in the TCS terminal. You will likely also have to do a ctrl-c afterwards. Type "tcs_nuke" on Hokulea, close the top-end DM screen and log out.
  13. Exit the DRs and nuke them (with -nogaia). The nuke can also be done using the "wfcamReduceNuke" button in the wfcamObs GUI. Log out.
  14. Exit wfcamObs on Kauwa and log out.
  15. Finally, send an email to ukirt_ops titled "WFCAM status" and report the status of WFCAM. Include that darks were taken and report the final frame number so the nighttime TSS is aware of this and can inform the observers.

Full daytime check without darks (TCS not necessary)

  1. Log into Ohi as "observer" (using "ssh -Y observer@ohi") and and type "wfcamControl". This will launch the WFCAM control GUI. Run up the low level software from the GUI by hitting the "start" button and check that all four cameras are running by clicking on "status". Ignore any beeps you get when hitting buttons on the GUI, this behaviour is quite normal.
  2. On Ohi, type "ocs_up" and wait for all the OCS screens to launch.
  3. Back on the WFCAM control GUI, hit the "add inst" button, wait for the sequence console to become "live", and then hit the "enable" button to activate the arrays.
  4. Send the "WFCAM flush" (optional) and then the "WFCAM readnoise measurement" sequence to the queue and start the queue, or send directly via "send for engineering" and start the sequence in the sequence console.
  5. While the read noise data are being taken, log onto Kauwa as "observer" (using "ssh -Y observer@kauwa") and type wfcamObs. From this GUI, hit the wfcamReduce button to start the DRs on all four WFDR machines. Check that the DRs can see the raw data and the read noise level as reported by the DRs.
  6. Assuming all is well, quickly run ompobslog on both Kauwa and Hokulea and ensure they see the data.
  7. Disable the arrays from the WFCAM control GUI (NB. please make sure that you get the response in the GUI logging pane that the arrays are powered off).
  8. Type "ocs_down" on Ohi, then "drama_nuke".
  9. Run down the WFCAM low level software, check the status to make sure the software has run down successfully, exit wfcamControl and log out. You can ignore any messages in the wfcamControl logging pane that say "Failed to unlink noticeboard file".
  10. Exit the DRs and nuke them (with -nogaia). The nuke can also be done using the "wfcamReduceNuke" button in the wfcamObs GUI. Log out.
  11. Exit wfcamObs on Kauwa and log out.
  12. Log into each wfacq machine as root. cd to /export/data/raw/wfcamN/<utdate> (where N is the number of the wfacq machine you are logged into) and delete both the raw data and the .*.ok files.
  13. Log into Kauwa as observer, cd to /ukirtdata/raw/wfcam/<utdate> and delete the .lastobs file.
Steps 12 and 13 ensure that the nighttime observers will start with a blank slate and that their first frame of the night will be frame 1, You do not need to carry out these steps, but if you don't, you must in form the TSS/observers or send email to ukirt_ops explaining that data files already exist for tonight.

Regular engineering tests (data in engineering directories)

  1. Log into Ohi as "observer" (using "ssh -Y observer@ohi") and and type "wfcamControlEng". This will launch the WFCAM control GUI. Run up the low level software from the GUI by hitting the "start" button and check that all four cameras are running by clicking on "status". Ignore any beeps you get when hitting buttons on the GUI, this behaviour is quite normal.
  2. From the GUI, hit the "ocs_up -simTel -eng" button which will launch the OCS.
  3. Again on the WFCAM control GUI, hit the "add inst" button, wait for the sequence console to become "live", and then hit the "enable" button to activate the arrays.
  4. Send the "WFCAM readnoise measurement" sequence to the queue and start the queue, or send directly via "send for engineering" and start the sequence in the sequence console.
  5. Log in as observer to each of the four WFDR machines. In each one, type "oracdr_wfcam_eng" followed by a nuke (-nogaia) and run the DR with "oracdr -loop flag -log sf -nodisplay -skip" or with the options you prefer.
  6. When finished, disable the arrays from the WFCAM control GUI (NB. please make sure that you get the response in the GUI logging pane that the cameras are powered off). You can ignore any messages in the wfcamControl logging pane that say "Failed to unlink noticeboard file".
  7. Hit the ocs_down button in the GUI (also best to type drama_nuke on Ohi afterwards as well) and then stop the low level software, checking the status to make sure the software has run down successfully. Exit the GUI and log out.
  8. Exit the DRs and nuke them (with -nogaia), Log out.

Potential problems and fixes

  • Countdown stuck at zero or low level software does not run up successfully: Refer to flow chart in fault 20061106.003
  • "add inst" does not activate the sequence console or the arrays won't enable: Usually a problem with the OCS. Run down the low level software, do an ocs_down and then drama_nuke on Ohi, run up the low level software, ocs_up and try again. This will usually clear the problem. Refer to the flow chart in 20061106.003 if there are problems with the low level software.
  • Data are taken successfully, but DR can't see the frames: One or more raw data disks are not mounted. Log into Kauwa as yourself, type "/local/bin/remount wfcamN" where N is the number of the camera that you need to remount" and when prompted enter your password. You should make sure the DR for the dismounted disk/s isn't running when this is done. If the remounting stops at some point, ctrl-c will get it to continue ands then repeat the command.
  • Ompobslog does not show data: the raw data disk on wfacq1 is not mounted. Follow instructions above.
  • Arrays do not power off: When disabling the arrays, you must make sure you get a response saying the cameras are powered off in the wfcamControl logging pane. If not, run down the low level software, fix any problems with reference to the flowchart in 20061106.003, run the low level software back up again, ocs_up, add inst, enable and then disable arrays. If you cannot disable arrays, then get someone at the summit to power off the arrays using the disable buttons on the controllers.
Contact: Luca Rizzi. Updated: Mon Apr 28 15:49:01 HST 2008

Return to top ^