Summary of Rules & Procedures Governing the Workings of the UK JCMT
TAG
Time Allocation Group
- Chair and group are approved by STFC.
- Membership, including Chair, normally number
6 individuals.
They are almost inevitably users of JCMT, expert in submillimetre
astronomy, and competent to judge the proposals of their peers in the
context of advancing the subject and the profile of JCMT.
-
They are assisted logistically by the Technical Secretary (usually
a JCMT support scientist), and (previously) the JCMT Secretary - collectively
called the 'Technical Secretariat'.
-
Informally, the 6 TAG members are divided by expertise into
'galactic' (G) and 'extra-galactic' (E) in rough proportion to
the type of applications received :
| Semester |
Applications |
| Galactic |
Extragalactic |
| 01b |
25 |
13 |
| 02a |
28 |
26 |
| 02b |
31 |
23 |
| 03a |
23 |
25 |
| 03b |
27 |
20 |
| 04a |
33 |
20 |
| 04b |
25 |
19 |
| 05a |
31 |
21 |
| 05b |
28 |
27 |
| 06b |
16 |
7 |
| 07a |
24 |
8 |
The G:E ratio was not worryingly different from 1:1 until the 2006
shutdown. However, in the two subsequent semesters - where we had no
continuum capability, and perhaps as a consequence of the start of the
Large Scale
Surveys - there has been a dramatic tilt towards Galactic programs.
This has meant our 'E'-assessors having to brush up on their G-alactic
astronomy  .
With SCUBA-2 in the offing it is expected that the
pendulum will swing back towards parity soon, and so this recent trend
did not warrant adjusting the composition of the TAG in the short term.
(There are one or two proposals in recent semesters to do Solar
System work that are not shown in this table).
The success rate amongst 'G' and 'E' proposals has been remarkably
similar through the 9 semesters 01b-05b : at 53+13% and
49+15%, respectively.
The TAG has moved away from
a strict division between successful and
unsuccessful proposals, and has adopted the scheme used by the
other national queues: of relative prioritization (ranking) and block
observing, wherein an observer from a highly-ranked program observes
according to standard flexible
scheduling rules.
- New TAG members are recommended to STFC by the Chair, and the Chair
recommends his/her successor. The Chair should alternate between those
of 'E' and 'G' types (see 4).
- Chair and general TAG membership terms are limited to 3 years, or 6 semesters.
- For PATT terms of reference see
http://www.stfc.ac.uk/Our+Research/4508.aspx
- The TAG meets each 6 months (semester) to assess each round of
proposals. The location is usually the home base of one of the members
to facilitate logistics, but it is the intent to take the TAG to as many
of the centres of UK submillimetre astronomy as possible.
- Behaviour of TAG must comply with Standards for Conduct in Public Life
(the
Nolan Report). See also Annex 3 of "Essential Information
for Members of Council and Advisory Bodies" (Jan 2002 - under the auspices
of PPARC).
- The members of the TAG receive nominal remuneration from
STFC for their efforts and expenses for attending the meeting
(use the STFC T&S form to claim).
Applications & Applicants
-
Definition of 'UK', vis-a-vis an applicant's nationality, is
anyone
funded >=50% by a UK institution.
- RCUH staff at JCMT should apply to PATT queue appropriate to their
nationality.
- The ITAC assesses 'regular' proposals from 'International' applicants
. . .
- . . . although the UKTAG may be requested to assess service
applications
from international applicants, since the UK queue, being the largest,
offers perhaps the only chance for these proposals to be completed.
- There is an 8-hour limit on the amount of time that may be requested
for heterodyne service applications.
There is no limit to the amount
of time that may be requested by 'regular' proposals to the
twice-yearly 'PATT' rounds.
- Proposals for UK service observing
received
between the dates of the PATT deadline (~15 Mar, ~15 Sep) and the TAG
meeting (~mid-May, ~mid-Nov) will be held in abeyance and assessed during the
TAG meeting, unless the proposal is particularly time-critical.
- The number of observers funded to travel may be 2 if the number of
shifts assigned is >=5.
Assignment of Referees & Assessors
- There are ~2 months between the deadline for receipt of proposals
and the TAG meeting to allocate the available observing time.
- Two members of the TAG are assigned to each proposal as 'Assessors'.
Starting in 03b, assignment of Assessors is done by the TAG Chair.
Care should be taken not to assign an Assessor from the same institute
as the PI, and to watch out for previous (professional)
relationships/collaborations between Assessors and PIs/CoIs, etc.
- Usually Assessors are assigned to proposals in their fields of
expertise (see 4) - and with luck the work load is divided
equally among the Assessors - but other factors may skew each
Assessor's specific load.
-
Each Assessor then has approximately N/3 proposals to assess;
each is 'First Assessor' on approximately N/6 and 'Second
Assessor' on N/6. The First Assessor is expected to provide the more
comprehensive assessment (a few paragraphs cf. a
couple from the second assessor).
-
The opinion of an expert, external referee is also sought for each
proposal. The referee is recommended by the First Assessor.
- Referees should be knowledgeable, ideally about both the
science and about submm astronomy : JCMT users are an obvious
source ! UK-based referees would be preferred in order to
imply some sense of self-regulation within our community, although,
for their particular expertise, referees based abroad are occasionally engaged.
- They will be given the opportunity to decline.
They will have ~3 weeks to report and many
will have other commitments, so prepare to find alternates.
- Do not recommend a referee from the same institute
as any of the PI/Co-Is or who may otherwise have a
relationship (good or bad) with any of the proposers that may
pervert their objectivity.
- Do not use Assessors as referees.
- Do not use
PATT or TAG members,
of any telescope TAG, to be referees - their work load
at this time of year is usually too heavy.
- Referees may be sought amongst current JCMT applicants on the
sensible grounds that they are
knowledgable about JCMT and its capabilities and have a vested
interest in the application process : or, more bluntly, if they
could spare the time and effort to submit a proposal they can also
surely referee one.
- The Chair should ensure that no referee is asked
more than twice in the same semester.
- Assignment of Assessors to service proposals is done by the Chair :
no (external) referees are required for service proposals.
Fair Play
In the interest of fairness and accountability, referees are offered the
protection of anonymity, and their comments - on the work of their peers,
after all - should be made in confidence if desired. To this end, PIs and
CoIs of proposals should not normally see referee reports, or even know
the names of the referee(s) involved. This must apply also to TAG members
who are PI or CoI on any proposal. Therefore certain additional
precautions must be taken when requesting and distributing the names of
external referees and their reports. Thus, while Assessors should normally
send their suggestions for referees to the Chair, for proposals involving
the Chair as PI/CoI the referee name should be sent directly to the
Technical Secretariat at JAC. For proposals involving the Tech.Sec, such
information should not reach him either and be communicated to the JCMT
Secretary.
These issues are securely addressed by the
Northstar
proposal handling system now in use.
In the procedures described here the symbol (*)
identifies actions that should still be carefully tailored to prevent sensitive
material or confidential reports from reaching members of the TAG or the
TechSec when they are involved as PI or CoI.
Assessment of Proposals
- Referees are given about 3 weeks to produce a report.
They are asked not to contact the PI directly with questions.
The First Assessor should co-ordinate any such questions to be posed to
the PI by the referee and any of the other assessors.
To avoid duplication of work (and the perceived harrassment of
the PI by the TAG) these e-mails, and their responses, should be copied to
both Assessors.
-
Reports from Assessors are usually produced only
a few days prior to the TAG meeting. Prior to adoption of the
Northstar
system, they were sent to
the Tech.Sec. (*) for circulation back to all
Assessors(*).
-
Assessors are expected to grade ALL proposals, and to assign a
weight to their ability to grade (see the
Appendix below).
This work should all be done blind of the other Assessors.
- Proposals are ranked by mean, weighted, normalized grade.
- The purpose of the TAG meeting is to confirm or change this ranking
through discussion based on the reports by Assessors and referees,
and to allocate the available observing time. Dynamic
grading was successfully introduced at the 04a meeting - the
TechSec's EXCEL spreadsheet of the grades being projected for all
(*) to see.
- TAG members (and the Tech.Sec) who are PI/Co-I on any proposal must
absent themselves during discussion of such proposals
(*).
- Feedback reports to PIs should be compiled by the primary Assessor
and distributed by the Chair in a timely manner after the meeting.
Long-term and Large programs
- Some proposals may require allocations of observing time over more
than the current semester for reasons of, eg, RA ranges, sequencing
with satellites, phase coverage of long-term variability. These
attract 'Long-Term' (LT) status.
PIs of LT programs must submit progress reports going into
subsequent semesters to justify continuation. Observing time for
Long-term proposals should be made available in subsequent (suitable)
semesters as agreed at earlier TAG meetings, subject to approval of
the progress report. Allocated time unused in any semester is lost.
- By a definition established in Sect 4.4(ii) of the PATT policy
meeting of 13 December 2002, "Encourag(ing) large programmes and
multi-telescope applications", a proposal requesting 8 'nights' or
more would be considered 'Large'. In the context of the
12-hour nights scheduled on JCMT, this is to be interpreted as 96
hours.
A 2 page (scientific) case for support is allowed for long-term,
multi-telescope and large programmes.
Timetable
The timetable is dictated by a couple of important dates. The deadline for
submission of proposals is usually 15 March or 15 September, and the TAG
meeting occurs mid-May or November, respectively.
[The ITAC meeting, at which final allocations are confirmed, is a week or
so after that].
So there are ~8 weeks to do all the work itemized below, but in general
terms the referees ought to be identified and invited to participate
within the first week or two, and given about 3 weeks to produce their
reports; that leaves about 3-4 weeks for the assessors (TAG members) to
complete their assessments and grades.
The timetable below is the TechSec's view of the process
(hence the odd use of pronouns) - may not make suitable general reading.
The events in the timetable are referred to the week number on which it should
occur, ideally.
The TechSec should have a JAC laptop and accessories and
a working knowledge of EXCEL to allow easy compilation of
grades, sorting by grade/rank, and production of hardcopy.
- The
Northstar system
collects the proposals, and makes them and the
combined reviews
available to the TAG and Tech.Sec., as appropriate
(*).
Week 1
- Ensure all UK applications are readable/printable.
- Make them available to the UKTAG members
and to DIANE.GREENING@stfc.ac.uk (STFC, Swindon office).
- (maybe ?) Print out a complete set if none is already available.
- Flag those eligible for RadioNet
funding.
- Check with UKTAG Chair for date of UKTAG meeting - it is
usually scheduled for a week or so before the ITAC & PATT meetings
in Swindon, and will occur therefore mid-May or Nov.
- Inform ITAC TechSec of UKTAG meeting date,
- Start planning/booking flights & hotels.
Local secretaries have previously obliged in some of these logistics.
- Make note of applications that have UKTAG members as PI/CoI.
Week 2
- Liaise with Chair/TAG members if necessary on the assignment
of Assessors/referees (see 19-23 above).
(NB : The Assessors or Chair will assign referees to proposals
involving the TechSec (*)).
- Ensure that the names and e-mail addresses of the referees are sent
to the Northstar administrator (Sam Hart)
- Ask TAG members to reply a.s.a.p. if they are uncomfortable with any
assignment.
- Check and/or edit the
cover letter that
will be sent to each referee requesting their assistance.
- Within the appropriate
Northstar
page
assign the referees. Send the
cover letter
directly to any who have previously acted as referees
(they will not receive a fresh invitation from
Northstar
).
The
cover letter
should give them 2-3 weeks to respond.
They are asked to submit/upload their response to
Northstar.
Week 3
- Wait for the referees's reports to come flooding in . . .
Week 4
- Request progress reports, including how many more shifts are needed,
from PIs of current long-term proposals.
Account for possible observing between now & end of semester.
Programs without such reports will, by rule, be discontinued!
- One week before the referee deadline send
a reminder to referees who have not
yet reported : update the e-mail addresses of referees.
Week 5 - 6
- Technical assessments are usually
assessed by JCMT support staff before the end of April/October
and uploaded into the
Northstar
reviews.
Check that they are there.
- Upon reaching the referee deadline, pester the recalcitrant referees
and/or ask TAG for alternates.
Week 6.5
- Request assessments and grades from TAG members. The Chair (2002)
has suggested this deadline be <1 week before meeting. They will
probably not respond immediately because all the referee's reports
aren't in . . . but they need to be reminded . . .
Week 7
- Collect Assessors grades and tabulate. You might have to wait until
the morning of the meeting for all of them.
Week 7.5
- Get provisional UK allocation from ITAC TechSec.
- In preparation for departure make copies of relevant supporting mail messages,
proposals & reviews,
and feedback from previous semesters.
- With departure nearing, this may be where you must leave it.
The rest, on the road, requires the laptop.
Week 8 :
On the road
Read the proposals & reports to be familiar with them all.
At the meeting
- The TAG members may provide you with last minute grades, or
corrections thereto. These must be entered into the (EXCEL)
spreadsheet and a
weighted mean grade for each proposal derived. Flag those with
individual grades differing by 2.5 or more for discussion.
- Make copies of this table for the TAG team.
A local TAG member may be able to help in use of local computing
networks to do this, but be prepared to use pen & ink & a copier . . .
- The discussion of the allocation of time to each proposal will be in
rank order,
starting with the highest ranked proposal. Keep copious notes of
comments regarding
shifts/hours, associated flex. time, weather grades, scheduling
peculiarities etc..
Dynamic grading by TAG members may be requested - having your laptop
plugged into a suitable projector will enable the TAG to view the
results.
- Ask the Chair to clarify relative grading of LTS programs
from last time.
- NB
- RadioNet-eligible
programs are to be given priority in the event
of a tie on the basis of scientific merit.
- The allocation at this meeting is provisional : the ITAC
meets after this TAG meeting and the amount of time it allocates
to the international proposals is unknown until its work is finished.
This then impacts the national queues pro rata. So there
may be small adjustments to make later to the total UK allocation.
- You should e-mail final allocations, including weighted scores,
weather bands,
flex.time & scheduling constraints to the JCMT scheduler
and to the ITAC TechSec (currently these are both IMC).
The scores are used to generate priorities for the
OMP.
Copy to DIANE.GREENING@stfc.ac.uk.
After the Meeting
- The TAG members refine their feedback reports for the PIs,
and the Chair will circulate this by e-mail. These should be
forwarded to the JCMT support scientist managing the UK queue
(the UK queue manager ought to be the TechSec !).
At the start of the new semester,
s/he will then be able to check the MSBs submitted to the
OMP
against the original proposal and against the TAG feedback to ensure
that only approved observations are scheduled and executed.
Current TAG membership
| Member |
e-mail |
First semester
of duty |
|
Stuart Lumsden
|
s.l.lumsden @ leeds.ac.uk
|
08B
Chair since 09B
|
|
Elizabeth Stanway
|
E.R.Stanway @ bristol.ac.uk
|
10B
|
|
Mark Swinbank
|
a.m.swinbank @ durham.ac.uk
|
11A
|
|
Andrew Blain
|
awb @ astro.caltech.edu
|
11A
|
|
Jane Buckle
|
j.buckle @ mrao.cam.ac.uk
|
12A
|
|
Mark Thompson
|
m.a.thompson @ herts.ac.uk
|
12A
|
|
|
JCMT Secretariat
|
|
|
Iain Coulson
|
i.coulson @ jach.hawaii.edu
|
01B
|
- Chair and general TAG membership terms are limited to 3 years
(normally 6 semesters).
- A table of past members is in preparation.
Appendix 1 : The Grading scheme
THE AIM IS TO GRADE PURELY ON SCIENTIFIC MERIT.
"The TAG must be looking
to support science that will generate the next 100+ citation papers
for JCMT, and to keep the community healthy." (anon)
Grades should be assigned according to this scheme :
- clear proposal with potentially seminal results
- potentially seminal project that hasn't been clearly explained
or has minor flaws; Is "do-able". Grade
is given for potential for good science, not proposers' ability
to explain it.
A "sexy" proposal might also make this grade occassionally
- solid proposal which may or may not get time depending on
competition for RA/Dec and time left after allocating higher ranked
proposals. Appropriate feedback should keep the project on course.
- proposal with some flaw. Appropriate feedback
could help it fare better next time.
- seriously flawed
Notes:
-
If a proposal is scientifically brilliant but requests weather that's not
the most sensible option (e.g. grade 3 to go deep, or grade 1 for only
850um work) then it should still get a high grade :
a more appropriate weather band for the requested science can be
allocated at the TAG meeting.
-
Requested observing style (e.g. backup vis-a-vis allocated) is *not* a
consideration - a
brilliant backup proposal should still get a high grade; a poor backup
proposal should *not* get grade 3 "because it was just for backup"!
The way time is allocated deals automatically with this issue and
grading should not be influenced by it.
-
If a proposal is brilliant but requests the wrong semester, we should
still give it a high grade. The feedback can then simply read "The TAG
thought this was excellent, but you requested the wrong semester, so
please apply again".
The TAG should not scare away potential new members of the JCMT community
just because they don't know the "rules".
Weights should be supplied with each grade :
- (statistical weight 1.00) "well-equipped to judge"
- (statistical weight 0.75) "adequately-equipped to judge"
- (statistical weight 0.25) "out of one's depth"
The means and s.d.'s of each Assessor's grades yield normalized
grades and the final score for each proposal is their weighted mean.
Iain Coulson i.coulson @ jach.hawaii.edu
Technical Secretary to the JCMT UKTAG
|
