PST Feedback (October 2006)

Summary from Joan Wrobel

At the 2006 October 2 deadline, a total of 217 proposals were received
via the PST.  These were split as 122 VLA and 95 GBT.  Feedback was
requested from 163 proposal editors, i.e., the individuals who
initiated the GBT and/or VLA proposals.  A one-week deadline was
advertised for feedback responses.  27 responses were received, mostly
within the first few days.

The table below gives a summary of the types of trouble reported.

     =======================================
     Number of Mentions  Trouble Bin Keyword
     =======================================
                      7  [improve docs]
                      6  [saving]
                      4  [browser-pdf]
                      3  [no service]
                      3  [abstract]
                      3  [emails]
                      2  [concurrency]
                      2  [GBT resources]
                      1  [VLA resources]
                      1  [overload]
                      1  [RA range bug]
                      1  [RA sort bug]
                      1  [view sessions bug]
                      1  [student bug]
     =======================================

The text of the 27 responses appears below.  Appropriate trouble
keywords have been added to the responses.  The responses contain
quite a few positive comments.  The most frequent enhancement request
was to be able to copy sources, resources and/or sessions.

Cheers.  - Joan

---------------------------------------------------------------------

[no service]

   The only problem I had was an inability to accees the system on
Sunday evening from about 10pm - midnight (pacific time).  I didn't
want to wait to Monday to submit.

------------------------------------------------------ End Response 1

[RA range bug]

  I submitted a proposal to do a mosaic of 14 pointings.  I could not
get the RA range to work. Each time I entered a number it came back to
me saying that it could not accept because the RA range was zero.

I got around the problem by getting the coordinates from NED, but then
it didnot even ask for an RA and DEC range.

Perhaps I misunderstood, the whole thing?

------------------------------------------------------ End Response 2

I have to say that was proposal tool that the NRAO use is really nice,
Is was easy and useful, because it keeps the previous proposal
history.

I only can give a positive feedback about it.  I don't know if I have
to do something more for the feedban on the NRAO PST, please make me
know if it is the case.

------------------------------------------------------ End Response 3

[browser-pdf]

I was unable to use Firefox to upload PDF Technical Justifications.  I
tried Firefox on both Linux and Mac computers, and was successful only
by switching to Safari on my Mac.

Otherwise, the tool worked well.

------------------------------------------------------ End Response 4

[saving] [abstract]

A few issues came up for me during the latest submission round (GBT
only):

- The tool gives warnings when I use rest frame lines that are not
  in-band.  Perhaps this is intentional, but I get heaps of warnings
  because I propose to observe lines at moderate to high redshift.

- Several times, the sessions tab failed to save my changes, even when
  I chose "Save".  I would resume working on the proposal at a later
  time to find that all of the carefully programmed sessions were
  simply gone.  Also, sometimes adding a session didn't take and I had
  to add it multiple times.

- The abstract field removes "<" and ">" symbols.

Otherwise, things went pretty smoothly.

------------------------------------------------------ End Response 5

[saving?]

I had a couple of problems with the PST this time:

(1) Time listed at the top of the menu appeared to be off by an hour,
making it look like I had more time until the deadline than in
reality.

(2) One one occasion, the PST went into a mode when it sat "thinking"
if I tried to click on any button.  I eventually gave up and
relaunched it from another Mozilla window, from which things worked
fine at once.  I'm not sure what caused this, unfortunately, so this
probably isn't too helpful for the E2E team.

------------------------------------------------------ End Response 6

[browser-pdf] [saving]

Well, you asked for this:

I was not impressed with the PST.  It took me two full days to submit
two proposals even thought I was prepared -- I estimate that it should
have taken me about 1/2 hour for each if the PST were working
correctly.

The PST has problems with PDF files and the browser that one uses to
upload them; this problem wasn't even mentioned in the PST application
home page itself.  One has to learn it the hard way it seems.

Moreover, several times over the course of 3 or 4 days of working with
the PST, information was lost because the SAVE command seemed to have
a mind of its own.

On one of the proposals I used 32 resources and that was a real chore
to enter the repetitive data from one similar resource to another; it
would be real nice if the user could set his/her own defaults as
regards resources.

NRAO personnel were quite helpful when problems are encountered, but I
think the problems that happened shouldn't be there in the first
place.  No wonder that neophyte radio researchers from other
disciplines think it too hard to propose to NRAO ....

------------------------------------------------------ End Response 7

[saving]

just to say that I managed to get my proposal in eventually but I
still did have problems even after my proposal had "returned" to the
list. I found that the only way to not get errors is to press "save"
after *every* single action in each tab. That seemed to work. I guess
by doing this I was re-writing over corrupted files which may have
been causing the problems.

Might be useful for others if they have similar problems.

------------------------------------------------------ End Response 8

[overload] [view sessions bug] [no service]

Here is my feedback on the tool:

I submitted 3 problems to the PST helpdesk when I was creating my
proposals.

1 The 'make PDF' function did not work at all during the first
  proposal creation.  I just got a little white x-term called gecko
  that sat there an tried to look innocent.  When I was doing my
  second proposal, the 'make PDF' function worked again.  helpdesk
  thinks this may have been related to traffic load.

2 The tool itself completely hung when I tried to upload a pdf file
  once for the sci justification.  I couldn't even save or log out and
  my browser stopped responding as well.  I restarted the browser and
  the PST and tried to access the proposal but I got only error
  messages.  I thought I had lost many hours worth of work and sent a
  panic note to the helpdesk.  After a few minutes I tried again
  through a different route and managed to get the propsal so the
  problem wasn't persistent.  Again, helpdesk thinks this is a traffic
  load problem.

  The above 2 problems dramatically increased my frustration level
  even though they were not serious (especially when I thought I had
  lost the proposal in the database).  I think more work needs to be
  done to handle the traffic load expected in the last 8 hours before
  a proposal deadline.

3 When I was editing sessions I suddenly got to a point where I could
  no longer see the source/resource pairs in each session at the
  bottom of the page.  I could edit the session and see what was
  inside but this was a pain.  helpdesk thinks this was an actual bug
  in the software and they are trying to reproduce the problem.

Besides the above problems, the tool worked well.  The tabs did not
bother me as much as I thought they would but the size of all the
boxes with information in them made it difficult to see what was on a
single page (I had 9 resources and 19 sessions).  If you can manage to
not have so much unused space between fields or not have so many
independent boxes (e.g. one for each resource or session), this would
help.  I would prefer a table layout listing all the resources and
sessions so I can compare and check them more easily.  Anything to
increase the information density on a page so you can relate what is
in the tool with what is needed would be helpful.

ps, On monday the PST web interface had at the top of the page in
large red letters a notice that the PST would go down at noon (or was
it 10am?) EDT for some updates and that everyone should be logged off
by then or lose their inputs.

I don't think update/upgrades should be made on the day of the
proposal submission.  We had 2 people reviewing the proposal in
Scotland and on the East Coast, and one person editing in Socorro
around this time.  The additional constraint posed by this update made
things even more difficult.

Did this even happen?  I was logged into the PST but made sure I was
not editing a proposal at the time the update was supposed to
happen. The tool didn't go down which makes me think this was just a
scary left-over message from another day.  If so, please ask the
developers to check the version of the tool and make sure such
messages are not on the screen just before the proposal deadline!

------------------------------------------------------ End Response 9

[emails] [GBT resources]

 I never got an email that my proposal was successfully read by a
human. Did it make it there alright?

 I also got a warning message (upon submission) from the PST that
K-band frequencies need to be below 22.0 GHz. I ignored this error
since such a restriction would preclude water observations, which are
obviously possible.  I am wondering if the PST read my frequencies
correctly? I look forward to some verification that my proposal makes
sense to a human.

------------------------------------------------------ End Response 10

[GBT resources]

I assume that people reported the center-frequency bug for the GBT
receivers?  Instead of just allowing just one (or two, etc) entries,
it made us put in center frequencies for all possible IFs.

------------------------------------------------------ End Response 11

[browser-pdf?] [saving] [no service]

  I successfully submitted my proposal but on the half way I was not
able to see my scientific justification file (although it was
successfully uploaded during filling the first page where we put
abstract). Then I was not able to save but I came out of PST. There
was a message in the screen that the system is going for an update and
an internal error occurred etc.. By then only 5- hours left for the
dead-line time. When I restarted most of the things were there, I
reloaded the scientific justification file and things went
successfully.

  Please take care of it in future versions for smooth submission, as
many like me, submit it at the last moment and such things would scare
us.

  BUT it was brilliantly simple and smooth. I even had not read the
help document but could successfully do it. Thanks a lot.

------------------------------------------------------ End Response 12

Just wanted to make a few comments about the new PST.  Hope they are
helpful.

First of all, I think it's really getting there.  A number of problems
I'd had with the original PST have been addressed.  I worked on 5
proposals this round and had a much easier time of things than in May
of this year.

I might be misremembering, but I was convinced the last version of the
PST allowed one to copy sources.  I had a scenario in which I had
multiple telescope pointings that were very similar in position; it
would have saved a little time to just copy one source and change the
minutes of RA and arcminutes of DEC and the source name, rather than
rewriting everything for each source.  Same goes for sessions.  I had
a few sessions for which the only thing that was different was the
source name; again it would have saved time to be able to just copy a
previous session.

I'll also reiterate an opinion that I've held since March of 2005 when
Dale Frail asked me take the first PST test ride: the 80 character
limitation for the title is a real pain and limiting.  This is the 2nd
time I've had a proposal whose working title was about 85-90
characters.  I think Dale told me this restriction was suggested by
the TAC and schedulers.

Finally, I found the requirement that the scientific justification had
to be a PDF file a really arbitrary one.  PostScript has served as a
standard justification format for quite a long time.  The link from
the PST guide did a fine job explaining how to make a quality PDF file
from the source LaTeX so this wasn't too big a deal.

In terms of a wish-list feature... as a co-author on a number of
proposals, I find that a number of the PIs make copies of these
proposals.  Then they sit there on MY proposal list taking up space.
I don't want to see these when trying to find a proposal.  The only
solution I can think of is to include a button that would change the
list to include only SUBMITTED proposals, then a button that would
display only DRAFT proposals, and finally, a button that displays the
default of ALL proposals.  There is currently the ability to sort on
Status, but then what if I want to also sort on PI Name to see which
first-author proposals I've already submitted?  I'm just now getting
to the number of proposals (15) where being selective in what I'm
seeing will become useful.

------------------------------------------------------ End Response 13

[abstract]

I received an email last night asking for feedback on the PST system
for NRAO proposals.  I found the PST to be very easy to use, and
well-documented.  I prefer it to the old LaTeX template that one had
to use before.  There was only one issue that I found to be
troublesome.  There seems to be some discrepancy between the word
count displayed for the abstract and the number of words that the
system thinks there is.  My abstract was displayed as being 197 words
long, but when I came to submit the proposal, I was told that the
abstract was too long, despite appearing to be less than the maximum
of 200 words.  I had to excise 5 or 6 words before the system would
accept the proposal.  It would be good if this discrepancy could be
addressed and fixed.  Other than that, I was very happy with the new
tool.

------------------------------------------------------ End Response 14

[saving]

 the tool for proposal is a good improvement for sharing the proposal
with the co-i in real time.

Still on my system (linux fedora core 4 - firefox) saving was quite an
issue and I lost two times my work. The option of autosaving before
session expires apparently did not work for me.  This is something you
may want to check.

Also this format makes impossible to cut and paste things from old
proposals. For example copying the sources and/or resources could be
useful for a project that was already proposed before.  An example may
be the JCMT proposal tool that organizes all the XML blocks in icons
that can be cut and paste even from the proposal to the observing
tool.

------------------------------------------------------ End Response 15

[improve docs]

my reaction is rarely positive to tools of this nature, but the NRAO
system is an exception. i thought it was excellent. we had a
last-minute calamity, but it would be churlish to deny the tool worked
well overall!

my only reservations concerned the part where one inputs noise levels
and the number of tracks for a source or group of sources. it was
difficult to understand when an update/save was being done. also, the
noise displayed on the submitted version corresponds to each source in
a group, so where i said "6hr for EGS-1 will reach 0.03mJy rms" or
words to that effect, the final submission says that 0.8hr will reach
0.03mJy rms since there are 7 sources in the group EGS-1. not a big
problem, but a new user of the tool (like me) will get caught out.

although a deadline is a deadline, i think it would be an idea to have
a formal 24hr grace period, after 6pm EDT, when calm has returned, to
routinely allow people to send minor corrections to their proposals
and have someone at NRAO (Susan, this last time, God bless her!) go
through the proposals and make those corrections. probably a day or so
of work, but i don't see the point of the proposals suffering for want
of a little pampering. it's just common sense. thankfully that seems
to exist in spades at NRAO already.

------------------------------------------------------ End Response 16

[improve docs]

I missed the info about introduction of the PST and had to learn it in
10 minutes, without having time to read the instructions. It was
possible, thus the system is fine, except that the final submission
procedure was unclear after completion of the pages.  I realized that
an extra submission button exists only when I listed my active
proposals.

------------------------------------------------------ End Response 17

[improve docs] [RA sort bug]

Well, you asked for feedback...

1. I really would have liked to be able to sort my sources by RA
and/or by Dec, but the sort function didn't seem to work.  I already
told Susan L. about it.

2. The other thing that would be really handy, for a proposal like
mine with a lot of sources at varying declinations, would be a chart
showing when all the sources are up.

3.  The bit about losing your work after 30 minutes of inactivity was
a pain in the neck!  I seem to get interrupted a lot, and it's really
nobody's fault.  I mean I only lost bits of work 3 times, so I suppose
it's not all that bad.  But still, we're not really used to losing our
work; NOAO's proposal tool doesn't have that problem, for example.  Is
there really no workaround?

------------------------------------------------------ End Response 18

[improve docs] [student bug]

I had a few comments regarding the new version of the PST after having
submitted proposals to the GBT for this, and previous semesters.

First off, I love the fact that you can simply copy an old proposal
into a new one, so as to avoid having to fill in all of the same
information a second time. This saved me alot of time, especially on
proposals with many co-authors.

It would be good if there was a clearer definition for GBT proposals
as to how long a proposal may be. On the website:

http://www.gb.nrao.edu/gbtprops/generalproposalinfo.shtml

it simply states:

"- Scientific justification should not exceed approximately 1000 words."

while on the GBT PST website it states:

"- Only one attachment is allowed and it must be limited in size to 5
MB or less. Note that large files will slow down the responsiveness of
the PST."

I could not immediately find any mention of the acceptable (or
recommended) number of pages for figures or science+technical case,
nor the acceptable font size. The new PST only accepts proposals where
the science justification is 4 pages.

I like the word count feature for writing the abstract.

There was one feature which I do not understand, that resulted in an error
when submitting one of my proposals this semester. One of the co-authors
on my proposal must have requested student funding on a previous proposal,
as his name appeared automatically came up in student support section.
There were no students on the proposal, so it led to an error when I tried
to submit.

Many thanks for the development of such a great online resource!

------------------------------------------------------ End Response 19

[saving] [concurrency]

thanks for providing us with the excellent submission tool.  It was
easy to use and fairly problem-free.

Two issues have been noted during the update process. Previously
stored and unaltered "sessions" were lost twice in the process while
altering other parts of the proposal. Small chance of user mistake
makes this likely a system issue.  No error messages, sessions would
just dissapear.  Also, an issue with multiple
investigators... Significant delay have been noted between the time
one user would sign off from editing proposal and the time system
would allow another user access to the proposal. We eliminated all
local problems related to this (erasing cookies and cache on both
users machines, re-loading pages and re-signing in; even closing the
browsers. Log out have been done using PST logout button,...). The
longest delay noted was close to 30 minutes.

Hope our feedback will help you to further improve the already
excellent PST.

------------------------------------------------------ End Response 20

[abstract] [improve docs] [emails]

Overall I found the PST to be very useful and easy to use, and I had
few troubles submitting my VLA proposal, but there are a few minor
issues that I would like to raise to help improve the tool.  I hope
these are helpful.

- Abstract Word Count - There seems to be a problem with the word
count for the abstract which meant I had to reduce my abstract below
the 200 words allowed.  I had an abstract with 198 words (the PSTs
count) but even though it displayed 198 it would not accept the
abstract until I had reduced it further.  I eventually found 194 was
the most words I was allowed.

- Ambiguous Error Messages - I entered my 92 sources into the PST
using the ASCII file method.  My positions were initially given with
four decimal places, which produced an error saying that "Minutes
should be less than 60," which made no sense to me.  After reviewing
the PST manual and with a bit of experimenting I found what the
problem was, but if the error message were clearer it would have saved
me some time submitting.  The error message for the abstract error
above was similarly nonsensical.

- Low Source Position Accuracy - In relation to the previous problem,
one decimal degree/hour is not very high source position accuracy.
While these exact numbers aren't going to be used for the actual
observations, it would still be good to be allowed higher accuracy

- Constant Page Reloads - While I was entering information into the
PST it seemed that most of the things I did triggered another page
reload.  If I only had one object/observing session and a few co-Is
this wouldn't be a problem, but I had 92 galaxies, 16 observing
sessions, and 18 co-Is to enter.  Given it would often take up to five
reloads to enter an observing session, this got very tedious
especially since the scroll position would reset every time it
reloaded.  While modern browsers and internet connections are much
better at this sort of thing than they use to be, you should still
minimise the number of times the page needs to reload.

- Show/Hide Comments - On the Sessions tab, you have an option to
show/hide the comments for each of the observing session.  With 16
sessions, to show all the comments I entered so they could be checked
I had to click that option 16 times, again more page reloads.  And if
I added more sessions then all the ones I'd already done would reset
to hide.  You should put a Show/Hide All button there to make it
easier

- Emails To Co-Is - My co-Is have been reporting that they didn't
receive a conformation of submission, which the PST manual says they
should receive (as the PI I did get one though).  This may just be
something you just haven't implemented yet.  I'm also not sure if any
of those who were not on the NRAO users database before I entered them
into the system received a message about registering in the database,
as it also says in the manual.

------------------------------------------------------ End Response 21

[emails]

I only had one problem with the tool.  I think the automated password
sending tool triggers some spam filters.  I have forgotten the
password for the account I set up at my previous institute and needed
to get another name set up.  Otherwise, it works well.

------------------------------------------------------ End Response 22

[browser-pdf]

Overall, I had a much better experience with the proposal tool this
time than last term when I used it. Everything just seemed to work
better. It was also very helpful to be able to apply for student
support simultaneously.

However, when I tried to upload my scientific justification, the tool
would not save it. I finally had to email the help staff in a panic,
and Susan Loveland (very promptly) uploaded my file for me. She
suggested that the problem may have had something to do with the
browser I was using, Safari; this is a problem that should probably be
addressed.

------------------------------------------------------ End Response 23

[improve docs] [saving]

I like the web-based proposal submission. Better, easier and more
modern than the old Latex-files.

Some small suggestions 

- Combining the sources with sessions can be a little confusing, and
maybe including an example in somewhere can be useful.

- If I remember correctly, the WSRT proposal tool can add RA and Dec
from NED to the sources. That is very convenient for large samples.

- Also, I added a source+session at the end, and it took me three
tries before the new source was accepted.

But I think the tool is a success!

------------------------------------------------------ End Response 24

[concurrency] {VLA resources]

We are a group from University of Barcelona who sent 2 proposals last
deadline using the Proposal Submission Tool (PST), and we think that:

1) The fact that the PST lets every Co-I enter and modify the proposal
is a very useful capability, but we had some problems. A few hours
before the deadline, the PI of the proposal was editing it in France,
and meanwhile other Co-Is tried to edit the proposal from Spain. The
system did not allow us to work at the same time, as was
expected. However, instead of warning the Co-Is about it, the result
was that the PST stopped working in France (where the PI was!), and
could not work any more on it until 1 hour later.  Of course, all of
us (PI, and Co-Is) were really worried.

We strongly suggest that the PST should warn a Co-I who is trying to
edit a proposal which is currently being edited by another Co-I, and
should never stop working for the Co-I who was first editing the
proposal.

2) A second aspect for improving the PST concerns the simultaneous
observation of two lines, like NH3(1,1) and NH3(2,2). These two lines
must be entered as different Resources, and thus the time assigned in
the Session Page is confusing, since one must write X hours for
NH3(1,1), and X hours for NH3(2,2), while one only wants X hours for
NH3(1,1) AND NH3(2,2).

It would be interesting that the simultaneously observed lines can be
edited as a single Resource.

------------------------------------------------------ End Response 25

I found the NRAO PST very useful and efficient. I had only a minor
problem that I reported in the "Comments" field in the following way:

We got the following Warning: "For session Sess-FOV2 there is a time
discrepancy between the sum of the resource/source times, 12.0, and
the session time, 6.0." In the "Scientific and Technical
Justification" we explain that we need 5 fields of view (FOV) to
observe the 8 sources (galaxy clusters) of the present
proposal. Actually, one FOV contains 3 sources (here grouped with the
name FOV1), another FOV contains 2 sources (group FOV2), the other 3
sources will be observed in 3 separate FOVs. Each source
(i.e. cluster) has to be observed for 6 hours in total. In the case of
FOV1 and FOV2 the total observing time is 6 hours, and of course not
18 or 12 hours, even if they contains 3 and 2 sources
respectively. For this Session ("Sess-FOV2") the "Total Session Time"
(fixed manually) is therefore 6 hours.

------------------------------------------------------ End Response 26

[improve docs]

I used the PST to submit a GBT proposal.  In general, I found the
system okay, but there were lots of fields that appeared to be
extraneous. It was unclear, for example, what the diffrerence between
the resource name and group were.  Or why both were needed.

Also, fields like "rest frequency" are fine, but the tool complains
when it is out of the observing band (apparently not comprehending the
concept of redshifted lines).

Finally, I found the whole "session" concept to be rather cumbersome.
A simple way to state that I want to observe a list of sources, all
with one observing setup, would be nice.  The system, as it is
currently constructed, basically requires the user to construct two
source lists: the original one, and then one with the observing
parameters attached. This could be a lot simpler.

------------------------------------------------------ End Response 27