Coordinated Development Required for Information Transfer to EVLA Scheduling & GBT DSS in 2009

NRAO Interactive Services Modification Request 8C109, February 2009



PSTandUserWebServices are documented here

1. Introduction

The purpose of this MR is to describe work to be coordinated between the GBT SDD and E2E Operations, to ensure successful delivery of the components required for the October 1, 2009 preliminary release of the GBT Dynamic Scheduling System (DSS). Development will be coordinated to ensure: 1) that each group can utilize the resources provided by the other group, 2) to promote reuse of software and prevent unnecessary duplication of effort, and 3) to demonstrate effective synergy between site-based and shared E2E software systems.

These deliverables are also required by the EVLA to support input into the scheduling process, and so prototype design and development must be vetted by EVLA software engineers.

Note that all of the deliverables from E2E to GBT must be in place by June 30, 2009 to accommodate the DSS code freeze in mid-August. Interfaces must be defined well before this deadline, by February 15, 2009, to set appropriate expectations for the developers in both domains.

2. Background

This MR captures the outcomes from a meeting held in CV on December 12, 2009, following the External Review of the DSS System on December 8 and 9. Attendees included: Amy Shelton, Nicole Radziwill, Dana Balser, Karen O'Neil, Paul Marganian, Mike McCarty, and Kai Groner in CV; Mark Clark in GB.

Updates have been made to this MR as the requirements and design has evolved.

3. Requirements

Prior to December 31, 2008 Ashish will provide Mike (mmccarty at nrao dot edu) with the up-to-date documentation for the CAS module, originally sent on 12/4/08, with additional information about the "groups" implementation however it turns out. Note that Mike received the original CAS release notes on December 9.

Requirements from Carl Bignell (GBT scheduler)

  • Need to be able to retain all of the historic information for reporting purposes. This includes keeping all people information unique, and also means that we need historical information regarding the institution and other information of investigators. As an example, assume someone proposed to use the GBT in 2007 and was affiliated with a US institution, but then moved in 2009 to a foreign institution. When we run a report for 2007 we need to count that person as "US" and in 2009 as "foreign". Currently Carl Bignell has this information, but I am unsure if it exists an any other database.

For C1 2009 (January 1 - February 15, 2009) - Ashish

  1. The DSS team is going with Django, which already has a CAS plugin.
    • For the BOS, write a Turbogears service for CAS authentication (ie. "CASify Turbogears"). Kai will investigate this with Mike's help.
  2. Adapt the GBT DSS application to use CAS-based authentication and move to Django.
  3. Ashish will prepare a web service to retrieve all contact information, keyed by person_id. The contact information should also return the proposal_id of any proposal that person_id is associated with, as well as whether they are the PI or not on that proposal.

For C2 2009 (February 15 - March 31, 2009) - Mike & Ashish

  • Develop 6 Web Services to retrieve proposal information, where a URL is used to access the following information which is then returned to the user as XML. The data should be retrieved keyed by the field identified in parenthesis. It is important to have some kind of secure connection here, either by a username/password that only other NRAO systems know, or by a key system, or by IP sniffing (developer's choice). We just want to make sure that the information cannot be intercepted over the web so people can snoop on each others' proposals. Before security is added, the GBT and EVLA developers should check and make sure they understand the XML output and are prepared to handle it - this will provide an early "interface check".
    1. Retrieve all person_id's for a given proposal, keyed by proposal_id or legacy_id
    2. Retrieve all sources for a given proposal, keyed by proposal_id or legacy_id
    3. Retrieve all resources for a given proposal, keyed by proposal_id or legacy_id
    4. Retrieve all sessions for a given proposal, keyed by proposal_id or legacy_id
    5. Retrieve all cover sheet information for a given proposal, keyed by proposal_id or legacy_id
    6. Retrieve all of the above information for a given proposal, keyed by proposal_id or legacy_id Note: Ashish, is this what you currently deliver to Carl to shuttle into his database? Perhaps we just need to expand on the current Carl interface.)
    7. Retrieve all Green Bank reservations in the BOS, keyed by a specific person_id, dated from t=0 (today) through t=t+1 year.
    8. Retrieve all Green Bank reservations in the BOS for a given date range, including person_id's.

For C3 2009 (April 1, 2009 - May 15, 2009) - Kai

  1. Merge the common user information tables used by the BOS and the PST
  2. Change the BOS Turbogears application to use the CAS authentication service Mike developed in C2
  3. Mike will test the reservations web service to make sure it still provides the needed information
  4. At the end of this development cycle, PST/DSS/BOS will all be accessing the same user database via CAS authentication

For C4 2009 (May 15, 2009 - June 30, 2009)

  • Catch up on anything not completed in previous three cycles

4. Design

Technical lead should provide a brief description of how this will be implemented in the code, or descriptions of the interfaces to use the completed web services, or LINKS to those descriptions.

During a conference call on June 9, 2009, it was originally decided that the CAS authentication account names (or 'user name') should be used for all user-related queries. However because the CAS authentication account name is not immutable, it is not an appropriate choice for universal identifier. As of a follow-up conference call on July 21, 2009, we will use the primary key from the PST Person table for all user-related queries.

4.2. BOS views of reservations

BOS reservation views will be generate in XML format using the schema below.


<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<nrao:user id="${person_id}" domestic="true" xmlns:py="http://purl.org/kid/ns#" xmlns:nrao="http://www.nrao.edu/namespaces/nrao">
    <nrao:reservation py:for="r in reservations" id="${r.res_id}">
        <nrao:startDate py:content="r.startDate" />
        <nrao:endDate   py:content="r.endDate" />
    </nrao:reservation>
</nrao:user>

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<nrao:reservations domestic="true" xmlns:py="http://purl.org/kid/ns#" xmlns:nrao="http://www.nrao.edu/namespaces/nrao">
    <nrao:reservation py:for="r in reservations" id="${r.res_id}" >
        <nrao:user id="${r.person_id}" />
        <nrao:startDate py:content="r.startDate" />
        <nrao:endDate   py:content="r.endDate" />
    </nrao:reservation>
</nrao:reservations>

5. Deployment Checklist

Check to make sure:
  • Updates to profile information on the BOS side should be successfully propagated and viewable on PST User Database side
  • Updates to profile information on the PST User Database side should be successfully propagated and viewable on BOS side

Does the PST documentation need to be updated? Do the users' expectations need to be set at all regarding these changes?

6. Test Plan

6.1 Internal Testing

6.2 Sponsor Testing

6.2.1 BOS deliverables

Sponsors can test BOS deliverables from C2 by viewing the following XML resources on a test server. The test server is not always running, contact mmccarty@nrao.edu to start the BOS test server.

I have sponsor tested and found two issues.
1) When you give each of the services a date prior to the year 1900 it gives lots of yucky errors. This may be trapped in the code that uses the web service, but the error should be a little more graceful.
2) When you make the start date later than the end date, it still gives you "" - which may be OK, but you should double check.
--Main.NicoleRadziwill -- 15 April 2009

6.2.2 PST deliverables

6.2.3 Existing my.nrao.edu services

6.3 Integration/Regression Tests


Signatures

APPROVED: I acknowledge that my request is fully contained in this MR, and if the Open Sky (or other NIS or PST developers) deliver exactly what I specified, I will be happy.

ACCEPTED: I acknowledge that I have validated the completed code according to the acceptance tests, and I am happy with the results.

Written - - - - -
Checked - - - - -
Approved by Scientific Sponsor - - - - -
Accepted/Delivered by Sponsor NicoleRadziwill -- 17 April 2009

Symbols:
  • Use %X% if MR is not complete (will display ALERT!)
  • Use %Y% if MR iscomplete (will display DONE)


Discussion Area

Referring to C2 2009, above:

  • It would be helpful to get the xml schema file for a proposal as soon as it is ready. EVLA will be able to do significant work once this schema is in hand, even if none of the services are ready.

  • Item #6, retrieving a full proposal by proposal ID, will probably be the one most used by EVLA. We will fetch a proposal for the purpose of initializing an observational setup.

  • It will probably be useful to ask for all proposals, perhaps as list of proposal IDs?, that were approved in a particular cycle. Even better might be all proposals, for telescope T, approved in cycle C.

-- DavidHarland - 06 Jan 2009

-- NicoleRadziwill - 16 Dec 2008
Topic revision: r18 - 2009-07-27, AmyShelton
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback