-- JimJacobs - 2011-01-12

Discussion on Asynchronous I/O and Possible Usage Patterns

  • Held 11 January 2011 at 10 a.m. in AOC

  • Abstract: The current design and implementation of the Asynchronous I/O enhancement to CASA was presented by Jim Jacobs and potential users of the new feature provided useful feedback.

Conclusions

1. The use of VisBuffer::msColumns() to directly access the columns of the table

The normal use of this function is to get at MS-wide information related to the current MS. Therefore, as long as the columns returned correspond to the same MS as the data currently in the VisBuffer there is no problem. In my current implementation I cached a pointer to the MeasurementSet object and then use it to create an ROMSColumns object which is returned to the caller. That would seem to meet the requirements.

2. Expand the prefetch columns facility to allow to allow control over

information prefetch for computed values (e.g., lsrFreq (freq) and lsrFreq (spw, freq), etc.).

3. Accessing the visibility

VBA offers ObservedVisibility and ObservedVisibilityCube as independent prefetch options (the same for the corrected and model flavors).

4. The double pass through a chunk

VisibilityIterator vi (ms);
VisBuffer vb (vi);

for (vi.originChunks();
      vi.moreChunks();
      vi.nextChunk()){

    for (vi.origin(); vi.more(); vi ++){

        DoSomethingWithSubChunk (vb);
     }

    for (vi.origin(); vi.more(); vi ++){

         DoSomethingElseWithSubChunk (vb);
     }

}

This won't work right now. I would have to build a mechanism to interrupt the VLAT thread and have it back up to the start of the subchunk. It's similar to what is done when the user calls originChunks now so it's certainly doable. It would add a little complexity to the code but should only impose a performance cost to code that actually does the double pass.

5. Using async I/O in situations requiring writing.

While the actual writing may not benefit from asynchronous I/O the corresponding reads might well benefit from using async i/o. There are two solutions to this. (A) The first is to add the ability to write to the async I/O implementation; this would require keeping sufficient information in the VisBuffer to allow writing to occur behind the actual position of the VisIter. (B) Another solution that might be as efficient would the use two of VIs using the semi-pseudocode below.

VisBuffer vbR (viR); // read only
VisBuffer vbW (viW); // write only

for (viR.originChunks(), viW.originChunks();
      viR.moreChunks(), viW.moreChunks();
      viR.nextChunk(), viW.nextChunk()){

    for (viR.origin(), viW.origin(); viR.more(), viW.more(); viR ++, viW++){

         Flags flags = calculateFlags (vbR);
         vbW.saveFlags (flags);

     }
}
Topic attachments
I Attachment Action Size Date Who Comment
async-io-interim.odpodp async-io-interim.odp manage 431 K 2011-01-12 - 10:31 JimJacobs Slides for the Async I/O Design/Implementation discussion of 11 January 2011
Topic revision: r1 - 2011-01-12, JimJacobs
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding NRAO Public Wiki? Send feedback