--
JimJacobs - 2011-01-12
Discussion on Asynchronous I/O and Possible Usage Patterns
- Held 11 January 2011 at 10 a.m. in AOC
- Abstract: The current design and implementation of the Asynchronous I/O enhancement to CASA was presented by Jim Jacobs and potential users of the new feature provided useful feedback.
Conclusions
1. The use of VisBuffer::msColumns() to directly access the columns of the table
The normal use of this function is to get at MS-wide information related to the current MS. Therefore, as long as the columns returned correspond to the same MS as the data currently in the
VisBuffer there is no problem. In my current implementation I cached a pointer to the
MeasurementSet object and then use it to create an ROMSColumns object which is returned to the caller. That would seem to meet the requirements.
2. Expand the prefetch columns facility to allow to allow control over
information prefetch for computed values (e.g., lsrFreq (freq) and
lsrFreq (spw, freq), etc.).
3. Accessing the visibility
VBA offers
ObservedVisibility and
ObservedVisibilityCube as independent
prefetch options (the same for the corrected and model flavors).
4. The double pass through a chunk
VisibilityIterator vi (ms);
VisBuffer vb (vi);
for (vi.originChunks();
vi.moreChunks();
vi.nextChunk()){
for (vi.origin(); vi.more(); vi ++){
DoSomethingWithSubChunk (vb);
}
for (vi.origin(); vi.more(); vi ++){
DoSomethingElseWithSubChunk (vb);
}
}
This won't work right now. I would have to build a mechanism to
interrupt the VLAT thread and have it back up to the start of the
subchunk. It's similar to what is done when the user calls originChunks
now so it's certainly doable. It would add a little complexity to the
code but should only impose a performance cost to code that actually
does the double pass.
5. Using async I/O in situations requiring writing.
While the actual writing may not benefit from asynchronous I/O the
corresponding reads might well benefit from using async i/o. There are
two solutions to this. (A) The first is to add the ability to write to
the async I/O implementation; this would require keeping sufficient
information in the
VisBuffer to allow writing to occur behind the actual
position of the
VisIter. (B) Another solution that might be as
efficient would the use two of VIs using the semi-pseudocode below.
VisBuffer vbR (viR); // read only
VisBuffer vbW (viW); // write only
for (viR.originChunks(), viW.originChunks();
viR.moreChunks(), viW.moreChunks();
viR.nextChunk(), viW.nextChunk()){
for (viR.origin(), viW.origin(); viR.more(), viW.more(); viR ++, viW++){
Flags flags = calculateFlags (vbR);
vbW.saveFlags (flags);
}
}