Below is the first version of intragroup alltoall channel proposal
as we agreed at the last meeting. I am still working on the bipartite
version of alltoall channel.

Arkady

*********************************************************************
Proposal for alltoall collective channel.

Rationale:
Many applications need to reorganize a distributed matrix over a group
of processes. This is the first of two alltoall channels that support
that. This alltoall channel redistributes data within a group. Many
platform specific features can be used to optimize this channel beyond
what users can do themselves as well as making more performance portable
for the users.


1. Changes to section 8.
Add
8.7 Alltoall Operation Channel

This section defines operations for creating and manipulating an
alltoall collective channel. The alltoall operation is analogous to the
scatter & gather performed together. Each process scatters its message
among processes of the alltoall channel group and then gathers all the
incomming messages from all group processes into a single buffer. The
send message buffer is evenly divided into small messages with the
number of small messages equal to the size of the group. Each small
message is sent to the corresponding process of the group where the i-th
process gets the i-th small message. The arriving small buffers are than
concatenated into a single buffer that is inserted by an implementation
into a recvoutbufiter. The order of dividing a send message into smaller
messages for individual processes is the same as the order of
concatenation of receiving small messages into alltoall channel receive
message as follows the ordering of the processes of the channel group.
All send and recv buffers are of the same size and are divisible by the
cardinality of the group and dataspec size.

MPIRT_ALLTOALL_CHANNEL_CREATE (name, inbufiter, outbufiter,
recvinbuffer, recvoutbufiter, group, qos, alltoallchannel)
IN name  name of the collective channel (MPIRT_STRING_NAME)
IN  inbufiter sender side input buffer iterator (MPIRT_BUFITER)
IN  outbufiter sender side output buffer iterator (MPIRT_BUFITER)
IN  recvinbufiter receiver side input buffer iterator (MPIRT_BUFITER)
IN  recvoutbufiter receiver side output buffer iterator (MPIRT_BUFITER)
IN  group  group over which the collective channel operates
(MPIRT_GROUP)
IN  qos  channel quality of service
(MPIRT_QOS_name-of-QOS-CHANNEL-class)
OUT  alltoallchannel alltoall collective channel (ref. to
MPIRT_ALLTOALL_CHANNEL)


[C and C++ will be added here.]

[Two pair of set/get operations for recvinbufiter and recvoutbufiter
will be added here.]

[The tables Miscellaneous definitions and bindings will be changed to
add alltoall channel.
One more table for alltoall channel will be added at the end.]

2. [Changes to Chapter 2.]
Add ALLTOALL_CHANNEL to table 2.1 (page 21) for meta classes
name-of-class, name-of-class-for-free-dup, name-of-class-for-attributes,
name-of-class-for-comparison, name-of-leaf-class,
name-of-leaf-class-for-create and name-of-channel-class.

Add Alltoall_Channel to figure 2.1 on page 22.

3. [Changes to Chapters 6 & 9.]
Add Alltoall channel to first footnote on page 117 and the sentence on
page 119 lines 28-31.
Add Alltoall channel to the sentence on page 162 lines 38-42.

4. [Changes to the indexes]
Alltoall channel and its functions should appear in the function and
entities indexes at the end of the document.

[I had checked the event model for alltoall channel and it works. No
changes required for it.]
[Alltoall channel is consistent with the variable buffer proposal.]