summaryrefslogtreecommitdiff
path: root/doc/rfc/rfc1045.txt
diff options
context:
space:
mode:
Diffstat (limited to 'doc/rfc/rfc1045.txt')
-rw-r--r--doc/rfc/rfc1045.txt7130
1 files changed, 7130 insertions, 0 deletions
diff --git a/doc/rfc/rfc1045.txt b/doc/rfc/rfc1045.txt
new file mode 100644
index 0000000..f42d07d
--- /dev/null
+++ b/doc/rfc/rfc1045.txt
@@ -0,0 +1,7130 @@
+
+
+Network Working Group David Cheriton
+Request for Comments: 1045 Stanford University
+ February 1988
+
+
+ VMTP: VERSATILE MESSAGE TRANSACTION PROTOCOL
+ Protocol Specification
+
+
+
+STATUS OF THIS MEMO
+
+This RFC describes a protocol proposed as a standard for the Internet
+community. Comments are encouraged. Distribution of this document is
+unlimited.
+
+
+OVERVIEW
+
+This memo specifies the Versatile Message Transaction Protocol (VMTP)
+[Version 0.7 of 19-Feb-88], a transport protocol specifically designed
+to support the transaction model of communication, as exemplified by
+remote procedure call (RPC). The full function of VMTP, including
+support for security, real-time, asynchronous message exchanges,
+streaming, multicast and idempotency, provides a rich selection to the
+VMTP user level. Subsettability allows the VMTP module for particular
+clients and servers to be specialized and simplified to the services
+actually required. Examples of such simple clients and servers include
+PROM network bootload programs, network boot servers, data sensors and
+simple controllers, to mention but a few examples.
+
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Table of Contents
+
+1. Introduction 1
+
+ 1.1. Motivation 2
+ 1.1.1. Poor RPC Performance 2
+ 1.1.2. Weak Naming 3
+ 1.1.3. Function Poor 3
+ 1.2. Relation to Other Protocols 4
+ 1.3. Document Overview 5
+
+2. Protocol Overview 6
+
+ 2.1. Entities, Processes and Principals 7
+ 2.2. Entity Domains 9
+ 2.3. Message Transactions 10
+ 2.4. Request and Response Messages 11
+ 2.5. Reliability 12
+ 2.5.1. Transaction Identifiers 13
+ 2.5.2. Checksum 14
+ 2.5.3. Request and Response Acknowledgment 14
+ 2.5.4. Retransmissions 15
+ 2.5.5. Timeouts 15
+ 2.5.6. Rate Control 18
+ 2.6. Security 19
+ 2.7. Multicast 21
+ 2.8. Real-time Communication 22
+ 2.9. Forwarded Message Transactions 24
+ 2.10. VMTP Management 25
+ 2.11. Streamed Message Transactions 25
+ 2.12. Fault-Tolerant Applications 28
+ 2.13. Packet Groups 29
+ 2.14. Runs of Packet Groups 31
+ 2.15. Byte Order 32
+ 2.16. Minimal VMTP Implementation 33
+ 2.17. Message vs. Procedural Request Handling 33
+ 2.18. Bibliography 34
+
+3. VMTP Packet Formats 37
+
+ 3.1. Entity Identifier Format 37
+ 3.2. Packet Fields 38
+
+
+
+
+
+
+
+Cheriton [page i]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ 3.3. Request Packet 45
+ 3.4. Response Packet 47
+
+4. Client Protocol Operation 49
+
+ 4.1. Client State Record Fields 49
+ 4.2. Client Protocol States 51
+ 4.3. State Transition Diagrams 51
+ 4.4. User Interface 52
+ 4.5. Event Processing 53
+ 4.6. Client User-invoked Events 54
+ 4.6.1. Send 54
+ 4.6.2. GetResponse 56
+ 4.7. Packet Arrival 56
+ 4.7.1. Response 58
+ 4.8. Management Operations 61
+ 4.8.1. HandleNoCSR 62
+ 4.9. Timeouts 64
+
+5. Server Protocol Operation 66
+
+ 5.1. Remote Client State Record Fields 66
+ 5.2. Remote Client Protocol States 66
+ 5.3. State Transition Diagrams 67
+ 5.4. User Interface 69
+ 5.5. Event Processing 70
+ 5.6. Server User-invoked Events 71
+ 5.6.1. Receive 71
+ 5.6.2. Respond 72
+ 5.6.3. Forward 73
+ 5.6.4. Other Functions 74
+ 5.7. Request Packet Arrival 74
+ 5.8. Management Operations 78
+ 5.8.1. HandleRequestNoCSR 79
+ 5.9. Timeouts 82
+
+6. Concluding Remarks 84
+
+I. Standard VMTP Response Codes 85
+
+II. VMTP RPC Presentation Protocol 87
+
+
+
+
+
+
+
+
+Cheriton [page ii]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ II.1. Request Code Management 87
+
+III. VMTP Management Procedures 89
+
+ III.1. Entity Group Management 100
+ III.2. VMTP Management Digital Signatures 101
+
+IV. VMTP Entity Identifier Domains 102
+
+ IV.1. Domain 1 102
+ IV.2. Domain 3 104
+ IV.3. Other Domains 105
+ IV.4. Decentralized Entity Identifier Allocation 105
+
+V. Authentication Domains 107
+
+ V.1. Authentication Domain 1 107
+ V.2. Other Authentication Domains 107
+
+VI. IP Implementation 108
+
+VII. Implementation Notes 109
+
+ VII.1. Mapping Data Structures 109
+ VII.2. Client Data Structures 111
+ VII.3. Server Data Structures 111
+ VII.4. Packet Group transmission 112
+ VII.5. VMTP Management Module 113
+ VII.6. Timeout Handling 114
+ VII.7. Timeout Values 114
+ VII.8. Packet Reception 115
+ VII.9. Streaming 116
+ VII.10. Implementation Experience 117
+
+VIII. UNIX 4.3 BSD Kernel Interface for VMTP 118
+
+Index 120
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page iii]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ List of Figures
+
+ Figure 1-1: Relation to Other Protocols 4
+ Figure 3-1: Request Packet Format 45
+ Figure 3-2: Response Packet Format 47
+ Figure 4-1: Client State Transitions 52
+ Figure 5-1: Remote Client State Transitions 68
+ Figure III-1: Authenticator Format 92
+ Figure VII-1: Mapping Client Identifier to CSR 109
+ Figure VII-2: Mapping Server Identifiers 110
+ Figure VII-3: Mapping Group Identifiers 111
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page iv]
+
+RFC 1045 VMTP February 1988
+
+
+1. Introduction
+
+The Versatile Message Transaction Protocol (VMTP) is a transport
+protocol designed to support remote procedure call (RPC) and general
+transaction-oriented communication. By transaction-oriented
+communication, we mean that:
+
+ - Communication is request-response: A client sends a request
+ for a service to a server, the request is processed, and the
+ server responds. For example, a client may ask for the next
+ page of a file as the service. The transaction is terminated
+ by the server responding with the next page.
+
+ - A transaction is initiated as part of sending a request to a
+ server and terminated by the server responding. There are no
+ separate operations for setting up or terminating associations
+ between clients and servers at the transport level.
+
+ - The server is free to discard communication state about a
+ client between transactions without causing incorrect behavior
+ or failures.
+
+The term message transaction (or transaction) is used in the reminder of
+this document for a request-response exchange in the sense described
+above.
+
+VMTP handles the error detection, retransmission, duplicate suppression
+and, optionally, security required for transport-level end-to-end
+reliability.
+
+The protocol is designed to provide a range of behaviors within the
+transaction model, including:
+
+ - Minimal two packet exchanges for short, simple transactions.
+
+ - Streaming of multi-packet requests and responses for efficient
+ data transfer.
+
+ - Datagram and multicast communication as an extension of the
+ transaction model.
+
+Example Uses:
+
+ - Page-level file access - VMTP is intended as the transport
+ level for file access, allowing simple, efficient operation on
+ a local network. In particular, VMTP is appropriate for use
+ by diskless workstations accessing shared network file
+
+
+Cheriton [page 1]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ servers.
+
+ - Distributed programming - VMTP is intended to provide an
+ efficient transport level protocol for remote procedure call
+ implementations, distributed object-oriented systems plus
+ message-based systems that conform to the request-response
+ model.
+
+ - Multicast communication with groups of servers to: locate a
+ specific object within the group, update a replicated object,
+ synchronize the commitment of a distributed transaction, etc.
+
+ - Distributed real-time control with prioritized message
+ handling, including datagrams, multicast and asynchronous
+ calls.
+
+The protocol is designed to operate on top of a simple unreliable
+datagram service, such as is provided by IP.
+
+
+1.1. Motivation
+
+VMTP was designed to address three categories of deficiencies with
+existing transport protocols in the Internet architecture. We use TCP
+as the key current transport protocol for comparison.
+
+
+1.1.1. Poor RPC Performance
+
+First, current protocols provide poor performance for remote procedure
+call (RPC) and network file access. This is attributable to three key
+causes:
+
+ - TCP requires excessive packets for RPC, especially for
+ isolated calls. In particular, connection setup and clear
+ generates extra packets over that needed for VMTP to support
+ RPC.
+
+ - TCP is difficult to implement, speaking purely from the
+ empirical experience over the last 10 years. VMTP was
+ designed concurrently with its implementation, with focus on
+ making it easy to implement and providing sensible subsets of
+ its functionality.
+
+ - TCP handles packet loss due to overruns poorly. We claim that
+ overruns are the key source of packet loss in a
+ high-performance RPC environment and, with the increasing
+
+
+Cheriton [page 2]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ performance of networks, will continue to be the key source.
+ (Older machines and network interfaces cannot keep up with new
+ machines and network interfaces. Also, low-end network
+ interfaces for high-speed networks have limited receive
+ buffering.)
+
+VMTP is designed for ease of implementation and efficient RPC. In
+addition, it provides selective retransmission with rate-based flow
+control, thus addressing all of the above issues.
+
+
+1.1.2. Weak Naming
+
+Second, current protocols provide inadequate naming of transport-level
+endpoints because the names are based on IP addresses. For example, a
+TCP endpoint is named by an Internet address and port identifier.
+Unfortunately, this makes the endpoint tied to a particular host
+interface, not specifically the process-level state associated with the
+transport-level endpoint. In particular, this form of naming causes
+problems for process migration, mobile hosts and multi-homed hosts.
+VMTP provides host-address independent names, thereby solving the above
+mentioned problems.
+
+In addition, TCP provides no security and reliability guarantees on the
+dynamically allocated names. In particular, other than well-known
+ports, (host-addr, port-id)-tuples can change meaning on reboot
+following a crash. VMTP provides large identifiers with guarantee of
+stability, meaning that either the identifier never changes in meaning
+or else remains invalid for a significant time before becoming valid
+again.
+
+
+1.1.3. Function Poor
+
+TCP does not support multicast, real-time datagrams or security. In
+fact, it only supports pair-wise, long-term, streamed reliable
+interchanges. Yet, multicast is of growing importance and is being
+developed for the Internet (see RFC 966 and 988). Also, a datagram
+facility with the same naming, transmission and reception facilities as
+the normal transport level is a powerful asset for real-time and
+parallel applications. Finally, security is a basic requirement in an
+increasing number of environments. We note that security is natural to
+implement at the transport level to provide end-to-end security (as
+opposed to (inter)network level security). Without security at the
+transport level, a transport level protocol cannot guarantee the
+standard transport level service definition in the presence of an
+intruder. In particular, the intruder can interject packets or modify
+
+
+Cheriton [page 3]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+packets while updating the checksum, making mockery out of the
+transport-level claim of "reliable delivery".
+
+In contrast, VMTP provides multicast, real-time datagrams and security,
+addressing precisely these weaknesses.
+
+In general, VMTP is designed with the next generation of communication
+systems in mind. These communication systems are characterized as
+follows. RPC, page-level file access and other request-response
+behavior dominates. In addition, the communication substrate, both
+local and wide-area, provides high data rates, low error rates and
+relatively low delay. Finally, intelligent, high-performance network
+interfaces are common and in fact required to achieve performance that
+approximates the network capability. However, VMTP is also designed to
+function acceptably with existing networks and network interfaces.
+
+
+1.2. Relation to Other Protocols
+
+VMTP is a transport protocol that fits into the layered Internet
+protocol environment. Figure 1-1 illustrates the place of VMTP in the
+protocol hierarchy.
+
+
+ +-----------+ +----+ +-----------------+ +------+
+ |File Access| |Time| |Program Execution| |Naming|... Application
+ +-----------+ +----+ +-----------------+ +------+ Layer
+ | | | | |
+ +-----------+-----------+-------------+------+
+ |
+ +------------------+
+ | RPC Presentation | Presentation
+ +------------------+ Layer
+ |
+ +------+ +--------+
+ | TCP | | VMTP | Transport
+ +------+ +--------+ Layer
+ | |
+ +-----------------------------------+
+ | Internet Protocol & ICMP | Internetwork
+ +-----------------------------------+ Layer
+
+ Figure 1-1: Relation to Other Protocols
+
+The RPC presentation level is not currently defined in the Internet
+suite of protocols. Appendix II defines a proposed RPC presentation
+level for use with VMTP and assumed for the definition of the VMTP
+management procedures. There is also a need for the definition of the
+
+
+Cheriton [page 4]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+Application layer protocols listed above.
+
+If internetwork services are not required, VMTP can be used without the
+IP layer, layered directly on top of the network or data link layers.
+
+
+1.3. Document Overview
+
+The next chapter gives an overview of the protocol, covering naming,
+message structure, reliability, flow control, streaming, real-time,
+security, byte-ordering and management. Chapter 3 describes the VMTP
+packet formats. Chapter 4 describes the client VMTP protocol operation
+in terms of pseudo-code for event handling. Chapter 5 describes the
+server VMTP protocol operation in terms of pseudo-code for event
+handling. Chapter 6 summarizes the state of the protocol, some
+remaining issues and expected directions for the future. Appendix I
+lists some standard Response codes. Appendix II describes the RPC
+presentation protocol proposed for VMTP and used with the VMTP
+management procedures. Appendix III lists the VMTP management
+procedures. Appendix IV proposes initial approaches for handling entity
+identification for VMTP. Appendix V proposes initial authentication
+domains for VMTP. Appendix VI provides some details for implementing
+VMTP on top of IP. Appendix VII provides some suggestions on host
+implementation of VMTP, focusing on data structures and support
+functions. Appendix VIII describes a proposed program interface for
+UNIX 4.3 BSD and its descendants and related systems.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 5]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+2. Protocol Overview
+
+VMTP provides an efficient, reliable, optionally secure transport
+service in the message transaction or request-response model with the
+following features:
+
+ - Host address-independent naming with provision for multiple
+ forms of names for endpoints as well as associated (security)
+ principals. (See Sections 2.1, 2.2, 3.1 and Appendix IV.)
+
+ - Multi-packet request and response messages, with a maximum
+ size of 4 megaoctets per message. (Sections 2.3 and 2.14.)
+
+ - Selective retransmission. (Section 2.13.) and rate-based flow
+ control to reduce overrun and the cost of overruns. (Section
+ 2.5.6.)
+
+ - Secure message transactions with provision for a variety of
+ encryption schemes. (Section 2.6.)
+
+ - Multicast message transactions with multiple response messages
+ per request message. (Section 2.7.)
+
+ - Support for real-time communication with idempotent message
+ transactions with minimal server overhead and state (Section
+ 2.5.3), datagram request message transactions with no
+ response, optional header-only checksum, priority processing
+ of transactions, conditional delivery and preemptive handling
+ of requests (Section 2.8)
+
+ - Forwarded message transactions as an optimization for certain
+ forms of nested remote procedure calls or message
+ transactions. (Section 2.9.)
+
+ - Multiple outstanding (asynchronous) message transactions per
+ client. (Section 2.11.)
+
+ - An integrated management module, defined with a remote
+ procedure call interface on top of VMTP providing a variety of
+ communication services (Section 2.10.)
+
+ - Simple subset implementation for simple clients and simple
+ servers. (Section 2.16.)
+
+This chapter provides an overview of the protocol as introduction to the
+basic ideas and as preparation for the subsequent chapters that describe
+the packet formats and event processing procedures in detail.
+
+
+Cheriton [page 6]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+In overview, VMTP provides transport communication between network-
+visible entities via message transactions. A message transaction
+consists of a request message sent by the client, or requestor, to a
+group of server entities followed by zero or more response messages to
+the client, at most one from each server entity. A message is
+structured as a message control portion and a segment data portion. A
+message is transmitted as one or more packet groups. A packet group is
+one or more packets (up to a maximum of 32 packets) grouped by the
+protocol for acknowledgment, sequencing, selective retransmission and
+rate control.
+
+Entities and VMTP operations are managed using a VMTP management
+mechanism that is accessed through a procedural interface (RPC)
+implemented on top of VMTP. In particular, information about a remote
+entity is obtained and maintained using the Probe VMTP management
+operation. Also, acknowledgment information and requests for
+retransmission are sent as notify requests to the management module.
+(In the following description, reference to an "acknowledgment" of a
+request or a response refers to a management-level notify operation that
+is acknowledging the request or response.)
+
+
+2.1. Entities, Processes and Principals
+
+VMTP defines and uses three main types of identifiers: entity
+identifiers, process identifiers and principal identifiers, each 64-bits
+in length. Communication takes place between network-visible entities,
+typically mapping to, or representing, a message port or procedure
+invocation. Thus, entities are the VMTP communication endpoints. The
+process associated with each entity designates the agent behind the
+communication activity for purposes of resource allocation and
+management. For example, when a lock is requested on a file, the lock
+is associated with the process, not the requesting entity, allowing a
+process to use multiple entity identifiers to perform operations without
+lock conflict between these entities. The principal associated with an
+entity specifies the permissions, security and accounting designation
+associated with the entity. The process and principal identifiers are
+included in VMTP solely to make these values available to VMTP users
+with the security and efficiency provided by VMTP. Only the entity
+identifiers are actively used by the protocol.
+
+Entity identifiers are required to have three properties;
+
+Uniqueness Each entity identifier is uniquely defined at any given
+ time. (An entity identifier may be reused over time.)
+
+Stability An entity identifier does not change between valid
+
+
+Cheriton [page 7]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ meanings without suitable provision for removing
+ references to the entity identifier. Certain entity
+ identifiers are strictly stable, (i.e. never changing
+ meaning), typically being administratively assigned
+ (although they need not be bound to a valid entity at
+ all times), often called well-known identifiers. All
+ other entity identifiers are required to be T-stable,
+ not change meaning without having remained invalid for
+ at least a time interval T.
+
+Host address independent
+ An entity identifier is unique independent of the host
+ address of its current host. Moreover, an entity
+ identifier is not tied to a single Internet host
+ address. An entity can migrate between hosts, reside on
+ a mobile host that changes Internet addresses or reside
+ on a multi-homed host. It is up to the VMTP
+ implementation to determine and maintain up to date the
+ host addresses of entities with which it is
+ communicating.
+
+The stability of entity identifiers guarantees that an entity identifier
+represents the same logical communication entity and principal (in the
+security sense) over the time that it is valid. For example, if an
+entity identifier is authenticated as having the privileges of a given
+user account, it continues to have those privileges as long as it is
+continuously valid (unless some explicit notice is provided otherwise).
+Thus, a file server need not fully authenticate the entity on every file
+access request. With T-stable identifiers, periodically checking the
+validity of an entity identifier with period less than T seconds detects
+a change in entity identifier validity.
+
+A group of entities can form an entity group, which is a set of zero or
+more entities identified by a single entity identifier. For example,
+one can have a single entity identifier that identifies the group of
+name servers. An entity identifier representing an entity group is
+drawn from the same name space as entity identifiers. However, single
+entity identifiers are flagged as such by a bit in the entity
+identifier, indicating that the identifier is known to identify at most
+one entity. In addition to the group bit, each entity identifier
+includes other standard type flags. One flag indicates whether the
+identifier is an alias for an entity in another domain (See Section 2.2
+below.). Another flag indicates, for an entity group identifier,
+whether the identifier is a restricted group or not. A restricted group
+is one in which an entity can be added only by another entity with group
+management authorization. With an unrestricted group, an entity is
+allowed to add itself. If an entity identifier does not represent a
+
+
+Cheriton [page 8]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+group, a type bit indicates whether the entity uses big-endian or
+little-endian data representation (corresponding to Motorola 680X0 and
+VAX byte orders, respectively). Further specification of the format of
+entity identifiers is contained in Section 3.1 and Appendix IV.
+
+An entity identifier identifies a Client, a Server or a group of
+Servers <1>. A Client is always identified by a T-stable identifier. A
+server or group of servers may be identified by a a T-stable identifier
+(group or single entity) or by strictly stable (statically assigned)
+entity group identifier. The same T-stable identifier can be used to
+identify a Client and Server simultaneously as long as both are
+logically associated with the same entity. The state required for
+reliable, secure communication between entities is maintained in client
+state records (CSRs), which include the entity identifier of the Client,
+its principal, its current or next transaction identifier and so on.
+
+
+2.2. Entity Domains
+
+An entity domain is an administration or an administration mechanism
+that guarantees the three required entity identifier properties of
+uniqueness, stability and host address independence for the entities it
+administers. That is, entity identifiers are only guaranteed to be
+unique and stable within one entity domain. For example, the set of all
+Internet hosts may function as one domain. Independently, the set of
+hosts local to one autonomous network may function as a separate domain.
+Each entity domain is identified by an entity domain identifier, Domain.
+Only entities within the same domain may communicate directly via VMTP.
+However, hosts and entities may participate in multiple entity domains
+simultaneously, possibly with different entity identifiers. For
+example, a file server may participate in multiple entity domains in
+order to provide file service to each domain. Each entity domain
+specifies the algorithms for allocation, interpretation and mapping of
+entity identifiers.
+
+Domains are necessary because it does not appear feasible to specify one
+universal VMTP entity identification administration that covers all
+entities for all time. Domains limit the number of entities that need
+to be managed to maintain the uniqueness and stability of the entity
+
+_______________
+
+<1> Terms such as Client, Server, Request, Response, etc. are
+capitalized in this document when they refer to their specific meaning
+in VMTP.
+
+
+Cheriton [page 9]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+name space. Domains can also serve to separate entities of different
+security levels. For instance, allocation of a unclassified entity
+identifier cannot conflict with secret level entity identifiers because
+the former is interpreted only in the unclassified domain, which is
+disjoint from the secret domain.
+
+It is intended that there be a small number of domains. In particular,
+there should be one (or a few) domains per installation "type", rather
+than per installation. For example, the Internet is expected to use one
+domain per security level, resulting in at most 8 different domains.
+Cluster-based internetwork architectures, those with a local cluster
+protocol distinct from the wide-area protocol, may use one domain for
+local use and one for wide-area use.
+
+Additional details on the specification of specific domains is provided
+in Appendix IV.
+
+
+2.3. Message Transactions
+
+The message transaction is the unit of interaction between a Client that
+initiates the transaction and one or more Servers. A message
+transaction starts with a request message generated by a client. At
+the service interface, a server becomes involved with a transaction by
+receiving and accepting the request. A server terminates its
+involvement with a transaction by sending a response message. In a
+group message transaction, the server entity designated by the client
+corresponds to a group of entities. In this case, each server in the
+group receives a copy of the request. In the client's view, the
+transaction is terminated when it receives the response message or, in
+the case of a group message transaction, when it receives the last
+response message. Because it is normally impractical to determine when
+the last response message has been received. the current transaction is
+terminated by VMTP when the next transaction is initiated.
+
+Within an entity domain, a transaction is uniquely identified by the
+tuple (Client, Transaction, ForwardCount). where Transaction is a
+32-bit number and ForwardCount is a 4-bit value. A Client uses
+monotonically increasing Transaction identifiers for new message
+transactions. Normally, the next higher transaction number, modulo
+2**32, is used for the next message transaction, although there are
+cases in which it skips a small range of Transaction identifiers. (See
+the description of the STI control flag.) The ForwardCount is used when
+a message transaction is forwarded and is zero otherwise.
+
+A Client generates a stream of message transactions with increasing
+transaction identifiers, directed at a diversity of Servers. We say a
+
+
+Cheriton [page 10]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+Client has a transaction outstanding if it has invoked a message
+transaction, but has not received the last Response (or possibly any
+Response). Normally, a Client has only one transaction outstanding at a
+time. However, VMTP allows a Client to have multiple message
+transactions outstanding simultaneously, supporting streamed,
+asynchronous remote procedure call invocations. In addition, VMTP
+supports nested calls where, for example, procedure A calls procedure B
+which calls procedure C, each on a separate host with different client
+entity identifiers for each call but identified with the same process
+and principal.
+
+
+2.4. Request and Response Messages
+
+A message transaction consists of a request message and one or more
+Response messages. A message is structured as message control block
+(MCB) and segment data, passed as parameters, as suggested below.
+
+ +-----------------------+
+ | Message Control Block |
+ +-----------------------+
+ +-----------------------------------+
+ | segment data |
+ +-----------------------------------+
+
+In the request message, the MCB specifies control information about the
+request plus an optional data segment. The MCB has the following
+format:
+ 0 1 2 3
+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ + ServerEntityId (8 octets) +
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | Flags | RequestCode |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ + CoresidentEntity (8 octets) +
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ > User Data (12 octets) <
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | MsgDelivery |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | SegmentSize |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+The ServerEntityId is the entity to which the Request MCB is to be sent
+(or was sent, in the case of reception). The Flags indicate various
+options in the request and response handling as well as whether the
+
+
+Cheriton [page 11]
+
+
+RFC 1045 VMTP February 1988
+
+
+CoresidentEntity, MsgDelivery and SegmentSize fields are in use. The
+RequestCode field specifies the type of Request. It is analogous to a
+packet type field of the Ethernet, acting as a switch for higher-level
+protocols. The CoresidentEntity field, if used, designates a subgroup
+of the ServerEntityId group to which the Request should be routed,
+namely those members that are co-resident with the specified entity (or
+entity group). The primary intended use is to specify the manager for a
+particular service that is co-resident with a particular entity, using
+the well-known entity group identifier for the service manager in the
+ServerEntityId field and the identifier for the entity in the
+CoresidentEntity field. The next 12 octets are user- or
+application-specified.
+
+The MsgDelivery field is optionally used by the RPC or user level to
+specify the portions of the segment data to transmit and on reception,
+the portions received. It provides the client and server with
+(optional) access to, and responsibility for, a simple selective
+transmission and reception facility. For example, a client may request
+retransmission of just those portions of the segment that it failed to
+receive as part of the original Response. The primary intended use is
+to support highly efficient multi-packet reading from a file server.
+Exploiting user-level selective retransmission using the MsgDelivery
+field, the file server VMTP module need not save multi-packet Responses
+for retransmission. Retransmissions, when needed, are instead handled
+directly from the file server buffers.
+
+The SegmentSize field indicates the size of the data segment, if
+present. The CoresidentEntity, MsgDelivery and SegmentSize fields are
+usable as additional user data if they are not otherwise used.
+
+The Flags field provides a simple mechanism for the user level to
+communicate its use of VMTP options with the VMTP module as well as for
+VMTP modules to communicate this use among themselves. The use of these
+options is generally fixed for each remote procedure so that an RPC
+mechanism using VMTP can treat the Flags as an integral part of the
+RequestCode field for the purpose of demultiplexing to the correct stub.
+
+A Response message control block follows the same format except the
+Response is sent from the Server to the Client and there is no
+Coresident Entity field (and thus 20 octets of user data).
+
+
+2.5. Reliability
+
+VMTP provides reliable, sequenced transfer of request and response
+messages as well as several variants, such as unreliable datagram
+requests. The reliability mechanisms include: transaction identifiers,
+
+
+Cheriton [page 12]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+checksums, positive acknowledgment of messages and timeout and
+retransmission of lost packets.
+
+
+2.5.1. Transaction Identifiers
+
+Each message transaction is uniquely identified by the pair (Client,
+Transaction). (We defer discussion of the ForwardCount field to Section
+2.9.) The 32-bit transaction identifier is initialized to a random
+value when the Client entity is created or allocated its entity
+identifier. The transaction identifier is incremented at the end of
+each message transaction. All Responses with the same specified
+(Client, Transaction) pair are associated with this Request.
+
+The transaction identifier is used for duplicate suppression at the
+Server. A Server maintains a state record for each Client for which it
+is processing a Request, identified by (Client, Transaction). A Request
+with the same (Client, Transaction) pair is discarded as a duplicate.
+(The ForwardCount field must also be equal.) Normally, this record is
+retained for some period after the Response is sent, allowing the Server
+to filter out subsequent duplicates of this Request. When a Request
+arrives and the Server does not have a state record for the sending
+Client, the Server takes one of three actions:
+
+ 1. The Server may send a Probe request, a simple query
+ operation, to the VMTP management module associated with the
+ requesting Client to determine the Client's current
+ Transaction identifier (and other information), initialize a
+ new state record from this information, and then process the
+ Request as above.
+
+ 2. The Server may reason that the Request must be a new request
+ because it does not have a state record for this Client if it
+ keeps these state records for the maximum packet lifetime of
+ packets in the network (plus the maximum VMTP retransmission
+ time) and it has not been rebooted within this time period.
+ That is, if the Request is not new either the Request would
+ have exceeded the maximum packet lifetime or else the Server
+ would have a state record for the Client.
+
+ 3. The Server may know that the Request is idempotent or can be
+ safely redone so it need not care whether the Request is a
+ duplicate or not. For example, a request for the current
+ time can be responded to with the current time without being
+ concerned whether the Request is a duplicate. The Response
+ is discarded at the Client if it is no longer of interest.
+
+
+
+Cheriton [page 13]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+2.5.2. Checksum
+
+Each VMTP packet contains a checksum to allow the receiver to detect
+corrupted packets independent of lower level checks. The checksum field
+is 32 bits, providing greater protection than the standard 16-bit IP
+checksum (in combination with an improved checksum algorithm). The
+large packets, high packet rates and general network characteristics
+expected in the future warrant a stronger checksum mechanism.
+
+The checksum normally covers both the VMTP header and the segment data.
+Optionally (for real-time applications), the checksum may apply only to
+the packet header, as indicated by the HCO control bit being set in the
+header. The checksum field is placed at the end of the packet to allow
+it to be calculated as part of a software copy or as part of a hardware
+transmission or reception packet processing pipeline, as expected in the
+next generation of network interfaces. Note that the number of header
+and data octets is an integral multiple of 8 because VMTP requires that
+the segment data be padded to be a multiple of 64 bits. The checksum
+field is appended after the padding, if any. The actual algorithm is
+described in Section 3.2.
+
+A zero checksum field indicates that no checksum was transmitted with
+the packet. VMTP may be used without a checksum only when there is a
+host-to-host error detection mechanism and the VMTP security facility is
+not being used. For example, one could rely on the Ethernet CRC if
+communication is restricted to hosts on the same Ethernet and the
+network interfaces are considered sufficiently reliable.
+
+
+2.5.3. Request and Response Acknowledgment
+
+VMTP assumes an unreliable datagram network and internetwork interface.
+To guarantee delivery of Requests and Response, VMTP uses positive
+acknowledgments, retransmissions and timeouts.
+
+A Request is normally acknowledged by receipt of a Response associated
+with the Request, i.e. with the same (Client, Transaction). With
+streamed message transactions, it may also be acknowledged by a
+subsequent Response that acknowledges previous Requests in addition to
+the transaction it explicitly identifies. A Response may be explicitly
+acknowledged by a NotifyVmtpServer operation requested of the manager
+for the Server. In the case of streaming, this is a cumulative
+acknowledgment, acknowledging all Responses with a lower transaction
+identifier as well.) In addition, with non-streamed communication, a
+subsequent Request from the same Client acknowledges Responses to all
+previous message transactions (at least in the sense that either the
+client received a Response or is no longer interested in Responses to
+
+
+Cheriton [page 14]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+those earlier message transactions). Finally, a client response timeout
+(at the server) acknowledges a Response at least in the sense that the
+server need not be prepared to retransmit the Response subsequently.
+Note that there is no end-to-end guarantee of the Response being
+received by the client at the application level.
+
+
+2.5.4. Retransmissions
+
+In general, a Request or Response is retransmitted periodically until
+acknowledged as above, up to some maximum number of retransmissions.
+VMTP uses parameters RequestRetries(Server) and ResponseRetries(Client)
+that indicate the number of retransmissions for the server and client
+respectively before giving up. We suggest the value 5 be used for both
+parameters based on our experience with VMTP and Internet packet loss.
+Smaller values (such as 3) could be used in low loss environments in
+which fast detection of failed hosts or communication channels is
+required. Larger values should be used in high loss environments where
+transport-level persistence is important.
+
+In a low loss environment, a retransmission only includes the MCB and
+not the segment data of the Request or Response, resulting in a single
+(short) packet on retransmission. The intended recipient of the
+retransmission can request selective retransmission of all or part of
+the segment data as necessary. The selective retransmission mechanism
+is described in Section 2.13.
+
+If a Response is specified as idempotent, the Response is neither
+retransmitted nor stored for retransmission. Instead, the Client must
+retransmit the Request to effectively get the Response retransmitted.
+The server VMTP module responds to retransmissions of the Request by
+passing the Request on to the server again to have it regenerate the
+Response (by redoing the operation), rather than saving a copy of the
+Response. Only Request packets for the last transaction from this
+client are passed on in this fashion; older Request packets from this
+client are discarded as delayed duplicates. If a Response is not
+idempotent, the VMTP module must ensure it has a copy of the Response
+for retransmission either by making a copy of the Response (either
+physically or copy-on-write) or by preventing the Server from continuing
+until the Response is acknowledged.
+
+
+2.5.5. Timeouts
+
+There is one client timer for each Client with an outstanding
+transaction. Similarly, there is one server timer for each Client
+transaction that is "active" at the server, i.e. there is a transaction
+
+
+Cheriton [page 15]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+record for a Request from the Client.
+
+When the client transmits a new Request (without streaming), the client
+timer is set to roughly the time expected for the Response to be
+returned. On timeout, the Request is retransmitted with the APG
+(Acknowledge Packet Group) bit set. The timeout is reset to the
+expected roundtrip time to the Server because an acknowledgment should
+be returned immediately unless a Response has been sent. The Request
+may also be retransmitted in response to receipt of a VMTP management
+operation indicating that selected portions of the Request message
+segment need to be retransmitted. With streaming, the timeout applies
+to the oldest outstanding message transaction in the run of outstanding
+message transactions. Without streaming, there is one message
+transaction in the run, reducing to the previous situation. After the
+first packet of a Response is received, the Client resets the timeout to
+be the time expected before the next packet in the Response packet group
+is received, assuming it is a multi-packet Response. If not, the timer
+is stopped. Finally, the client timer is used to timeout waiting for
+second and subsequent Responses to a multicast Request.
+
+The client timer is set at different times to four different values:
+
+TC1(Server) The expected time required to receive a Response from
+ the Server. Set on initial Request transmission plus
+ after its management module receives a NotifyVmtpClient
+ operation, acknowledging the Request.
+
+TC2(Server) The estimated round trip delay between the client and
+ the server. Set when retransmitting after receiving no
+ Response for TC1(Server) time and retransmitting the
+ Request with the APG bit set.
+
+TC3(Server) The estimated maximum expected interpacket time for
+ multi-packet Responses from the Server. Set when
+ waiting for subsequent Response packets within a packet
+ group before timing out.
+
+TC4 The time to wait for additional Responses to a group
+ Request after the first Response is received. This is
+ specified by the user level.
+
+These values are selected as follows. TC1 can be set to TC2 plus a
+constant, reflecting the time within which most servers respond to most
+requests. For example, various measurements of VMTP usage at Stanford
+indicate that 90 percent of the servers respond in less than 200
+milliseconds. Setting TC1 to TC2 + 200 means that most Requests receive
+a Response before timing out and also that overhead for retransmission
+
+
+Cheriton [page 16]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+for long running transactions is insignificant. A sophisticated
+implementation may make the estimation of TC1 further specific to the
+Server.
+
+TC2 may be estimated by measuring the time from when a Probe request is
+sent to the Server to when a response is received. TC2 can also be
+measured as the time between the transmission of a Request with the APG
+bit set to receipt of a management operation acknowledging receipt of
+the Request.
+
+When the Server is an entity group, TC1 and TC2 should be the largest of
+the values for the members of the group that are expected to respond.
+This information may be determined by probing the group on first use
+(and using the values for the last responses to arrive). Alternatively,
+one can resort to default values.
+
+TC3 is set initially to 10 times the transmission time for the maximum
+transmission unit (MTU) to be used for the Response. A sophisticated
+implementation may record TC3 per Server and refine the estimate based
+on measurements of actual interpacket gaps. However, a tighter estimate
+of TC3 only improves the reaction time when a packet is lost in a packet
+group, at some cost in unnecessary retransmissions when the estimate
+becomes overly tight.
+
+The server timer, one per active Client, takes on the following values:
+
+TS1(Client) The estimated maximum expected interpacket time. Set
+ when waiting for subsequent Request packets within a
+ packet group before timing out.
+
+TS2(Client) The time to wait to hear from a client before
+ terminating the server processing of a Request. This
+ limits the time spent processing orphan calls, as well
+ as limiting how out of date the server's record of the
+ Client state can be. In particular, TS2 should be
+ significantly less than the minimum time within which it
+ is reasonable to reuse a transaction identifier.
+
+TS3(Client) Estimated roundtrip time to the Client,
+
+TS4(Client) The time to wait after sending a Response (or last
+ hearing from a client) before discarding the state
+ associated with the Request which allows it to filter
+ duplicate Request packets and regenerate the Response.
+
+TS5(Client) The time to wait for an acknowledgment after sending a
+ Response before retransmitting the Response, or giving
+
+
+Cheriton [page 17]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ up (after some number of retransmissions).
+
+TS1 is set the same as TC3.
+
+The suggested value for TS2 is TC1 + 3*TC2 for this server, giving the
+Client time to timeout waiting for a Response and retransmit 3 Request
+packets, asking for acknowledgments.
+
+TS3 is estimated the same as TC1 except that refinements to the estimate
+use measurements of the Response-to-acknowledgment times.
+
+In the general case, TS4 is set large enough so that a Client issuing a
+series of closely-spaced Requests to the same Server reuses the same
+state record at the Server end and thus does not incur the overhead of
+recreating this state. (The Server can recreate the state for a Client
+by performing a Probe on the Client to get the needed information.) It
+should also be set low enough so that the transaction identifier cannot
+wrap around and so that the Server does not run out of CSR's. We
+suggest a value in the range of 500 milliseconds. However, if the
+Server accepts non-idempotent Requests from this Client without doing a
+Probe on the Client, the TS4 value for this CSR is set to at least 4
+times the maximum packet lifetime.
+
+TS5 is TS3 plus the expected time for transmission and reception of the
+Response. We suggest that the latter be calculated as 3 times the
+transmission time for the Response data, allowing time for reception,
+processing and transmission of an acknowledgment at the Client end. A
+sophisticated implementation may refine this estimate further over time
+by timing acknowledgments to Responses.
+
+
+2.5.6. Rate Control
+
+VMTP is designed to deal with the present and future problem of packet
+overruns. We expect overruns to be the major cause of dropped packets
+in the future. A client is expected to estimate and adjust the
+interpacket gap times so as to not overrun a server or intermediate
+nodes. The selective retransmission mechanism allows the server to
+indicate that it is being overrun (or some intermediate point is being
+overrun). For example, if the server requests retransmission of every
+Kth block, the client should assume overrun is taking place and increase
+the interpacket gap times. The client passes the server an indication
+of the interpacket gap desired for a response. The client may have to
+increase the interval because packets are being dropped by an
+intermediate gateway or bridge, even though it can handle a higher rate.
+A conservative policy is to increase the interpacket gap whenever a
+packet is lost as part of a multi-packet packet group.
+
+
+Cheriton [page 18]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+The provision of selective retransmission allows the rate of the client
+and the server to "push up" against the maximum rate (and thus lose
+packets) without significant penalty. That is, every time that packet
+transmission exceeds the rate of the channel or receiver, the recovery
+cost to retransmit the dropped packets is generally far less than
+retransmitting from the first dropped packet.
+
+The interpacket gap is expressed in 1/32nd's of the MTU packet
+transmission time. The minimum interpacket gap is 0 and the maximum gap
+that can be described in the protocol is 8 packet times. This places a
+limit on the slowest receivers that can be efficiently used on a
+network, at least those handling multi-packet Requests and Responses.
+This scheme also limits the granularity of adjustment. However, the
+granularity is relative to the speed of the network, as opposed to an
+absolute time. For entities on different networks of significantly
+different speed, we assume the interconnecting gateways can buffer
+packets to compensate<2>. With different network speeds and intermediary
+nodes subject to packet loss, a node must adjust the interpacket gap
+based on packet loss. The interpacket gap parameter may be of limited
+use.
+
+
+2.6. Security
+
+VMTP provides an (optional) secure mode that protects against the usual
+security threats of peeking, impostoring, message tampering and replays.
+Secure VMTP must be used to guarantee any of the transport-level
+reliability properties unless it is guaranteed that there are no
+intruders or agents that can modify packets and update the packet
+checksums. That is, non-secure VMTP provides no guarantees in the
+presence of an intelligent intruder.
+
+The design closely follows that described by Birrell [1]. Authenticated
+information about a remote entity, including an encryption/decryption
+key, is obtained and maintained using a VMTP management operation, the
+authenticated Probe operation, which is executed as a non-secure VMTP
+message transaction. If a server receives a secure Request for which
+the server has no entity state, it sends a Probe request to the VMTP
+
+_______________
+
+<2> Gateways must also employ techniques to preserve or intelligently
+modify (if appropriate) the interpacket gaps. In particular, they must
+be sure not to arbitrarily remove interpacket gaps as a result of their
+forwarding of packets.
+
+
+Cheriton [page 19]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+management module of the client, "challenging" it to provide an
+authenticator that both authenticates the client as being associated
+with a particular principal as well as providing a key for
+encryption/decryption. The principal can include a real and effective
+principal, as used in UNIX <3>. Namely, the real principal is the
+principal on whose behalf the Request is being performed whereas the
+effective principal is the principal of the module invoking the request
+or remote procedure call.
+
+Peeking is prevented by encrypting every Request and Response packet
+with a working Key that is shared between Client and Server.
+Impostoring and replays are detected by comparing the Transaction
+identifier with that stored in the corresponding entity state record
+(which is created and updated by VMTP as needed). Message tampering is
+detected by encryption of the packet including the Checksum field. An
+intruder cannot update the checksum after modifying the packet without
+knowing the Key. The cost of fully encrypting a packet is close to the
+cost of generating a cryptographic checksum (and of course, encryption
+is needed in the general case), so there is no explicit provision for
+cryptographic checksum without packet encryption.
+
+A Client determines the Principal of the Server and acquires an
+authenticator for this Server and Principal using a higher level
+protocol. The Server cannot decrypt the authenticator or the Request
+packets unless it is in fact the Principal expected by the Client.
+
+An encrypted VMTP packet is flagged by the EPG bit in the VMTP packet
+header. Thus, encrypted packets are easily detected and demultiplexed
+from unencrypted packets. An encrypted VMTP packet is entirely
+encrypted except for the Client, Version, Domain, Length and Packet
+Flags fields at the beginning of the packet. Client identifiers can be
+assigned, changed and used to have no real meaning to an intruder or to
+only communicate public information (such as the host Internet address).
+They are otherwise just a random means of identification and
+demultiplexing and do not therefore divulge any sensitive information.
+Further secure measures must be taken at the network or data link levels
+if this information or traffic behavior is considered sensitive.
+
+VMTP provides multiple authentication domains as well as an encryption
+qualifier to accommodate different encryption algorithms and their
+
+_______________
+
+<3> Principal group membership must be obtained, if needed, by a
+higher level protocol.
+
+
+Cheriton [page 20]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+corresponding security/performance trade-offs. (See Appendix V.) A
+separate key distribution and authentication protocol is required to
+handle generation and distribution of authenticators and keys. This
+protocol can be implemented on top of VMTP and can closely follow the
+Birrell design as well.
+
+Security is optional in the sense that messages may be secure or
+non-secure, even between consecutive message transactions from the same
+client. It is also optional in that VMTP clients and servers are not
+required to implement secure VMTP (although they are required to respond
+intelligently to attempts to use secure VMTP). At worst, a Client may
+fail to communicate with a Server if the Server insists on secure
+communication and the Client does not implement security or vice versa.
+However, a failure to communicate in this case is necessary from a
+security standpoint.
+
+
+2.7. Multicast
+
+The Server entity identifier in a message transaction can identify an
+entity group, in which case the Request is multicast to every Entity in
+this group (on a best-efforts basis). The Request is retransmitted
+until at least one Response is received (or an error timeout occurs)
+unless it is a datagram Request. The Client can receive multiple
+Responses to the Request.
+
+The VMTP service interface does not directly provide reliable multicast
+because it is expensive to provide, rarely needed by applications, and
+can be implemented by applications using the multiple Response feature.
+However, the protocol itself is adequate for reliable multicast using
+positive acknowledgments. In particular, a sophisticated Client
+implementation could maintain a list of members for each entity group of
+interest and retransmit the Request until acknowledged by all members.
+No modifications are required to the Server implementations.
+
+VMTP supports a simple form of subgroup addressing. If the CRE bit is
+set in a Request, the Request is delivered to the subgroup of entities
+in the Server group that are co-resident with one or more entities in
+the group (or individual entity) identified by the CoresidentEntity
+field of the Request. This is commonly used to send to the manager
+entity for a particular entity, where Server specifies the group of such
+managers. Co-resident means "using the same VMTP module", and logically
+on the same network host. In particular, a Probe request can be sent to
+the particular VMTP management module for an entity by specifying the
+VMTP management group as the Server and the entity in question as the
+CoResidentEntity.
+
+
+
+Cheriton [page 21]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+As an experimental aspect of the protocol, VMTP supports the Server
+sending a group Response which is sent to the Client as well as members
+of the destination group of Servers to which the original Request was
+sent. The MDG bit indicates whether the Client is a member of this
+group, allowing the Server module to determine whether separately
+addressed packet groups are required to send the Response to both the
+Client and the Server group. Normally, a Server accepts a group
+Response only if it has received the Request and not yet responded to
+the Client. Also, the Server must explicitly indicate it wants to
+accept group Responses. Logically, this facility is analogous to
+responding to a mail message sent to a distribution list by sending a
+copy of the Response to the distribution list.
+
+
+2.8. Real-time Communication
+
+VMTP provides three forms of support for real-time communication, in
+addition to its standard facilities, which make it applicable to a wide
+range of real-time applications. First, a priority is transmitted in
+each Request and Response which governs the priority of its handling.
+The priority levels are intended to correspond roughly to:
+
+ - urgent/emergency.
+
+ - important
+
+ - normal
+
+ - background.
+
+with additional gradations for each level. The interpretation and
+implementation of these priority levels is otherwise host-specific, e.g.
+the assignment to host processing priorities.
+
+Second, datagram Requests allow the Client to send a datagram to another
+entity or entity group using the VMTP naming, transmission and delivery
+mechanism, but without blocking, retransmissions or acknowledgment.
+(The client can still request acknowledgment using the APG bit although
+the Server does not expect missing portions of a multi-packet datagram
+Request to be retransmitted even if some are not received.) A datagram
+Request in non-streamed mode supersedes all previous Requests from the
+same Client. A datagram Request in stream mode is queued (if necessary)
+after previous datagram Requests on the same stream. (See Section
+2.11.)
+
+Finally, VMTP provides several control bit flags to modify the handling
+of Requests and Responses for real-time requirements. First, the
+
+
+Cheriton [page 22]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+conditional message delivery (CMD) flag causes a Request to be discarded
+if the recipient is not waiting for it when it arrives, similarly for
+the Response. This option allows a client to send a Request that is
+contingent on the server being able to process it immediately. The
+header checksum only (HCO) flag indicates that the checksum has been
+calculated only on the VMTP header and not on the data segment.
+Applications such as voice and video can avoid the overhead of
+calculating the checksum on data whose utility is insensitive to typical
+bit errors without losing protection on the header information.
+Finally, the No Retransmission (NRT) flag indicates that the recipient
+of a message should not ask for retransmission if part of the message is
+missing but rather either use what was received or discard it.
+
+None of these facilities introduce new protocol states. In fact, the
+total processing overhead in the normal case is a bit flag test for CMD,
+HCO or NRT plus assignment of priority on packet transmission and
+reception. (In fact, CMD and NRT are not tested in the normal case.)
+The additional code complexity is minimal. We feel that the overhead
+for providing these real-time facilities is minimal and that these
+facilities are both important and adequate for a wide class of real-time
+applications.
+
+Several of the normal facilities of VMTP appear useful for real-time
+applications. First, multicast is useful for distributed, replicated
+(fault-tolerant) real-time applications, allowing efficient state query
+and update for (for example) sensors and control state. Second, the DGM
+or idempotent flag for Responses has some real-time benefits, namely: a
+Request is redone to get the latest values when the Response is lost,
+rather than just returning the old values. The desirability of this
+behavior is illustrated by considering a request for the current time of
+day. An idempotent handling of this request gives better accuracy in
+returning the current time in the case that a retransmission is
+necessary. Finally, the request-response semantics (in the absence of
+streaming) of each new Request from a Client terminating the previous
+message transactions from that Client, if any, provides the "most recent
+is most important" handling of processing that most real-time
+applications require.
+
+In general, a key design goal of VMTP was provide an efficient
+general-purpose transport protocol with the features required for
+real-time communication. Further experience is required to determine
+whether this goal has been achieved.
+
+
+
+
+
+
+
+Cheriton [page 23]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+2.9. Forwarded Message Transactions
+
+A Server may invoke another Server to handle a Request. It is fairly
+common for the invocation of the second Server to be the last action
+performed by the first Server as part of handling the Request. For
+example, the original Server may function primarily to select a process
+to handle the Request. Also, the Server may simply check the
+authorization on the Request. Describing this situation in the context
+of RPC, a nested remote procedure call may be the last action in the
+remote procedure and the return parameters are exactly those of the
+nested call. (This situation is analogous to tail recursion.)
+
+As an optimization to support this case, VMTP provides a Forward
+operation that allows the server to send the nested Request to the other
+server and have this other server respond directly to the Client.
+
+If the message transaction being forwarded was not multicast, not secure
+or the two Servers are the same principal and the ForwardCount of the
+Request is less than the maximum forward count of 15, the Forward
+operation is implemented by the Server sending a Request onto the next
+Server with the forwarded Request identified by the same Client and
+Transaction as the original Request and a ForwardCount one greater than
+the Request received from the Client. In this case, the new Server
+responds directly to the Client. A forwarded Request is illustrated in
+the following figure.
+
+ +---------+ Request +----------+
+ | Client +---------------->| Server 1 |
+ +---------+ +----------+
+ ^ |
+ | | forwarded Request
+ | V
+ | Response +----------+
+ +----------------------| Server 2 |
+ +----------+
+
+If the message transaction does not meet the above requirements, the
+Server's VMTP module issues a nested call and simply maps the returned
+Response to a Response to original Request without further Server-level
+processing. In this case, the only optimization over a user-level
+nested call is one fewer VMTP service operation; the VMTP module handles
+the return to the invoking call directly. The Server may also use this
+form of forwarding when the Request is part of a stream of message
+transactions. Otherwise, it must wait until the forwarded message
+transaction completes before proceeding with the subsequent message
+transactions in the stream.
+
+
+
+Cheriton [page 24]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+Implementation of the user-level Forward operation is optional,
+depending on whether the server modules require this facility. Handling
+an incoming forwarded Request is a minor modification of handling a
+normal incoming Request. In particular, it is only necessary to examine
+the ForwardCount field when the Transaction of the Request matches that
+of the last message transaction received from the Client. Thus, the
+additional complexity in the VMTP module for the required forwarding
+support is minimal; the complexity is concentrated in providing a highly
+optimized user-level Forward primitive, and that is optional.
+
+
+2.10. VMTP Management
+
+VMTP management includes operations for creating, deleting, modifying
+and querying VMTP entities and entity groups. VMTP management is
+logically implemented by a VMTP management server module that is invoked
+using a message transaction addressed to the Server, VMTP_MANAGER_GROUP,
+a well-known group entity identifier, in conjunction with Coresident
+Entity mechanism introduced in Section 2.7. A particular Request may
+address the local module, the module managing a particular entity, the
+set of modules managing those entities contained in a specific group or
+all management modules, as appropriate.
+
+The VMTP management procedures are specified in Appendix III.
+
+
+2.11. Streamed Message Transactions
+
+Streamed message transactions refer to two or more message transactions
+initiated by a Client before it receives the response to the first
+message transaction, with each transaction being processed and responded
+to in order but asynchronous relative to the initiation of the
+transactions. A Client streams messages transactions, and thereby has
+multiple message transactions outstanding, by sending them as part of a
+single run of message transactions. A run of message transactions is a
+sequence of message transactions with the same Client and Server and
+consecutive Transaction identifiers, with all but the first and last
+Requests and Responses flagged with the NSR (Not Start Run) and NER
+(Not End Run) control bits. (Conversely, the first Request and
+Response does not have the NSR set and the last Request and Response
+does not have the NER bit set.) The message transactions in a run use
+
+
+
+
+
+
+
+
+Cheriton [page 25]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+consecutive transaction identifiers (except if the STI bit <4> is used
+in one, in which case the transaction identifier for the next message
+transaction is 256 greater, rather than 1).
+
+The Client retains a record for each outstanding transaction until it
+gets a Response or is timed out in error. The record provides the
+information required to retransmit the Request. On retransmission
+timeout, the client retransmits the last Request for which it has not
+received a Response the same as is done with non-streamed communication.
+(I.e. there need be only one timeout for all the outstanding message
+transactions associated with a single client.)
+
+The consecutive transaction identifiers within a run of message
+transactions are used as sequence numbers for error control. The Server
+handles each message transaction in the sequence specified by its
+transaction identifier. When it receives a message transaction that is
+not marked as the beginning of a run, it checks that it previously
+received a message transaction with the predecessor transaction
+identifier, either 1 less than the current one or 256 less if the
+previous one had the STI bit set. If not, the Server sends a
+NotifyVmtpClient operation to the Client's manager indicating either:
+(1) the first message transaction was not fully received, or else (2) it
+has no record of the last one received. If the NRT control flag is set,
+it does not await nor expect retransmission but proceeds with handling
+this Request. This flag is used primarily when datagram Requests are
+used as part of a stream of message transactions. If NRT was not
+specified, the Client must retransmit from the first message transaction
+not fully received (either at all or in part) before the Server can
+proceed with handling this run of Requests or else restart the run of
+message transactions.
+
+The Client expects to receive the Responses in a consecutive sequence,
+using the Transaction identifier to detect missing Responses. Thus, the
+Server must return Responses in sequence except possibly for some gaps,
+as follows. The Server can specify in the PGcount field in a Response,
+the number of consecutively previous Responses that this Response
+
+
+
+
+_______________
+
+<4> The STI bit is used by the Client to effectively allocate 255
+transaction identifiers for use by the Server in returning a large
+Response or stream of Responses.
+
+
+Cheriton [page 26]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+corresponds to, up to a maximum of 255 previous Responses <5>. Thus,
+for example, a Response with Transaction identifier 46 and PGcount 3
+represents Responses 43, 44, 45 and 46. This facility allows the Server
+to eliminate sending Responses to Requests that require no Response,
+effectively batching the Responses into one. It also allows the Server
+to effectively maintain strictly consecutive sequencing when the Client
+has skipped 256 Transaction identifiers using the STI bit and the Server
+does not have that many Responses to return.
+
+If the Client receives a Response that is not consecutive, it
+retransmits the Request(s) for which the Response(s) is/are missing
+(unless, of course, the corresponding Requests were sent as datagrams).
+The Client should wait at the end of a run of message transactions for
+the last one to complete.
+
+When a Server receives a Request with the NSR bit clear and a higher
+transaction identifier than it currently has for the Client, it
+terminates all processing and discards Responses associated with the
+previous Requests. Thus, a stream of message transactions is
+effectively aborted by starting a new run, even if the Server was in the
+middle of handling the previous run.
+
+Using a mixture of datagram and normal Requests as part of a stream of
+message transactions, particularly with the use of the NRT bit, can lead
+to complex behavior under packet loss. It is recommended that a run of
+message transactions be all of one type to avoid problems, i.e. all
+normal or all datagrams. Finally, when a Server forwards a Request that
+is part of a run, it must suspend further processing of the subsequent
+Requests until the forwarded Request has been handled, to preserve order
+of processing. The simplest handling of this situation is to use a real
+nested call when forwarding with streamed message transactions.
+
+Flow control of streamed message transactions relies on rate control at
+the Client plus receipt (or non-receipt) of management notify operations
+indicating the presence of overrunning. A Client must reduce the number
+of outstanding message transactions at the Server when it receives a
+NotifyVmtpServer operation with the MSGTRANS_OVERFLOW ResponseCode. The
+transact parameter indicates the last packet group that was accepted.
+
+
+_______________
+
+<5> PGcount actually corresponds to packet groups which are described
+in Section 2.13. This (simplified) description is accurate when there
+is one Request or Response per packet group.
+
+
+Cheriton [page 27]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+The implementation of multiple outstanding message transactions requires
+the ability to record, timeout and buffer multiple outstanding message
+transactions at the Client end as well as the Server end. However, this
+facility is optional for both the Client and the Server. Client systems
+with heavy-weight processes and high network access cost are most likely
+to benefit from this facility. Servers that serve a wide variety of
+client machines should implement streaming to accommodate these types of
+clients.
+
+
+2.12. Fault-Tolerant Applications
+
+One approach to fault-tolerant systems is to maintain a log of all
+messages sent at each node and replay the messages at a node when the
+node fails, after restarting it from the last checkpoint <6>. As an
+experimental facility, VMTP provides a Receive Sequence Number field in
+the NotifyVmtpClient and NotifyVmtpServer operations as well as the Next
+Receive Sequence (NRS) flag in the Response packet to allow a sender to
+log a receive sequence number with each message sent, allowing the
+packets to be replayed at a recovering node in the same sequence as they
+were originally received, thereby recovering to the same state as
+before.
+
+Basically, each sending node maintains a receive sequence number for
+each receiving node. On sending a Request to a node, it presume that
+the receive sequence number is one greater than the one it has recorded
+for that node. If not, the receiving node sends a notify operation
+indicating the receive sequence number assigned the Request. The NRS in
+the Response confirms that the Request message was the next receive
+sequence number, so the sender can detect if it failed to receive the
+notify operation in the previous case. With Responses, the packets are
+ordered by the Transaction identifier except for multicast message
+transactions, in which there may be multiple Responses with the same
+identification. In this case, NotifyVmtpServer operations are used to
+provide receive sequence numbers.
+
+This experimental extension of the protocol is focused on support for
+fault-tolerant real-time distributed systems required in various
+critical applications. It may be removed or extended, depending on
+further investigations.
+
+_______________
+
+<6> The sender-based logging is being investigated by Willy Zwaenepoel
+of Rice University.
+
+
+Cheriton [page 28]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+2.13. Packet Groups
+
+A message (whether Request or Response) is sent as one or more packet
+groups. A packet group is one or more packets, each containing the same
+transaction identification and message control block. Each packet is
+formatted as below with the message control block logically embedded in
+the VMTP header.
+
+ +------------------------------------++---------------------+
+ | VMTP Header || |
+ +------------+-----------------------|| segment data |
+ |VMTP Control| Message Control Block || |
+ +------------+-----------------------++---------------------+
+
+The some fields of the VMTP control portion of the packet and data
+segment portion can differ between packets within the same packet group.
+
+The segment data portion of a packet group represents up to 16
+kilooctets of the segment specified in the message control block. The
+portion contained in each packet is indicated by the PacketDelivery
+field contained in the VMTP header. The PacketDelivery field as a bit
+mask has a similar interpretation to the MsgDelivery field in that each
+bit corresponds to a segment data block of 512 octets. The
+PacketDelivery field limits a packet group to 16 kilooctets and a
+maximum of 32 VMTP packets (with a minimum of 1 packet). Data can be
+sent in fewer packets by sending multiple data blocks per packet. We
+require that the underlying datagram service support delivery of (at
+minimum) the basic 580 octet VMTP packet <7>. To illustrate the use of
+the PacketDelivery field, consider for example the Ethernet which has a
+MTU of 1536 octets. so one would send 2 512-octet segment data blocks
+per packet. (In fact, if a third block is last in the segment and less
+than 512 octets and fits in the packet without making it too big, an
+Ethernet packet could contain three data blocks. Thus, an Ethernet
+packet group for a segment of size 0x1D00 octets (14.5 blocks) and
+MsgDelivery 0x000074FF consists of 6 packets indicated as follows <8>.
+
+_______________
+
+<7> Note that with a 20 octet IP header, a VMTP packet is 600
+octets. We propose the convention that any host implementing VMTP
+implicitly agrees to accept IP/VMTP packets of at least 600 octets.
+
+<8> We use the C notation 0xHHHH to represent a hexadecimal number.
+
+
+Cheriton [page 29]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Packet
+ Delivery 1 1 1 1 1 1 1 1 0 0 1 0 1 0 1 0 0 0 0 0 0 . . .
+ 0000 0400 0800 0C00 1000 1400 1800 1C00
+ +----+----+----+----+----+----+----+-+
+ Segment |....|....|....|....|....|....|....|.|
+ +----+----+----+----+----+----+----+-+
+ : : : : : : : / / :
+ v v v v v v v /| v
+ +----+----+----+----+ +----+ +---+
+ Packets | 1 | 2 | 3 | 4 | | 5 | | 6 |
+ +----+----+----+----+ +----+ +---+
+
+Each '.' is 256 octets of data. The PacketDelivery masks for the 6
+packets are: 0x00000003, 0x0000000C, 0x00000030, 0x000000C0, 0x00001400
+and 0x00006000, indicating the segment blocks contained in each of the
+packets. (Note that the delivery bits are in little endian order.)
+
+A packet group is sent as a single "blast" of packets with no explicit
+flow control. However, the sender should estimate and transmit at a
+rate of packet transmission to avoid congesting the network or
+overwhelming the receiver, as described in Section 2.5.6. Packets in a
+packet group can be sent in any order with no change in semantics.
+
+When the first packet of a packet group is received (assuming the Server
+does not decide to discard the packet group), the Server saves a copy of
+the VMTP packet header, indicates it is currently receiving a packet
+group, initializes a "current delivery mask" (indicating the data in the
+segment received so far) to 0, accepts this packet (updating the current
+delivery mask) and sets the timer for the packet group. Subsequent
+packets in the packet group update the current delivery mask.
+
+Reception of a packet group is terminated when either the current
+delivery mask indicates that all the packets in the packet group have
+been received or the packet group reception timer expires (set to TC3 or
+TS1). If the packet group reception timer expires, if the NRT bit is
+set in the Control flags then the packet group is discarded if not
+complete unless MDM is set. In this case, the MsgDelivery field in the
+message control block is set to indicate the segment data blocks
+actually received and the message control block and segment data
+received is delivered to application level.
+
+If NRT is not set and not all data blocks have been received, a
+NotifyVmtpClient (if a Request) or NotifyVmtpServer (if a Response) is
+sent back with a PacketDelivery field indicating the blocks received.
+The source of the packet group is then expected to retransmit the
+missing blocks. If not all blocks of a Request are received after
+RequestAckRetries(Client) retransmissions, the Request is discarded and
+
+
+Cheriton [page 30]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+a NotifyVmtpClient operation with an error response code is sent to the
+client's manager unless MDM is set. With a Response, there are
+ResponseAckRetries(Server) retransmissions and then, if MDM is not set,
+the requesting entity is returned the message control block with an
+indication of the amount of segment data received extending contiguously
+from the start of the segment. E.g. if the sender sent 6 512-octet
+blocks and only the first two and the last two arrived, the receiver
+would be told that 1024 octets were received. The ResponseCode field is
+set to BAD_REPLY_SEGMENT. (Note that VMTP is only able to indicate the
+specific segment blocks received if MDM is set.)
+
+The parameters RequestAckRetries(Client) and ResponseAckRetries(Server)
+could be set on a per-client and per-server basis in a sophisticated
+implementation based on knowledge of packet loss.
+
+If the APG flag is set, a NotifyVmtpClient or NotifyVmtpServer
+operation is sent back at the end of the packet group reception,
+depending on whether it is a Request or a Response.
+
+At minimum, a Server should check that each packet in the packet group
+contains the same Client, Server, Transaction identifier and SegmentSize
+fields. It is a protocol error for any field other than the Checksum,
+packet group control flags, Length and PacketDelivery in the VMTP header
+to differ between any two packets in one packet group. A packet group
+containing a protocol error of this nature should be discarded.
+
+Notify operations should be sent (or invoked) in the manager whenever
+there is a problem with a unicast packet. i.e. negative acknowledgments
+are always sent in this case. In the case of problems with multicast
+packets, the default is to send nothing in response to an error
+condition unless there is some clear reason why no other node can
+respond positively. For example, the packet might be a Probe for an
+entity that is known to have been recently existing on the receiving
+host but now invalid and could not have migrated. In this case, the
+receiving host responds to the Probe indicating the entity is
+nonexistent, knowing that no other host can respond to the Probe. For
+packets and packet groups that are received and processed without
+problems, a Notify operation is invoked only if the APG bit is set.
+
+
+2.14. Runs of Packet Groups
+
+A run of packet groups is a sequence of packet groups, all Request
+packets or all Response packets, with the same Client and consecutive
+transaction identifiers, all but the first and last packets flagged with
+the NSR (Not Start Run) and NER (Not End Run) control bits. When each
+packet group in the run corresponds to a single Request or Response, it
+
+
+Cheriton [page 31]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+is identical to a run of message transactions. (See Section 2.11)
+However, a Request message or a Response message may consists of up to
+256 packet groups within a run, for a maximum of 4 megaoctets of segment
+data. A message that is continued in the next packet group in the run
+is flagged in the current packet group by the CMG flag. Otherwise, the
+next packet group in the run (if any) is treated as a separate Request
+or Response.
+
+Normally, each Request and Response message is sent as a single packet
+group and each run consists of a single packet group. In this case
+neither NSR or NER are set. For multi-packet group messages, the
+PacketDelivery mask in the i-th packet group of a message corresponds to
+the portion of the segment offset by i-1 times 16 kilooctets,
+designating the the first packet group to have i = 1.
+
+
+2.15. Byte Order
+
+For purposes of transmission and reception, the MCB is treated as
+consisting of 8 32-bit fields and the segment is a sequence of bytes.
+VMTP transmits the MCB in big-endian order, performing byte-swapping, if
+necessary, before transmission. A little-endian host must byte-swap the
+MCB on reception. (The data segment is transmitted as a sequence of
+bytes with no reordering.) The byte order of the sender of a message is
+indicated by the LEE bit in the entity identifier for the sender, the
+Client field if a Request and the Server field if a Response. The
+sender and receiver of a message are required to agree in some higher
+level protocol (such as an RPC presentation protocol) on who does
+further swapping of the MCB and data segment if required by the types of
+the data actually being transmitted. For example, the segment data may
+contain a record with 8-bit, 16-bit and 32-bit fields, so additional
+transformation is required to move the segment from a host of one byte
+order to another.
+
+VMTP to date has used a higher-level presentation protocol in which
+segment data is sent in the native order of the sending host and
+byte-swapped as necessary by the receiving host. This approach
+minimizes the byte-swapping overhead between machines of common byte
+order (including when the communication is transparently local to one
+host), avoids a strong bias in the protocol to one byte-order, and
+allows for the sending entity to be sending to a group of hosts with
+different byte orders. (Note that the byte-swap overhead for the MCB is
+minimal.) The presentation-level overhead is minimal because most
+common operations, such as file access operations, have parameters that
+fit the MCB and data segment data types exactly.
+
+
+
+
+Cheriton [page 32]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+2.16. Minimal VMTP Implementation
+
+A minimal VMTP client needs to be able to send a Request packet group
+and receive a Response packet group as well as accept and respond to
+Requests sent to its management module, including Probe and NotifyClient
+operations. It may also require the ability to invoke Probe and Notify
+operations to locate a Server and acknowledge responses. (the latter
+only if it is involved in transactions that are not idempotent or
+datagram message transactions. However, a simple sensor, for example,
+can transmit VMTP datagram Requests indicating its current state with
+even less mechanism.) The minimal client thus requires very little code
+and is suitable as a basis for (e.g.) a network boot loader.
+
+A minimal VMTP server implements idempotent, non-encrypted message
+transactions, possibly with no segment data support. It should use an
+entity state record for each Request but need only retain it while
+processing the Request. Without segment data larger than a packet,
+there is no need for any timers, buffering (outside of immediate request
+processing) or queuing. In particular, it needs only as many records as
+message transactions it handles simultaneously (e.g. 1). The entity
+state record is required to recognize and respond to Request
+retransmissions during request processing.
+
+The minimal server need only receive Requests and and be able to send
+Response packets. It need have only a minimal management module
+supporting Probe operations. (Support for the NotifyVmtpClient
+operation is only required if it does not respond immediately to a
+Request.) Thus the VMTP support for say a time server, sensor, or
+actuator can be extremely simple. Note that the server need never issue
+a Probe operation if it uses the host address of the Request for the
+Response and does not require the Client information returned by the
+Probe operation. The minimal server should also support reception of
+forwarded Requests.
+
+
+2.17. Message vs. Procedural Request Handling
+
+A request-response protocol can be used to implement two forms of
+semantics on reception. With procedural handling of a Request, a
+Request is handled by a process associated with the Server that
+effectively takes on the identity of the calling process, treating the
+Request message as invoking a procedure, and relinquishing its
+association to the calling process on return. VMTP supports multiple
+nested calls spanning multiple machines. In this case, the distributed
+call stack that results is associated with a single process from the
+standpoint of authentication and resource management, using the
+ProcessId field supported by VMTP. The entity identifiers effectively
+
+
+Cheriton [page 33]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+link these call frames together. That is, the Client field in a Request
+is effectively the return link to the previous call frame.
+
+With message handling of a Request, a Request message is queued for a
+server process. The server process dequeues, reads, processes and
+responds to the Request message, executing as a separate process.
+Subsequent Requests to the same server are queued until the server asks
+to receive the next Request.
+
+Procedural semantics have the advantage of allowing each Request (up to
+the resource limits of the Server) to execute concurrently at the
+Server, with Request-specific synchronization. Message semantics have
+the advantage that Requests are serialized at the Server and that the
+request processing logically executes with the priority, protection and
+independent execution of a separate process. Note that procedural and
+message handling of a request appear no differently to the client
+invoking the message transaction, except possibly for differences in
+performance.
+
+We view the two Request handling approaches as appropriate under
+different circumstances. VMTP supports both models.
+
+
+2.18. Bibliography
+
+The basic protocol is similar to that used in the original form of the V
+kernel [3, 4] as well as the transport protocol of Birrell and
+Nelson's [2] remote procedure call mechanism. An earlier version of the
+protocol was described in SIGCOMM'86 [6]. The rate-based flow control
+is similar to the techniques of Netblt [9]. The support for idempotency
+draws, in part, on the favorable experience with idempotency in the V
+distributed system. Its use was originally inspired by the Woodstock
+File Server [11]. The multicast support draws on the multicast
+facilities in V [5] and is designed to work with, and is now implemented
+using, the multicast extensions to the Internet [8] described in RFC 966
+and 988. The secure version of the protocol is similar to that
+described by Birrell [1] for secure RPC. The use of runs of packet
+groups is similar to Fletcher and Watson's delta-T protocol [10]. The
+use of "management" operations implemented using VMTP in place of
+specialized packet types is viewed as part of a general strategy of
+using recursion to simplify protocol architectures [7].
+
+Finally, this protocol was designed, in part, to respond to the
+requirements identified by Braden in RFC 955. We believe that VMTP
+satisfies the requirements stated in RFC 955.
+
+
+
+
+Cheriton [page 34]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+
+[1] A.D. Birrell, "Secure Communication using Remote Procedure
+ Calls", ACM. Trans. on Computer Systems 3(1), February, 1985.
+
+
+[2] A. Birrell and B. Nelson, "Implementing Remote Procedure Calls",
+ ACM Trans. on Computer Systems 2(1), February, 1984.
+
+
+[3] D.R. Cheriton and W. Zwaenepoel, "The Distributed V Kernel and its
+ Performance for Diskless Workstations", In Proceedings of the 9th
+ Symposium on Operating System Principles, ACM, 1983.
+
+
+[4] D.R. Cheriton, "The V Kernel: A Software Base for Distributed
+ Systems", IEEE Software 1(2), April, 1984.
+
+
+[5] D.R. Cheriton and W. Zwaenepoel, "Distributed Process Groups in
+ the V Kernel", ACM Trans. on Computer Systems 3(2), May, 1985.
+
+
+[6] D.R. Cheriton, "VMTP: A Transport Protocol for the Next
+ Generation of Communication Systems", In Proceedings of
+ SIGCOMM'86, ACM, Aug 5-7, 1986.
+
+
+[7] D.R. Cheriton, "Exploiting Recursion to Simplify an RPC
+ Communication Architecture", in preparation, 1988.
+
+
+[8] D.R. Cheriton and S.E. Deering, "Host Groups: A Multicast
+ Extension for Datagram Internetworks", In 9th Data Communication
+ Symposium, IEEE Computer Society and ACM SIGCOMM, September, 1985.
+
+
+[9] D.D. Clark and M. Lambert and L. Zhang, "NETBLT: A Bulk Data
+ Transfer Protocol", Technical Report RFC 969, Defense Advanced
+ Research Projects Agency, 1985.
+
+
+[10] J.G. Fletcher and R.W. Watson, "Mechanism for a Reliable Timer-
+ based Protocol", Computer Networks 2:271-290, 1978.
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 35]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+
+
+[11] D. Swinehart and G. McDaniel and D. Boggs, "WFS: A Simple File
+ System for a Distributed Environment", In Proc. 7th Symp.
+ Operating Systems Principles, 1979.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 36]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+3. VMTP Packet Formats
+
+VMTP uses 2 basic packet formats corresponding to Request packets and
+Response packets. These packet formats are identical in most of the
+fields to simplify the implementation.
+
+We first describe the entity identifier format and the packet fields
+that are used in general, followed by a detailed description of each of
+the packet formats. These fields are described below in detail. The
+individual packet formats are described in the following subsections.
+The reader and VMTP implementor may wish to refer to Chapters 4 and 5
+for a description of VMTP event handling and only refer to this detailed
+description as needed.
+
+
+3.1. Entity Identifier Format
+
+The 64-bit non-group entity identifiers have the following substructure.
+
+ 0 1 2 3
+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |R| |L|R|
+ |A|0|E|E| Domain-specific structure
+ |E| |E|S|
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ Domain-specific structure |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+The field meanings are as follows:
+
+RAE Remote Alias Entity - the entity identifier identifies
+ an entity that is acting as an alias for some entity
+ outside this entity domain. This bit is used by
+ higher-level protocols. For instance, servers may take
+ extra security and protection measures with aliases.
+
+GRP Group - 0, for non-group entity identifiers.
+
+LEE Little-Endian Entity - the entity transmits data in
+ little-endian (VAX) order.
+
+RES Reserved - must be 0.
+
+The 64-bit entity group identifiers have the following substructure.
+
+
+
+
+Cheriton [page 37]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ 0 1 2 3
+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |R| |U|R|
+ |A|1|G|E| Domain-specific structure
+ |E| |P|S|
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ Domain-specific structure |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+The field meanings are as follows:
+
+RAE Remote Alias Entity - same as for non-group entity
+ identifier.
+
+GRP Group - 1, for entity group identifiers.
+
+UGP Unrestricted Group - no restrictions are placed on
+ joining this group. I.e. any entity can join limited
+ only by implementation resources.
+
+RES Reserved - must be 0.
+
+The all-zero entity identifier is reserved and guaranteed to be
+unallocated in all domains. In addition, a domain may reserve part of
+the entity identifier space for statically allocated identifiers.
+However, this is domain-specific.
+
+Description of currently defined entity identifier domains is provided
+in Appendix IV.
+
+
+3.2. Packet Fields
+
+Client 64-bit identifier for the client entity associated with
+ this packet. The structure, allocation and binding of
+ this identifier is specific to the specified Domain. An
+ entity identifier always includes 4 types bits as
+ specified in Section 3.1.
+
+Version The 3-bit identifier specifying the version of the
+ protocol. Current version is version 0.
+
+Domain The 13-bit identifier specifying the naming and
+ administration domain for the client and server named in
+ the packet.
+
+
+
+Cheriton [page 38]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+Packet Flags: 3 bits. (The normal case has none of the flags set.)
+
+ HCO Header checksum only - checksum has only been calculated
+ on the header. This is used in some real-time
+ applications where the strict correctness of the data is
+ not needed.
+
+ EPG Encrypted packet group - part of a secure message
+ transaction.
+
+ MPG Multicast packet group - packet was multicast on
+ transmission.
+
+Length A 13-bit field that specifies the number of 32-bit words
+ in the segment data portion of the packet (if any),
+ excluding the checksum field. (Every VMTP packet is
+ required to be a multiple of 64 bits, possibly by
+ padding out the segment data.) The minimum legal Length
+ is 0, the maximum length is 4096 and it must be an even
+ number.
+
+Control Flags: 9 bits. (The normal case has none of the flags set.)
+
+ NRS Next Receive Sequence - the associated Request message
+ (in a Response) or previous Response (if a Request) was
+ received consecutive with the last Request from this
+ entity. That is, there was no interfering messages
+ received.
+
+ APG Acknowledge Packet Group - Acknowledge packet group on
+ receipt. If a Request, send back a Request to the
+ client's manager providing an update on the state of the
+ transaction as soon as the request packet group is
+ received, independent of the response being available.
+ If a Response, send an update to the server's manager as
+ soon as possible after response packet group is received
+ providing an update on the state of the transaction at
+ the client
+
+ NSR Not Start Run - 1 if this packet is not part of the
+ first packet group of a run of packet groups.
+
+ NER Not End Run - 1 if this packet is not part of the last
+ packet group of a run of packet groups.
+
+ NRT No Retransmission - do not ask for retransmissions of
+ this packet group if not all received within timeout
+
+
+Cheriton [page 39]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ period, just deliver or discard.
+
+ MDG Member of Destination Group - this packet is sent to a
+ group and the client is a member of this group.
+
+ CMG Continued Message - the message (Request or Response) is
+ continued in the next packet group. The next packet
+ group has to be part of the same run of packet groups.
+
+ STI Skip Transaction Identifiers - the next transaction
+ identifier that the Client plans to use is the current
+ transaction plus 256, if part of the same run and at
+ least this big if not. In a Request, this authorizes
+ the Server to send back up to 256 packet groups
+ containing the Response.
+
+ DRT Delay Response Transmission - set by request sender if
+ multiple responses are expected (as indicated by the MRD
+ flag in the RequestCode) and it may be overrun by
+ multiple responses. The responder(s) should then
+ introduce a short random delay in sending the Response
+ to minimize the danger of overrunning the Client. This
+ is normally only used for responding to multicast
+ Requests where the Client may be receiving a large
+ number of Responses, as indicated by the MRD flag in the
+ Request flags. Otherwise, the Response is sent
+ immediately.
+
+RetransmitCount:
+ 3 bits - the ordinal number of transmissions of this
+ packet group prior to this one, modulo 8. This field is
+ used in estimation of roundtrip times. This count may
+ wrap around during a message transaction. However, it
+ should be sufficient to match acknowledgments and
+ responses with a particular transmission.
+
+ForwardCount: 4 bits indicating the number of times this Request has
+ been forwarded. The original Request is always sent
+ with a ForwardCount of 0.
+
+Interpacket Gap: 8 bits.
+ Indicates the recommended time to use between subsequent
+ packet transmissions within a multi-packet packet group
+ transmission. The Interpacket Gap time is in 1/32nd of
+ a network packet transmission time for a packet of size
+ MTU for the node. (Thus, the maximum gap time is 8
+ packet times.)
+
+
+Cheriton [page 40]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+PGcount: 8 bits
+ The number of packet groups that this packet group
+ represents in addition to that specified by the
+ Transaction field. This is used in acknowledging
+ multiple packet groups in streamed communication.
+
+Priority 4-bit identifier for priority for the processing of this
+ request both on transmission and reception. The
+ interpretation is:
+
+ 1100 urgent/emergency
+
+ 1000 important
+
+ 0000 normal
+
+ 0100 background
+
+ Viewing the higher-order bit as a sign bit (with 1
+ meaning negative), low values are high priority and high
+ values are low priority. The low-order 2 bits indicate
+ additional (lower) gradations for each level.
+
+Function Code: 1 bit - types of VMTP packets. If the low-order bit of
+ the function code is 0, the packet is sent to the
+ Server, else it is sent to the Client.
+
+ 0 Request
+
+ 1 Response
+
+Transaction: 32 bits:
+ Identifier for this message transaction.
+
+PacketDelivery: 32 bits:
+ Delivery indicates the segment blocks contained in this
+ packet. Each bit corresponds to one 512-octet block of
+ segment data. A 1 bit in the i-th bit position
+ (counting the LSB as 0) indicates the presence of the
+ i-th segment block.
+
+Server: 64 bits
+ Entity identifier for the server or server group
+ associated with this transaction. This is the receiver
+ when a Request packet and the sender when a Response
+ packet.
+
+
+
+Cheriton [page 41]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+Code: 32 bits The Request Code and Response Code, set either at the
+ user level or VMTP level depending on use and packet
+ type. Both the Request and Response codes include 8
+ high-order bits from the following set of control bits:
+
+ CMD Conditional Message Delivery - only deliver the request
+ or response if the receiving entity is waiting for it at
+ the time of delivery, otherwise drop the message.
+
+ DGM DataGram Message - indicates that the message is being
+ sent as a datagram. If a Request message, do not wait
+ for reply, or retransmit. If a Response message, treat
+ this message transaction as idempotent.
+
+ MDM Message Delivery Mask - indicates that the MsgDelivery
+ field is being used. Otherwise, the MsgDelivery field
+ is available for general use.
+
+ SDA Segment Data Appended - segment data is appended to the
+ message control block, with the total size of the
+ segment specified by the SegmentSize field. Otherwise,
+ the segment data is null and the SegmentSize field is
+ not used by VMTP and available for user- or RPC-level
+ uses.
+
+ CRE CoResident Entity - indicates that the CoResidentEntity
+ field in the message should be interpreted by VMTP.
+ Otherwise, this field is available for additional user
+ data.
+
+ MRD Multiple Responses Desired - multiple Responses are
+ desired to to this Request if it is multicast.
+ Otherwise, the VMTP module can discard subsequent
+ Responses after the first Response.
+
+ PIC Public Interface Code - Values for Code with this bit
+ set are reserved for definition by the VMTP
+ specification and other standard protocols defined on
+ top of VMTP.
+
+ RES Reserved for future use. Must be 0.
+
+CoResidentEntity
+ 64-bit Identifier for an entity or group of entities
+ with which the Server entity or entities must be
+ co-resident, i.e. route only to entities (identified by
+ Server) on the same host(s) as that specified by
+
+
+Cheriton [page 42]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ CoResidentEntity, Only meaningful if CRE is set in the
+ Code field.
+
+User Data 12 octets Space in the header for the VMTP user to
+ specify user-specific control and data.
+
+MsgDelivery: 32 bits
+ The segment blocks being transmitted (in total) in this
+ packet group following the conventions for the
+ PacketDelivery field. This field is ignored by the
+ protocol and treated as an additional user data field if
+ MDM is 0. On transmission, the user level sets the
+ MsgDelivery to indicate those portions of the segment to
+ be transmitted. On receipt, the MsgDelivery field is
+ modified by the VMTP module to indicate the segment data
+ blocks that were actually received before the message
+ control block is passed to the user or RPC level. In
+ particular, the kernel does not discard the packet group
+ if segment data blocks are missing. A Server or Client
+ entity receiving a message with a MsgDelivery in use
+ must check the field to ensure adequate delivery and
+ retry the operation if necessary.
+
+SegmentSize: 32 bits
+ Size of segment in octets, up to a maximum of 16
+ kilooctets without streaming and 4 megaoctets with
+ streaming, if SDA is set. Otherwise, this field is
+ ignored by the protocol and treated as an additional
+ user data field.
+
+Segment Data: 0-16 kilooctets
+ 0 octets if SDA is 0, else the portion of the segment
+ corresponding to the Delivery Mask, limited by the
+ SegmentSize and the MTU, padded out to a multiple of 64
+ bits.
+
+Checksum: 32 bits.
+ The 32-bit checksum for the header and segment data.
+
+
+The VMTP checksum algorithm <9> develops a 32-bit checksum by computing
+
+_______________
+
+<9> This algorithm and description are largely due to Steve Deering of
+Stanford University.
+
+
+Cheriton [page 43]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+two 16-bit, ones-complement sums (like IP), each covering different
+parts of the packet. The packet is divided into clusters of 16 16-bit
+words. The first, third, fifth,... clusters are added to the first sum,
+and the second, fourth, sixth,... clusters are added to the second sum.
+Addition stops at the end of the packet; there is no need to pad out to
+a cluster boundary (although it is necessary that the packet be an
+integral multiple of 64 bits; padding octets may have any value and are
+included in the checksum and in the transmitted packet). If either of
+the resulting sums is zero, it is changed to 0xFFFF. The two sums are
+appended to the transmitted packet, with the first sum being transmitted
+first. Four bytes of zero in place of the checksum may be used to
+indicate that no checksum was computed.
+
+The 16-bit, ones-complement addition in this algorithm is the same as
+used in IP and, therefore, subject to the same optimizations. In
+particular, the words may be added up 32-bits at a time as long as the
+carry-out of each addition is added to the sum on the following
+addition, using an "add-with-carry" type of instruction. (64-bit or
+128-bit additions would also work on machines that have registers that
+big.)
+
+A particular weakness of this algorithm (shared by IP) is that it does
+not detect the erroneous swapping of 16-bit words, which may easily
+occur due to software errors. A future version of VMTP is expected to
+include a more secure algorithm, but such an algorithm appears to
+require hardware support for efficient execution.
+
+Not all of these fields are used in every packet. The specific packet
+formats are described below. If a field is not mentioned in the
+description of a packet type, its use is assumed to be clear from the
+above description.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 44]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+3.3. Request Packet
+
+The Request packet (or packet group) is sent from the client to the
+server or group of servers to solicit processing plus the return of zero
+or more responses. A Request packet is identified by a 0 in the LSB of
+the fourth 32-bit word in the packet.
+
+ 0 1 2 3
+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ + Client (8 octets) +
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |Ver | |H|E|M| |
+ |sion | Domain |C|P|P| Length |
+ | | |O|G|G| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |N|A|N|N|N|M|C|S|D|Retra|Forward| Inter- | |R|R|R| |
+ |R|P|S|E|R|D|M|T|R|nsmit| Count | Packet | Prior |E|E|E|0|
+ |S|G|R|R|T|G|G|I|T|Count| | Gap | -ity |S|S|S| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | Transaction |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | PacketDelivery |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ + Server (8 octets) +
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |C|D|M|S|R|C|M|P| |
+ |M|G|D|D|E|R|R|I| RequestCode |
+ |D|M|M|A|S|E|D|C| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ + CoResidentEntity (8 octets) +
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ > User Data (12 octets) <
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | MsgDelivery |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | SegmentSize |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ > segment data, if any <
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | Checksum |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+ Figure 3-1: Request Packet Format
+
+The fields of the Request packet are set according to the semantics
+described in Section 3.2 with the following qualifications.
+
+
+Cheriton [page 45]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+InterPacketGap The estimated interpacket gap time the client would like
+ for the Response packet group to be sent by the Server
+ in responding to this Request.
+
+Transaction Identifier for transaction, at least one greater than
+ the previously issued Request from this Client.
+
+Server Server to which this Request is destined.
+
+RequestCode Request code for this request, indicating the operation
+ to perform.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 46]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+3.4. Response Packet
+
+The Response packet is sent from the Server to the Client in response to
+a Request, identified by a 1 in the LSB of the fourth 32-bit word in the
+packet.
+
+ 0 1 2 3
+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ + Client (8 octets) +
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |Ver | |H|E|M| |
+ |sion | Domain |C|P|P| Length |
+ | | |O|G|G| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |N|A|N|N|N|R|C|S|R|Retra|Forward| | |R|R|R| |
+ |R|P|S|E|R|E|M|T|E|nsmit| Count | PGcount | Prior |E|E|E|1|
+ |S|G|R|R|T|S|G|I|S|Count| | | -ity |S|S|S| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | Transaction |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | PacketDelivery |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ + Server (8 octets) +
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ |C|D|M|S|R|R|R|R| |
+ |M|G|D|D|E|E|E|E| ResponseCode |
+ |D|M|M|A|S|S|S|S| |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ > UserData (20 octets) <
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | MsgDelivery |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | Segment Size |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ > segment data, if any <
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | Checksum |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+ Figure 3-2: Response Packet Format
+
+The fields of the Response packet are set according to the semantics
+described in Section 3.2 with the following qualifications.
+
+Client, Version, Domain, Transaction
+ Match those in the Request packet group to which this is
+
+
+Cheriton [page 47]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ a response.
+
+STI 1 if this Response is using one or more of the
+ transaction identifiers skipped by the Client after the
+ Request to which this is a Response. STI in the Request
+ essentially allocates up to 256 transaction identifiers
+ for the Server to use in a run of Response packet
+ groups.
+
+RetransmitCount The retransmit count from the last Request packet
+ received to which this is a response.
+
+ForwardCount The number of times the corresponding Request was
+ forwarded before this Response was generated.
+
+PGcount The number of consecutively previous packet groups that
+ this response is acknowledging in addition to the one
+ identified by the Transaction identifier.
+
+Server Server sending this response. This may differ from that
+ originally specified in the Request packet if the
+ original Server was a server group, or the request was
+ forwarded.
+
+The next two chapters describes the protocol operation using these
+packet formats, with the the Client and the Server portions described
+separately.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 48]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+4. Client Protocol Operation
+
+This chapter describes the operation of the client portion of VMTP in
+terms of the procedures for handling VMTP user events, packet reception
+events, management operations and timeout events. Note that the client
+portion of VMTP is separable from the server portion. It is feasible to
+have a node that only implements the client end of VMTP.
+
+To simplify the description, we define a client state record (CSR) plus
+some standard utility routines.
+
+
+4.1. Client State Record Fields
+
+In the following protocol description, there is one client state record
+(CSR) per (client,transaction) outstanding message transaction. Here is
+a suggested set of fields.
+
+Link Link to next CSR when queued in one of the transmission,
+ timeout or message queues.
+
+QueuePtr Pointer to queue head in which this CSR is contained or
+ NULL if none. Queue could be one of transmission queue,
+ timeout queue, server queue or response queue.
+
+ProcessIdentification
+ The process identification and address space.
+
+Priority Priority for processing, network service, etc.
+
+State One of the client states described below.
+
+FinishupFunc Procedure to be executed on the CSR when it is completes
+ its processing in transmission or timeout queues.
+
+TimeoutCount Time to remain in timeout queue.
+
+TimeoutLimit User-specified time after which the message transaction
+ is aborted. The timeout is infinite if set to zero.
+
+RetransCount Number of retransmissions since last hearing from the
+ Server.
+
+LastTransmitTime
+ The time at which the last packet was sent. This field
+ is used to calculate roundtrip times, using the
+ RetransmitCount to match the responding packet to a
+
+
+Cheriton [page 49]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ particular transmission. I.e. Response or management
+ NotifyVmtpClient operation to Request and a management
+ NotifyVmtpServer operation to a Response.
+
+TimetoLive Time to live to be used on transmission of IP packets.
+
+TransmissionMask
+ Bit mask indicating the portions of the segment to
+ transmit. Set before entering the transmission queue
+ and cleared incrementally as the 512-byte segment blocks
+ of the segment are transmitted.
+
+LocalClientLink Link to next CSR hashing to same hash index in the
+ ClientMap.
+
+LocalClient Entity identifier for client when this CSR is used to
+ send a Request packet.
+
+LocalTransaction
+ Transaction identifier for current message transaction
+ the local client has outstanding.
+
+LocalPrincipal Account identification, possibly including key and key
+ timeout.
+
+LocalDelivery Bit mask of segment blocks that have not been
+ acknowledged in the Request or have been received in the
+ Response, depending on the state.
+
+ResponseQueue Queue of CSR's representing the queued Responses for
+ this entity.
+
+VMTP Header Prototype VMTP header, used to generate and store the
+ header portion of a Request for transmission and
+ retransmission on timeout.
+
+SegmentDesc Description of the segment data associated with the CSR,
+ either the area storing the original Request data, the
+ area for receiving Request data, or the area storing the
+ Response data that is returned.
+
+HostAddr The network or internetwork host address to which the
+ Client last transmitted. This field also indicates the
+ type of the address, e.g. IP, Ethernet, etc.
+
+Note: the CSR can be combined with a light-weight process descriptor
+with considerable benefit if the process is designed to block when it
+
+
+Cheriton [page 50]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+issues a message transaction. In particular, by combining the two
+descriptors, the implementation saves time because it only needs to
+locate and queue one descriptor with various operations (rather than
+having to locate two descriptors). It also saves space, given that the
+VMTP header prototype provides space such as the user data field which
+may serve to store processor state for when the process is preempted.
+Non-preemptive blocking can use the process stack to store the processor
+state so only a program counter and stack pointer may be required in the
+process descriptor beyond what we have described. (This is the approach
+used in the V kernel.)
+
+
+4.2. Client Protocol States
+
+A Client State Record records the state of message transaction generated
+by this host, identified by the (Client, Transaction) values in the CSR.
+As a client originating a transaction, it is in one of the following
+states.
+
+AwaitingResponse
+ Waiting for a Response packet group to arrive with the
+ same (Client,Transaction) identification.
+
+ReceivingResponse
+ Waiting for additional packets in the Response packet
+ group it is currently receiving.
+
+"Other" Not waiting for a response, which can be Processing or
+ some other operating system state, or one of the Server
+ states if it also acts as a server.
+
+This covers all the states for a client.
+
+
+4.3. State Transition Diagrams
+
+The client state transitions are illustrated in Figure 4-1. The client
+goes into the state AwaitingResponse on sending a request unless it is a
+datagram request. In the AwaitingResponse state, it can timeout and
+retry and eventually give up and return to the processing state unless
+it receives a Response. (A NotifyVmtpClient operation resets the
+timeout but does not change the state.) On receipt of a single packet
+response, it returns to the processing state. Otherwise, it goes to
+ReceivingResponse state. After timeout or final response packet is
+received, the client returns to the processing state. The processing
+state also includes any other state besides those associated with
+issuing a message transaction.
+
+
+Cheriton [page 51]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ +------------+
+ | Processing |<--------------------|
+ | |<-------------| |
+ | |<---| | |
+ +|------^--^-+ Single Last |
+ Transmit | | Packet Response |
+ | | | Response Packet |
+ | | | | | |
+ +-DGM->+ Timeout | | Final timeout
+ | | | | |
+ +V-----------+ | +-----------+
+ | Awaiting |----+ | Receiving |->Response-+
+ | Response |->Response->| Response | |
+ | | (multi- | |<----------+
+ +-|--------^-+ packet) +----------^+
+ V | | |
+ +-Timeout+ +>Timeout+
+
+ Figure 4-1: Client State Transitions
+
+
+4.4. User Interface
+
+The RPC or user interface to VMTP is implementation-dependent and may
+use systems calls, functions or some other mechanism. The list of
+requests that follow is intended to suggest the basic functionality that
+should be available.
+
+Send( mcb, timeout, segptr, segsize )
+ Initiate a message transaction to the server and request
+ message specified by mcb and return a response in mcb,
+ if it is received within the specified timeout period
+ (or else return USER_TIMEOUT in the Code field). The
+ segptr parameter specifies the location from which the
+ segment data is sent and the location into which the
+ response data is to be delivered. The segsize field
+ indicates the maximum length of this area.
+
+GetResponse( responsemcb, timeout, segptr, segsize )
+ Get the next response sent to this client as part of the
+ current message transaction, returning the segment data,
+ if any, into the memory specified by segptr and segsize.
+
+This interface assumes that there is a client entity associated with the
+invoking process that is to be used with these operations. Otherwise,
+the client entity must be specified as an additional parameter.
+
+
+
+Cheriton [page 52]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+4.5. Event Processing
+
+The following events may occur in the VMTP client:
+
+ - User Requests
+
+ * Send
+
+ * GetResponse
+
+ - Packet Arrival
+
+ * Response Packet
+
+ * Request
+
+ The minimal Client implementation handles Request packets for
+ its VMTP management (server) module and sends NotifyVmtpClient
+ requests in response to others, indicating the specified
+ server does not exist.
+
+ - Management Operation - NotifyVmtpClient
+
+ - Timeouts
+
+ * Client Retransmission Timeout
+
+The handling of these events is described in detail in the following
+subsections.
+
+We first describe some conventions and procedures used in the
+description. A field of the received packet is indicated as (for
+example) p.Transaction, for the Transaction field. Optional portions of
+the code, such as the streaming handling code are prefixed with a "|" in
+the first column.
+
+MapClient( client )
+ Return pointer to CSR for client with the specified
+ clientId, else NULL.
+
+SendPacketGroup( csr )
+ Send the packet group (Request, Response) according to
+ that specified by the CSR.
+
+NotifyClient( csr, p, code )
+ Invoke the NotifyVmtpClient operation with the
+ parameters csr.RemoteClient, p.control,
+
+
+Cheriton [page 53]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ csr.ReceiveSeqNumber, csr.RemoteTransaction and
+ csr.RemoteDelivery, and code. If csr is NULL, use
+ p.Client, p.Transaction and p.PacketDelivery instead and
+ the global ReceiveSequenceNumber, if supported. This
+ function simplifies the description over calling
+ NotifyVmtpClient directly in the procedural
+ specification below. (See Appendix III.)
+
+NotifyServer( csr, p, code )
+ Invoke the NotifyVmtpServer operation with the
+ parameters p.Server, csr.LocalClient,
+ csr.LocalTransaction, csr.LocalDelivery and code. Use
+ p.Client, P.Transaction and 0 for the clientId, transact
+ and delivery parameters if csr is NULL. This function
+ simplifies the description over calling NotifyVmtpServer
+ directly in the procedural specification below. (See
+ Appendix III.)
+
+DGMset(p) True if DGM bit set in packet (or csr) else False.
+ (Similar functions are used for other bits.)
+
+Timeout( csr, timeperiod, func )
+ Set or reset timer on csr record for timeperiod later
+ and invoke func if the timeout expires.
+
+
+4.6. Client User-invoked Events
+
+A user event occurs when a VMTP user application invokes one of the VMTP
+interface procedures.
+
+
+4.6.1. Send
+
+Send( mcb, timeout, segptr, segsize )
+ map to main CSR for this client.
+ increment csr.LocalTransaction
+ Init csr and check parameters and segment if any.
+ Set SDA if sending appended data.
+ Flush queued replies from previous transaction, if any.
+ if local non-group server then
+ deliver locally
+ await response
+ return
+ if GroupId(server) then
+ Check for and deliver to local members.
+ if CRE request and non-group local CR entity then
+
+
+Cheriton [page 54]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ await response
+ return
+ endif
+ set MDG if member of this group.
+ endif
+ clear csr.RetransCount
+ set csr.TransmissionMask
+ set csr.TimeLimit to timeout
+ set csr.HostAddr for csr.Server
+ SendPacketGroup( csr )
+ if DGMset(csr) then
+ return
+ endif
+ set csr.State to AwaitingResponse
+ Timeout( rootcsr, TC1(csr.Server), LocalClientTimeout )
+ return
+end Send
+
+Notes:
+
+ 1. Normally, the HostAddr is extracted from the ServerHost
+ cache, which maps server entity identifiers to host
+ addresses. However, on cache miss, the client first queries
+ the network using the ProbeEntity operation, as specified in
+ Appendix III, determining the host address from the Response.
+ The ProbeEntity operation is handled as a separate message
+ transaction by the Client.
+
+The stream interface incorporates a parameter to pass a responseHandler
+procedure that is invoked when the message transaction completes.
+
+StreamSend( mcb, timeout, segptr, segsize, responseHandler )
+ map to main CSR for this client.
+| Allocate a new csr if root in use.
+| lastcsr := First csr for last request.
+| if STIset(lastcsr)
+| csr.LocalTransaction := lastcsr.LocalTransaction + 256
+| else
+| csr.LocalTransaction := lastcsr.LocalTransaction + 1
+ Init csr and check parameters and segment if any.
+ . . . ( rest is the same as for the normal Send)
+
+Notes:
+
+ 1. Each outstanding message transaction is represented by a CSR
+ queued on the root CSR for this client entity. The root CSR
+ is used to handle timeouts, etc. On timeout, the last packet
+
+
+Cheriton [page 55]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ from the last packet group is retransmitted (with or without
+ the segment data).
+
+
+4.6.2. GetResponse
+
+GetResponse( req, timeout, segptr, segsize )
+ csr := CurrentCSR;
+ if responses queued then return next response
+ (in req, segptr to max of segsize )
+ if timeout is zero then return KERNEL_TIMEOUT error
+ set state to AWAITING_RESPONSE
+ Timeout( csr, timeout, ReturnKernelTimeout );
+end GetResponse
+
+Notes:
+
+ 1. GetResponse is only used with multicast Requests, which is
+ the only case in which multiple (different) Responses should
+ be received.
+
+ 2. A response must remain queued until the next message
+ transaction is invoked to filter out duplicates of this
+ response.
+
+ 3. If the response is incomplete (only relevant if a
+ multi-packet response), then the client may wait for the
+ response to be fully received, including issuing requests for
+ retransmission (using NotifyVmtpServer operations) before
+ returning the response.
+
+ 4. As an optimization, a response may be stored in the CSR of
+ the client. In this case, the response must be transferred
+ to a separate buffer (for duplicate suppression) before
+ waiting for another response. Using this optimization, a
+ response buffer is not allocated in the common case of the
+ client receiving only one response.
+
+
+4.7. Packet Arrival
+
+In general, on packet reception, a packet is mapped to the client state
+record, decrypted if necessary using the key in the CSR. It then has
+its checksum verified and then is transformed to the right byte order.
+The packet is then processed fully relative to its packet function code.
+It is discarded immediately if it is addressed to a different domain
+than the domain(s) in which the receiving host participates.
+
+
+Cheriton [page 56]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+For each of the 2 packet types, we assume a procedure called with a
+pointer p to the VMTP packet and psize, the size of the packet in
+octets. Thus, generic packet reception is:
+
+if not LocalDomain(p.Domain) then return;
+
+csr := MapClient( p.Client )
+
+if csr is NULL then
+ HandleNoCsr( p, psize )
+ return
+
+if Secure(p) then
+ if SecureVMTP not supported then
+ { Assume a Request. }
+ if not Multicast(p) then
+ NotifyClient(NULL, p, SECURITY_NOT_SUPPORTED )
+ return
+ endif
+| Decrypt( csr.Key, p, psize )
+
+if p.Checksum not null then
+ if not VerifyChecksum(p, psize) then return;
+if OppositeByteOrder(p) then ByteSwap( p, psize )
+if psize not equal sizeof(VmtpHeader) + 4*p.Length then
+ NotifyClient(NULL, p, VMTP_ERROR )
+ return
+Invoke Procedure[p.FuncCode]( csr, p, psize )
+Discard packet and return
+
+Notes:
+
+ 1. The Procedure[p.FuncCode] refers to one of the 2 procedures
+ corresponding to the two different packet types of VMTP,
+ Requests and Responses.
+
+ 2. In all the following descriptions, a packet is discarded on
+ "return" unless otherwise stated.
+
+ 3. The procedure HandleNoCSR is a management routine that
+ allocates and initializes a CSR and processes the packet or
+ else sends an error indication to the sender of the packet.
+ This procedure is described in greater detail in Section
+ 4.8.1.
+
+
+
+
+
+Cheriton [page 57]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+4.7.1. Response
+
+This procedure handles incoming Response packets.
+
+HandleResponse( csr, p, psize )
+ if not LocalClient( csr ) then
+ if Multicast then return
+| if Migrated( p.Client ) then
+| NotifyServer(csr, p ENTITY_MIGRATED )
+| else
+ NotifyServer(csr, p, ENTITY_NOT_HERE )
+ return
+ endif
+
+ if NSRset(p) then
+ if Streaming not supported then
+ NotifyServer(csr, p, STREAMING_NOT_SUPPORTED )
+ return STREAMED_RESPONSE
+| Find csr corresponding to p.Transaction
+| if none found then
+| NotifyServer(csr, p, BAD_TRANSACTION_ID )
+| return
+ else
+ if csr.LocalTransaction not equal p.Transaction then
+ NotifyServer(csr, p, BAD_TRANSACTION_ID )
+ return
+ endif
+ Locate reply buffer rb for this p.Server
+ if found then
+ if rb.State is not ReceivingResponse then
+ { Duplicate }
+ if APGset(p) or NERset(p) then
+ { Send Response to stop response packets. }
+ NotifyServer(csr, p, RESPONSE_DISCARDED )
+ endif
+ return
+ endif
+ { rb.State is ReceivingRequest}
+ if new segment data then retain in CSR segment area.
+ if packetgroup not complete then
+ Timeout( rb, TC3(p.Server), LocalClientTimeout )
+ return;
+ endif
+ goto EndPacketGroup
+ endif
+ { Otherwise, a new response message. }
+
+
+
+Cheriton [page 58]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ if (NSRset(p) or NERset(p)) and NoStreaming then
+ NotifyServer(csr, p, VMTP_ERROR )
+ return
+| if NSRset(p) then
+| { Check consecutive with previous packet group }
+| Find last packet group CSR from p.Server.
+| if p.Transaction not
+| lastcsr.RemoteTransaction+1 mod 2**32 then
+| { Out of order packet group }
+| NotifyServer(csr, p, BAD_TRANSACTION_ID)
+| return
+| endif
+| if lastcsr not completed then
+| NotifyServer(lastcsr, p, RETRY )
+| endif
+| if CMG(lastcsr) then
+| Add segment data to lastcsr Response
+| Notify lastcsr with new packet group.
+| Clear lastcsr.VerifyInterval
+| else
+| if lastcsr available then
+| use it for this packet group
+| else allocate and initialize new CSR
+| Save message and segment data in new CSR area.
+| endif
+| else { First packet group }
+ Allocate and init reply buffer rb for this response.
+ if allocation fails then
+ NotifyServer(csr, p, BUSY )
+ return
+ Set rb.State to ReceivingResponse
+ Copy message and segment data to rb's segment area
+ and set rb.PacketDelivery to that delivered.
+ Save p.Server host address in ServerHost cache.
+ endif
+ if packetgroup not complete then
+ Timeout( rb, TS1(p.Client), LocalClientTimeout )
+ return;
+ endif
+endPacketGroup:
+ { We have received last packet in packet group. }
+ if APGset(p) then NotifyServer(csr, p, OK )
+| if NERset(p) and CMGset(p) then
+| Queue waiting for continuation packet group.
+| Timeout( rb, TC2(rb.Server), LocalClientTimeout )
+| return
+| endif
+
+
+Cheriton [page 59]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ { Deliver response message. }
+ Deliver response to Client, or queue as appropriate.
+end HandleResponse
+
+Notes:
+
+ 1. The mechanism for handling streaming is optional and can be
+ replaced with the tests for use of streaming. Note that the
+ server should never stream at the Client unless the Client
+ has streamed at the Server or has used the STI control bit.
+ Otherwise, streamed Responses are a protocol error.
+
+ 2. As an optimization, a Response can be stored into the CSR for
+ the Client rather than allocating a separate CSR for a
+ response buffer. However, if multiple responses are handled,
+ the code must be careful to perform duplicate detection on
+ the Response stored there as well as those queued. In
+ addition, GetResponse must create a queued version of this
+ Response before allowing it to be overwritten.
+
+ 3. The handling of Group Responses has been omitted for brevity.
+ Basically, a Response is accepted if there has been a Request
+ received locally from the same Client and same Transaction
+ that has not been responded to. In this case, the Response
+ is delivered to the Server or queued.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 60]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+4.8. Management Operations
+
+VMTP uses management operations (invoked as remote procedure calls) to
+effectively acknowledge packet groups and request retransmissions. The
+following routine is invoked by the Client's management module on
+request from the Server.
+
+NotifyVmtpClient( clientId,ctrl,receiveSeqNumber,transact,delivery,code)
+ Get csr for clientId
+ if none then return
+ if RemoteClient( csr ) and not NotifyVmtpRemoteClient then
+ return
+| else (for streaming)
+| Find csr with same LocalTransaction as transact
+| if csr is NULL then return
+ if csr.State not AwaitingResponse then return
+ if ctrl.PGcount then ack previous packet groups.
+ select on code
+ case OK:
+ Notify ack'ed segment blocks from delivery
+ Clear csr.RetransCount;
+ Timeout( csr, TC1(csr.Server), LocalClientTimeout )
+ return
+ case RETRY:
+ Set csr.TransmissionMask to missing segment blocks,
+ as specified by delivery
+ SendPacketGroup( csr )
+ Timeout( csr, TC1(csr.Server), LocalClientTimeout )
+ case RETRY_ALL
+ Set csr.TransmissionMask to retransmit all blocks.
+ SendPacketGroup( csr )
+ Timeout( csr, TC1(csr.Server), LocalClientTimeout )
+| if streaming then
+| Restart transmission of packet groups,
+| starting from transact+1
+ return
+ case BUSY:
+ if csr.TimeLimit exceeded then
+ Set csr.Code to USER_TIMEOUT
+ return Response to application
+ return;
+ Set csr.TransmissionMask for full retransmission
+ Clear csr.RetransCount
+ Timeout( csr, TC1(csr.Server), LocalClientTimeout )
+ return
+ case ENTITY_MIGRATED:
+ Get new host address for entity
+
+
+Cheriton [page 61]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Set csr.TransmissionMask for full retransmission
+ Clear csr.RetransCount
+ SendPacketGroup( csr )
+ Timeout( csr, TC1(csr.Server), LocalClientTimeout )
+ return
+
+ case STREAMING_NOT_SUPPORTED:
+ Record that server does not support streaming
+ if CMG(csr) then forget this packet group
+ else resend Request as separate packet group.
+ return
+ default:
+ Set csr.Code to code
+ return Response to application
+ return;
+ endselect
+end NotifyVmtpClient
+
+Notes:
+
+ 1. The delivery parameter indicates the segment blocks received
+ by the Server. That is, a 1 bit in the i-th position
+ indicates that the i-th segment block in the segment data of
+ the Request was received. All subsequent NotifyVmtpClient
+ operations for this transaction should be set to acknowledge
+ a superset of the segment blocks in this packet. In
+ particular, the Client need not be prepared to retransmit the
+ segment data once it has been acknowledged by a Notify
+ operation.
+
+
+4.8.1. HandleNoCSR
+
+HandleNoCSR is called when a packet arrives for which there is no CSR
+matching the client field of the packet.
+
+HandleNoCSR( p, psize )
+ if Secure(p) then
+ if SecureVMTP not supported then
+ { Assume a Request }
+ if not Multicast(p) then
+ NotifyClient(NULL,p,SECURITY_NOT_SUPPORTED)
+ return
+ endif
+ HandleRequestNoCSR( p, psize )
+ return
+ endif
+
+
+Cheriton [page 62]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ if p.Checksum not null then
+ if not VerifyChecksum(p, psize) then return;
+ if OppositeByteOrder(p) then ByteSwap( p, psize )
+ if psize not equal sizeof(VmtpHeader) + 4*p.Length then
+ NotifyClient(NULL, p, VMTP_ERROR )
+ return
+
+ if p.FuncCode is Response then
+| if Migrated( p.Client ) then
+| NotifyServer(csr, p ENTITY_MIGRATED )
+| else
+ NotifyServer(csr, p, NONEXISTENT_ENTITY )
+ return
+ endif
+
+ if p.FuncCode is Request then
+ HandleRequestNoCSR( p, psize )
+ return
+end HandleNoCSR
+
+Notes:
+
+ 1. The node need only check to see if the client entity has
+ migrated if in fact it supports migration of entities.
+
+ 2. The procedure HandleRequestNoCSR is specified in Section
+ 5.8.1. In the minimal client version, it need only handle
+ Probe requests and can do so directly without allocating a
+ new CSR.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 63]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+4.9. Timeouts
+
+A client with a message transaction in progress has a single timer
+corresponding to the first unacknowledged request message. (In the
+absence of streaming, this request is also the last request sent.) This
+timeout is handled as follows:
+
+LocalClientTimeout( csr )
+ select on csr.State
+ case AwaitingResponse:
+ if csr.RetransCount > MaxRetrans(csr.Server) then
+ terminate Client's message transactions up to
+ and including the current message transaction.
+ set return code to KERNEL_TIMEOUT
+ return
+ increment csr.RetransCount
+ Resend current packet group with APG set.
+ Timeout( csr, TC2(csr.Server), LocalClientTimeout )
+ return
+ case ReceivingResponse:
+ if DGMset(csr) or csr.RetransCount > Max then
+ if MDMset(csr) then
+ Set MCB.MsgDeliveryMask to blocks received.
+ else
+ Set csr.Code to BAD_REPLY_SEGMENT
+ return to user Client
+ endif
+ increment csr.RetransCount
+ NotifyServer with RETRY
+ Timeout( csr, TC3(csr.Server), LocalClientTimeout )
+ return
+ end select
+end LocalClientTimeout
+
+Notes:
+
+ 1. A Client can only request retransmission of a Response if the
+ Response is not idempotent. If idempotent, it must
+ retransmit the Request. The Server should generally support
+ the MsgDeliveryMask for Requests that it treats as idempotent
+ and that require multi-packet Responses. Otherwise, there is
+ no selective retransmission for idempotent message
+ transactions.
+
+ 2. The current packet group is the last one transmitted. Thus,
+ with streaming, there may be several packet groups
+ outstanding that precede the current packet group.
+
+
+Cheriton [page 64]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ 3. The Request packet group should be retransmitted without the
+ segment data, resulting in a single short packet in the
+ retransmission. The Server must then send a
+ NotifyVmtpClient with a RETRY or RETRY_ALL code to get the
+ segment data transmitted as needed. This strategy minimizes
+ the overhead on the network and the server(s) for
+ retransmissions.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 65]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+5. Server Protocol Operation
+
+This section describes the operation of the server portion of the
+protocol in terms of the procedures for handling VMTP user events,
+packet reception events and timeout events. Each server is assumed to
+implement the client procedures described in the previous chapter.
+(This is not strictly necessary but it simplifies the exposition.)
+
+
+5.1. Remote Client State Record Fields
+
+The CSR for a server is extended with the following fields, in addition
+to the ones listed for the client version.
+
+RemoteClient Identifier for remote client that sent the Request that
+ this CSR is handling.
+
+RemoteClientLink
+ Link to next CSR hashing to same hash index in the
+ ClientMap.
+
+RemoteTransaction
+ Transaction identifier for Request from remote client.
+
+RemoteDelivery The segment blocks received so far as part of a Request
+ or yet to be acknowledged as part of a Response.
+
+VerifyInterval Time interval since there was confirmation that the
+ remote Client was still valid.
+
+RemotePrincipal Account identification, possibly including key and key
+ timeout for secure communication.
+
+
+5.2. Remote Client Protocol States
+
+A CSR in the server end is in one of the following states.
+
+AwaitingRequest Waiting for a Request packet group. It may be marked as
+ waiting on a specific Client, or on any Client.
+
+ReceivingRequest
+ Waiting to receive additional Request packets in a
+ multi-packet group Request.
+
+Responded The Response has been sent and the CSR is timing out,
+ providing duplicate suppression and retransmission (if
+
+
+Cheriton [page 66]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ the Response was not idempotent).
+
+ResponseDiscarded
+ Response has been acknowledged or has timed out so
+ cannot be retransmitted. However, duplicates are still
+ filtered and CSR can be reused for new message
+ transaction.
+
+Processing Executing on behalf of the Client.
+
+Forwarded The message transaction has been forwarded to another
+ Server that is to respond directly to the Client.
+
+
+5.3. State Transition Diagrams
+
+The CSR state transitions in the server are illustrated in Figure 5-1.
+The CSR generally starts in the AwaitingRequest state. On receipt of a
+Request, the Server either has an up-to-date CSR for the Client or else
+it sends a Probe request (as a separate VMTP message transaction) to the
+VMTP management module associated with the Client. In the latter case,
+the processing of the Request is delayed until a Response to the Probe
+request is received. At that time, the CSR information is brought up to
+date and the Request is processed. If the Request is a single-packet
+request, the CSR is then set in the Processing state to handle the
+request. Otherwise (a multi-packet Request), the CSR is put into the
+ReceivingResponse state, waiting to receive subsequent Request packets
+that constitute the Request message. It exits the ReceivingRequest
+state on timeout or on receiving the last Request packet. In the former
+case, the request is delivered with an indication of the portion
+received, using the MsgDelivery field if MDM is set. After request
+processing is complete, either the Response is sent and the CSR enters
+the Responded state or the message transaction is forwarded and the CSR
+enters the Forwarded state.
+
+In the Responded state, if the Response is not marked as idempotent, the
+Response is retransmitted on receipt of a retransmission of the
+corresponding Request, on receipt of a NotifyVmtpServer operation
+requesting retransmission or on timeout at which time APG is set,
+requesting an acknowledgment from the Client. The Response is
+retransmitted some maximum number of times at which time the Response is
+discarded and the CSR is marked accordingly. If a Request or a
+NotifyVmtpServer operation is received expecting retransmission of the
+Response after the CSR has entered the ResponseDiscarded state, a
+NotifyVmtpClient operation is sent back (or invoked in the Client
+management module) indicating that the response was discarded unless the
+Request was multicast, in which case no action is taken. After a
+
+
+Cheriton [page 67]
+
+
+RFC 1045 VMTP February 1988
+
+
+ (Retransmit Forwarded Request and NotifyVmtpClient)
+ Request/
+ Ack/
+ +Timeout+
+ V |
+ +-|-------^-+
+ | |
+ +-Time-| Forwarded |<-------------+
+ | out +-----------+ |
+ | |
+ | (Retransmit Response) |
+ | Request |
+ V Ack |
+ | +-Timeout-+ |
+ | V | |
+ +---------+ Ack/ +|---------^+ |
+ +-Time-|Response |<-Timeout--| Responded | |
+ | out |Discarded| +----^------+ |
+ | +---------+ | |
+ | +------------+ | |
+ | | |->-Send Response-+ |
+ | | |->-forward Request--------+
+ +->| Processing |<----------------------+
+ | | |<----------------+ |
+ | | |<---| | |
+ | +-|--------^-+ | Last |
+ | Receive | | Request |
+ | | Timeout Single Packet |
+ | | | Packet | Timeout
+ | | | Request ^ ^
+ | | | ^ +|-----|--+
+ | +-V--------|-+ | |Receiving|<-+Time
+ +->| Awaiting |->--+->Request->| Request |--+ out
+ | Request | | (multi- +---------+
+ +------|-----+ ^ packet)
+ Request |
+ | Response
+ Send Probe to
+ | Probe
+ +---V----+ |
+ |Awaiting| ^
+ |Response|-->--+
+ |to Probe|
+ +--------+
+
+ Figure 5-1: Remote Client State Transitions
+
+timeout corresponding to the time required to filter out duplicates, the
+
+Cheriton [page 68]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+CSR returns either to the AwaitingRequest state or to the Processing
+state. Note that "Ack" refers to acknowledgment by a Notify operation.
+
+A Request that is forwarded leaves the CSR in the Forwarded state. In
+the Forwarded state, the forwarded Request is retransmitted
+periodically, expecting NotifyRemoteClient operations back from the
+Server to which the Request was forwarded, analogous to the Client
+behavior in the AwaitingResponse state. In this state, a
+NotifyRemoteClient from this Server acknowledges the Request or asks
+that it be retransmitted or reports an error. A retransmission of the
+Request from the Client causes a NotifyVmtpClient to be returned to the
+Client if APG is set. The CSR leaves the Forwarded state after timing
+out in the absence of NotifyRemoteClient operations from the forward
+Server or on receipt of a NotifyRemoteClient operation indicating the
+forward Server has sent a Response and received an acknowledgement. It
+then enters the ResponseDiscarded state.
+
+Receipt of a new Request from the same Client aborts the current
+transaction, independent of its state, and initiates a new transaction
+unless the new Request is part of a run of message transactions. If it
+is part of a run of message transactions, the handling follows the state
+diagram except the new Request is not Processed until there has been a
+response sent to the previous transaction.
+
+
+5.4. User Interface
+
+The RPC or user interface to VMTP is implementation-dependent and may
+use systems calls, functions or some other mechanism. The list of
+requests that follow is intended to suggest the basic functionality that
+should be available.
+
+AcceptMessage( reqmcb, segptr, segsize, client, transid, timeout )
+ Accept a new Request message in the specified reqmcb
+ area, placing the segment data, if any, in the area
+ described by segptr and segsize. This returns the
+ Server in the entityId field of the reqmcb and actual
+ segment size in the segsize parameters. It also returns
+ the Client and Transaction for this message transaction
+ in the corresponding parameters. This procedure
+ supports message semantics for request processing. When
+ a server process executes this call, it blocks until a
+ Request message has been queued for the server.
+ AcceptMessage returns after the specified timeout period
+ if a message has not been received by that time.
+
+RespondMessage( responsemcb, client, transid, segptr )
+
+
+Cheriton [page 69]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Respond to the client with the specified response
+ message and segment, again with message semantics.
+
+RespondCall( responsemcb, segptr )
+ Respond to the client with the specified response
+ message and segment, with remote procedure call
+ semantics. This procedure does not return. The
+ lightweight process that executes this procedure is
+ matched to a stack, program counter, segment area and
+ priority from the information provided in a
+ ModifyService call, as specified in Appendix III.
+
+ForwardMessage( requestmcb, transid, segptr, segsize, forwardserver )
+ Forward the client to the specified forwardserver with
+ the request specified in mcb.
+
+ForwardCall( requestmcb, segptr, segsize, forwardserver )
+ Forward the client transaction to the specified
+ forwardserver with the request specified by requestmcb.
+ This procedure does not return.
+
+GetRemoteClientId()
+ Return the entityId for the remote client on whose
+ behave the process is executing. This is only
+ applicable in the procedure call model of request
+ handling.
+
+GetForwarder( client )
+ Return the entity that forwarded this Request, if any.
+
+GetProcess( client )
+ Return an identifier for the process associated with
+ this client entity-id.
+
+GetPrincipal( client )
+ Return the principal associated with this client
+ entity-id.
+
+
+5.5. Event Processing
+
+The following events may occur in VMTP servers.
+
+ - User Requests
+
+ * Receive
+
+
+
+Cheriton [page 70]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ * Respond
+
+ * Forward
+
+ * GetForwarder
+
+ * GetProcess
+
+ * GetPrincipal
+
+ - Packet Arrival
+
+ * Request Packet
+
+ - Management Operations
+
+ * NotifyVmtpServer
+
+ - Timeouts
+
+ * Client State Record Timeout
+
+The handling of these events is described in detail in the following
+subsections. The conventions of the previous chapter are followed,
+including the use of the various subroutines in the description.
+
+
+5.6. Server User-invoked Events
+
+A user event occurs when a VMTP server invokes one of the VMTP interface
+procedures.
+
+
+5.6.1. Receive
+
+AcceptMessage(reqmcb, segptr, segsize, client, transid, timeout)
+ Locate server's request queue.
+ if request is queued then
+ Remember CSR associated with this Request.
+ return Request in reqmcb, segptr and segsize
+ and client and transaction id.
+ Wait on server's request queue for next request
+ up time timeout seconds.
+end ReceiveCall
+
+Notes:
+
+
+
+Cheriton [page 71]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ 1. If a multi-packet Request is partially received at the time
+ of the AcceptMessage, the process waits until it completes.
+
+ 2. The behavior of a process accepting a Request as a
+ lightweight thread is similar except that the process
+ executes using the Request data logically as part of the
+ requesting Client process.
+
+
+5.6.2. Respond
+
+RespondCall is described as one case of the Respond transmission
+procedure; RespondMessage is similar.
+
+RespondCall( responsemcb, responsesegptr )
+ Locate csr for this client.
+ Check segment data accessible, if any
+ if local client then
+ Handle locally
+ return
+ endif
+ if responsemcb.Code is RESPONSE_DISCARDED then
+ Mark as RESPONSE_DISCARDED
+ return
+ SendPacketGroup( csr )
+ set csr.State to Responded.
+ if DGM reply then { Idempotent }
+ release segment data
+ Timeout( csr, TS4(csr.Client), FreeCsr );
+ else { Await acknowledgement or new Request else ask for ack. }
+ Timeout( csr, TS5(csr.Client), RemoteClientTimeout )
+end RespondCall
+
+Notes:
+
+ 1. RespondMessage is similar except the Server process must be
+ synchronized with the release of the segment data (if any).
+
+ 2. The non-idempotent Response with segment data is sent first
+ without a request for an acknowledgement. The Response is
+ retransmitted after time TS5(client) if no acknowledgment or
+ new Request is received from the client in the meantime. At
+ this point, the APG bit is sent.
+
+ 3. The MCB of the Response is buffered in the client CSR, which
+ remains for TS4 seconds, sufficient to filter old duplicates.
+ The segment data (if any) must be retained intact until: (1)
+
+
+Cheriton [page 72]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ after transmission if idempotent or (2) after acknowledged or
+ timeout has occurred if not idempotent. Techniques such as
+ copy-on-write might be used to keep a copy of the Response
+ segment data without incurring the cost of a copy.
+
+
+5.6.3. Forward
+
+Forwarding is logically initiating a new message transaction between the
+Server (now acting as a Client) and the server to which the Request is
+forwarded. When the second server returns a Response, the same Response
+is immediately returned to the Client. The forwarding support in VMTP
+preserves these semantics while providing some performance optimizations
+in some cases.
+
+ForwardCall( req, segptr, segsize, forwardserver )
+ Locate csr for this client.
+ Check segment data accessible, if any
+
+ if local client or Request was multicast or secure
+ or csr.ForwardCount == 15 then
+ Handle as a new Send operation
+ return
+ if forwardserver is local then
+ Handle locally
+ return
+ Set csr.funccode to Request
+ Increment csr.ForwardCount
+ Set csr.State to Responded
+ SendPacketGroup( csr ) { To ForwardServer }
+ Timeout( csr, TS4(csr.Client), FreeAlien )
+end ForwardCall
+
+Notes:
+
+ 1. A Forward is logically a new call or message transaction. It
+ must be really implemented as a new message transaction if
+ the original Request was multicast or secure (with the
+ optional further refinement that it can be used with a secure
+ message transaction when the Server and ForwardServer are the
+ same principal and the Request was not multicast).
+
+ 2. A Forward operation is never handled as an idempotent
+ operation because it requires knowledge that the
+ ForwardServer will treat the forwarded operation as
+ idempotent as well. Thus, a Forward operation that includes
+ a segment should set APG on the first transmission of the
+
+
+Cheriton [page 73]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ forwarded Request to get an acknowledgement for this data.
+ Once the acknowledgement is received, the forwarding Server
+ can discard the segment data, leaving only the basic CSR to
+ handle retransmissions from the Client.
+
+
+5.6.4. Other Functions
+
+GetRemoteClient is a simple local query of the CSR. GetProcess and
+GetPrincipal also extract this information from the CSR. A server
+module may defer the Probe callback to the Client to get that
+information until it is requested by the Server (assuming it is not
+using secure communication and duplicate suppression is adequate without
+callback.) GetForwarder is implemented as a callback to the Client,
+using a GetRequestForwarder VMTP management operation. Additional
+management procedures for VMTP are described in Appendix III.
+
+
+5.7. Request Packet Arrival
+
+The basic packet reception follows that described for the Client
+routines. A Request packet is handled by the procedure HandleRequest.
+
+HandleRequest( csr, p, psize )
+
+ if LocalClient(csr) then
+ { Forwarded Request on local Client }
+ if csr.LocalTransaction != p.Transaction then return
+ if csr.State != AwaitingResponse then return
+ if p.ForwardCount < csr.ForwardCount then
+ Discard Request and return.
+ Find a CSR for Client as a remote Client.
+ if not found then
+ if packet group complete then
+ handle as a local message transaction
+ return
+ Allocate and init CSR
+ goto newTransaction
+ { Otherwise part of current transaction }
+ { Handle directly below. }n
+ if csr.RemoteTransaction = p.Transaction then
+ { Matches current transaction }
+ if OldForward(p.ForwardCount,csr.ForwardCount) then
+ return
+ if p.ForwardCount > csr.ForwardCount then
+ { New forwarded transaction }
+ goto newTransaction
+
+
+Cheriton [page 74]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ { Otherwise part of current transaction }
+ if csr.State = ReceivingRequest then
+ if new segment data then retain in CSR segment area.
+ if Request not complete then
+ Timeout( csr, TS1(p.Client), RemoteClientTimeout )
+ return;
+ endif
+ goto endPacketGroup
+ endif
+ if csr.State is Responded then
+ { Duplicate }
+ if csr.Code is RESPONSE_DISCARDED
+ and Multicast(p) then
+ return
+ endif
+ if not DGM(csr) then { Not idempotent }
+ if SegmentData(csr) then set APG
+ { Resend Response or Request, if Forwarded }
+ SendPacketGroup( csr )
+ timeout=if SegmentData(csr) then TS5(csr.Client)
+ else TS4(csr.Client)
+ Timeout( csr, timeout, RemoteClientTimeout )
+ return
+ { Else idempotent - fall thru to newTransaction }
+ else { Presume it is a retransmission }
+ NotifyClient( csr, p, OK )
+ return
+ else if OldTransaction(csr.RemoteTransact,p.Transaction) then
+ return
+ { Otherwise, a new message transaction. }
+newTransaction:
+ Abort handling of previous transactions for this Client.
+
+ if (NSRset(p) or NERset(p)) and NoStreaming then
+ NotifyClient( csr, p, STREAMING_NOT_SUPPORTED )
+ return
+| if NSRset(p) then { Streaming }
+| { Check that consecutive with previous packet group }
+| Find last packet group CSR from this client.
+| if p.Transaction not lastcsr.RemoteTransaction+1 mod 2**32
+| and not STIset(lastcsr) or
+| p.Transaction not lastcsr.RemoteTransaction+256 mod **32
+| then
+| { Out of order packet group }
+| NotifyClient(csr, p, BAD_TRANSACTION_ID )
+| return
+| endif
+
+
+Cheriton [page 75]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+| if lastcsr not completed then
+| NotifyClient( lastcsr, p, RETRY )
+| endif
+| if lastcsr available then use it for this packet group
+| else allocate and initialize new CSR
+| if CMG(lastcsr) then
+| Add segment data to lastcsr Request
+| Keep csr as record of this packet group.
+| Clear lastcsr.VerifyInterval
+| endif
+| else { First packet group }
+ if MultipleRemoteClients(csr) then ScavengeCsrs(p.Client)
+ Set csr.RemoteTransaction, csr.Priority
+ Copy message and segment data to csr's segment area
+ and set csr.PacketDelivery to that delivered.
+ Clear csr.PacketDelivery
+ Clear csr.VerifyInterval
+ SaveNetworkAddress( csr, p )
+ endif
+ if packetgroup not complete then
+ Timeout( csr, TS3(p.Client), RemoteClientTimeout )
+ return;
+ endif
+endPacketGroup:
+ { We have received complete packet group. }
+ if APG(p) then NotifyClient( csr, p, OK )
+ endif
+| if NERset(p) and CMG(p) then
+| Queue waiting for continuation packet group.
+| Timeout( csr, TS3(csr.Client), RemoteClientTimeout )
+| return
+| endif
+ { Deliver request message. }
+ if GroupId(csr.Server) then
+ For each server identified by csr.Server
+ Replicate csr and associated data segment.
+ if CMDset(csr) and Server busy then
+ Discard csr and data
+ else
+ Deliver or invoke csr for each Server.
+ if not DGMset(csr) then queue for Response
+ else Timeout( csr, TS4(csr.Client), FreeCsr )
+ endfor
+ else
+ if CMDset(csr) and Server busy then
+ Discard csr and data
+ else
+
+
+Cheriton [page 76]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Deliver or invoke csr for this server.
+ if not DGMset(csr) then queue for Response
+ else Timeout( csr, TS4(csr.Client), FreeCsr )
+ endif
+end HandleRequest
+
+Notes:
+
+ 1. A Request received that specifies a Client that is a local
+ entity should be a Request forwarded by a remote server to a
+ local Server.
+
+ 2. An alternative structure for handling a Request sent to a
+ group when there are multiple local group members is to
+ create a remote CSR for each group member on reception of the
+ first packet and deliver a copy of each packet to each such
+ remote CSR as each packet arrives.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 77]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+5.8. Management Operations
+
+VMTP uses management operations (invoked as remote procedure calls) to
+effectively acknowledge packet groups and request retransmissions. The
+following routine is invoked by the Server's management module on
+request from the Client.
+
+NotifyVmtpServer(server,clientId,transact,delivery,code)
+ Find csr with same RemoteTransaction and RemoteClient
+ as clientId and transact.
+ if not found or csr.State not Responded then return
+ if DGMset(csr) then
+ if transmission of Response in progress then
+ Abort transmission
+ if code is migrated then
+ restart transmission with new host addr.
+ if Retry then Report protocol error
+ return
+ endif
+ select on code
+ case RETRY:
+ if csr.RetransCount > MaxRetrans(clientId) then
+ if response data segment then
+ Discard data and mark as RESPONSE_DISCARDED
+| if NERset(csr) and subsequent csr then
+| Deallocate csr and use later csr for
+| future duplicate suppression
+| endif
+ return
+ endif
+ increment csr.RetransCount
+ Set csr.TransmissionMask to missing segment blocks,
+ as specified by delivery
+ SendPacketGroup( csr )
+ Timeout( csr, TS3(csr.Client), RemoteClientTimeout )
+ case BUSY:
+ if csr.TimeLimit exceeded then
+ if response data segment then
+ Discard data and mark as RESPONSE_DISCARDED
+| if NERset(csr) and subsequent csr then
+| Deallocate csr and use later csr for
+| future duplicate suppression
+| endif
+ endif
+ endif
+ Set csr.TransmissionMask for full retransmission
+ Clear csr.RetransCount
+
+
+Cheriton [page 78]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Timeout( csr, TS3(csr.Server), RemoteClientTimeout )
+ return
+
+ case ENTITY_MIGRATED:
+ Get new host address for entity
+ Set csr.TransmissionMask for full retransmission
+ Clear csr.RetransCount
+ SendPacketGroup( csr )
+ Timeout( csr, TS3(csr.Server), RemoteClientTimeout )
+ return
+
+ case default:
+ Abort transmission of Response if in progress.
+ if response data segment then
+ Discard data and mark as RESPONSE_DISCARDED
+ if NERset(csr) and subsequent csr then
+ Deallocate csr and use later csr for
+ future duplicate suppression
+ endif
+ return
+ endselect
+end NotifyVmtpServer
+
+Notes:
+
+ 1. A NotifyVmtpServer operation requesting retransmission of
+ the Response is acceptable only if the Response was not
+ idempotent. When the Response is idempotent, the Client must
+ be prepared to retransmit the Request to effectively request
+ retransmission of the Response.
+
+ 2. A NotifyVmtpServer operation may be received while the
+ Response is being transmitted. If an error return, as an
+ efficiency, the transmission should be aborted, as suggested
+ when the Response is a datagram.
+
+ 3. A NotifyVmtpServer operation indicating OK or an error
+ allows the Server to discard segment data and not provide for
+ subsequent retransmission of the Response.
+
+
+5.8.1. HandleRequestNoCSR
+
+When a Request is received from a Client for which the node has no CSR,
+the node allocates and initializes a CSR for this Client and does a
+callback to the Client's VMTP management module to get the Principal,
+Process and other information associated with this Client. It also
+
+
+Cheriton [page 79]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+checks that the TransactionId is correct in order to filter out
+duplicates.
+
+HandleRequestNoCSR( p, psize )
+| if Secure(p) then
+| Allocate and init CSR
+| SaveSourceHostAddr( csr, p )
+| ProbeRemoteClient( csr, p, AUTH_PROBE )
+| if no response or error then
+| delete CSR
+| return
+| Decrypt( csr.Key, p, psize )
+| if p.Checksum not null then
+| if not VerifyChecksum(p, psize) then return;
+| if OppositeByteOrder(p) then ByteSwap( p, psize )
+| if psize not equal sizeof(VmtpHeader) + 4*p.Length then
+| NotifyClient(NULL, p, VMTP_ERROR )
+| return
+| HandleRequest( csr, p, psize )
+| return
+ if Server does not exist then
+ NotifyClient( csr, p, NONEXISTENT_ENTITY )
+ return
+ endif
+ if security required by server then
+ NotifyClient(csr, p, SECURITY_REQUIRED )
+ return
+ endif
+ Allocate and init CSR
+ SaveSourceHostAddr( csr, p );
+ if server requires Authentication then
+ ProbeRemoteClient( csr, p, AUTH_PROBE )
+ if no response or error then
+ delete CSR
+ return
+ endif
+ { Setup immediately as a new message transaction }
+ set csr.Server to p.Server
+ set csr.RemoteTransaction to p.Transaction-1
+
+ HandleRequest( csr, p, psize )
+ endif
+
+Notes:
+
+ 1. A Probe request is always handled as a Request not requiring
+ authentication so it never generates a callback Probe to the
+
+
+Cheriton [page 80]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Client.
+
+ 2. If the Server host retains remote client CSR's for longer
+ than the maximum packet lifetime and the Request
+ retransmission time, and the host has been running for at
+ least that long, then it is not necessary to do a Probe
+ callback unless the Request is secure. A Probe callback can
+ take place when the Server asks for the Process or
+ PrincipalId associated with the Client.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 81]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+5.9. Timeouts
+
+The server must implement a timeout for remote client CSRs. There is a
+timeout for each CSR in the server.
+
+RemoteClientTimeout( csr )
+ select on csr.State
+ case Responded:
+ if RESPONSE_DISCARDED then
+ mark as timed out
+ Make a candidate for reuse.
+ return
+ if csr.RetransCount > MaxRetrans(Client) then
+ discard Response
+ mark CSR as RESPONSE_DISCARDED
+ Timeout(csr, TS4(Client), RemoteClientTimeout)
+ return
+ increment csr.RetransCount
+ { Retransmit Response or forwarded Request }
+ Set APG to get acknowledgement.
+ SendPacketGroup( csr )
+ Timeout( csr, TS3(Client), RemoteClientTimeout )
+ return
+ case ReceivingRequest:
+ if csr.RetransCount > MaxRetrans(csr.Client)
+ or DGMset(csr) or NRTset(csr) then
+ Modify csr.segmentSize and csr.MsgDelivery
+ to indicate packets received.
+ if MDMset(csr) then
+ Invoke processing on Request
+ return
+ else
+ discard Request and reuse CSR
+ (Note: Need not remember Request discarded.)
+ return
+ increment csr.RetransCount
+ NotifyClient( csr, p, RETRY )
+ Timeout( csr, TS3(Client), RemoteClientTimeout )
+ return
+ default:
+ Report error - invalid state for RemoteClientTimeout
+ endselect
+end RemoteClientTimeout
+
+Notes:
+
+ 1. When a CSR in the Responded state times out after discarding
+
+
+Cheriton [page 82]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ the Response, it can be made available for reuse, either by
+ the same Client or a different one. The CSR should be kept
+ available for reuse by the Client for as long as possible to
+ avoid unnecessary callback Probes.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 83]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+6. Concluding Remarks
+
+This document represents a description of the current state of the VMTP
+design. We are currently engaged in several experimental
+implementations to explore and refine all aspects of the protocol.
+Preliminary implementations are running in the UNIX 4.3BSD kernel and in
+the V kernel.
+
+Several issues are still being discussed and explored with this
+protocol. First, the size of the checksum field and the algorithm to
+use for its calculation are undergoing some discussion. The author
+believes that the conventional 16-bit checksum used with TCP and IP is
+too weak for future high-speed networks, arguing for at least a 32-bit
+checksum. Unfortunately, there appears to be limited theory covering
+checksum algorithms that are suitable for calculation in software.
+
+Implementation of the streaming facilities of VMTP is still in progress.
+This facility is expected to be important for wide-area, long delay
+communication.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 84]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+I. Standard VMTP Response Codes
+
+The following are the numeric values of the response codes used in VMTP.
+
+0 OK
+
+1 RETRY
+
+2 RETRY_ALL
+
+3 BUSY
+
+4 NONEXISTENT_ENTITY
+
+5 ENTITY_MIGRATED
+
+6 NO_PERMISSION
+
+7 NOT_AWAITING_MSG
+
+8 VMTP_ERROR
+
+9 MSGTRANS_OVERFLOW
+
+10 BAD_TRANSACTION_ID
+
+11 STREAMING_NOT_SUPPORTED
+
+12 NO_RUN_RECORD
+
+13 RETRANS_TIMEOUT
+
+14 USER_TIMEOUT
+
+15 RESPONSE_DISCARDED
+
+16 SECURITY_NOT_SUPPORTED
+
+17 BAD_REPLY_SEGMENT
+
+18 SECURITY_REQUIRED
+
+19 STREAMED_RESPONSE
+
+20 TOO_MANY_RETRIES
+
+21 NO_PRINCIPAL
+
+
+Cheriton [page 85]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+22 NO_KEY
+
+23 ENCRYPTION_NOT_SUPPORTED
+
+24 NO_AUTHENTICATOR
+
+25-63 Reserved for future VMTP assignment.
+
+Other values of the codes are available for use by higher level
+protocols. Separate protocol documents will specify further standard
+values.
+
+Applications are free to use values starting at 0x00800000 (hex) for
+application-specific return values.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 86]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+II. VMTP RPC Presentation Protocol
+
+For complete generality, the mapping of the procedures and the
+parameters onto VMTP messages should be defined by a RPC presentation
+protocol. In the absence of an accepted standard protocol, we define an
+RPC presentation protocol for VMTP as follows.
+
+Each procedure is assigned an identifying Request Code. The Request
+code serves effectively the same as a tag field of variant record,
+identifying the format of the Request and associated Response as a
+variant of the possible message formats.
+
+The format of the Request for a procedure is its Request Code followed
+by its parameters sequentially in the message control block until it is
+full.
+
+The remaining parameters are sent as part of the message segment data
+formatted according to the XDR protocol (RFC ??). In this case, the
+size of the segment is specified in the SegmentSize field.
+
+The Response for a procedure consists of a ResponseCode field followed
+by the return parameters sequentially in the message control block,
+except if there is a parameter returned that must be transmitted as
+segment data, its size is specified in the SegmentSize field and the
+parameter is stored in the SegmentData field.
+
+Attributes associated with procedure definitions should indicate the
+Flags to be used in the RequestCode. Request Codes are assigned as
+described below.
+
+
+II.1. Request Code Management
+
+Request codes are divided into Public Interface Codes and
+application-specific, according to whether the PIC value is set. An
+interface is a set of request codes representing one service or module
+function. A public interface is one that is to be used in multiple
+independently developed modules. In VMTP, public interface codes are
+allocated in units of 256 structured as
+
+ +-------------+----------------+-------------------+
+ | ControlFlags| Interface | Version/Procedure |
+ +-------------+----------------+-------------------+
+ 8 bits 16 bits 8 bits
+
+An interface is free to allocate the 8 bits for version and procedure as
+desired. For example, all 8 bits can be used for procedures. A module
+requiring more than 256 Version/Procedure values can be allocated
+
+Cheriton [page 87]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+multiple Interface values. They need not be consecutive Interface
+values.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 88]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+III. VMTP Management Procedures
+
+Standard procedures are defined for VMTP management, including creation,
+deletion and query of entities and entity groups, probing to get
+information about entities, and updating message transaction information
+at the client or the server.
+
+The procedures are implemented by the VMTP manager that constitutes a
+portion of every complete VMTP module. Each procedure is invoked by
+sending a Request to the VMTP manager that handles the entity specified
+in the operation or the local manager. The Request sent using the
+normal Send operation with the Server specified as the well-known entity
+group VMTP_MANGER_GROUP, using the CoResident Entity mechanism to direct
+the request to the specific manager that should handle the Request.
+(The ProbeEntity operation is multicast to the VMTP_MANAGER_GROUP if the
+host address for the entity is not known locally and the host address is
+determined as the host address of the responder. For all other
+operations, a ProbeEntity operation is used to determine the host
+address if it is not known.) Specifying co-resident entity 0 is
+interpreted as the co-resident with the invoking process. The
+co-resident entity identifier may also specify a group in which case,
+the Request is sent to all managers with members in this group.
+
+The standard procedures with their RequestCode and parameters are listed
+below with their semantics. (The RequestCode range 0xVV000100 to
+0xVV0001FF is reserved for use by the VMTP management routines, where VV
+is any choice of control flags with the PIC bit set. The flags are set
+below as required for each procedure.)
+
+0x05000101 - ProbeEntity(CREntity, entityId, authDomain) -> (code,
+ <staterec>)
+ Request and return information on the specified entity
+ in the specified authDomain, sending the Request to the
+ VMTP management module coresident with CREntity. An
+ error return is given if the requested information
+ cannot be provided in the specified authDomain. The
+ <staterec> returned is structured as the following
+ fields.
+
+ Transaction identifier
+ The current or next transaction
+ identifier being used by the probed
+ entity.
+
+ ProcessId: 64 bits
+ Identifier for client process. The
+ meaning of this is specified as part of
+
+
+Cheriton [page 89]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ the Domain definition.
+
+ PrincipalId The identifier for the principal or
+ account associated with the process
+ specified by ProcessId. The meaning of
+ this field is specified as part of the
+ Domain definition.
+
+ EffectivePrincipalId
+ The identifier for the principal or
+ account associated with the Client port,
+ which may be different from the
+ PrincipalId especially if this is an
+ nested call. The meaning of this field
+ is specified as part of the Domain
+ definition.
+
+ The code field indicates whether this is an error
+ response or not. The codes and their interpretation
+ are:
+
+ OK
+ No error. Probe was completed OK.
+
+ NONEXISTENT_ENTITY
+ Specified entity does not exist.
+
+ ENTITY_MIGRATED
+ The entity has migrated and is no longer at the host to
+ which the request was sent.
+
+ NO_PERMISSION
+ Entity has refused to provide ProbeResponse.
+
+ VMTP_ERROR
+ The Request packet group was in error relative to the
+ VMTP protocol specification.
+
+ "default"
+ Some type of error - discard ProbeResponse.
+
+0x0D000102 - AuthProbeEntity(CREntity,entityId,authDomain,randomId) ->
+ (code,ProbeAuthenticator,EncryptType,EntityAuthenticator)
+
+ Request authentication of the entity specified by
+ entityId from the VMTP manager coresident with CREntity
+ in authDomain authentication domain, returning the
+
+
+Cheriton [page 90]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ information contained in the return parameters. The
+ fields are set the same as that specified for the basic
+ ProbeResponse except as noted below.
+
+ ProbeAuthenticator
+ 20 bytes consisting of the EntityId, the
+ randomId and the probed Entity's current
+ Transaction value plus a 32-bit checksum
+ for these two fields (checksummed using
+ the standard packet Checksum algorithm),
+ all encrypted with the Key supplied in
+ the Authenticator.
+
+ EncryptType An identifier that identifies the
+ variant of encryption method being used
+ by the probed Entity for packets it
+ transmits and packets it is able to
+ receive. (See Appendix V.) The
+ high-order 8 bits of the EncryptType
+ contain the XOR of the 8 octets of the
+ PrincipalId associated with private key
+ used to encrypt the EntityAuthenticator.
+ This value is used by the requestor or
+ Client as an aid in locating the key to
+ decrypt the authenticator.
+
+ EntityAuthenticator
+ (returned as segment data) The
+ ProcessId, PrincipalId,
+ EffectivePrincipal associated with the
+ ProbedEntity plus the private
+ encryption/decryption key and its
+ lifetime limit to be used for
+ communication with the Entity. The
+ authenticator is encrypted with a
+ private key associated with the Client
+ entity such that it can be neither read
+ nor forged by a party not trusted by the
+ Client Entity. The format of the
+ Authenticator in the message segment is
+ shown in detail in Figure III-1.
+
+ Key: 64 bits Encryption key to be used for encrypting
+ and decrypting packets sent to and
+ received from the probed Entity. This
+ is the "working" key for packet
+ transmissions. VMTP only uses private
+
+
+Cheriton [page 91]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ +-----------------------------------------------+
+ | ProcessId (8 octets) |
+ +-----------------------------------------------+
+ | PrincipalId (8 octets) |
+ +-----------------------------------------------+
+ | EffectivePrincipalId (8 octets) |
+ +-----------------------------------------------+
+ | Key (8 octets) |
+ +-----------------------------------------------+
+ | KeyTimeLimit |
+ +-----------------------------------------------+
+ | AuthDomain |
+ +-----------------------------------------------+
+ | AuthChecksum |
+ +-----------------------------------------------+
+
+ Figure III-1: Authenticator Format
+
+ key encryption for data transmission.
+
+ KeyTimeLimit: 32 bits
+ The time in seconds since Dec. 31st,
+ 1969 GMT at which one should cease to
+ use the Key.
+
+ AuthDomain: 32 bits
+ The authentication domain in which to
+ interpret the principal identifiers.
+ This may be different from the
+ authDomain specified in the call if the
+ Server cannot provide the authentication
+ information in the request domain.
+
+ AuthChecksum: 32 bits
+ Contains the checksum (using the same
+ Checksum algorithm as for packet) of
+ KeyTimeLimit, Key, PrincipalId and
+ EffectivePrincipalId.
+
+ Notes:
+
+ 1. A authentication Probe Request and Response
+ are sent unencrypted in general because it is
+ used prior to there being a secure channel.
+ Therefore, specific fields or groups of
+ fields checksummed and encrypted to prevent
+ unauthorized modification or forgery. In
+
+
+Cheriton [page 92]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ particular, the ProbeAuthenticator is
+ checksummed and encrypted with the Key.
+
+ 2. The ProbeAuthenticator authenticates the
+ Response as responding to the Request when
+ its EntityId, randomId and Transaction values
+ match those in the Probe request. The
+ ProbeAutenticator is bound to the
+ EntityAutenticator by being encrypted by the
+ private Key contained in that authenticator.
+
+ 3. The authenticator is encrypted such that it
+ can be decrypted by a private key, known to
+ the Client. This authenticator is presumably
+ obtained from a key distribution center that
+ the Client trusts. The AuthChecksum prevents
+ undetected modifications to the
+ authenticator.
+
+0x05000103 - ProbeEntityBlock( entityId ) -> ( code, entityId )
+ Check whether the block of 256 entity identifiers
+ associated with this entityId are in use. The entityId
+ returned should match that being queried or else the
+ return value should be ignored and the operation redone.
+
+0x05000104 - QueryVMTPNode( entityId ) -> (code, MTU, flags, authdomain,
+ domains, authdomains, domainlist)
+ Query the VMTP management module for entityId to get
+ various module- or node-wide parameters, including: (1)
+ MTU - Maximum transmission unit or packet size handled
+ by this node. (2) flags- zero or more of the following
+ bit fields:
+
+ 1 Handles streamed Requests.
+
+ 2 Can issue streamed message transactions
+ for clients.
+
+ 4 Handles secure Requests.
+
+ 8 Can issue secure message transactions.
+
+ The authdomain indicates the primary authentication
+ domain supported. The domains and authdomains
+ parameters indicate the number of entity domains and
+ authentication domains supported by this node, which are
+ listed in the data segment parameter domainlist if
+
+
+Cheriton [page 93]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ either parameter is non-zero. (All the entity domains
+ precede the authentication domains in the data segment.)
+
+0x05000105 - GetRequestForwarder( CREntity, entityId1 ) -> (code,
+ entityId2, principal, authDomain)
+ Return the forwarding server's entity identifer and
+ principal for the forwarder of entityId1. CREntity
+ should be zero to get the local VMTP management module.
+
+0x05000106 - CreateEntity( entityId1 ) -> ( code, entityId2 )
+ Create a new entity and return its entity identifier in
+ entityId2. The entity is created local to the entity
+ specified in entityId1 and local to the requestor if
+ entityId1 is 0.
+
+0x05000107 - DeleteEntity( entityId ) -> ( code )
+ Delete the entity specified by entityId, which may be a
+ group. If a group, the deletion is only on a best
+ efforts basis. The client must take additional measures
+ to ensure complete deletion if required.
+
+0x0D000108 -QueryEntity( entityId ) -> ( code, descriptor )
+ Return a descriptor of entityId in arg of a maximum of
+ segmentSize bytes.
+
+0x05000109 - SignalEntity( entityId, arg )->( code )
+ Send the signal specified by arg to the entity specified
+ by entityId. (arg is 32 bits.)
+
+0x0500010A - CreateGroup(CREntity,entityGroupId,entityId,perms)->(code)
+ Request that the VMTP manager local to CREntity create
+ an new entity group, using the specified entityGroupId
+ with entityId as the first member and permissions
+ "perms", a 32-bit field described later. The invoker is
+ registered as a manager of the new group, giving it the
+ permissions to add or remove members. (Normally
+ CREntity is 0, indicating the VMTP manager local to the
+ requestor.)
+
+0x0500010B - AddToGroup(CREntity, entityGroupId, entityId,
+ perms)->(code)
+ Request that the VMTP manager local to CREntity add the
+ specified entityId to the entityGroupId with the
+ specified permissions. If entityGroupId specifies a
+ restricted group, the invoker must have permission to
+ add members to the group, either because the invoker is
+
+
+Cheriton [page 94]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ a manager of the group or because it was added to the
+ group with the required permissions. If CREntity is 0,
+ then the local VMTP manager checks permissions and
+ forwards the request with CREntity set to entityId and
+ the entityId field set to a digital signature (see
+ below) of the Request by the VMTP manager, certifying
+ that the Client has the permissions required by the
+ Request. (If entityGroupId specifies an unrestricted
+ group, the Request can be sent directly to the handling
+ VMTP manager by setting CREntity to entityId.)
+
+0x0500010C - RemoveFromGroup(CREntity, entityGroupId, entityId)->(code)
+ Request that the VMTP manager local to CREntity remove
+ the specified entityId from the group specified by
+ entityGroupId. Normally CREntity is 0, indicating the
+ VMTP manager local to the requestor. If CREntity is 0,
+ then the local VMTP manager checks permissions and
+ forwards the request with CREntity set to entityId and
+ the entityId field a digital signature of the Request by
+ the VMTP manager, certifying that the Client has the
+ permissions required by the Request.
+
+0x0500010D - QueryGroup( entityId )->( code, record )...
+ Return information on the specified entity. The
+ Response from each responding VMTP manager is (code,
+ record). The format of the record is (memberCount,
+ member1, member2, ...). The Responses are returned on a
+ best efforts basis; there is no guarantee that responses
+ from all managers with members in the specified group
+ will be received.
+
+0x0500010E - ModifyService(entityId,flags,count,pc,threadlist)->(code,
+ count)
+ Modify the service associated with the entity specified
+ by entityId. The flags may indicate a message service
+ model, in which case the call "count" parameter
+ indicates the maximum number of queued messages desired;
+ the return "count" parameter indicates the number of
+ queued message allowed. Alternatively, the "flags"
+ parameters indicates the RPC thread service model, in
+ which case "count" threads are requested, each with an
+ inital program counter as specified and stack, priority
+ and message receive area indicated by the threadlist.
+ In particular, "threadlist" consists of "count" records
+ of the form
+ (priority,stack,stacksize,segment,segmentsize), each one
+ assigned to one of the threads. Flags defined for the
+
+
+Cheriton [page 95]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ "flags" parameter are:
+
+ 1 THREAD_SERVICE - otherwise the message
+ model.
+
+ 2 AUTHENTICATION_REQUIRED - Sent a Probe
+ request to determine principal
+ associated with the Client, if not
+ known.
+
+ 4 SECURITY_REQUIRED - Request must be
+ encrypted or else reject.
+
+ 8 INCREMENTAL - treat the count value as
+ an increment (or decrement) relative to
+ the current value rather than an
+ absolute value for the maximum number of
+ queued messages or threads.
+
+ In the thread model, the count must be a positive
+ increment or else 0, which disables the service. Only a
+ count of 0 terminates currently queued requests or
+ in-progress request handling.
+
+0x4500010F -
+ NotifyVmtpClient(client,cntrl,recSeq,transact,delivery,code)->()
+
+ Update the state associated with the transaction
+ specified by client and transact, an entity identifier
+ and transaction identifier, respectively. This
+ operation is normally used only by another VMTP
+ management module. (Note that it is a datagram
+ operation.) The other parameters are as follows:
+
+ ctrl A 32-bit value corresponding to 4th
+ 32-bit word of the VMTP header of a
+ Response packet that would be sent in
+ response to the Request that this is
+ responding to. That is, the control
+ flags, ForwardCount, RetransmitCount and
+ Priority fields match those of the
+ Request. (The NRS flag is set if the
+ receiveSeqNumber field is used.) The
+ PGCount subfield indicates the number of
+ previous Request packet groups being
+ acknowledged by this Notify operation.
+ (The bit fields that are reserved in
+
+
+Cheriton [page 96]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ this word in the header are also
+ reserved here and must be zero.)
+
+ recSeq Sequence number of reception at the
+ Server if the NRS flag is set in the
+ ctrl parameter, otherwise reserved and
+ zero. (This is used for sender-based
+ logging of message activity for replay
+ in case of failure - an optional
+ facility.)
+
+ delivery Indicates the segment blocks of the
+ packet group have been received at the
+ Server.
+
+ code indicates the action the client should
+ take, as described below.
+
+ The VMTP management module should take action on this
+ operation according to the code, as specified below.
+
+ OK Do nothing at this time, continue
+ waiting for the response with a reset
+ timer.
+
+ RETRY Retransmit the request packet group
+ immediately with at least the segment
+ blocks that the Server failed to
+ receive, the complement of those
+ indicated by the delivery parameter.
+
+ RETRY_ALL Retransmit the request packet group
+ immediately with at least the segment
+ blocks that the Server failed to
+ receive, as indicated by the delivery
+ field plus all subsequently transmitted
+ packets that are part of this packet
+ run. (The latter is applicable only for
+ streamed message transactions.)
+
+ BUSY The server was unable to accept the
+ Request at this time. Retry later if
+ desired to continue with the message
+ transaction.
+
+ NONEXISTENT_ENTITY
+ Specified Server entity does not exist.
+
+
+Cheriton [page 97]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ ENTITY_MIGRATED The server entity has migrated and is no
+ longer at the host to which the request
+ was sent. The Server should attempt to
+ determine the new host address of the
+ Client using the VMTP management
+ ProbeEntity operation (described
+ earlier).
+
+ NO_PERMISSION Server has not authorized reception of
+ messages from this client.
+
+ NOT_AWAITING_MSG
+ The conditional message delivery bit was
+ set for the Request packet group and the
+ Server was not waiting for it so the
+ Request packet group was discarded.
+
+ VMTP_ERROR The Request packet group was in error
+ relative to the VMTP protocol
+ specification.
+
+ BAD_TRANSACTION_ID
+ Transaction identifier is old relative
+ to the transaction identifier held for
+ the Client by the Server.
+
+ STREAMING_NOT_SUPPORTED
+ Server does not support multiple
+ outstanding message transactions from
+ the same Client, i.e. streamed message
+ transactions.
+
+ SECURITY_NOT_SUPPORTED
+ The Request was secure and this Server
+ does not support security.
+
+ SECURITY_REQUIRED
+ The Server is refusing the Request
+ because it was not encrypted.
+
+ NO_RUN_RECORD Server has no record of previous packets
+ in this run of packet groups. This can
+ occur if the first packet group is lost
+ or if the current packet group is sent
+ significantly later than the last one
+ and the Server has discarded its client
+ state record.
+
+
+Cheriton [page 98]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+0x45000110 - NotifyVmtpServer(server,client,transact,delivery,code)->()
+ Update the server state associated with the transaction
+ specified by client and transact, an entity identifier
+ and transaction identifier, respectively. This
+ operation is normally used only by another VMTP
+ management module. (Note that it is a datagram
+ operation.) The other parameters are as follows:
+
+ delivery Indicates the segment blocks of the
+ Response packet group that have been
+ received at the Client.
+
+ code indicates the action the Server should
+ take, as listed below.
+
+ The VMTP management module should take action on this
+ operation according to the code, as specified below.
+
+ OK Client is satisfied with Response data.
+ The Server can discard the response
+ data, if any.
+
+ RETRY Retransmit the Response packet group
+ immediately with at least the segment
+ blocks that the Client failed to
+ receive, as indicated by the delivery
+ parameter. (The delivery parameter
+ indicates those segment blocks received
+ by the Client).
+
+ RETRY_ALL Retransmit the Response packet group
+ immediately with at least the segment
+ blocks that the Client failed to
+ receive, as indicated by the (complement
+ of) the delivery parameter. Also,
+ retransmit all Response packet groups
+ send subsequent to the specified packet
+ group.
+
+ NONEXISTENT_ENTITY
+ Specified Client entity does not exist.
+
+ ENTITY_MIGRATED The Client entity has migrated and is no
+ longer at the host to which the response
+ was sent.
+
+ RESPONSE_DISCARDED
+
+
+Cheriton [page 99]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ The Response was discarded and no longer
+ of interest to the Client. This may
+ occur if the conditional message
+ delivery bit was set for the Response
+ packet group and the Client was not
+ waiting for it so the Response packet
+ group was discarded.
+
+ VMTP_ERROR The Response packet group was in error
+ relative to the VMTP protocol
+ specification.
+
+0x41000111 -
+ NotifyRemoteVmtpClient(client,ctrl,recSeq,transact,delivery,code->()
+
+ The same as NotifyVmtpClient except the co-resident
+ addressing is not used. This operation is used to
+ update client state that is remote when a Request is
+ forwarded.
+
+Note the use of the CRE bit in the RequestCodes to route the request to
+the correct VMTP management module(s) to handle the request.
+
+
+III.1. Entity Group Management
+
+An entity in a group has a set of permissions associated with its
+membership, controling whether it can add or remove others, whether it
+can remove itself, and whether others can remove it from the group. The
+permissions for entity groups are as follows:
+VMTP_GRP_MANAGER 0x00000001 { Manager of group. }
+VMTP_REM_BY_SELF 0x00000002 { Can be removed self. }
+VMTP_REM_BY_PRIN 0x00000004 { Can be rem'ed by same principal}
+VMTP_REM_BY_OTHE 0x00000008 { Can be removed any others. }
+VMTP_ADD_PRIN 0x00000010 { Can add by same principal. }
+VMTP_ADD_OTHE 0x00000020 { Can add any others. }
+VMTP_REM_PRIN 0x00000040 { Can remove same principal. }
+VMTP_REM_OTHE 0x00000080 { Can remove any others. }
+
+To remove an entity from a restricted group, the invoker must have
+permission to remove that entity and the entity must have permissions
+that allow it to be removed by that entity. With an unrestricted group,
+only the latter condition applies.
+
+With a restricted group, a member can only be added by another entity
+with the permissions to add other entities. The creator of a group is
+given full permissions on a group. A entity adding another entity to a
+
+
+Cheriton [page 100]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+group can only give the entity it adds a subset of its permissions.
+With unrestricted groups, any entity can add itself to the group. It
+can also add other entities to the group providing the entity is not
+marked as immune to such requests. (This is an implementation
+restriction that individual entities can impose.)
+
+
+III.2. VMTP Management Digital Signatures
+
+As mentioned above, the entityId field of the AddToGroup and
+RemoveFromGroup is used to transmit a digital signature indicating the
+permission for the operation has been checked by the sending kernel.
+The digital signature procedures have not yet been defined. This field
+should be set to 0 for now to indicate no signature after the CREntity
+parameter is set to the entity on which the operation is to be
+performed.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 101]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+IV. VMTP Entity Identifier Domains
+
+VMTP allows for several disjoint naming domains for its endpoints. The
+64-bit entity identifier is only unique and meaningful within its
+domain. Each domain can define its own algorithm or mechanism for
+assignment of entity identifiers, although each domain mechanism must
+ensure uniqueness, stability of identifiers and host independence.
+
+
+IV.1. Domain 1
+
+For initial use of VMTP, we define the domain with Domain identifier 1
+as follows:
+
+ +-----------+----------------+------------------------+
+ | TypeFlags | Discriminator | Internet Address |
+ +-----------+----------------+------------------------+
+ 4 bits 28 bits 32 bits
+
+The Internet address is the Internet address of the host on which this
+entity-id is originally allocated. The Discriminator is an arbitrary
+value that is unique relative to this Internet host address. In
+addition, the host must guarantee that this identifier does not get
+reused for a long period of time after it becomes invalid. ("Invalid"
+means that no VMTP module considers in bound to an entity.) One
+technique is to use the lower order bits of a 1 second clock. The clock
+need not represent real-time but must never be set back after a crash.
+In a simple implementation, using the low order bits of a clock as the
+time stamp, the generation of unique identifiers is overall limited to
+no more than 1 per second on average. The type flags were described in
+Section 3.1.
+
+An entity may migrate between hosts. Thus, an implementation can
+heuristically use the embedded Internet address to locate an entity but
+should be prepared to maintain a cache of redirects for migrated
+entities, plus accept Notify operations indicating that migration has
+occurred.
+
+Entity group identifiers in Domain 1 are structured in one of two forms,
+depending on whether they are well-known or dynamically allocated
+identifiers. A well-known entity identifier is structured as:
+
+ +-----------+----------------+------------------------+
+ | TypeFlags | Discriminator |Internet Host Group Addr|
+ +-----------+----------------+------------------------+
+ 4 bits 28 bits 32 bits
+
+
+
+Cheriton [page 102]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+with the second high-order bit (GRP) set to 1. This form of entity
+identifier is mapped to the Internet host group address specified in the
+low-order 32 bits. The Discriminator distinguishes group identifiers
+using the same Internet host group. Well-known entity group identifiers
+should be allocated to correspond to the basic services provided by
+hosts that are members of the group, not specifically because that
+service is provided by VMTP. For example, the well-known entity group
+identifier for the domain name service should contain as its embedded
+Internet host group address the host group for Domain Name servers.
+
+A dynamically allocated entity identifier is structured as:
+
+ +-----------+----------------+------------------------+
+ | TypeFlags | Discriminator | Internet Host Addr |
+ +-----------+----------------+------------------------+
+ 4 bits 28 bits 32 bits
+
+with the second high-order bit (GRP) set to 1. The Internet address in
+the low-order 32 bits is a Internet address assigned to the host that
+dynamically allocates this entity group identifier. A dynamically
+allocated entity group identifier is mapped to Internet host group
+address 232.X.X.X where X.X.X are the low-order 24 bits of the
+Discriminator subfield of the entity group identifier.
+
+We use the following notation for Domain 1 entity identifiers <10> and
+propose it use as a standard convention.
+
+ <flags>-<discriminator>-<Internet address>
+
+where <flags> are [X]{BE,LE,RG,UG}[A]
+
+ X = reserved
+ BE = big-endian entity
+ LE = little-endian entity
+ RG = restricted group
+ UG = unrestricted group
+ A = alias
+
+and <discriminator> is a decimal integer and <Internet address> is in
+standard dotted decimal IP address notation.
+
+Examples:
+
+_______________
+
+<10> This notation was developed by Steve Deering.
+
+
+Cheriton [page 103]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+BE-25593-36.8.0.49 is big-endian entity #25593 created on host
+ 36.8.0.49.
+
+RG-1-224.0.1.0 is the well-known restricted VMTP managers group.
+
+UG-565338-36.8.0.77 is unrestricted entity group #565338 created on host
+ 36.8.0.77.
+
+LEA-7823-36.8.0.77 is a little-endian alias entity #7823 created on host
+ 36.8.0.77.
+
+This notation makes it easy to communicate and understand entity
+identifiers for Domain 1.
+
+The well-known entity identifiers specified to date are:
+
+VMTP_MANAGER_GROUP RG-1-224.0.1.0
+ Managers for VMTP operations.
+
+VMTP_DEFAULT_BECLIENT BE-1-224.0.1.0
+ Client entity identifier to use when a (big-endian) host
+ has not determined or been allocated any client entity
+ identifiers.
+
+VMTP_DEFAULT_LECLIENT LE-1-224.0.1.0
+ Client entity identifier to use when a (little-endian)
+ host has not determined or been allocated any client
+ entity identifiers.
+
+Note that 224.0.1.0 is the host group address assigned to VMTP and to
+which all VMTP hosts belong.
+
+Other well-known entity group identifiers will be specified in
+subsequent extensions to VMTP and in higher-level protocols that use
+VMTP.
+
+
+IV.2. Domain 3
+
+Domain 3 is reserved for embedded systems that are restricted to a
+single network and are independent of IP. Entity identifiers are
+allocated using the decentralized approach described below. The mapping
+of entity group identifiers is specific to the type of network being
+used and not defined here. In general, there should be a simple
+algorithmic mapping from entity group identifier to multicast address,
+similar to that described for Domain 1. Similarly, the values for
+default client identifier are specific to the type of network and not
+
+
+Cheriton [page 104]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+defined here.
+
+
+IV.3. Other Domains
+
+Definition of additional VMTP domains is planned for the future.
+Requests for allocation of VMTP Domains should be addressed to the
+Internet protocol administrator.
+
+
+IV.4. Decentralized Entity Identifier Allocation
+
+The ProbeEntityBlock operation may be used to determine whether a block
+of entity identifiers is in use. ("In use" means valid or reserved by a
+host for allocation.) This mechanism is used to detect collisions in
+allocation of blocks of entity identifiers as part of the implementation
+of decentralized allocation of entity identifiers. (Decentralized
+allocation is used in local domain use of VMTP such as in embedded
+systems- see Domain 3.)
+
+Basically, a group of hosts can form a Domain or sub-Domain, a group of
+hosts managing their own entity identifier space or subspace,
+respectively. As an example of a sub-Domain, a group of hosts in Domain
+1 all identified with a particular host group address can manage the
+sub-Domain corresponding to all entity identifiers that contain that
+host group address. The ProbeEntityBlock operation is used to allocate
+the random bits of these identifiers as follows.
+
+When a host requires a new block of entity identifiers, it selects a new
+block (randomly or by some choice algorithm) and then multicasts a
+ProbeEntityBlock request to the members of the (sub-)Domain some R
+times. If no response is received after R (re)transmissions, the host
+concludes that it is free to use this block of identifiers. Otherwise,
+it picks another block and tries again.
+
+Notes:
+
+ 1. A block of 256 identifiers is specified by an entity
+ identifier with the low-order 8 bits all zero.
+
+ 2. When a host allocates an initial block of entity identifiers
+ (and therefore does not yet have a specified entity
+ identifier to use) it uses VMTP_DEFAULT_BECLIENT (if
+ big-endian, else VMTP_DEFAULT_LECLIENT if little-endian) as
+ its client identifier in the ProbeEntityBlock Request and a
+ transaction identifier of 0. As soon as it has allocated a
+ block of entity identifiers, it should use these identifiers
+
+
+Cheriton [page 105]
+
+
+
+ RFC 1045 VMTP February 1988
+
+
+ for all subsequent communication. The default client
+ identifier values are defined for each Domain.
+
+ 3. The set of hosts using this decentralized allocation must not
+ be subject to network partitioning. That is, the R
+ transmissions must be sufficient to ensure that every host
+ sees the ProbeEntityBlock request and (reliably) sends a
+ response. (A host that detects a collision can retransmit
+ the response multiple times until it sees a new
+ ProbeEntityBlock operation from the same host/Client up to a
+ maximum number of times.) For instance, a set of machines
+ connected by a single local network may able to use this type
+ of allocation.
+
+ 4. To guarantee T-stability, a host must prevent reuse of a
+ block of identifiers if any of the identifiers in the block
+ are currently valid or have been valid less than T seconds
+ previously. To this end, a host must remember recently used
+ identifiers and object to their reuse in response to a
+ ProbeEntityBlock operation.
+
+ 5. Care is required in a VMTP implementation to ensure that
+ Probe operations cannot be discarded due to lack of buffer
+ space or queued or delayed so that a response is not
+ generated quickly. This is required not only to detect
+ collisions but also to provide accurate roundtrip estimates
+ as part of ProbeEntity operations.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 106]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+V. Authentication Domains
+
+A VMTP authentication domain defines the format and interpretation for
+principal identifiers and encryption keys. In particular, an
+authentication domain must specify a means by which principal
+identifiers are allocated and guaranteed unique and stable. The
+currently defined authentication domains are as follows (0 is reserved).
+
+Ideally, all entities within one entity domain are also associated with
+one authentication domain. However, authentication domains are
+orthogonal to entity domains. Entities within one domain may have
+different authentication domains. (In this case, it is generally
+necessary to have some correspondence between principals in the
+different domains.) Also, one entity identifier may be associated with
+multiple authentication domains. Finally, one authentication domain may
+be used across multiple entity domains.
+
+
+V.1. Authentication Domain 1
+
+A principal identifier is structured as follows.
+
+ +---------------------------+------------------------+
+ | Internet Address | Local User Identifier |
+ +---------------------------+------------------------+
+ 32 bits 32 bits
+
+The Internet Address may specify an individual host (such as a UNIX
+machine) or may specify a host group address corresponding to a cluster
+of machines operating under a single adminstration. In both cases,
+there is assumed to be an adminstration associated with the embedded
+Internet address that guarantees the uniqueness and stability of the
+User Identifier relative to the Internet address. In particular, that
+administration is the only one authorized to allocate principal
+identifiers with that Internet address prefix, and it may allocate any
+of these identifiers.
+
+In authentication domain 1, the standard EncryptionQualifiers are:
+
+0 Clear text - no encryption.
+
+1 use 64-bit CBC DES for encryption and decryption.
+
+
+V.2. Other Authentication Domains
+
+Other authentication domains will be defined in the future as needed.
+
+
+
+Cheriton [page 107]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+VI. IP Implementation
+
+VMTP is designed to be implemented on the DoD IP Internet Datagram
+Protocol (although it may also be implemented as a local network
+protocol directly in "raw" network packets.)
+
+VMTP is assigned the protocol number 81.
+
+With a 20 octet IP header and one segment block, a VMTP packet is 600
+octets. By convention, any host implementing VMTP implicitly agrees to
+accept VMTP/IP packets of at least 600 octets.
+
+VMTP multicast facilities are designed to work with, and have been
+implemented using, the multicast extensions to the Internet [8]
+described in RFC 966 and 988. The wide-scale use of full VMTP/IP
+depends on the availability of IP multicast in this form.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 108]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+VII. Implementation Notes
+
+The performance and reliability of a protocol in operation is highly
+dependent on the quality of its implementation, in addition to the
+"intrinsic" quality of the protocol design. One of the design goals of
+the VMTP effort was to produce an efficiently implementable protocol.
+The following notes and suggestions are based on experience with
+implementing VMTP in the V distributed system and the UNIX 4.3 BSD
+kernel. The following is described for a client and server handling
+only one domain. A multi-domain client or server would replicate these
+structures for each domain, although buffer space may be shared.
+
+
+VII.1. Mapping Data Structures
+
+The ClientMap procedure is implemented using a hash table that maps to
+the Client State Record whether this entity is local or remote, as shown
+in Figure VII-1.
+
+ +---+---+--------------------------+
+ ClientMap | | x | |
+ +---+-|-+--------------------------+
+ | +--------------+ +--------------+
+ +-->| LocalClient |--->| LocalClient |
+ +--------------+ +--------------+
+ | RemoteClient | | RemoteClient |-> ...
+ +--------------+ +--------------+
+ | | | |
+ | | | |
+ +--------------+ +--------------+
+
+ Figure VII-1: Mapping Client Identifier to CSR
+
+Local clients are linked through the LocalClientLink, similarly for the
+RemoteClientLink. Once a CSR with the specified Entity Id is found,
+some field or flag indicates whether it is identifying a local or remote
+Entity. Hash collisions are handled with the overflow pointers
+LocalClientLink and RemoteClientLink (not shown) in the CSR for the
+LocalClient and RemoteClient fields, respectively. Note that a CSR
+representing an RPC request has both a local and remote entity
+identifier mapping to the same CSR.
+
+The Server specified in a Request is mapped to a server descriptor using
+the ServerMap (with collisions handled by the overflow pointer.). The
+server descriptor is the root of a queue of CSR's for handling requests
+plus flags that modify the handling of the Request. Flags include:
+
+
+
+Cheriton [page 109]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ +-------+---+-------------------------+
+ ServerMap | | x | |
+ +-------+-|-+-------------------------+
+ | +--------------+
+ | | OverflowLink |
+ | +--------------+
+ +-->| Server |
+ +--------------+
+ | Flags | Lock |
+ +--------------+
+ | Head Pointer |
+ +--------------+
+ | Tail Pointer |
+ +--------------+
+
+ Figure VII-2: Mapping Server Identifiers
+
+THREAD_QUEUE Request is to be invoked directly as a remote procedure
+ invocation, rather than by a server process in the
+ message model.
+
+AUTHENTICATION_REQUIRED
+ Sent a Probe request to determine principal associated
+ with the Client, if not known.
+
+SECURITY_REQUIRED
+ Request must be encrypted or else reject.
+
+REQUESTS_QUEUED Queue contains waiting requests, rather than free CSR's.
+ Queue this request as well.
+
+SERVER_WAITING The server is waiting and available to handle incoming
+ Request immediately, as required by CMD.
+
+Alternatively, the Server identifiers can be mapped to a CSR using the
+MapToClient mechanism with a pointer in the CSR refering to the server
+descriptor, if any. This scheme is attractive if there are client CSR's
+associated with a service to allow it to communicate as a client using
+VMTP with other services.
+
+Finally, a similar structure is used to expand entity group identifiers
+to the local membership, as shown in Figure VII-3. A group identifier
+is hashed to an index in the GroupMap. The list of group descriptors
+rooted at that index in the GroupMap contains a group descriptor for
+each local member of the group. The flags are the group permissions
+defined in Appendix III.
+
+
+
+Cheriton [page 110]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ +-------+---+----------------------------------+
+ GroupMap | | x | |
+ +-------+-|-+----------------------------------+
+ | +--------------+
+ | | OverflowLink |
+ | +--------------+
+ +-->|EntityGroupId |
+ +--------------+
+ | Flags |
+ +--------------+
+ | Member Entity|
+ +--------------+
+
+ Figure VII-3: Mapping Group Identifiers
+
+Note that the same pool of descriptors could be used for the server and
+group descriptors given that they are similar in size.
+
+
+VII.2. Client Data Structures
+
+Each client entity is represented as a client state record. The CSR
+contains a VMTP header as well as other bookkeeping fields, including
+timeout count, retransmission count, as described in Section 4.1. In
+addition, there is a timeout queue, transmission queue and reception
+queue. Finally, there is a ServerHost cache that maps from server
+entity-id records to host address, estimated round trip time,
+interpacket gap, MTU size and (optimally) estimated processing time for
+this server entity.
+
+
+VII.3. Server Data Structures
+
+The server maintains a heap of client state records (CSR), one for each
+(Client, Transaction). (If streams are not supported, there is, at
+worst, a CSR per Client with which the server has communicated with
+recently.) The CSR contains a VMTP header as well as various
+bookkeeping fields including timeout count, retransmission count. The
+server maintains a hash table mapping of Client to CSR as well as the
+transmission, timeout and reception queues. In a VMTP module
+implementing both the client and server functions, the same timeout
+queue and transmission queue are used for both.
+
+
+
+
+
+
+
+Cheriton [page 111]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+VII.4. Packet Group transmission
+
+The procedure SendPacketGroup( csr ) transmits the packet group
+specified by the record CSR. It performs:
+
+ 1. Fragmentation of the segment data, if any, into packets.
+ (Note, segment data flagged by SDA bit.)
+
+ 2. Modifies the VMTP header for each packet as required e.g.
+ changing the delivery mask as appropriate.
+
+ 3. Computes the VMTP checksum.
+
+ 4. Encrypts the appropriate portion of the packet, if required.
+
+ 5. Prepends and appends network-level header and trailer using
+ network address from ServerHost cache, or from the responding
+ CSR.
+
+ 6. Transmits the packet with the interpacket gap specified in
+ the cache. This may involve round-robin scheduling between
+ hosts as well as delaying transmissions slightly.
+
+ 7. Invokes the finish-up procedure specified by the CSR record,
+ completing the processing. Generally, this finish-up
+ procedure adds the record to the timeout queue with the
+ appropriate timeout queue.
+
+The CSR includes a 32-bit transmission mask that indicates the portions
+of the segment to transmit. The SendPacketGroup procedure is assumed to
+handle queuing at the network transmission queue, queuing in priority
+order according to the priority field specified in the CSR record.
+(This priority may be reflected in network transmission behavior for
+networks that support priority.)
+
+The SendPacketGroup procedure only looks at the following fields of a
+CSR
+
+ - Transmission mask
+
+ - FuncCode
+
+ - SDA
+
+ - Client
+
+ - Server
+
+
+Cheriton [page 112]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ - CoResidentEntity
+
+ - Key
+
+It modifies the following fields
+
+ - Length
+
+ - Delivery
+
+ - Checksum
+
+In the case of encrypted transmission, it encrypts the entire packet,
+not including the Client field and the following 32-bits.
+
+If the packet group is a Response, (i.e. lower-order bit of function
+code is 1) the destination network address is determined from the
+Client, otherwise the Server. The HostAddr field is set either from the
+ServerHost cache (if a Request) or from the original Request if a
+Response, before SendPacketGroup is called.
+
+The CSR includes a timeout and TTL fields indicating the maximum time to
+complete the processing and the time-to-live for the packets to be
+transmitted.
+
+SendPacketGroup is viewed as the right functionality to implement for
+transmission in an "intelligent" network interface.
+
+Finally, it appears preferable to be able to assume that all portions of
+the segment remain memory-resident (no page faults) during transmission.
+In a demand-paged systems, some form of locking is required to keep the
+segment data in memory.
+
+
+VII.5. VMTP Management Module
+
+The implementation should implement the management operations as a
+separate module that is invoked from within the VMTP module. When a
+Request is received, either from the local user level or the network,
+for the VMTP management module, the management module is invoked as a
+remote or local procedure call to handle this request and return a
+response (if not a datagram request). By registering as a local server,
+the management module should minimize the special-case code required for
+its invocation. The management module is basically a case statement
+that selects the operation based on the RequestCode and then invokes the
+specified management operation. The procedure implementing the
+management operation, especially operations like NotifyVmtpClient and
+
+
+Cheriton [page 113]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+NotifyVmtpServer, are logically part of the VMTP module because they
+require full access to the basic data structures of the VMTP
+implementation.
+
+The management module should be implemented so that it can respond
+quickly to all requests, particularly since the timing of management
+interactions is used to estimate round trip time. To date, all
+implementations of the management module have been done at the kernel
+level, along with VMTP proper.
+
+
+VII.6. Timeout Handling
+
+The timeout queue is a queue of CSR records, ordered by timeout count,
+as specified in the CSR record. On entry into the timeout queue, the
+CSR record has the timeout field set to the time (preferable in
+milliseconds or similar unit) to remain in the queue plus the finishup
+field set to the procedure to execute on removal on timeout from the
+queue. The timeout field for a CSR in the queue is the time relative to
+the record preceding it in the queue (if any) at which it is to be
+removed. Some system-specific mechanism decrements the time for the
+record at the front of the queue, invoking the finishup procedure when
+the count goes to zero.
+
+Using this scheme, a special CSR is used to timeout and scan CSR's for
+non-recently pinged CSR's. That is, this CSR times out and invokes a
+finishup procedure that scans for non-recently pinged CSR that are
+"AwaitingResponse" and signals the request processing entity and deletes
+the CSR. It then returns to the timeout queue.
+
+The timeout mechanism tends to be specific to an operating system. The
+scheme described may have to be adapted to the operating system in which
+VMTP is to be implemented.
+
+This mechanism handles client request timeout and client response
+timeout. It is not intended to handle interpacket gaps given that these
+times are expected to be under 1 millisecond in general and possibly
+only a few microseconds.
+
+
+VII.7. Timeout Values
+
+Roundtrip timeout values are estimated by matching Responses or
+NotifyVmtpClient Requests to Request transmission, relying on the
+retransmitCount to identify the particular transmission of the Request
+that generated the response. A similar technique can be used with
+Responses and NotifyVmtpServer Requests. The retransmitCount is
+
+
+Cheriton [page 114]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+incremented each time the Response is sent, whether the retransmission
+was caused by timeout or retransmission of the Request.
+
+The ProbeEntity request is recommended as a basic way of getting
+up-to-date information about a Client as well as predictable host
+machine turnaround in processing a request. (VMTP assumes and requires
+an efficient, bounded response time implementation of the ProbeEntity
+operation.)
+
+Using this mechanism for measuring RTT, it is recommended that the
+various estimation and smoothing techniques developed for TCP RTT
+estimation be adapted and used.
+
+
+VII.8. Packet Reception
+
+Logically a network packet containing a VMTP packet is 5 portions:
+
+ - network header, possibly including lower-level headers
+
+ - VMTP header
+
+ - data segment
+
+ - VMTP checksum
+
+ - network trailer, etc.
+
+It may be advantageous to receive a packet fragmented into these
+portions, if supported by the network module. In this case, ideally the
+VMTP header may be received directly into a CSR, the data segment into a
+page that can be mapped, rather than copied, to its final destination,
+with VMTP checksum and network header in a separate area (used to
+extract the network address corresponding to the sender).
+
+Packet reception is described in detail by the pseudo-code in Section
+4.7.
+
+With a response, normally the CSR has an associated segment area
+immediately available so delivery of segment data is immediate.
+Similarly, server entities should be "armed" with CSR's with segment
+areas that provide for immediate delivery of requests. It is reasonable
+to discard segment data that cannot be immediately delivered in this
+way, providing that clients and servers are able to preallocate CSR's
+with segment areas for requests and responses. In particular, a client
+should be able to provide some number of additional CSR's for receiving
+multiple responses to a multicast request.
+
+
+Cheriton [page 115]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+The CSR data structure is intended to be the interface data structure
+for an intelligent network interface. For reception, the interface is
+"armed" with CSR's that may point to segment areas in main memory, into
+which it can deliver a packet group. Ideally, the interface handles all
+the processing of all packets, interacting with the host after receiving
+a complete Request or Response packet group. An implementation should
+use an interface based on SendPacketGroup(CSR) and
+ReceivePacketGroup(CSR) to facilitate the introduction of an intelligent
+network interface.
+
+ReceivePacketGroup(csr) provides the interface with a CSR descriptor and
+zero or more bytes of main memory to receive segment data. The CSR
+describes whether it is to receive responses (and if so, for which
+client) or requests (and if so for which server).
+
+The procedure ReclaimCSR(CSR) reclaims the specified record from the
+interface before it has been returned after receiving the specified
+packet group.
+
+A finishup procedure is set in the CSR to be invoked when the CSR is
+returned to the host by the normal processing sequence in the interface.
+Similarly, the timeout parameter is set to indicate the maximum time the
+host is providing for the routine to perform the specified function.
+The CSR and associated segment memory is returned to the host after the
+timeout period with an indication of progress after the timeout period.
+It is not returned earlier.
+
+
+VII.9. Streaming
+
+The implementation of streaming is optional in both VMTP clients and
+servers. Ideally, all performance-critical servers should implement
+streaming. In addition, clients that have high context switch overhead,
+network access overhead or expect to be communicating over long delay
+links should also implement streaming.
+
+A client stream is implemented by allocating a CSR for each outstanding
+message transaction. A stream of transactions is handled similarly to
+multiple outstanding transactions from separate clients except for the
+interaction between consecutive numbered transactions in a stream.
+
+For the server VMTP module, streamed message transactions to a server
+are queued (if accepted) subordinate to the first unprocessed CSR
+corresponding to this Client. Thus, streamed transactions from a given
+Client are always performed in the order specified by the transaction
+identifiers.
+
+
+
+Cheriton [page 116]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+If a server does not implement streaming, it must refuse streamed
+message transactions using the NotifyVmtpClient operation. Also, all
+client VMTP's that support streaming must support the streamed interface
+to a server that does not support streaming. That is, it must perform
+the message transactions one at a time. Consequently, a program that
+uses the streaming interface to a non-streaming server experiences
+degraded performance, but not failure.
+
+
+VII.10. Implementation Experience
+
+The implementation experience to date includes a partial implementation
+(minus the streaming and full security) in the V kernel plus a similar
+preliminary implementation in the 4.3 BSD Unix kernel. In the V kernel
+implementation, the CSR's are part of the (lightweight) process
+descriptor.
+
+The V kernel implementation is able to perform a VMTP message
+transaction with no data segment between two Sun-3/75's connected by 10
+Mb Ethernet in 2.25 milliseconds. It is also able to transfer data at
+4.7 megabits per second using 16 kilobyte Requests (but null checksums.)
+The UNIX kernel implementation running on Microvax II's achieves a basic
+message transaction time of 9 milliseconds and data rate of 1.9 megabits
+per second using 16 kilobyte Responses. This implementation is using
+the standard VMTP checksum.
+
+We hope to report more extensive implementation experience in future
+revisions of this document.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 117]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+VIII. UNIX 4.3 BSD Kernel Interface for VMTP
+
+UNIX 4.3 BSD includes a socket-based design for program interfaces to a
+variety of protocol families and types of protocols (streams,
+datagrams). In this appendix, we sketch an extension to this design to
+support a transaction-style protocol. (Some familiarity with UNIX 4.2/3
+IPC is assumed.) Several extensions are required to the system
+interface, rather than just adding a protocol, because no provision was
+made for supporting transaction protocols in the original design. These
+extensions include a new "transaction" type of socket plus new system
+calls invoke, getreply, probeentity, recreq, sendreply and forward.
+
+A socket of type transaction bound to the VMTP protocol type
+IPPROTO_VMTP is created by the call
+
+ s = socket(AF_INET, SOCK_TRANSACT, VMTP);
+
+This socket is bound to an entity identifier by
+
+ bind(s, &entityid, sizeof(entityid));
+
+The first address/port bound to a socket is considered its primary name
+and is the one used on packet transmission. A message transaction is
+invoked between the socket named by s and the Server specified by mcb by
+
+ invoke(s, mcb, segptr, seglen, timeout );
+
+The mcb is a message control block whose format was described in Section
+2.4. The message control block specifies the request to send plus the
+destination Server. The response message control block returned by the
+server is stored in mcb when invoke returns. The invoking process is
+blocked until a response is received or the message transaction times
+out unless the request is a datagram request. (Non-blocking versions
+with signals on completion could also be provided, especially with a
+streaming implementation.)
+
+For multicast message transactions (sent to an entity group), the next
+response to the current message transaction (if it arrives in less than
+timeout milliseconds) is returned by
+
+ getreply( s, mcb, segptr, maxseglen, timeout );
+
+The invoke operation sent to an entity group completes as soon as the
+first response is received. A request is retransmitted until the first
+reply is received (assuming the request is not a datagram). Thus, the
+system does not retransmit while getreply is timing out even if no
+replies are available.
+
+
+Cheriton [page 118]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+The state of an entity associated with entityId is probed using
+
+ probeentity( entityId, state );
+
+A UNIX process acting as a VMTP server accepts a Request by the
+operation
+
+ recvreq(s, mcb, segptr, maxseglen );
+
+The request message for the next queued transaction request is returned
+in mcb, plus the segment data of maximum length maxseglen, starting at
+segptr in the address space. On return, the message control block
+contains the values as set in invoke except: (1) the Client field
+indicates the Client that sent the received Request message. (2) the
+Code field indicates the type of request. (3) the MsgDelivery field
+indicates the portions of the segment actually received within the
+specified segment size, if MDM is 1 in the Code field. A segment block
+is marked as missing (i.e. the corresponding bit in the MsgDelivery
+field is 0) unless it is received in its entirety or it is all of the
+data in last segment contained in the segment.
+
+To complete a transaction, the reply specified by mcb is sent to the
+client specified by the MCB using
+
+ sendreply(s, mcb, segptr );
+
+The Client field of the MCB indicates the client to respond to.
+
+Finally, a message transaction specified by mcb is forwarded to
+newserver as though it were sent there by its original invoker using
+
+ forward(s, mcb, segptr, timeout );
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 119]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+Index
+
+ Acknowledgment 14
+ APG 16, 31, 39
+ Authentication domain 20
+
+ Big-endian 9
+
+ Checksum 14, 43
+ Checksum, not set 44
+ Client 7, 10, 38
+ Client timer 16
+ CMD 42, 110
+ CMG 32, 40
+ Co-resident entity 25
+ Code 42
+ CoResidentEntity 42, 43
+ CRE 21, 42
+
+ DGM 42
+ Digital signature, VMTP management 95, 101
+ Diskless workstations 2
+ Domain 9, 38
+ Domain 1 102
+ Domain 3 104
+
+ Entity 7
+ Entity domain 9
+ Entity group 8
+ Entity identifier 37
+ Entity identifier allocation 105
+ Entity identifier, all-zero 38
+ EPG 20, 39
+
+ Features 6
+ ForwardCount 24
+ Forwarding 24
+ FunctionCode 41
+
+ Group 8
+ Group message transaction 10
+ Group timeouts 16
+ GRP 37
+
+ HandleNoCSR 62
+ HandleRequestNoCSR 79
+ HCO 14, 23, 39
+
+
+Cheriton [page 120]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Host independence 8
+
+ Idempotent 15
+ Interpacket gap 18, 40
+ IP 108
+
+ Key 91
+
+ LEE 32, 37
+ Little-endian 9
+
+ MCB 118
+ MDG 22, 40
+ MDM 30, 42
+ Message control block 118
+ Message size 6
+ Message transaction 7, 10
+ MPG 39
+ MsgDelivery 43
+ MSGTRANS_OVERFLOW 27
+ Multicast 4, 21, 120
+ Multicast, reliable 21
+
+ Naming 6
+ Negative acknowledgment 31
+ NER 25, 31, 39
+ NRT 26, 30, 39
+ NSR 25, 27, 31, 39
+
+ Object-oriented 2
+ Overrun 18
+
+ Packet group 7, 29, 39
+ Packet group run 31
+ PacketDelivery 29, 31, 41
+ PGcount 26, 41
+ PIC 42
+ Principal 11
+ Priority 41
+ Process 11
+ ProcessId 89
+ Protocol number,IP 108
+
+ RAE 37
+ Rate control 18
+ Real-time 2, 4
+ Realtime 22
+
+
+Cheriton [page 121]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ Reliability 12
+ Request message 10
+ RequestAckRetries 30
+ RequestRetries 15
+ Response message 10
+ ResponseAckRetries 31
+ ResponseRetries 15
+ Restricted group 8
+ Retransmission 15
+ RetransmitCount 17
+ Roundtrip time 17
+ RPC 2
+ Run 31, 39
+ Run, message transactions 25
+
+ SDA 42
+ Security 4, 19
+ Segment block 41
+ Segment data 43
+ SegmentSize 42, 43
+ Selective retransmission 18
+ Server 7, 10, 41
+ Server group 8
+ Sockets, VMTP 118
+ STI 26, 40
+ Streaming 25, 55
+ Strictly stable 8
+ Subgroups 21
+
+ T-stable 8
+ TC1(Server) 16
+ TC2(Server) 16
+ TC3(Server) 16
+ TC4 16
+ TCP 2
+ Timeouts 15
+ Transaction 10, 41
+ Transaction identification 10
+ TS1(Client) 17
+ TS2(Client) 17
+ TS3(Client) 17
+ TS4(Client) 17
+ TS5(Client) 17
+ Type flags 8
+
+ UNIX interface 118
+ Unrestricted group 8, 38
+
+
+Cheriton [page 122]
+
+
+
+RFC 1045 VMTP February 1988
+
+
+ NotifyVmtpClient 7, 26, 27, 30
+ NotifyVmtpServer 7, 14, 30
+ User Data 43
+
+ Version 38
+ VMTP Management digital signature 95, 101
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Cheriton [page 123]
+