doc: Add RFC documents

1 files changed, 1515 insertions, 0 deletions

diff --git a/doc/rfc/rfc5473.txt b/doc/rfc/rfc5473.txt
new file mode 100644
index 0000000..ae3c874
--- /dev/null
+++ b/doc/rfc/rfc5473.txt
@@ -0,0 +1,1515 @@
+
+
+
+
+
+
+Network Working Group                                          E. Boschi
+Request for Comments: 5473                                Hitachi Europe
+Category: Informational                                          L. Mark
+                                                         Fraunhofer IFAM
+                                                               B. Claise
+                                                     Cisco Systems, Inc.
+                                                              March 2009
+
+
+       Reducing Redundancy in IP Flow Information Export (IPFIX)
+                  and Packet Sampling (PSAMP) Reports
+
+Status of This Memo
+
+   This memo provides information for the Internet community.  It does
+   not specify an Internet standard of any kind.  Distribution of this
+   memo is unlimited.
+
+Copyright Notice
+
+   Copyright (c) 2009 IETF Trust and the persons identified as the
+   document authors.  All rights reserved.
+
+   This document is subject to BCP 78 and the IETF Trust's Legal
+   Provisions Relating to IETF Documents in effect on the date of
+   publication of this document (http://trustee.ietf.org/license-info).
+   Please review these documents carefully, as they describe your rights
+   and restrictions with respect to this document.
+
+   This document may contain material from IETF Documents or IETF
+   Contributions published or made publicly available before November
+   10, 2008.  The person(s) controlling the copyright in some of this
+   material may not have granted the IETF Trust the right to allow
+   modifications of such material outside the IETF Standards Process.
+   Without obtaining an adequate license from the person(s) controlling
+   the copyright in such materials, this document may not be modified
+   outside the IETF Standards Process, and derivative works of it may
+   not be created outside the IETF Standards Process, except to format
+   it for publication as an RFC or to translate it into languages other
+   than English.
+
+Abstract
+
+   This document describes a bandwidth saving method for exporting Flow
+   or packet information using the IP Flow Information eXport (IPFIX)
+   protocol.  As the Packet Sampling (PSAMP) protocol is based on IPFIX,
+   these considerations are valid for PSAMP exports as well.
+
+
+
+
+Boschi, et al.               Informational                      [Page 1]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   This method works by separating information common to several Flow
+   Records from information specific to an individual Flow Record.
+   Common Flow information is exported only once in a Data Record
+   defined by an Options Template, while the rest of the specific Flow
+   information is associated with the common information via a unique
+   identifier.
+
+Table of Contents
+
+   1. Introduction ....................................................3
+      1.1. IPFIX Documents Overview ...................................3
+      1.2. PSAMP Documents Overview ...................................4
+   2. Terminology .....................................................4
+      2.1. Terminology Summary Table ..................................5
+      2.2. IPFIX Flows versus PSAMP Packets ...........................5
+   3. Specifications for Bandwidth-Saving Information Export ..........5
+      3.1. Problem Statement and High-Level Solution ..................6
+      3.2. Data Reduction Technique ...................................7
+   4. Transport Protocol Choice .......................................8
+      4.1. PR-SCTP ....................................................9
+      4.2. UDP ........................................................9
+      4.3. TCP ........................................................9
+   5. commonPropertiesID Management ...................................9
+   6. The Collecting Process Side ....................................10
+      6.1. UDP .......................................................11
+      6.2. TCP .......................................................12
+   7. Advanced Techniques ............................................12
+      7.1. Multiple Data Reduction ...................................12
+      7.2. Cascading Common Properties ...............................15
+   8. Export and Evaluation Considerations ...........................15
+      8.1. Transport Protocol Choice .................................16
+      8.2. Reduced Size Encoding .....................................16
+      8.3. Efficiency Gain ...........................................16
+   9. Security Considerations ........................................17
+   10. Acknowledgments ...............................................17
+   11. References ....................................................17
+      11.1. Normative References .....................................17
+      11.2. Informative References ...................................18
+   Appendix A. Examples ..............................................19
+      A.1. Per-Flow Data Reduction ...................................19
+      A.2. Per-Packet Data Reduction .................................23
+      A.3. Common Properties Withdrawal Message ......................26
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                      [Page 2]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+1.  Introduction
+
+   The IPFIX working group has specified a protocol to export IP Flow
+   information [RFC5101].  This protocol is designed to export
+   information about IP traffic Flows and related measurement data,
+   where a Flow is defined by a set of key attributes (e.g., source and
+   destination IP address, source and destination port, etc.).  However,
+   thanks to its template mechanism, the IPFIX protocol can export any
+   type of information, as long as the Information Element is specified
+   in the IPFIX information model [RFC5101] or registered with IANA.
+
+   Regardless of the fields' contents, Flow Records with common
+   properties export the same fields in every single Data Record.  These
+   common properties may represent values common to a collection of
+   Flows or packets, or values that are invariant over time.  Note that
+   the common properties don't represent the list of Flow Keys, which
+   are used to define a Flow definition; however, the common properties
+   may contain some of the Flow Keys.  The reduction of redundant data
+   from the export stream can result in a significant reduction of the
+   transferred data.
+
+   This document specifies a way to export these invariant or common
+   properties only once, while the rest of the Flow-specific properties
+   are exported in regular Data Records.  Unique common properties
+   identifiers are used to link Data Records and the common attributes.
+
+   The proposed method is applicable to IPFIX Flow and to PSAMP per-
+   packet information, without any changes to both the IPFIX and PSAMP
+   protocol specifications.
+
+1.1.  IPFIX Documents Overview
+
+   The IPFIX protocol [RFC5101] provides network administrators with
+   access to IP Flow information.  The architecture for the export of
+   measured IP Flow information out of an IPFIX exporting process to a
+   collecting process is defined in the IPFIX Architecture [RFC5470],
+   per the requirements defined in RFC 3917 [RFC3917].  The IPFIX
+   Architecture [RFC5470] specifies how IPFIX Data Records and templates
+   are carried via a congestion-aware transport protocol from IPFIX
+   exporting processes to IPFIX collecting processes.  IPFIX has a
+   formal description of IPFIX Information Elements, their names, types,
+   and additional semantic information, as specified in the IPFIX
+   information model [RFC5102].  Finally, the IPFIX applicability
+   statement [RFC5472] describes what type of applications can use the
+   IPFIX protocol and how they can use the information provided.  It
+   furthermore shows how the IPFIX framework relates to other
+   architectures and frameworks.
+
+
+
+
+Boschi, et al.               Informational                      [Page 3]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+1.2.  PSAMP Documents Overview
+
+   The document "A Framework for Packet Selection and Reporting"
+   [RFC5474] describes the PSAMP framework for network elements to
+   select subsets of packets by statistical and other methods, and to
+   export a stream of reports on the selected packets to a collector.
+   The set of packet selection techniques (sampling, filtering, and
+   hashing) supported by PSAMP is described in "Sampling and Filtering
+   Techniques for IP Packet Selection" [RFC5475].  The PSAMP protocol
+   [RFC5476] specifies the export of packet information from a PSAMP
+   exporting process to a PSAMP collecting process.  Like IPFIX, PSAMP
+   has a formal description of its Information Elements, their names,
+   types, and additional semantic information.  The PSAMP information
+   model is defined in [RFC5477].  Finally, [PSAMP-MIB] describes the
+   PSAMP Management Information Base.
+
+2.  Terminology
+
+   IPFIX-specific terminology used in this document is defined in
+   Section 2 of the IPFIX protocol specification [RFC5101] and Section 3
+   of the PSAMP protocol specification [RFC5476].  As in [RFC5101] and
+   [RFC5476], these IPFIX-specific terms have the first letter of a word
+   capitalized when used in this document.
+
+   In addition, the following new terms are defined in this document:
+
+   commonPropertiesID:  The commonPropertiesID is an identifier of a set
+      of common properties that is locally unique per Observation Domain
+      and Transport Session.  Typically, this Information Element is
+      used to link to information reported in separate Data Records.
+      See the IPFIX information model [RFC5102] for the Information
+      Element definition.
+
+   Common Properties:  Common Properties are a collection of one or more
+      attributes shared by a set of different Flow Records.  Each set of
+      Common Properties is uniquely identifiable by means of a
+      commonPropertiesID.
+
+   Specific Properties:  Specific Properties are a collection of one or
+      more attributes reported in a Flow Record that are not included in
+      the Common Properties defined for that Flow Record.
+
+   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+   document are to be interpreted as described in RFC 2119 [RFC2119].
+
+
+
+
+
+
+Boschi, et al.               Informational                      [Page 4]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+2.1.  Terminology Summary Table
+
+   +------------------+---------------------------------------------+
+   |                  |                 Contents                    |
+   |                  +--------------------+------------------------+
+   |       Set        |      Template      |         Record         |
+   +------------------+--------------------+------------------------+
+   |     Data Set     |          /         |     Data Record(s)     |
+   +------------------+--------------------+------------------------+
+   |   Template Set   | Template Record(s) |           /            |
+   +------------------+--------------------+------------------------+
+   | Options Template | Options Template   |           /            |
+   |       Set        | Record(s)          |                        |
+   +------------------+--------------------+------------------------+
+
+                         Terminology Summary Table
+
+   A Data Set is composed of Data Record(s).  No Template Record is
+   included.  A Template Record or an Options Template Record defines
+   the Data Record.
+
+   A Template Set contains only Template Record(s).
+
+   An Options Template Set contains only Options Template Record(s).
+
+2.2.  IPFIX Flows versus PSAMP Packets
+
+   As described in the PSAMP protocol specification [RFC5476], the major
+   difference between IPFIX and PSAMP is that the IPFIX protocol exports
+   Flow Records while the PSAMP protocol exports Packet Records.  From a
+   pure export point of view, IPFIX will not distinguish a Flow Record
+   composed of several packets aggregated together from a Flow Record
+   composed of a single packet.  So, the PSAMP export can be seen as a
+   special IPFIX Flow Record containing information about a single
+   packet.
+
+   For this document's clarity, the term Flow Record represents a
+   generic term expressing an IPFIX Flow Record or a PSAMP Packet
+   Record, as foreseen by its definition.  However, when appropriate, a
+   clear distinction between Flow Record or Packet Record will be made.
+
+3.  Specifications for Bandwidth-Saving Information Export
+
+   Several Flow Records often share a set of Common Properties.
+   Repeating the information about these Common Properties for every
+   Flow Record introduces a huge amount of redundancy.  This document
+   proposes a method to reduce this redundancy.
+
+
+
+
+Boschi, et al.               Informational                      [Page 5]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   The PSAMP specifications are used for the export of per-packet
+   information, exporting the specific observed packet in an IPFIX Flow
+   Record.  This can be considered as a special Flow Record case,
+   composed of a single packet.  Therefore, the method described in this
+   document is also applicable to per-packet data reduction, e.g., for
+   export of One-Way Delay (OWD) measurements (see Appendix), trajectory
+   sampling, etc.
+
+3.1.  Problem Statement and High-Level Solution
+
+   Consider a set of properties "A", e.g., common sourceAddressA and
+   sourcePortA, equivalent for each Flow Record exported.  Figure 1
+   shows how this information is repeated with classical IPFIX Flow
+   Records, expressing the waste of bandwidth to export redundant
+   information.
+
+   +----------------+-------------+---------------------------+
+   | sourceAddressA | sourcePortA |     <Flow1 information>   |
+   +----------------+-------------+---------------------------+
+   | sourceAddressA | sourcePortA |     <Flow2 information>   |
+   +----------------+-------------+---------------------------+
+   | sourceAddressA | sourcePortA |     <Flow3 information>   |
+   +----------------+-------------+---------------------------+
+   | sourceAddressA | sourcePortA |     <Flow4 information>   |
+   +----------------+-------------+---------------------------+
+   |      ...       |     ...     |            ...            |
+   +----------------+-------------+---------------------------+
+
+        Figure 1: Common and Specific Properties Exported Together
+
+   Figure 2 shows how this information is exported when applying the
+   specifications of this document.  The Common Properties are separated
+   from the Specific Properties for each Flow Record.  The Common
+   Properties would be exported only once in a specific Data Record
+   (defined by an Options Template), while each Flow Record contains a
+   pointer to the Common Properties A, along with its Flow-specific
+   information.  In order to maintain the relationship between these
+   sets of properties, we introduce indices (in this case, the index for
+   properties A) for the Common Properties that are unique for all
+   Common Properties entries within an Observation Domain.  The purpose
+   of the indices is to serve as a "key" identifying "rows" of the
+   Common Properties table.  The rows are then referenced by the
+   Specific Properties by using the appropriate value for the Common
+   Properties identifier.
+
+
+
+
+
+
+
+Boschi, et al.               Informational                      [Page 6]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   +------------------------+-----------------+-------------+
+   | index for properties A | sourceAddressA  | sourcePortA |
+   +------------------------+-----------------+-------------+
+   |          ...           |      ...        |     ...     |
+   +------------------------+-----------------+-------------+
+
+
+   +------------------------+---------------------------+
+   | index for properties A |     <Flow1 information>   |
+   +------------------------+---------------------------+
+   | index for properties A |     <Flow2 information>   |
+   +------------------------+---------------------------+
+   | index for properties A |     <Flow3 information>   |
+   +------------------------+---------------------------+
+   | index for properties A |     <Flow4 information>   |
+   +------------------------+---------------------------+
+
+       Figure 2: Common and Specific Properties Exported Separately
+
+   This unique export of the Common Properties results in a decrease of
+   the bandwidth requirements for the path between the Exporter and the
+   Collector.
+
+3.2.  Data Reduction Technique
+
+   The IPFIX protocol [RFC5101] is Template based.  Templates define how
+   data should be exported, describing data fields together with their
+   type and meaning.  IPFIX specifies two types of Templates: the
+   Template Record and the Options Template Record.  The difference
+   between the two is that the Options Template Record includes the
+   notion of scope, defining how to scope the applicability of the Data
+   Record.  The scope, which is only available in the Options Template
+   Record, gives the context of the reported Information Elements in the
+   Data Records.  The Template Records and Options Template Records are
+   necessary to decode the Data Records.  Indeed, by only looking at the
+   Data Records themselves, it is impossible to distinguish a Data
+   Record defined by Template Record from a Data Record defined by an
+   Options Template Record.  To export information more efficiently,
+   this specification proposes to group Flow Records by their common
+   properties.  We define Common Properties as a collection of
+   attributes shared by a set of different Flow Records.
+
+   An implementation using the proposed specification MUST follow the
+   IPFIX transport protocol specifications defined in the IPFIX protocol
+   [RFC5101].
+
+
+
+
+
+
+Boschi, et al.               Informational                      [Page 7]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   As explained in Figure 3, the information is split into two parts,
+   using two different Data Records.  Common Properties MUST be exported
+   via Data Records defined by an Options Template Record.  Like
+   Template Records, they MUST be sent only once per SCTP association or
+   TCP connection, and MUST be sent reliably via SCTP if SCTP is the
+   transport protocol.  These properties represent values common to
+   several Flow Records (e.g., IP source and destination address).  The
+   Common Properties Data Records MUST be sent prior to the
+   corresponding Specific Properties Data Records.  The Data Records
+   reporting Specific Properties MUST be associated with the Data
+   Records reporting the Common Properties using a unique identifier for
+   the Common Properties, the commonPropertiesID Information Element
+   [RFC5102].  The commonPropertiesID Information Element MUST be
+   included in the scope of the Options Template Record, and also
+   included in the associated Template Record.
+
+   +---------------------------+     +---------------------+
+   | Common Properties         |     | Specific Properties |  Template
+   | Options Template Record   |     | Template Record     |  Definition
+   |                           |     |                     |
+   | scope: commonPropertiesID |     | commonPropertiesID  |
+   | Common Properties         |     | Specific Properties |
+   +------------+--------------+     +----------+----------+
+   .............|...............................|.......................
+                |                               |
+   +------------v-------------+      +----------v----------+
+   | Common Properties        |      | Specific Properties |+  Exported
+   | Data Record              |------> Data Records        ||  Data
+   +--------------------------+      +---------------------+|  Records
+                                      +---------------------+
+
+          Figure 3: Template Record and Data Record Dependencies
+
+   From the IPFIX protocol, there are no differences between the per-
+   Flow or per-packet data reduction, except maybe the terminology where
+   the Specific Properties could be called packet Specific Properties in
+   the previous figure.
+
+4.  Transport Protocol Choice
+
+   This document follows the IPFIX transport protocol specifications
+   defined in the IPFIX protocol [RFC5101].  However, depending on the
+   transport protocol choice, this document imposes some additional
+   constraints.  If Partial Reliable Stream Control Transmission
+   Protocol (PR-SCTP) [RFC3758] is selected as the IPFIX protocol, the
+   following PR-SCTP subsection specifications MUST be respected.  If
+   UDP is selected as the IPFIX protocol, the following UDP subsection
+
+
+
+
+Boschi, et al.               Informational                      [Page 8]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   specifications MUST be respected.  If TCP is selected as the IPFIX
+   protocol, the following TCP subsection specifications MUST be
+   respected.
+
+4.1.  PR-SCTP
+
+   The active Common Properties MUST be sent after the SCTP association
+   establishment and before the corresponding Specific Properties Data
+   Records.  In the case of SCTP association re-establishment, all
+   active Common Properties MUST be resent before the corresponding
+   Specific Properties Data Records.
+
+   The Common Properties Data Records MUST be sent reliably.
+
+4.2.  UDP
+
+   Common Properties Data Records MUST be resent on a regular basis.
+   The periodicity MUST be configurable.  The default value for the
+   frequency of Common Properties transmission (refresh timeout) is 10
+   minutes.
+
+   The Exporting Process SHOULD transmit the Common Properties
+   definition in advance of any Data Record that uses these Common
+   Properties to help ensure that the Collector has the Common
+   Properties definition before receiving the first associated Data
+   Record.
+
+   If a commonPropertiesID is not used anymore, the Exporting Process
+   stops resending the related Common Properties Data Record.  The old
+   commonPropertiesID MUST NOT be used until its lifetime (see
+   Section 6.1) has expired.
+
+4.3.  TCP
+
+   Common Properties MUST be sent after the TCP connection
+   establishment, and before the corresponding Specific Properties Data
+   Records.  In the case of TCP connection re-establishment, all active
+   Common Properties MUST be resent before the corresponding Specific
+   Properties Data Records.
+
+5.  commonPropertiesID Management
+
+   The commonPropertiesID is an identifier of a set of common properties
+   that is locally unique per Observation Domain and Transport Session.
+   The Exporting Process MUST manage the commonPropertiesIDs allocations
+   for its Observation Domains and Transport Session.  Different
+
+
+
+
+
+Boschi, et al.               Informational                      [Page 9]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   Observation Domains from the same Exporter MAY use the same
+   commonPropertiesID value to refer to different sets of Common
+   Properties.
+
+   The commonPropertiesID values MAY be assigned sequentially, but it is
+   NOT REQUIRED.  Particular commonPropertiesID ranges or values MAY
+   have explicit meanings for the IPFIX Device.  For example,
+   commonPropertiesID values may be assigned based on the result of a
+   hash function, etc.
+
+   Using a 64-bit commonPropertiesID Information Element allows the
+   export of 2**64 active sets of Common Properties, per Observation
+   Domain and per Transport Session.
+
+   commonPropertiesIDs that are not used anymore SHOULD be withdrawn.
+   The Common Properties Withdrawal message is a Data Record defined by
+   an Options Template consisting of only one scope field -- namely, the
+   commonPropertiesID (with a type of 137 [RFC5102]) and no non-scope
+   fields.
+
+   0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |         Set ID = 3            |      Length = 14 octets       |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |       Template ID N           |       Field Count = 1         |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Scope Field count = 1    |0|  commonPropertiesID = 137   |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |  Scope 1 Field Length = 8     |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+              Figure 4: Common Properties Withdrawal Message
+
+   If UDP is selected as the transport protocol, the Common Properties
+   Withdrawal messages MUST NOT be used, as this method is inefficient
+   due to the unreliable nature of UDP.
+
+6.  The Collecting Process Side
+
+   This section describes the Collecting Process when using SCTP and PR-
+   SCTP as the transport protocol.  Any necessary changes to the
+   Collecting Process, specifically related to TCP or UDP transport
+   protocols, are specified in the subsections.
+
+   The Collecting Process MUST store the commonPropertiesID information
+   for the duration of the association so that it can interpret the
+   corresponding Data Records that are received in subsequent Data Sets.
+
+
+
+Boschi, et al.               Informational                     [Page 10]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   The Collecting Process can either store the Data Records as they
+   arrive, without reconstructing the initial Flow Record, or
+   reconstruct the initial Flow Record.  In the former case, there might
+   be less storage capacity required at the Collector side.  In the
+   latter case, the Collector job is more complex and time-consuming due
+   to the higher resource demand for record processing in real time.
+
+   If the Collecting Process has received the Specific Properties Data
+   Record before the associated Common Properties Data Record, the
+   Collecting Process SHOULD store the Specific Properties Data Record
+   and await the retransmission or out-of-order arrival of the Common
+   Properties Data Record.
+
+   commonPropertiesIDs are unique per SCTP association and per
+   Observation Domain.  If the Collecting Process receives an Options
+   Template Record with a scope containing a commonPropertiesID that has
+   already been received but that has not previously been withdrawn
+   (i.e., a commonPropertiesID from the same Exporter Observation Domain
+   received on the SCTP association), then the Collecting Process MUST
+   shut down the association.
+
+   When an SCTP association is closed, the Collecting Process MUST
+   discard all commonPropertiesIDs received over that association and
+   stop decoding IPFIX Messages that use those commonPropertiesIDs.
+
+   If a Collecting Process receives a Common Properties Withdrawal
+   message, the Collecting Process MUST delete the corresponding Common
+   Properties associated with the specific SCTP association and specific
+   Observation Domain, and stop interpreting Data Records referring to
+   those Common Properties.  The receipt of Data Records referring to
+   Common Properties that have been withdrawn MUST be ignored and SHOULD
+   be logged by the Collecting Process.
+
+   If the Collecting Process receives a Common Properties Withdrawal
+   message for Common Properties that it has not received before on this
+   SCTP association, it MUST reset the SCTP association and discard the
+   IPFIX Message, and it SHOULD log the error as it does for malformed
+   IPFIX Messages.
+
+6.1.  UDP
+
+   The Collecting Process MUST associate a lifetime with each Common
+   Property received via UDP.  Common Properties not refreshed by the
+   Exporting Process within the lifetime are expired at the Collecting
+   Process.
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 11]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   If the Common Properties are not refreshed before that lifetime has
+   expired, the Collecting Process MUST discard the corresponding
+   definition of the commonPropertiesID and any current and future
+   associated Data Records.  In this case, an alarm MUST be logged.
+
+   The Collecting Process MUST NOT decode any further Data Records that
+   are associated with the expired Common Properties.  If a Common
+   Property is refreshed with a definition that differs from the
+   previous definition, the Collecting Process SHOULD log a warning and
+   replace the previously received Common Property with the new one.
+   The Common Property lifetime at the Collecting Process MUST be at
+   least 3 times higher than the refresh timeout of the Template used to
+   export the Common Property definition, configured on the Exporting
+   Process.
+
+   The Collecting Process SHOULD accept Data Records without the
+   associated Common Properties required to decode the Data Record.  If
+   the Common Properties have not been received at the time Data Records
+   are received, the Collecting Process SHOULD store the Data Records
+   for a short period of time and decode them after the Common
+   Properties definitions are received.  The short period of time MUST
+   be lower than the lifetime of definitions associated with identifiers
+   considered unique within the UDP session.
+
+6.2.  TCP
+
+   When the TCP connection is reset, either gracefully or abnormally,
+   the Collecting Processes MUST delete all commonPropertiesID values
+   and associated Common Properties data corresponding to that
+   connection.
+
+   If a Collection Process receives a Common Properties Withdrawal
+   message, the Collection Process MUST expire the related Common
+   Properties data.
+
+7.  Advanced Techniques
+
+7.1.  Multiple Data Reduction
+
+   A Flow Record can refer to one or more Common Properties sets; the
+   use of multiple Common Properties can lead to more efficient exports.
+   When sets of Common Properties are identified in the data, it may be
+   found that there is more than one set of non-overlapping properties.
+
+   Note that in the case of multiple Common Properties in one Data
+   Record, the different sets of Common Properties MUST be disjoint
+   (i.e., MUST NOT have Information Elements in common) to avoid
+   potential collisions.
+
+
+
+Boschi, et al.               Informational                     [Page 12]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   Consider a set of properties "A", e.g., common sourceAddressA and
+   sourcePortA, and another set of properties "B", e.g.,
+   destinationAddressB and destinationPortB.  Figure 5 shows how this
+   information is repeated with classical IPFIX export in several Flow
+   Records.
+
+   +--------+--------+---------+---------+---------------------+
+   |srcAddrA|srcPortA|destAddrB|destPortB| <Flow1 information> |
+   +--------+--------+---------+---------+---------------------+
+   |srcAddrA|srcPortA|destAddrC|destPortC| <Flow2 information> |
+   +--------+--------+---------+---------+---------------------+
+   |srcAddrD|srcPortD|destAddrB|destPortB| <Flow3 information> |
+   +--------+--------+---------+---------+---------------------+
+   |srcAddrD|srcPortD|destAddrC|destPortC| <Flow4 information> |
+   +--------+--------+---------+---------+---------------------+
+   |   ...  |   ...  |   ...   |   ...   |        ...          |
+   +--------+--------+---------+---------+---------------------+
+
+        Figure 5: Common and Specific Properties Exported Together
+
+   Besides A and B, other sets of Properties might be repeated as well
+   (e.g., Properties C and D in the figure above).
+
+   We can separate the Common Properties into properties A composed of
+   sourceAddressA and sourcePortA, properties D composed of
+   sourceAddressD and sourcePortD, properties B composed of
+   destinationAddressB and destinationPortB, and properties C composed
+   of destinationAddressC and destinationPortC.  These four records can
+   be expanded to four combinations of Data Records to reduce redundancy
+   without the need to define four complete sets of Common Properties
+   (see the figure below).  The more Common Properties sets that are
+   defined, the more combinations that are available.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 13]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   +-------------------+-----------------+-------------+
+   | index for prop. A | sourceAddressA  | sourcePortA |
+   +-------------------+-----------------+-------------+
+   | index for prop. D | sourceAddressD  | sourcePortD |
+   +-------------------+-----------------+-------------+
+
+   +-------------------+---------------------+------------------+
+   | index for prop. B | destinationAddressB | destinationPortB |
+   +-------------------+---------------------+------------------+
+   | index for prop. C | destinationAddressC | destinationPortC |
+   +-------------------+---------------------+------------------+
+
+   +------------------+------------------+-----------------------+
+   |index for prop. A |index for prop. B |  <Flow1 information>  |
+   +------------------+------------------+-----------------------+
+   |index for prop. A |index for prop. C |  <Flow2 information>  |
+   +------------------+------------------+-----------------------+
+   |index for prop. D |index for prop. B |  <Flow3 information>  |
+   +------------------+------------------+-----------------------+
+   |index for prop. D |index for prop. C |  <Flow4 information>  |
+   +------------------+------------------+-----------------------+
+
+     Figure 6: Multiple Common (above) and Specific Properties (below)
+                            Exported Separately
+
+   The advantage of the multiple Common Properties is that the objective
+   of reducing the bandwidth is met while the number of indices is kept
+   to a minimum.  Defining an extra index for all records would not save
+   bandwidth in the case of Figure 5 and is generally a less efficient
+   solution.
+
+   If a set of Flow Records share multiple sets of Common Properties,
+   multiple commonPropertiesID instances MAY be used to increase export
+   efficiency even further, as displayed in Figure 7.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 14]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   +--------------------------- +     +---------------------+
+   | Common Properties          |     | Specific Properties | Template
+   | Options Template Record    |     | Template Record     | Definition
+   |                            |     |                     |
+   | Scope: commonPropertiesID1 |     | commonPropertiesID1 |
+   | Scope: commonPropertiesID2 |     | commonPropertiesID2 |
+   | Common Properties          |     | Specific Properties |
+   +------------+---------------+     +---------+-----------+
+   .............|...............................|.......................
+                |                               |
+   +------------v-------------+      +----------v----------+
+   | Common Properties        |      | Specific Properties |+  Exported
+   | Data Record              |------> Data Records        ||  Data
+   +------------------------- +      +---------------------+|  Records
+                                      +---------------------+
+
+                     Figure 7: Multiple Data Reduction
+
+7.2.  Cascading Common Properties
+
+   An Exporting Process MUST NOT export any set of Common Properties
+   that contains, either directly or via other cascaded Common
+   Properties, references to itself in its own definition (i.e., a
+   circular definition).  When the Collecting Process receives Common
+   Properties that reference other Common Properties, it MUST resolve
+   the references to Common Properties.  If the Common Properties aren't
+   available at the time Data Records are received, the Collecting
+   Process SHOULD store the Data Records for a short period of time and
+   decode them after the Common Properties are received.
+
+   If the Collecting Process could not decode a cascading Common
+   Properties definition because the referenced Common Properties are
+   not available before the short period of time, then the Collecting
+   Process SHOULD log the error.
+
+   If the Collecting Process could not decode a cascading Common
+   Properties definition because it detects a circular definition, then
+   the Collecting Process SHOULD log the error.
+
+   Information Element ordering MUST be preserved when creating and
+   expanding Common Properties.
+
+8.  Export and Evaluation Considerations
+
+   The objective of the method specified in this document is the
+   reduction in the amount of measurement data that has to be
+   transferred from the Exporter to the Collector.  Note that the
+   efficiency of this method may vary, as discussed in this section.  In
+
+
+
+Boschi, et al.               Informational                     [Page 15]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   addition, there might be less storage capacity required at the
+   Collector side if the Collector decides to store the Data Records as
+   they arrive, without reconstructing the initial Flow Record.
+
+   On the other hand, this method requires additional resources on both
+   the Exporter and the Collector.  The Exporter has to manage Common
+   Properties information and to assign commonPropertiesID values.  The
+   Collector has to process records described by two templates instead
+   of just one.  Additional effort is also required when post processing
+   the measurement data, in order to correlate Flow Records with Common
+   Properties information.
+
+8.1.  Transport Protocol Choice
+
+   The proposed method is most effective using a reliable transport
+   protocol for the transfer of the Common Properties.  Therefore, the
+   use of PR-SCTP with full reliability or TCP is recommended for the
+   transmission of IPFIX Messages containing Common Properties.  Note
+   that use of UDP is less efficient for the transmission of Common
+   Properties, as they have to be resent regularly.
+
+8.2.  Reduced Size Encoding
+
+   The transfer of the commonPropertiesIDs originates some overhead and
+   might even increase the amount of exported data if the length of the
+   commonPropertiesID field is not shorter than the length of the
+   replaced fields.
+
+   In cases where the range of the commonPropertiesID can be restricted,
+   it is RECOMMENDED to apply reduced size encoding to the
+   commonPropertiesID to achieve a further gain in bandwidth efficiency.
+
+8.3.  Efficiency Gain
+
+   While the goal of this specification is to reduce the bandwidth, the
+   efficiency might be limited.  Indeed, the efficiency gain is based on
+   the abundance of redundant information in Flows and would be directly
+   proportional to the reuse of the defined commonPropertiesID values,
+   with a theoretical limit where all the Data Records would use a
+   single commonPropertiesID.  In other words, the more we reuse a
+   commonPropertiesID value, the better the efficiency gain.  While the
+   Exporting Process can evaluate the direct gain for the Flow Records
+   to be exported, it cannot predict whether future Flow Records would
+   contain the information specified by active commonPropertiesID
+   values.  This implies that the efficiency factor of this
+   specification is higher for specific applications where filtering is
+   involved, such as one-way delay or trajectory sampling.
+
+
+
+
+Boschi, et al.               Informational                     [Page 16]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   Note that this technique might even lead to an increase in bandwidth
+   usage under certain conditions.  Taking into account the overhead of
+   exporting the commonPropertiesID values, if the commonPropertiesID
+   values are not used in future Data Records, this technique would
+   actually increase the export bandwidth.  A typical case would be the
+   assignments of Common Properties based on past observed traffic,
+   hoping that future Flows would contain the same characteristics.
+
+   The efficiency gain depends also on the difference between the length
+   of the replaced fields and the length of the commonPropertiesID.  The
+   shorter the length of the commonPropertiesID is (with respect to the
+   total length of the Common Properties fields), the bigger the gain
+   is.
+
+   The example in Appendix A.2 below uses IPFIX to export measurement
+   data for each received packet.  In that case, for a Flow of 1000
+   packets, the amount of data can be decreased more than 26 percent.
+
+9.  Security Considerations
+
+   The same security considerations as for the IPFIX protocol [RFC5101]
+   apply.
+
+10.  Acknowledgments
+
+   The authors would like to thank Guido Pohl for initiating this work
+   and for his contribution to early versions of this document.  Thanks
+   also to Andrew Johnson, Gehrard Muenz, Brian Trammell, and Paul
+   Aitken for their comments and feedback.
+
+11.  References
+
+11.1.  Normative References
+
+   [RFC2119]    Bradner, S., "Key words for use in RFCs to Indicate
+                Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+   [RFC5101]    Claise, B., "Specification of the IP Flow Information
+                Export (IPFIX) Protocol for the Exchange of IP Traffic
+                Flow Information", RFC 5101, January 2008.
+
+   [RFC5102]    Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
+                Meyer, "Information Model for IP Flow Information
+                Export", RFC 5102, January 2008.
+
+   [RFC5476]    Claise, B., Ed., "Packet Sampling (PSAMP) Protocol
+                Specifications", RFC 5476, March 2009.
+
+
+
+
+Boschi, et al.               Informational                     [Page 17]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+11.2.  Informative References
+
+   [PSAMP-MIB]  Dietz, T., Ed. and B. Claise, "Definitions of Managed
+                Objects for Packet Sampling", Work in Progress,
+                June 2006.
+
+   [RFC3550]    Schulzrinne, H., Casner, S., Frederick, R., and V.
+                Jacobson, "RTP: A Transport Protocol for Real-Time
+                Applications", STD 64, RFC 3550, July 2003.
+
+   [RFC3758]    Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P.
+                Conrad, "Stream Control Transmission Protocol (SCTP)
+                Partial Reliability Extension", RFC 3758, May 2004.
+
+   [RFC3917]    Quittek, J., Zseby, T., Claise, B., and S. Zander,
+                "Requirements for IP Flow Information Export (IPFIX)",
+                RFC 3917, October 2004.
+
+   [RFC4960]    Stewart, R., "Stream Control Transmission Protocol",
+                RFC 4960, September 2007.
+
+   [RFC5470]    Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
+                "Architecture for IP Flow Information Export", RFC 5470,
+                March 2009.
+
+   [RFC5472]    Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP
+                Flow Information Export (IPFIX) Applicability",
+                RFC 5472, March 2009.
+
+   [RFC5474]    Duffield, N., Ed., "A Framework for Packet Selection and
+                Reporting", RFC 5474, March 2009.
+
+   [RFC5475]    Zseby, T., Molina, M., Duffield, N., Niccolini, S., and
+                F. Raspall, "Sampling and Filtering Techniques for IP
+                Packet Selection", RFC 5475, March 2009.
+
+   [RFC5477]    Dietz, T., Claise, B., Aitken, P., Dressler, F., and G.
+                Carle, "Information Model for Packet Sampling Exports",
+                RFC 5477, March 2009.
+
+
+
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 18]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+Appendix A.  Examples
+
+A.1.  Per-Flow Data Reduction
+
+   In this section, we show how Flow information can be exported
+   efficiently using the method described in this document.  Let's
+   suppose we have to periodically export data about two IPv6 Flows.
+
+   In this example, we report the following information:
+
+   Flow|        dstIPv6Address                 | dst- |nPkts|nBytes
+       |                                       | Port |     |
+   ----------------------------------------------------------------
+    A  |2001:DB8:80AD:5800:0058:0800:2023:1D71 |  80  | 30  |  6000
+       |                                       |      |     |
+    A  |2001:DB8:80AD:5800:0058:0800:2023:1D71 |  80  | 50  |  9500
+       |                                       |      |     |
+    B  |2001:DB8:80AD:5800:0058:00AA:00B7:AF2B | 1932 | 60  |  8000
+       |                                       |      |     |
+    A  |2001:DB8:80AD:5800:0058:0800:2023:1D71 |  80  | 40  |  6500
+       |                                       |      |     |
+    A  |2001:DB8:80AD:5800:0058:0800:2023:1D71 |  80  | 60  |  9500
+       |                                       |      |     |
+    B  |2001:DB8:80AD:5800:0058:00AA:00B7:AF2B | 1932 | 54  |  7600
+
+                    Figure 8: Flow Information Example
+
+   The Common Properties in this case are the destination IPv6 address
+   and the destination port.  We first define an Options Template that
+   contains the following Information Elements:
+
+   o  Scope: commonPropertiesID in [RFC5102], with a type of 137 and a
+      length of 8 octets.
+
+   o  The destination IPv6 address: destinationIPv6Address in [RFC5102],
+      with a type of 28 and a length of 16 octets.
+
+   o  The destination port: destinationTransportPort in [RFC5102], with
+      a type of 11, and a length of 2 octets.
+
+   Figure 9 shows the Options Template defining the Common Properties
+   with commonPropertiesID as scope:
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 19]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+     0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |         Set ID = 3            |      Length = 24 octets       |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |       Template ID = 257       |       Field Count = 3         |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Scope Field count = 1    |0|  commonPropertiesID = 137   |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |  Scope 1 Field Length = 8     |0|  destinationIPv6Address = 28|
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 16        |0|destinationTransportPort = 11|
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 2         |        (Padding)              |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+               Figure 9: Common Properties Options Template
+
+   The Specific Properties Template consists of the information not
+   contained in the Options Templates, i.e., Flow-specific information;
+   in this case, the number of packets and the number of bytes to be
+   reported.  Additionally, this Template contains the
+   commonPropertiesID.  In Data Records, the value of this field will
+   contain one of the unique indices of the Option Records exported
+   before.  It contains the following Information Elements (see also
+   Figure 10):
+
+   o  commonPropertiesID with a length of 8 octets.
+
+   o  The number of packets of the Flow: inPacketDeltaCount in
+      [RFC5102], with a length of 4 octets.
+
+   o  The number of octets of the Flow: inOctetDeltaCount in [RFC5102],
+      with a length of 4 octets.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 20]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+     0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |         Set ID = 2            |      Length = 20 octets       |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |       Template ID = 258       |       Field Count = 3         |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |0|  commonPropertiesID = 137   |       Field Length = 8        |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |0|    inPacketDeltaCount = 2   |       Field Length = 4        |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |0|    inOctetDeltaCount = 1    |       Field Length = 4        |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+                  Figure 10: Specific Properties Template
+
+   Considering the data shown at the beginning of this example, the
+   following two Data Records will be exported:
+
+   Common-      |           dstAddress                    | dst-
+   PropertiesID |                                         | Port
+   -------------+-----------------------------------------+-------
+       101      | 2001:DB8:80AD:5800:0058:0800:2023:1D71  |  80
+                |                                         |
+       102      | 2001:DB8:80AD:5800:0058:00AA:00B7:AF2B  | 1932
+
+                                 Figure 11
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 21]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   The Data Records reporting the Common Properties will look like:
+
+   0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |         Set ID = 257          |      Length = 60 octets       |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                                                               |
+   +-                             101                             -+
+   |                                                               |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                                                               |
+   +-                                                             -+
+   |                                                               |
+   +-           2001:DB8:80AD:5800:0058:0800:2023:1D71            -+
+   |                                                               |
+   +-                                                             -+
+   |                                                               |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |            80                 |                               |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-                             -+
+   |                              102                              |
+   +-                             -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                               |                               |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                              -+
+   |                                                               |
+   +-                                                             -+
+   |            2001:DB8:80AD:5800:0058:00AA:00B7:AF2B             |
+   +-                                                             -+
+   |                                                               |
+   +-                             -+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                               |          1932                 |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+            Figure 12: Data Records reporting Common Properties
+
+   The Data Records will in turn be:
+
+   commonPropertiesID  |  inPacketDeltaCount  | inOctetDeltaCount
+   ---------------------------------------------------------------
+           101         |          30          |       6000
+           101         |          50          |       9500
+           102         |          60          |       8000
+           101         |          40          |       6500
+           101         |          60          |       9500
+           102         |          54          |       7600
+
+                                 Figure 13
+
+
+
+Boschi, et al.               Informational                     [Page 22]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   Figure 14 shows the first Data Record listed in the table:
+
+    0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |          Set ID = 258         |          Length = 16          |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                                                               |
+   +-                             101                             -+
+   |                                                               |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |               30              |             6000              |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+            Figure 14: Data Record reporting Common Properties
+
+A.2.  Per-Packet Data Reduction
+
+   An example of the per-packet data reduction is the measurement of
+   One-Way Delay (OWD), where the exact same specific packet must be
+   observed at the source and destination of the path to be measured.
+   The OWD is computed by subtracting the time of observation of the
+   same packet at the two end-points with synchronized clocks.  As the
+   OWD is measured for a specific application on which a Service Level
+   Agreement (SLA) is bound, this translates into the observation of
+   multiple packets with Specific Properties.  In order to match the
+   identical packet at both Observation Points, a series of packets with
+   a set of properties (for example, all the packets of a specific
+   source and destination IP addresses, of a specific Diffserv codepoint
+   (DSCP) value, and of a specific destination transport port) must be
+   observed at both ends of the measurements.  This implies that the
+   source and destination must export a series of Flow Records composed
+   of two types of information: some common information for all packets,
+   and some unique information about each packet in order to generate a
+   unique identifier for each packet passing this Observation Point (for
+   example, a hash value on the invariant fields of the packet).  So,
+   the source and destination composing the measurement's end-points can
+   individually and independently apply the redundancy technique
+   described in this document in order to save some bandwidth for their
+   respective Flow Records exports.
+
+   The Templates required for exporting measurement data of this kind
+   are illustrated in the figures below.  Figure 15 shows the Options
+   Template containing the information concerning Flows using the
+   commonPropertiesID as scope.  In the Common Properties Template, we
+   export the following Information Elements:
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 23]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   o  The source IPv4 Address: sourceIPv4Address in [RFC5102], with a
+      type of 8 and a length of 4 octets.
+
+   o  The destination IPv4 Address: destinationIPv4Address in [RFC5102],
+      with a type of 12 and a length of 4 octets.
+
+   o  The Class of Service field: ClassOfServiceIPv4 in [RFC5102], with
+      a type of 5 and a length of 1 octet.
+
+   o  The Protocol Identifier: protocolIdentifier in [RFC5102], with a
+      type of 4 and a length of 1 octet.
+
+   o  The source port: sourceTransportPort in [RFC5102], with a type of
+      7 and a length of 2 octets.
+
+   o  The destination port: destinationTransportPort in [RFC5102], with
+      a type of 11 and a length of 2 octets.
+
+   The commonPropertiesID Information Element is used as the Scope
+   Field.
+
+     0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |         Set ID = 3            |      Length = 40 octets       |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |       Template ID = 256       |       Field Count = 7         |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Scope Field count = 1    |0|  commonPropertiesID = 137   |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |  Scope 1 Field Length = 4     |0|    sourceIPv4Address = 8    |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 4         |0| destinationIPv4Address = 12 |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 4         |0|  classOfServiceIPv4 = 5     |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 1         |0|  protocolIdentifier = 4     |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 1         |0|  transportSourcePort = 7    |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 2         |0|transportDestinationPort = 11|
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Field Length = 2         |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+                Figure 15: Example Flow Properties Template
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 24]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   For passive OWD measurement, the Packet Properties Template or
+   Specific Properties Template consists of at least the timestamp and
+   packet ID.  Additionally, this template contains a commonPropertiesID
+   field to associate the packet with a Flow.
+
+   Figure 16 displays the template with the packet properties.  In this
+   example, we export the following Information Elements:
+
+   o  commonPropertiesID.  In this case, reduced size encoding is used,
+      and the Information Element is declared with a length of 4 octets
+      instead of 8.
+
+   o  The packet timestamp: observationTimeMilliseconds in the PSAMP
+      information model [RFC5477], with a type of 323 and a length of 8
+      octets.
+
+   o  digestHashValue in the PSAMP information model [RFC5477], with a
+      type of 326 and a length of 8 octets.
+
+   o  The packet length: ipTotalLength in the IPFIX information model
+      [RFC5102], with a type of 224 and a length of 8 octets.
+
+      0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |         Set ID = 2            |      Length = 36 octets       |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |       Template ID = 257       |       Field Count = 4         |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |0|  commonPropertiesID = 137   |       Field Length = 4        |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |0| observationTimeMillis.= 323 |       Field Length = 8        |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |0|    digestHashValue = 326    |       Field Length = 8        |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+    |0|    ipTotalLength = 224      |       Field Length = 8        |
+    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+               Figure 16: Example Packet Properties Template
+
+   At the collection point, packet records from the two measurement
+   points are gathered and correlated by means of the packet ID.  The
+   resulting delay Data Records are exported in a similar manner as the
+   packet data.  One-Way Delay data is associated with Flow information
+   by the commonPropertiesID field.  The OWD properties contain the
+   Packet Pair ID (which is the packet ID of the two contributing packet
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 25]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   records), the timestamp of the packet passing the reference monitor
+   point in order to reconstruct a time series, the calculated delay
+   value, and the commonPropertiesID.
+
+   In this example, using IPFIX to export the measurement data for each
+   received packet, 38 bytes have to be transferred (sourceAddressV4=4,
+   destinationAddressV4=4, classOfServiceV4=1, protocolIdentifier=1,
+   sourceTransportPort=2, destinationTransportPort=2,
+   observationTimeMilliseconds=8, digestHashValue=8, ipTotalLength=8).
+   Without considering the IPFIX protocol overhead, a Flow of 1000
+   packets produces 38000 bytes of measurement data.  Using the proposed
+   optimization, each packet produces an export of only 28 bytes
+   (observationTimeMilliseconds=8, digestHashValue=8, ipTotalLength=8,
+   commonPropertiesID=4).  The export of the Flow information produces
+   18 bytes (sourceAddressV4=4, destinationAddressV4=4,
+   classOfServiceV4=1, protocolIdentifier=1, sourceTransportPort=2,
+   destinationTransportPort=2, commonPropertiesID=4).  For a Flow of
+   1000 packets, this sums to 28018 bytes.  This is a decrease of more
+   than 26 percent.
+
+A.3.  Common Properties Withdrawal Message
+
+   This section shows an example commonPropertiesID Withdrawal message.
+   Figure 17 depicts the Options Template Record with the
+   commonPropertiesID as unique scope field, and no non-scope fields.
+
+     0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |         Set ID = 3            |      Length = 14 octets       |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |       Template ID 259         |       Field Count = 1         |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |      Scope Field count = 1    |0|  commonPropertiesID 137     |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |  Scope 1 Field Length = 8     |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+         Figure 17: Example Common Properties Withdrawal Template
+
+
+
+
+
+
+
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 26]
+
+RFC 5473                  Reducing Redundancy                 March 2009
+
+
+   Figure 18 shows the Option Data Record withdrawing commonPropertiesID
+   N:
+
+    0                   1                   2                   3
+    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |         Set ID = 259          |      Length = 12 octets       |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+   |                                                               |
+   +-                              N                              -+
+   |                                                               |
+   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+            Figure 18: Record Withdrawing commonPropertiesID N
+
+Authors' Addresses
+
+   Elisa Boschi
+   Hitachi Europe
+   c/o ETH Zurich
+   Gloriastrasse 35
+   8092 Zurich
+   Switzerland
+
+   Phone: +41 44 6327057
+   EMail: elisa.boschi@hitachi-eu.com
+
+
+   Lutz Mark
+   Fraunhofer IFAM
+   Wiener Str. 12
+   28359 Bremen
+   Germany
+
+   Phone: +49 421 2246206
+   EMail: lutz.mark@ifam.fraunhofer.de
+
+
+   Benoit Claise
+   Cisco Systems, Inc.
+   De Kleetlaan 6a b1
+   Diegem  1813
+   Belgium
+
+   Phone: +32 2 704 5622
+   EMail: bclaise@cisco.com
+
+
+
+
+
+Boschi, et al.               Informational                     [Page 27]
+