summaryrefslogtreecommitdiff
path: root/doc/rfc/rfc7290.txt
diff options
context:
space:
mode:
Diffstat (limited to 'doc/rfc/rfc7290.txt')
-rw-r--r--doc/rfc/rfc7290.txt1739
1 files changed, 1739 insertions, 0 deletions
diff --git a/doc/rfc/rfc7290.txt b/doc/rfc/rfc7290.txt
new file mode 100644
index 0000000..2be75c6
--- /dev/null
+++ b/doc/rfc/rfc7290.txt
@@ -0,0 +1,1739 @@
+
+
+
+
+
+
+Internet Engineering Task Force (IETF) L. Ciavattone
+Request for Comments: 7290 AT&T Labs
+Category: Informational R. Geib
+ISSN: 2070-1721 Deutsche Telekom
+ A. Morton
+ AT&T Labs
+ M. Wieser
+ Technical University Darmstadt
+ July 2014
+
+
+ Test Plan and Results for Advancing RFC 2680 on the Standards Track
+
+Abstract
+
+ This memo provides the supporting test plan and results to advance
+ RFC 2680, a performance metric RFC defining one-way packet loss
+ metrics, along the Standards Track. Observing that the metric
+ definitions themselves should be the primary focus rather than the
+ implementations of metrics, this memo describes the test procedures
+ to evaluate specific metric requirement clauses to determine if the
+ requirement has been interpreted and implemented as intended. Two
+ completely independent implementations have been tested against the
+ key specifications of RFC 2680.
+
+Status of This Memo
+
+ This document is not an Internet Standards Track specification; it is
+ published for informational purposes.
+
+ This document is a product of the Internet Engineering Task Force
+ (IETF). It represents the consensus of the IETF community. It has
+ received public review and has been approved for publication by the
+ Internet Engineering Steering Group (IESG). Not all documents
+ approved by the IESG are a candidate for any level of Internet
+ Standard; see Section 2 of RFC 5741.
+
+ Information about the current status of this document, any errata,
+ and how to provide feedback on it may be obtained at
+ http://www.rfc-editor.org/info/rfc7290.
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 1]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+Copyright Notice
+
+ Copyright (c) 2014 IETF Trust and the persons identified as the
+ document authors. All rights reserved.
+
+ This document is subject to BCP 78 and the IETF Trust's Legal
+ Provisions Relating to IETF Documents
+
+ (http://trustee.ietf.org/license-info) in effect on the date of
+ publication of this document. Please review these documents
+ carefully, as they describe your rights and restrictions with respect
+ to this document. Code Components extracted from this document must
+ include Simplified BSD License text as described in Section 4.e of
+ the Trust Legal Provisions and are provided without warranty as
+ described in the Simplified BSD License.
+
+ This document may contain material from IETF Documents or IETF
+ Contributions published or made publicly available before November
+ 10, 2008. The person(s) controlling the copyright in some of this
+ material may not have granted the IETF Trust the right to allow
+ modifications of such material outside the IETF Standards Process.
+ Without obtaining an adequate license from the person(s) controlling
+ the copyright in such materials, this document may not be modified
+ outside the IETF Standards Process, and derivative works of it may
+ not be created outside the IETF Standards Process, except to format
+ it for publication as an RFC or to translate it into languages other
+ than English.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 2]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+Table of Contents
+
+ 1. Introduction ....................................................3
+ 1.1. Requirements Language ......................................4
+ 1.2. RFC 2680 Coverage ..........................................5
+ 2. A Definition-Centric Metric Advancement Process .................5
+ 3. Test Configuration ..............................................5
+ 4. Error Calibration and RFC 2680 ..................................9
+ 4.1. Clock Synchronization Calibration ..........................9
+ 4.2. Packet Loss Determination Error ...........................10
+ 5. Predetermined Limits on Equivalence ............................10
+ 6. Tests to Evaluate RFC 2680 Specifications ......................11
+ 6.1. One-Way Loss: ADK Sample Comparison .......................11
+ 6.1.1. 340B/Periodic Cross-Implementation Results .........12
+ 6.1.2. 64B/Periodic Cross-Implementation Results ..........14
+ 6.1.3. 64B/Poisson Cross-Implementation Results ...........15
+ 6.1.4. Conclusions on the ADK Results for One-Way
+ Packet Loss ........................................16
+ 6.2. One-Way Loss: Delay Threshold .............................16
+ 6.2.1. NetProbe Results for Loss Threshold ................17
+ 6.2.2. Perfas+ Results for Loss Threshold .................17
+ 6.2.3. Conclusions for Loss Threshold .....................17
+ 6.3. One-Way Loss with Out-of-Order Arrival ....................17
+ 6.4. Poisson Sending Process Evaluation ........................19
+ 6.4.1. NetProbe Results ...................................19
+ 6.4.2. Perfas+ Results ....................................20
+ 6.4.3. Conclusions for Goodness-of-Fit ....................22
+ 6.5. Implementation of Statistics for One-Way Loss .............23
+ 7. Conclusions for a Revision of RFC 2680 .........................23
+ 8. Security Considerations ........................................24
+ 9. Acknowledgements ...............................................24
+ 10. Appendix - Network Configuration and Sample Commands ..........25
+ 11. References ....................................................28
+ 11.1. Normative References .....................................28
+ 11.2. Informative References ...................................29
+
+1. Introduction
+
+ The IETF IP Performance Metrics (IPPM) working group has considered
+ how to advance their metrics along the Standards Track since 2001.
+
+ The renewed work effort sought to investigate ways in which the
+ measurement variability could be reduced in order to thereby simplify
+ the problem of comparison for equivalence. As a result, there is
+ consensus (captured in [RFC6576]) that equivalent results from
+ independent implementations of metric specifications are sufficient
+ evidence that the specifications themselves are clear and
+ unambiguous; it is the parallel concept of protocol interoperability
+
+
+
+Ciavattone, et al. Informational [Page 3]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ for metric specifications. The advancement process either (1)
+ produces confidence that the metric definitions and supporting
+ material are clearly worded and unambiguous or (2) identifies ways in
+ which the metric definitions should be revised to achieve clarity.
+ It is a non-goal to compare the specific implementations themselves.
+
+ The process also permits identification of options described in the
+ metric RFC that were not implemented, so that they can be removed
+ from the advancing specification (this is an aspect more typical of
+ protocol advancement along the Standards Track).
+
+ This memo's purpose is to implement the current approach for
+ [RFC2680] and document the results.
+
+ In particular, this memo documents consensus on the extent of
+ tolerable errors when assessing equivalence in the results. In
+ discussions, the IPPM working group agreed that the test plan
+ and procedures should include the threshold for determining
+ equivalence, and this information should be available in advance of
+ cross-implementation comparisons. This memo includes procedures for
+ same-implementation comparisons to help set the equivalence
+ threshold.
+
+ Another aspect of the metric RFC advancement process is the
+ requirement to document the work and results. The procedures of
+ [RFC2026] are expanded in [RFC5657], including sample implementation
+ and interoperability reports. This memo follows the template in
+ [RFC6808] for the report that accompanies the protocol action request
+ submitted to the Area Director, including a description of the test
+ setup, procedures, results for each implementation, and conclusions.
+
+ The conclusion reached is that [RFC2680], with modifications, should
+ be advanced on the Standards Track. The revised text of RFC 2680
+ [LOSS-METRIC] is ready for review but awaits work in progress to
+ update the IPPM Framework [RFC2330]. Therefore, this memo documents
+ the information to support the advancement of [RFC2680], and the
+ approval of a revision of RFC 2680 is left for future action.
+
+1.1. Requirements Language
+
+ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+ "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
+ document are to be interpreted as described in RFC 2119 [RFC2119].
+ Some of these key words were used in [RFC2680], but there are no
+ requirements specified in this memo.
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 4]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+1.2. RFC 2680 Coverage
+
+ This plan is intended to cover all critical requirements and sections
+ of [RFC2680].
+
+ Note that there are only five relevant instances of the requirement
+ term "MUST" in [RFC2680], outside of the boilerplate and [RFC2119]
+ reference; the instance of "MUST" in the Security Considerations
+ section of [RFC2680] is not a basis for implementation equivalence
+ comparisons.
+
+ Statements in RFC 2680 that have the character of requirements may be
+ included if the community reaches consensus that the wording implies
+ a requirement. At least one instance of an implied requirement has
+ been found in Section 3.6 of [RFC2680].
+
+2. A Definition-Centric Metric Advancement Process
+
+ The process described in Section 3.5 of [RFC6576] takes as a first
+ principle that the metric definitions, embodied in the text of the
+ RFCs, are the objects that require evaluation and possible revision
+ in order to advance to the next step on the Standards Track. This
+ memo follows that process.
+
+3. Test Configuration
+
+ One metric implementation used was NetProbe version 5.8.5 (an earlier
+ version is used in the WIPM system and deployed worldwide [WIPM]).
+ NetProbe uses UDP packets of variable size and can produce test
+ streams with Periodic [RFC3432] or Poisson [RFC2330] sample
+ distributions.
+
+ The other metric implementation used was Perfas+ version 3.1,
+ developed by Deutsche Telekom [Perfas]. Perfas+ uses UDP unicast
+ packets of variable size (but also supports TCP and multicast). Test
+ streams with Periodic, Poisson, or uniform sample distributions may
+ be used.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 5]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ Figure 1 shows a view of the test path as each implementation's test
+ flows pass through the Internet and the Layer 2 Tunneling Protocol
+ version 3 (L2TPv3) [RFC3931] tunnel IDs (1 and 2), based on Figure 1
+ of [RFC6576].
+
+ +------------+ +------------+
+ | Imp 1 | ,---. | Imp 2 |
+ +------------+ / \ +-------+ +------------+
+ | V100 ^ V200 / \ | Tunnel| | V300 ^ V400
+ | | ( ) | Head | | |
+ +--------+ +------+ | |__| Router| +----------+
+ |Ethernet| |Tunnel| |Internet | +---B---+ |Ethernet |
+ |Switch |--|Head |-| | | |Switch |
+ +-+--+---+ |Router| | | +---+---+--+--+--+----+
+ |__| +--A---+ ( ) |Network| |__|
+ \ / |Emulat.|
+ U-turn \ / |"netem"| U-turn
+ V300 to V400 `-+-' +-------+ V100 to V200
+
+
+
+ Implementations ,---. +--------+
+ +~~~~~~~~~~~/ \~~~~~~| Remote |
+ +------->-----F2->-| / \ |->---. |
+ | +---------+ | Tunnel ( ) | | |
+ | | transmit|-F1->-| ID 1 | | |->. | |
+ | | Imp 1 | +~~~~~~~~~| |~~~~| | | |
+ | | receive |-<--+ | | | F1 F2 |
+ | +---------+ | |Internet | | | | |
+ *-------<-----+ F1 | | | | | |
+ +---------+ | | +~~~~~~~~~| |~~~~| | | |
+ | transmit|-* *-| | | |<-* | |
+ | Imp 2 | | Tunnel ( ) | | |
+ | receive |-<-F2-| ID 2 \ / |<----* |
+ +---------+ +~~~~~~~~~~~\ /~~~~~~| Switch |
+ `-+-' +--------+
+
+ Illustrations of a test setup with a bidirectional tunnel.
+ The upper diagram emphasizes the VLAN connectivity and
+ geographical location (where "Imp #" is the sender and
+ receiver of implementation 1 or 2 -- either Perfas+ or
+ NetProbe in this test). The lower diagram shows example
+ flows traveling between two measurement implementations.
+ For simplicity, only two flows are shown, and the netem
+ emulator is omitted (it would appear before or after the
+ Internet, depending on the flow).
+
+ Figure 1
+
+
+
+Ciavattone, et al. Informational [Page 6]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ The testing employs the L2TPv3 [RFC3931] tunnel between test sites on
+ the Internet. The tunnel IP and L2TPv3 headers are intended to
+ conceal the test equipment addresses and ports from hash functions
+ that would tend to spread different test streams across parallel
+ network resources, with likely variation in performance as a result.
+
+ At each end of the tunnel, one pair of VLANs encapsulated in the
+ tunnel are looped back so that test traffic is returned to each test
+ site. Thus, test streams traverse the L2TP tunnel twice but appear
+ to be one-way tests from the point of view of the test equipment.
+
+ The network emulator is a host running Fedora 14 Linux [FEDORA], with
+ IP forwarding enabled and the "netem" Network emulator as part of the
+ Fedora Kernel 2.6.35.11 [NETEM] loaded and operating. The standard
+ kernel is "tickless", replacing the previous periodic timer (250 Hz,
+ with 4 ms uncertainty) interrupts with on-demand interrupts.
+ Connectivity across the netem/Fedora host was accomplished by
+ bridging Ethernet VLAN interfaces together with "brctl" commands
+ (e.g., eth1.100 <-> eth2.100). The netem emulator was activated on
+ one interface (eth1) and only operated on test streams traveling in
+ one direction. In some tests, independent netem instances operated
+ separately on each VLAN. See the Appendix for more details.
+
+ The links between the netem emulator host, the router, and the switch
+ were found to be 100BaseTX-HD (100 Mbps half duplex), as reported by
+ "mii-tool" [MII-TOOL] when testing was complete. The use of half
+ duplex was not intended but probably added a small amount of delay
+ variation that could have been avoided in full-duplex mode.
+
+ Each individual test was run with common packet rates (1 pps, 10 pps)
+ Poisson/Periodic distributions, and IP packet sizes of 64, 340, and
+ 500 bytes.
+
+ For these tests, a stream of at least 300 packets was sent from
+ source to destination in each implementation. Periodic streams (as
+ per [RFC3432]) with 1-second spacing were used, except as noted.
+
+ As required in Section 2.8.1 of [RFC2680], packet Type-P must be
+ reported. The packet Type-P for this test was IP-UDP with Best
+ Effort Differentiated Services Code Point (DSCP). These headers were
+ encapsulated according to the L2TPv3 specification [RFC3931] and were
+ unlikely to influence the treatment received as the packets traversed
+ the Internet.
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 7]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ With the L2TPv3 tunnel in use, the metric name for the testing
+ configured here (with respect to the IP header exposed to Internet
+ processing) is:
+
+ Type-IP-protocol-115-One-way-Packet-Loss-<StreamType>-Stream
+
+ With (Section 3.2 of [RFC2680]) metric parameters:
+
+ + Src, the IP address of a host (12.3.167.16 or 193.159.144.8)
+
+ + Dst, the IP address of a host (193.159.144.8 or 12.3.167.16)
+
+ + T0, a time
+
+ + Tf, a time
+
+ + lambda, a rate in reciprocal seconds
+
+ + Thresh, a maximum waiting time in seconds (see Section 2.8.2 of
+ [RFC2680])
+
+ Metric Units: A sequence of pairs; the elements of each pair are:
+
+ + T, a time, and
+
+ + L, either a zero or a one
+
+ The values of T in the sequence are monotonically increasing.
+ Note that T would be a valid parameter of *singleton*
+ Type-P-One-way-Packet-Loss and that L would be a valid value of
+ Type-P-One-way-Packet-Loss (see Section 3.3 of [RFC2680]).
+
+ Also, Section 2.8.4 of [RFC2680] recommends that the path SHOULD be
+ reported. In this test setup, most of the path details will be
+ concealed from the implementations by the L2TPv3 tunnels; thus, a
+ more informative path traceroute can be conducted by the routers at
+ each location.
+
+ When NetProbe is used in production, a traceroute is conducted in
+ parallel at the outset of measurements.
+
+ Perfas+ does not support traceroute.
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 8]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ IPLGW#traceroute 193.159.144.8
+
+ Type escape sequence to abort.
+ Tracing the route to 193.159.144.8
+
+ 1 12.126.218.245 [AS 7018] 0 msec 0 msec 4 msec
+ 2 cr84.n54ny.ip.att.net (12.123.2.158) [AS 7018] 4 msec 4 msec
+ cr83.n54ny.ip.att.net (12.123.2.26) [AS 7018] 4 msec
+ 3 cr1.n54ny.ip.att.net (12.122.105.49) [AS 7018] 4 msec
+ cr2.n54ny.ip.att.net (12.122.115.93) [AS 7018] 0 msec
+ cr1.n54ny.ip.att.net (12.122.105.49) [AS 7018] 0 msec
+ 4 n54ny02jt.ip.att.net (12.122.80.225) [AS 7018] 4 msec 0 msec
+ n54ny02jt.ip.att.net (12.122.80.237) [AS 7018] 4 msec
+ 5 192.205.34.182 [AS 7018] 0 msec
+ 192.205.34.150 [AS 7018] 0 msec
+ 192.205.34.182 [AS 7018] 4 msec
+ 6 da-rg12-i.DA.DE.NET.DTAG.DE (62.154.1.30) [AS 3320] 88 msec 88 msec
+ 88 msec
+ 7 217.89.29.62 [AS 3320] 88 msec 88 msec 88 msec
+ 8 217.89.29.55 [AS 3320] 88 msec 88 msec 88 msec
+ 9 * * *
+
+ NetProbe Traceroute
+
+ It was only possible to conduct the traceroute for the measured path
+ on one of the tunnel-head routers (the normal trace facilities of the
+ measurement systems are confounded by the L2TPv3 tunnel
+ encapsulation).
+
+4. Error Calibration and RFC 2680
+
+ An implementation is required to report calibration results on clock
+ synchronization per Section 2.8.3 of [RFC2680] (also required in
+ Section 3.7 of [RFC2680] for sample metrics).
+
+ Also, it is recommended to report the probability that a packet
+ successfully arriving at the destination network interface is
+ incorrectly designated as lost due to resource exhaustion in
+ Section 2.8.3 of [RFC2680].
+
+4.1. Clock Synchronization Calibration
+
+ For NetProbe and Perfas+ clock synchronization test results, refer to
+ Section 4 of [RFC6808].
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 9]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+4.2. Packet Loss Determination Error
+
+ Since both measurement implementations have resource limitations, it
+ is theoretically possible that these limits could be exceeded and a
+ packet that arrived at the destination successfully might be
+ discarded in error.
+
+ In previous test efforts [ADV-METRICS], NetProbe produced six
+ multicast streams with an aggregate bit rate over 53 Mbit/s, in order
+ to characterize the one-way capacity of an emulator based on NIST
+ Net. Neither the emulator nor the pair of NetProbe implementations
+ used in this testing dropped any packets in these streams.
+
+ The maximum load used here between any two NetProbe implementations
+ was 11.5 Mbit/s divided equally among three unicast test streams. We
+ concluded that steady resource usage does not contribute error
+ (additional loss) to the measurements.
+
+5. Predetermined Limits on Equivalence
+
+ In this section, we provide the numerical limits on comparisons
+ between implementations in order to declare that the results are
+ equivalent and that the tested specification is therefore clear.
+
+ A key point is that the allowable errors, corrections, and confidence
+ levels only need to be sufficient to detect any misinterpretation of
+ the tested specification that would indicate diverging
+ implementations.
+
+ Also, the allowable error must be sufficient to compensate for
+ measured path differences. It was simply not possible to measure
+ fully identical paths in the VLAN-loopback test configuration used,
+ and this practical compromise must be taken into account.
+
+ For Anderson-Darling K-sample (ADK) [ADK] comparisons, the required
+ confidence factor for the cross-implementation comparisons SHALL be
+ the smallest of:
+
+ o 0.95 confidence factor at 1-packet resolution, or
+
+ o the smallest confidence factor (in combination with resolution) of
+ the two same-implementation comparisons for the same test
+ conditions (if the number of streams is sufficient to allow such
+ comparisons).
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 10]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ For Anderson-Darling Goodness-of-Fit (ADGoF) [RADGOF] comparisons,
+ the required level of significance for the same-implementation
+ Goodness-of-Fit (GoF) SHALL be 0.05 or 5%, as specified in
+ Section 11.4 of [RFC2330]. This is equivalent to a 95% confidence
+ factor.
+
+6. Tests to Evaluate RFC 2680 Specifications
+
+ This section describes some results from production network (cross-
+ Internet) tests with measurement devices implementing IPPM metrics
+ and a network emulator to create relevant conditions, to determine
+ whether the metric definitions were interpreted consistently by
+ implementors.
+
+ The procedures are similar to those contained in Appendix A.1 of
+ [RFC6576] for one-way delay.
+
+6.1. One-Way Loss: ADK Sample Comparison
+
+ This test determines if implementations produce results that appear
+ to come from a common packet loss distribution, as an overall
+ evaluation of Section 3 of [RFC2680] ("A Definition for Samples of
+ One-way Packet Loss"). Same-implementation comparison results help
+ to set the threshold of equivalence that will be applied to cross-
+ implementation comparisons.
+
+ This test is intended to evaluate measurements in Sections 2, 3, and
+ 4 of [RFC2680].
+
+ By testing the extent to which the counts of one-way packet loss on
+ different test streams of two [RFC2680] implementations appear to be
+ from the same loss process, we reduce comparison steps because
+ comparing the resulting summary statistics (as defined in Section 4
+ of [RFC2680]) would require a redundant set of equivalence
+ evaluations. We can easily check whether the single statistic in
+ Section 4 of [RFC2680] was implemented and report on that fact.
+
+ 1. Configure an L2TPv3 path between test sites, and each pair of
+ measurement devices to operate tests in their designated pair of
+ VLANs.
+
+ 2. Measure a sample of one-way packet loss singletons with two or
+ more implementations, using identical options and network
+ emulator settings (if used).
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 11]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ 3. Measure a sample of one-way packet loss singletons with *four or
+ more* instances of the *same* implementations, using identical
+ options, noting that connectivity differences SHOULD be the same
+ as for cross-implementation testing.
+
+ 4. If less than ten test streams are available, skip to step 7.
+
+ 5. Apply the ADK comparison procedures (see Appendix B of
+ [RFC6576]), and determine the resolution and confidence factor
+ for distribution equivalence of each same-implementation
+ comparison and each cross-implementation comparison.
+
+ 6. Take the coarsest resolution and confidence factor for
+ distribution equivalence from the same-implementation pairs, or
+ the limit defined in Section 5 above, as a limit on the
+ equivalence threshold for these experimental conditions.
+
+ 7. Compare the cross-implementation ADK performance with the
+ equivalence threshold determined in step 5 to determine if
+ equivalence can be declared.
+
+ The metric parameters varied for each loss test, and they are listed
+ first in each sub-section below.
+
+ The cross-implementation comparison uses a simple ADK analysis
+ [RTOOL] [RADK], where all NetProbe loss counts are compared with all
+ Perfas+ loss results.
+
+ In the results analysis of this section:
+
+ o All comparisons used 1-packet resolution.
+
+ o No correction factors were applied.
+
+ o The 0.95 confidence factor (and ADK criterion for t.obs < 1.960
+ for cross-implementation comparison) was used.
+
+6.1.1. 340B/Periodic Cross-Implementation Results
+
+ Tests described in this section used:
+
+ o IP header + payload = 340 octets
+
+ o Periodic sampling at 1 packet per second
+
+ o Test duration = 1200 seconds (during April 7, 2011, EDT)
+
+
+
+
+
+Ciavattone, et al. Informational [Page 12]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ The netem emulator was set for 100 ms constant delay, with a 10% loss
+ ratio. In this experiment, the netem emulator was configured to
+ operate independently on each VLAN; thus, the emulator itself is a
+ potential source of error when comparing streams that traverse the
+ test path in different directions.
+
+ =======================================
+
+ A07bps_loss <- c(114, 175, 138, 142, 181, 105) (NetProbe)
+ A07per_loss <- c(115, 128, 136, 127, 139, 138) (Perfas+)
+
+ > A07bps_loss <- c(114, 175, 138, 142, 181, 105)
+ > A07per_loss <- c(115, 128, 136, 127, 139, 138)
+ >
+ > A07cross_loss_ADK <- adk.test(A07bps_loss, A07per_loss)
+ > A07cross_loss_ADK
+ Anderson-Darling k-sample test.
+
+ Number of samples: 2
+ Sample sizes: 6 6
+ Total number of values: 12
+ Number of unique values: 11
+
+ Mean of Anderson Darling Criterion: 1
+ Standard deviation of Anderson Darling Criterion: 0.6569
+
+ T = (Anderson Darling Criterion - mean)/sigma
+
+ Null Hypothesis: All samples come from a common population.
+
+ t.obs P-value extrapolation
+ not adj. for ties 0.52043 0.20604 0
+ adj. for ties 0.62679 0.18607 0
+ >
+
+ =======================================
+
+ The cross-implementation comparisons pass the ADK criterion
+ (t.obs < 1.960).
+
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 13]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+6.1.2. 64B/Periodic Cross-Implementation Results
+
+ Tests described in this section used:
+
+ o IP header + payload = 64 octets
+
+ o Periodic sampling at 1 packet per second
+
+ o Test duration = 300 seconds (during March 24, 2011, EDT)
+
+ The netem emulator was set for 0 ms constant delay, with a 10% loss
+ ratio.
+
+ =======================================
+
+ > M24per_loss <- c(42,34,35,35) (Perfas+)
+ > M24apd_23BC_loss <- c(27,39,29,24) (NetProbe)
+ > M24apd_loss23BC_ADK <- adk.test(M24apd_23BC_loss,M24per_loss)
+ > M24apd_loss23BC_ADK
+ Anderson-Darling k-sample test.
+
+ Number of samples: 2
+ Sample sizes: 4 4
+ Total number of values: 8
+ Number of unique values: 7
+
+ Mean of Anderson Darling Criterion: 1
+ Standard deviation of Anderson Darling Criterion: 0.60978
+
+ T = (Anderson Darling Criterion - mean)/sigma
+
+ Null Hypothesis: All samples come from a common population.
+
+ t.obs P-value extrapolation
+ not adj. for ties 0.76921 0.16200 0
+ adj. for ties 0.90935 0.14113 0
+
+ Warning: At least one sample size is less than 5.
+ p-values may not be very accurate.
+ >
+
+ =======================================
+
+ The cross-implementation comparisons pass the ADK criterion.
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 14]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+6.1.3. 64B/Poisson Cross-Implementation Results
+
+ Tests described in this section used:
+
+ o IP header + payload = 64 octets
+
+ o Poisson sampling at lambda = 1 packet per second
+
+ o Test duration = 1200 seconds (during April 27, 2011, EDT)
+
+ The netem configuration was 0 ms delay and 10% loss, but there were
+ two passes through an emulator for each stream, and loss emulation
+ was present for 18 minutes of the 20-minute (1200-second) test.
+
+ =======================================
+
+ A27aps_loss <- c(91,110,113,102,111,109,112,113) (NetProbe)
+ A27per_loss <- c(95,123,126,114) (Perfas+)
+
+ A27cross_loss_ADK <- adk.test(A27aps_loss, A27per_loss)
+
+ > A27cross_loss_ADK
+ Anderson-Darling k-sample test.
+
+ Number of samples: 2
+ Sample sizes: 8 4
+ Total number of values: 12
+ Number of unique values: 11
+
+ Mean of Anderson Darling Criterion: 1
+ Standard deviation of Anderson Darling Criterion: 0.65642
+
+ T = (Anderson Darling Criterion - mean)/sigma
+
+ Null Hypothesis: All samples come from a common population.
+
+ t.obs P-value extrapolation
+ not adj. for ties 2.15099 0.04145 0
+ adj. for ties 1.93129 0.05125 0
+
+ Warning: At least one sample size is less than 5.
+ p-values may not be very accurate.
+ >
+
+ =======================================
+
+ The cross-implementation comparisons barely pass the ADK criterion at
+ 95% = 1.960 when adjusting for ties.
+
+
+
+Ciavattone, et al. Informational [Page 15]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+6.1.4. Conclusions on the ADK Results for One-Way Packet Loss
+
+ We conclude that the two implementations are capable of producing
+ equivalent one-way packet loss measurements based on their
+ interpretation of [RFC2680].
+
+6.2. One-Way Loss: Delay Threshold
+
+ This test determines if implementations use the same configured
+ maximum waiting time delay from one measurement to another under
+ different delay conditions and correctly declare packets arriving in
+ excess of the waiting time threshold as lost.
+
+ See Section 2.8.2 of [RFC2680].
+
+ 1. Configure an L2TPv3 path between test sites, and each pair of
+ measurement devices to operate tests in their designated pair of
+ VLANs.
+
+ 2. Configure the network emulator to add 1 second of one-way
+ constant delay in one direction of transmission.
+
+ 3. Measure (average) one-way delay with two or more implementations,
+ using identical waiting time thresholds (Thresh) for loss set at
+ 3 seconds.
+
+ 4. Configure the network emulator to add 3 seconds of one-way
+ constant delay in one direction of transmission equivalent to
+ 2 seconds of additional one-way delay (or change the path delay
+ while the test is in progress, when there are sufficient packets
+ at the first delay setting).
+
+ 5. Repeat/continue measurements.
+
+ 6. Observe that the increase measured in step 5 caused all packets
+ with 2 seconds of additional delay to be declared lost and that
+ all packets that arrive successfully in step 3 are assigned a
+ valid one-way delay.
+
+ The common parameters used for tests in this section are:
+
+ o IP header + payload = 64 octets
+
+ o Poisson sampling at lambda = 1 packet per second
+
+ o Test duration = 900 seconds total (March 21, 2011 EDT)
+
+
+
+
+
+Ciavattone, et al. Informational [Page 16]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ The netem emulator settings added constant delays as specified in the
+ procedure above.
+
+6.2.1. NetProbe Results for Loss Threshold
+
+ In NetProbe, the loss threshold was implemented uniformly over all
+ packets as a post-processing routine. With the loss threshold set at
+ 3 seconds, all packets with one-way delay >3 seconds were marked
+ "Lost" and included in the Lost Packet list with their transmission
+ time (as required in Section 3.3 of [RFC2680]). This resulted in
+ 342 packets designated as lost in one of the test streams (with
+ average delay = 3.091 sec).
+
+6.2.2. Perfas+ Results for Loss Threshold
+
+ Perfas+ uses a fixed loss threshold, which was not adjustable during
+ this study. The loss threshold is approximately one minute, and
+ emulation of a delay of this size was not attempted. However, it is
+ possible to implement any delay threshold desired with a
+ post-processing routine and subsequent analysis. Using this method,
+ 195 packets would be declared lost (with average delay = 3.091 sec).
+
+6.2.3. Conclusions for Loss Threshold
+
+ Both implementations assume that any constant delay value desired can
+ be used as the loss threshold, since all delays are stored as a pair
+ <Time, Delay> as required in [RFC2680]. This is a simple way to
+ enforce the constant loss threshold envisioned in [RFC2680] (see
+ Section 2.8.2 of [RFC2680]). We take the position that the
+ assumption of post-processing is compliant and that the text of the
+ revision of RFC 2680 should be revised slightly to include this
+ point.
+
+6.3. One-Way Loss with Out-of-Order Arrival
+
+ Section 3.6 of [RFC2680] indicates, with a lowercase "must" in the
+ text, that implementations need to ensure that reordered packets are
+ handled correctly. In essence, this is an implied requirement
+ because the correct packet must be identified as lost if it fails to
+ arrive before its delay threshold under all circumstances, and
+ reordering is always a possibility on IP network paths. See
+ [RFC4737] for the definition of reordering used in IETF
+ standard-compliant measurements.
+
+ The netem emulator can produce packet reordering because each
+ packet's delay is drawn from an independent distribution. Here,
+ significant delay (2000 ms) and delay variation (1000 ms) were
+
+
+
+
+Ciavattone, et al. Informational [Page 17]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ sufficient to produce packet reordering. Using the procedure
+ described in Section 6.1, the netem emulator was set to introduce 10%
+ loss while reordering was present.
+
+ The tests described in this section used:
+
+ o IP header + payload = 64 octets
+
+ o Periodic sampling = 1 packet per second
+
+ o Test duration = 600 seconds (during May 2, 2011, EDT)
+
+ =======================================
+
+ > Y02aps_loss <- c(53,45,67,55) (NetProbe)
+ > Y02per_loss <- c(59,62,67,69) (Perfas+)
+ > Y02cross_loss_ADK <- adk.test(Y02aps_loss, Y02per_loss)
+ > Y02cross_loss_ADK
+ Anderson-Darling k-sample test.
+
+ Number of samples: 2
+ Sample sizes: 4 4
+ Total number of values: 8
+ Number of unique values: 7
+
+ Mean of Anderson Darling Criterion: 1
+ Standard deviation of Anderson Darling Criterion: 0.60978
+
+ T = (Anderson Darling Criterion - mean)/sigma
+
+ Null Hypothesis: All samples come from a common population.
+
+ t.obs P-value extrapolation
+ not adj. for ties 1.11282 0.11531 0
+ adj. for ties 1.19571 0.10616 0
+
+ Warning: At least one sample size is less than 5.
+ p-values may not be very accurate.
+ >
+
+ =======================================
+
+ The test results indicate that extensive reordering was present.
+ Both implementations capture the extensive delay variation between
+ adjacent packets. In NetProbe, packet arrival order is preserved in
+ the raw measurement files, so an examination of arrival packet
+ sequence numbers also reveals reordering.
+
+
+
+
+Ciavattone, et al. Informational [Page 18]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ Despite extensive continuous packet reordering present in the
+ transmission path, the distributions of loss counts from the two
+ implementations pass the ADK criterion at 95% = 1.960.
+
+6.4. Poisson Sending Process Evaluation
+
+ Section 3.7 of [RFC2680] indicates that implementations need to
+ ensure that their sending process is reasonably close to a classic
+ Poisson distribution when used. Much more detail on sample
+ distribution generation and Goodness-of-Fit testing is specified in
+ Section 11.4 of [RFC2330] and the Appendix of [RFC2330].
+
+ In this section, each implementation's Poisson distribution is
+ compared with an idealistic version of the distribution available in
+ the base functionality of the R-tool for Statistical Analysis [RTOOL]
+ and performed using the Anderson-Darling Goodness-of-Fit test package
+ (ADGofTest) [RADGOF]. The Goodness-of-Fit criterion derived from
+ [RFC2330] requires a test statistic value AD <= 2.492 for 5%
+ significance. The Appendix of [RFC2330] also notes that there may be
+ difficulty satisfying the ADGofTest when the sample includes many
+ packets (when 8192 were used, the test always failed, but smaller
+ sets of the stream passed).
+
+ Both implementations were configured to produce Poisson distributions
+ with lambda = 1 packet per second and to assign received packet
+ timestamps in the measurement application (above the UDP layer; see
+ the calibration results in Section 4 of [RFC6808] for error
+ assessment).
+
+6.4.1. NetProbe Results
+
+ Section 11.4 of [RFC2330] suggests three possible measurement points
+ to evaluate the Poisson distribution. The NetProbe analysis uses
+ "user-level timestamps made just before or after the system call for
+ transmitting the packet".
+
+ The statistical summary for two NetProbe streams is below:
+
+ =======================================
+
+ > summary(a27ms$s1[2:1152])
+ Min. 1st Qu. Median Mean 3rd Qu. Max.
+ 0.0100 0.2900 0.6600 0.9846 1.3800 8.6390
+ > summary(a27ms$s2[2:1152])
+ Min. 1st Qu. Median Mean 3rd Qu. Max.
+ 0.010 0.280 0.670 0.979 1.365 8.829
+
+ =======================================
+
+
+
+Ciavattone, et al. Informational [Page 19]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ We see that both of the means are near the specified lambda = 1.
+
+ The results of ADGoF tests for these two streams are shown below:
+
+ =======================================
+
+ > ad.test( a27ms$s1[2:101], pexp, 1)
+
+ Anderson-Darling GoF Test
+
+ data: a27ms$s1[2:101] and pexp
+ AD = 0.8908, p-value = 0.4197
+ alternative hypothesis: NA
+
+ > ad.test( a27ms$s1[2:1001], pexp, 1)
+
+ Anderson-Darling GoF Test
+
+ data: a27ms$s1[2:1001] and pexp
+ AD = 0.9284, p-value = 0.3971
+ alternative hypothesis: NA
+
+ > ad.test( a27ms$s2[2:101], pexp, 1)
+
+ Anderson-Darling GoF Test
+
+ data: a27ms$s2[2:101] and pexp
+ AD = 0.3597, p-value = 0.8873
+ alternative hypothesis: NA
+
+ > ad.test( a27ms$s2[2:1001], pexp, 1)
+
+ Anderson-Darling GoF Test
+
+ data: a27ms$s2[2:1001] and pexp
+ AD = 0.6913, p-value = 0.5661
+ alternative hypothesis: NA
+
+ =======================================
+
+ We see that both sets of 100 packets and 1000 packets from two
+ different streams (s1 and s2) all passed the AD <= 2.492 criterion.
+
+6.4.2. Perfas+ Results
+
+ Section 11.4 of [RFC2330] suggests three possible measurement points
+ to evaluate the Poisson distribution. The Perfas+ analysis uses
+ "wire times for the packets as recorded using a packet filter".
+
+
+
+Ciavattone, et al. Informational [Page 20]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ However, due to limited access at the Perfas+ side of the test setup,
+ the captures were made after the Perfas+ streams traversed the
+ production network, adding a small amount of unwanted delay variation
+ to the wire times (and possibly error due to packet loss).
+
+ The statistical summary for two Perfas+ streams is below:
+
+ =======================================
+
+ > summary(a27pe$p1)
+ Min. 1st Qu. Median Mean 3rd Qu. Max.
+ 0.004 0.347 0.788 1.054 1.548 4.231
+ > summary(a27pe$p2)
+ Min. 1st Qu. Median Mean 3rd Qu. Max.
+ 0.0010 0.2710 0.7080 0.9696 1.3740 7.1160
+
+ =======================================
+
+ We see that both of the means are near the specified lambda = 1.
+
+ The results of ADGoF tests for these two streams are shown below:
+
+ =======================================
+
+ > ad.test(a27pe$p1, pexp, 1 )
+
+ Anderson-Darling GoF Test
+
+ data: a27pe$p1 and pexp
+ AD = 1.1364, p-value = 0.2930
+ alternative hypothesis: NA
+
+ > ad.test(a27pe$p2, pexp, 1 )
+
+ Anderson-Darling GoF Test
+
+ data: a27pe$p2 and pexp
+ AD = 0.5041, p-value = 0.7424
+ alternative hypothesis: NA
+
+ > ad.test(a27pe$p1[1:100], pexp, 1 )
+
+ Anderson-Darling GoF Test
+
+ data: a27pe$p1[1:100] and pexp
+ AD = 0.7202, p-value = 0.5419
+ alternative hypothesis: NA
+
+
+
+
+Ciavattone, et al. Informational [Page 21]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ > ad.test(a27pe$p1[101:193], pexp, 1 )
+
+ Anderson-Darling GoF Test
+
+ data: a27pe$p1[101:193] and pexp
+ AD = 1.4046, p-value = 0.201
+ alternative hypothesis: NA
+
+ > ad.test(a27pe$p2[1:100], pexp, 1 )
+
+ Anderson-Darling GoF Test
+
+ data: a27pe$p2[1:100] and pexp
+ AD = 0.4758, p-value = 0.7712
+ alternative hypothesis: NA
+
+ > ad.test(a27pe$p2[101:193], pexp, 1 )
+
+ Anderson-Darling GoF Test
+
+ data: a27pe$p2[101:193] and pexp
+ AD = 0.3381, p-value = 0.9068
+ alternative hypothesis: NA
+
+ >
+
+ =======================================
+
+ We see that sets of 193, 100, and 93 packets from two different
+ streams (p1 and p2) all passed the AD <= 2.492 criterion.
+
+6.4.3. Conclusions for Goodness-of-Fit
+
+ Both NetProbe and Perfas+ implementations produce adequate Poisson
+ distributions according to the Anderson-Darling Goodness-of-Fit at
+ the 5% significance (1-alpha = 0.05, or 95% confidence level).
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 22]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+6.5. Implementation of Statistics for One-Way Loss
+
+ We check to see which statistics were implemented and report on those
+ facts, noting that Section 4 of [RFC2680] does not specify the
+ calculations exactly and only gives some illustrative examples.
+
+ NetProbe Perfas+
+
+ Type-P-One-way-Packet-Loss-Average yes yes
+ (this is more commonly referred
+ to as "loss ratio")
+
+ Implementation of RFC 2680 Section 4 Statistics
+
+ We note that implementations refer to this metric as a loss ratio,
+ and this is an area for likely revision of the text to make it more
+ consistent with widespread usage.
+
+7. Conclusions for a Revision of RFC 2680
+
+ This memo concludes that [RFC2680] should be advanced on the
+ Standards Track and recommends the following edits to improve the
+ text (which are not deemed significant enough to affect maturity).
+
+ o Revise Type-P-One-way-Packet-Loss-Ave to
+ Type-P-One-way-Delay-Packet-Loss-Ratio.
+
+ o Regarding implementation of the loss delay threshold
+ (Section 6.2), the assumption of post-processing is compliant, and
+ the text of the revision of RFC 2680 should be revised slightly to
+ include this point.
+
+ o The IETF has reached consensus on guidance for reporting metrics
+ [RFC6703], and this memo should be referenced in a revision of
+ RFC 2680 to incorporate recent experience where appropriate.
+
+ We note that there are at least two errata for [RFC2680], and it
+ appears that these minor revisions should be incorporated in a
+ revision of RFC 2680.
+
+ The authors that revise [RFC2680] should review all errata filed at
+ the time the document is being written. They should not rely upon
+ this document to indicate all relevant errata updates.
+
+ We recognize the existence of BCP 170 [RFC6390], which provides
+ guidelines for development of documents describing new performance
+ metrics. However, the advancement of [RFC2680] represents fine-
+ tuning of long-standing specifications based on experience that
+
+
+
+Ciavattone, et al. Informational [Page 23]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ helped to formulate BCP 170, and material that satisfies some of the
+ requirements of [RFC6390] can be found in other RFCs, such as the
+ IPPM Framework [RFC2330]. Thus, no specific changes to address
+ BCP 170 guidelines are recommended for a revision of RFC 2680.
+
+8. Security Considerations
+
+ The security considerations that apply to any active measurement of
+ live networks are relevant here as well. See [RFC4656] and
+ [RFC5357].
+
+9. Acknowledgements
+
+ The authors thank Lars Eggert for his continued encouragement to
+ advance the IPPM metrics during his tenure as AD Advisor.
+
+ Nicole Kowalski supplied the needed Customer Premises Equipment (CPE)
+ router for the NetProbe side of the test setup and graciously managed
+ her testing in spite of issues caused by dual-use of the router.
+ Thanks, Nicole!
+
+ The "NetProbe Team" also acknowledges many useful discussions on
+ statistical interpretation with Ganga Maguluri.
+
+ Constructive comments and helpful reviews were also provided by Bill
+ Cerveny, Joachim Fabini, and Ann Cerveny.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 24]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+10. Appendix - Network Configuration and Sample Commands
+
+ This Appendix provides some background information on the host
+ configuration and sample tc commands for the "netem" network
+ emulator, as described in Section 3 and Figure 1 of this memo. These
+ details are also applicable to the test plan in [RFC6808].
+
+ The host interface and configuration are shown below. Due to the
+ limit of 72 characters per line, line breaks were added to the "tc"
+ commands in the output below.
+
+ [system@dell4-4 ~]$ su
+ Password:
+ [root@dell4-4 system]# service iptables save
+ iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]
+ [root@dell4-4 system]# service iptables stop
+ iptables: Flushing firewall rules: [ OK ]
+ iptables: Setting chains to policy ACCEPT: nat filter [ OK ]
+ iptables: Unloading modules: [ OK ]
+ [root@dell4-4 system]# brctl show
+ bridge name bridge id STP enabled interfaces
+ virbr0 8000.000000000000 yes
+ [root@dell4-4 system]# ifconfig eth1.300 0.0.0.0 promisc up
+ [root@dell4-4 system]# ifconfig eth1.400 0.0.0.0 promisc up
+ [root@dell4-4 system]# ifconfig eth2.400 0.0.0.0 promisc up
+ [root@dell4-4 system]# ifconfig eth2.300 0.0.0.0 promisc up
+ [root@dell4-4 system]# brctl addbr br300
+ [root@dell4-4 system]# brctl addif br300 eth1.300
+ [root@dell4-4 system]# brctl addif br300 eth2.300
+ [root@dell4-4 system]# ifconfig br300 up
+ [root@dell4-4 system]# brctl addbr br400
+ [root@dell4-4 system]# brctl addif br400 eth1.400
+ [root@dell4-4 system]# brctl addif br400 eth2.400
+ [root@dell4-4 system]# ifconfig br400 up
+ [root@dell4-4 system]# brctl show
+ bridge name bridge id STP enabled interfaces
+ br300 8000.0002b3109b8a no eth1.300
+ eth2.300
+ br400 8000.0002b3109b8a no eth1.400
+ eth2.400
+ virbr0 8000.000000000000 yes
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 25]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ [root@dell4-4 system]# brctl showmacs br300
+ port no mac addr is local? ageing timer
+ 2 00:02:b3:10:9b:8a yes 0.00
+ 1 00:02:b3:10:9b:99 yes 0.00
+ 1 00:02:b3:c4:c9:7a no 0.52
+ 2 00:02:b3:cf:02:c6 no 0.52
+ 2 00:0b:5f:54:de:81 no 0.01
+ [root@dell4-4 system]# brctl showmacs br400
+ port no mac addr is local? ageing timer
+ 2 00:02:b3:10:9b:8a yes 0.00
+ 1 00:02:b3:10:9b:99 yes 0.00
+ 2 00:02:b3:c4:c9:7a no 0.60
+ 1 00:02:b3:cf:02:c6 no 0.42
+ 2 00:0b:5f:54:de:81 no 0.33
+ [root@dell4-4 system]# tc qdisc add dev eth1.300 root netem
+ delay 100ms
+
+ [root@dell4-4 system]# ifconfig eth1.200 0.0.0.0 promisc up
+ [root@dell4-4 system]# vconfig add eth1 100
+ Added VLAN with VID == 100 to IF -:eth1:-
+
+ [root@dell4-4 system]# ifconfig eth1.100 0.0.0.0 promisc up
+
+ [root@dell4-4 system]# vconfig add eth2 100
+ Added VLAN with VID == 100 to IF -:eth2:-
+
+ [root@dell4-4 system]# ifconfig eth2.100 0.0.0.0 promisc up
+ [root@dell4-4 system]# ifconfig eth2.200 0.0.0.0 promisc up
+ [root@dell4-4 system]# brctl addbr br100
+ [root@dell4-4 system]# brctl addif br100 eth1.100
+ [root@dell4-4 system]# brctl addif br100 eth2.100
+ [root@dell4-4 system]# ifconfig br100 up
+ [root@dell4-4 system]# brctl addbr br200
+ [root@dell4-4 system]# brctl addif br200 eth1.200
+ [root@dell4-4 system]# brctl addif br200 eth2.200
+ [root@dell4-4 system]# ifconfig br200 up
+ [root@dell4-4 system]# brctl show
+ bridge name bridge id STP enabled interfaces
+ br100 8000.0002b3109b8a no eth1.100
+ eth2.100
+ br200 8000.0002b3109b8a no eth1.200
+ eth2.200
+ br300 8000.0002b3109b8a no eth1.300
+ eth2.300
+ br400 8000.0002b3109b8a no eth1.400
+ eth2.400
+ virbr0 8000.000000000000 yes
+
+
+
+
+Ciavattone, et al. Informational [Page 26]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ [root@dell4-4 system]# brctl showmacs br100
+ port no mac addr is local? ageing timer
+ 2 00:02:b3:10:9b:8a yes 0.00
+ 1 00:02:b3:10:9b:99 yes 0.00
+ 1 00:0a:e4:83:89:07 no 0.19
+ 2 00:0b:5f:54:de:81 no 0.91
+ 2 00:e0:ed:0f:72:86 no 1.28
+ [root@dell4-4 system]# brctl showmacs br200
+ port no mac addr is local? ageing timer
+ 2 00:02:b3:10:9b:8a yes 0.00
+ 1 00:02:b3:10:9b:99 yes 0.00
+ 2 00:0a:e4:83:89:07 no 1.14
+ 2 00:0b:5f:54:de:81 no 1.87
+ 1 00:e0:ed:0f:72:86 no 0.24
+ [root@dell4-4 system]# tc qdisc add dev eth1.100 root netem
+ delay 100ms
+ [root@dell4-4 system]#
+
+ =====================================================================
+
+ Some sample tc command lines controlling netem and its impairments
+ are given below.
+
+ tc qdisc add dev eth1.100 root netem loss 0%
+ tc qdisc add dev eth1.200 root netem loss 0%
+ tc qdisc add dev eth1.300 root netem loss 0%
+ tc qdisc add dev eth1.400 root netem loss 0%
+
+ Add delay and delay variation:
+ tc qdisc change dev eth1.100 root netem delay 100ms 50ms
+ tc qdisc change dev eth1.200 root netem delay 100ms 50ms
+ tc qdisc change dev eth1.300 root netem delay 100ms 50ms
+ tc qdisc change dev eth1.400 root netem delay 100ms 50ms
+
+ Add delay, delay variation, and loss:
+ tc qdisc change dev eth1 root netem delay 2000ms 1000ms loss 10%
+
+ =====================================================================
+
+
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 27]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+11. References
+
+11.1. Normative References
+
+ [RFC2026] Bradner, S., "The Internet Standards Process --
+ Revision 3", BCP 9, RFC 2026, October 1996.
+
+ [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
+ Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+ [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
+ "Framework for IP Performance Metrics", RFC 2330,
+ May 1998.
+
+ [RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
+ Packet Loss Metric for IPPM", RFC 2680, September 1999.
+
+ [RFC3432] Raisanen, V., Grotefeld, G., and A. Morton, "Network
+ performance measurement with periodic streams", RFC 3432,
+ November 2002.
+
+ [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M.
+ Zekauskas, "A One-way Active Measurement Protocol
+ (OWAMP)", RFC 4656, September 2006.
+
+ [RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov,
+ S., and J. Perser, "Packet Reordering Metrics", RFC 4737,
+ November 2006.
+
+ [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
+ Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
+ RFC 5357, October 2008.
+
+ [RFC5657] Dusseault, L. and R. Sparks, "Guidance on Interoperation
+ and Implementation Reports for Advancement to Draft
+ Standard", BCP 9, RFC 5657, September 2009.
+
+ [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New
+ Performance Metric Development", BCP 170, RFC 6390,
+ October 2011.
+
+ [RFC6576] Geib, R., Morton, A., Fardid, R., and A. Steinmitz, "IP
+ Performance Metrics (IPPM) Standard Advancement Testing",
+ BCP 176, RFC 6576, March 2012.
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 28]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting
+ IP Network Performance Metrics: Different Points of View",
+ RFC 6703, August 2012.
+
+ [RFC6808] Ciavattone, L., Geib, R., Morton, A., and M. Wieser, "Test
+ Plan and Results Supporting Advancement of RFC 2679 on the
+ Standards Track", RFC 6808, December 2012.
+
+11.2. Informative References
+
+ [ADK] Scholz, F. and M. Stephens, "K-Sample Anderson-Darling
+ Tests of Fit, for Continuous and Discrete Cases",
+ University of Washington, Technical Report No. 81,
+ May 1986.
+
+ [ADV-METRICS]
+ Morton, A., "Lab Test Results for Advancing Metrics on the
+ Standards Track", Work in Progress, October 2010.
+
+ [FEDORA] "Fedora", <http://fedoraproject.org/>.
+
+ [LOSS-METRIC]
+ Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton,
+ Ed., "A One-Way Loss Metric for IPPM", Work in Progress,
+ July 2014.
+
+ [MII-TOOL]
+ Hinds, D., Becker, D., and B. Eckenfels, "Linux System
+ Administrator's Manual", February 2013,
+ <http://man7.org/linux/man-pages/man8/mii-tool.8.html>.
+
+ [NETEM] Linux Foundation, "netem",
+ <http://www.linuxfoundation.org/collaborate/workgroups/
+ networking/netem>.
+
+ [Perfas] Heidemann, C., "Qualitaet in IP-Netzen Messverfahren",
+ published by ITG Fachgruppe, 2nd meeting 5.2.3,
+ November 2001, <www.itg523.de/oeffentlich/01nov/
+ Heidemann_QOS_Messverfahren.pdf>.
+
+ [RADGOF] Bellosta, C., "ADGofTest: Anderson-Darling Goodness-of-Fit
+ Test. R package version 0.3.", R-Package Version 0.3,
+ December 2011, <http://cran.r-project.org/web/packages/
+ ADGofTest/index.html>.
+
+ [RADK] Scholz, F., "ADK: Anderson-Darling K-Sample Test and
+ Combinations of Such Tests. R package version 1.0.", 2008.
+
+
+
+
+Ciavattone, et al. Informational [Page 29]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+ [RFC3931] Lau, J., Townsley, M., and I. Goyret, "Layer Two Tunneling
+ Protocol - Version 3 (L2TPv3)", RFC 3931, March 2005.
+
+ [RTOOL] R Development Core Team, "R: A Language and Environment
+ for Statistical Computing", ISBN 3-900051-07-0, 2014,
+ <http://www.R-project.org/>.
+
+ [WIPM] AT&T, "AT&T Global IP Network", 2014,
+ <http://ipnetwork.bgtmo.ip.att.net/pws/index.html>.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 30]
+
+RFC 7290 Standards Track Tests for RFC 2680 July 2014
+
+
+Authors' Addresses
+
+ Len Ciavattone
+ AT&T Labs
+ 200 Laurel Avenue South
+ Middletown, NJ 07748
+ USA
+
+ Phone: +1 732 420 1239
+ EMail: lencia@att.com
+
+
+ Ruediger Geib
+ Deutsche Telekom
+ Heinrich Hertz Str. 3-7
+ Darmstadt 64295
+ Germany
+
+ Phone: +49 6151 58 12747
+ EMail: Ruediger.Geib@telekom.de
+
+
+ Al Morton
+ AT&T Labs
+ 200 Laurel Avenue South
+ Middletown, NJ 07748
+ USA
+
+ Phone: +1 732 420 1571
+ Fax: +1 732 368 1192
+ EMail: acmorton@att.com
+ URI: http://home.comcast.net/~acmacm/
+
+
+ Matthias Wieser
+ Technical University Darmstadt
+ Darmstadt
+ Germany
+
+ EMail: matthias_michael.wieser@stud.tu-darmstadt.de
+
+
+
+
+
+
+
+
+
+
+
+Ciavattone, et al. Informational [Page 31]
+