Loss Episode Metrics for IPPM

Packet loss in the Internet is a complex phenomenon due to the bursty nature of traffic and congestion processes, influenced by both end-users and applications, and the operation of transport protocols such as TCP. For these reasons, the simplest model of packet loss--the single parameter Bernoulli (independent) loss model--does not represent the complexity of packet loss over periods of time. Correspondingly, a single loss metric--the average packet loss ratio over some period of time--arising, e.g., from a stream of Poisson probes as in is not sufficient to determine the effect of packet loss on traffic in general. Moving beyond single parameter loss models, Markovian and Markov-modulated loss models involving transitions between a good and bad state, each with an associated loss rate, have been proposed by Gilbert and more generally by Elliot . In principle, Markovian models can be formulated over state spaces involving patterns of loss of any desired number of packets. However further increase in the size of the state space makes such models cumbersome both for parameter estimation (accuracy decreases) and prediction in practice (due to computational complexity and sensitivity to parameter inaccuracy). In general, the relevance and importance of particular models can change in time, e.g. in response to the advent of new applications and services. For this reason we are drawn to empirical metrics that do not depend on a particular model for their interpretation. An empirical measure of packet loss complexity, the index of dispersion of counts (IDC), comprise, for each t >0, the ratio v(t) / a(t) of the variance v(t) and average a(t) of the number of losses over successive measurement windows of a duration t. However, a full characterization of packet loss over time requires specification of the IDC for each window size t>0. In the standards arena, loss pattern sample metrics are defined in . Following the Gilbert-Elliot model, burst metrics specific for VoIP that characterize complete episodes of lost, transmitted and discarded packets are defined in All these considerations motivate formulating empirical metrics of one-way packet loss that provide the simplest generalization of the successful that can capture deviations from independent packet loss in a robust model-independent manner, and, to define efficient measurement methodologies for these metrics.

The losses experienced by the packet stream can be viewed as occurring in loss episodes, i.e., maximal set of consecutively lost packets. This memo describes one-way loss episode metrics: their frequency and average duration. Although the average loss ratio can be expressed in terms of these quantities, they go further in characterizing the statistics of the patterns of packet loss within the stream of probes. This is useful information in understanding the effect of packet losses on application performance, since different applications can have different sensitivities to patterns of loss, being sensitive not only to the long term average loss rate, but how losses are distributed in time. As an example: MPEG video traffic may be sensitive to loss involving the I-frame in a group of pictures, but further losses within an episode of sufficiently short duration have no further impact; the damage is already done. The loss episode metrics presented here have the following useful properties: the metrics are empirical and do not depend on an underlying model; e.g., the loss process is not assumed to be Markovian. On the other hand, it turns out that the metrics of this memo can be related to the special case of the Gilbert Model parameters; see Section 7. the metric units can be directly compared with applications or user requirements or tolerance for network loss performance, in the frequency and duration of loss episodes, as well as the usual packet loss ratio, which can be recovered from the loss episode metrics upon dividing the average loss episode duration by the loss episode frequency. the metrics provide the smallest possible increment in complexity beyond, but in the spirit of, the IPPM average packet loss ratio metrics i.e., moving from a single metric (average packet loss ratio) to a pair of metrics (loss episode frequency and average loss episode duration). The document also describes a probing methodology under which loss episode metrics are to be measured. The methodology comprises sending probe packets in pairs, where packets within each probe pair have a fixed separation, and the time between pairs takes the form of a geometric distributed number multiplied by the same separation. This can be regarded a generalization of Poisson probing where the probes are pairs rather than single packets as in , and also of geometric probing described in . However, it should be distinguished from back to back packet pairs whose change in separation on traversing a link is used to probe bandwidth. In this document, the separation between the packets in a pair is the temporal resolution at which different loss episodes are to be distinguished. The methodology does not measure episodes of loss of consecutive background packets on the measured path. One key feature of this methodology is its efficiency: it estimates the average length of loss episodes without directly measuring the complete episodes themselves. Instead, this information is encoded in the observed relative frequencies of the 4 possible outcomes arising from the loss or successful transmission of each of the two packets of the probe pairs. This is distinct from the approach of that reports on directly measured episodes. The metrics defined in this memo are "derived metrics", according to Section 6.1 of the IPPM framework. They are based on the singleton loss metric defined in Section 2 of .

Section 2 defines the fundamental singleton metric for the possible outcomes of a probe pair: Type-P-One-way-Bi-Packet-Loss. Section 3 defines sample sets of this metric derived from a general probe stream: Type-P-One-way-Bi-Packet-Loss-Stream. Section 4 defines the prime example of the Bi-Packet-Loss-Stream metrics, specifically Type-P-One-way-Bi-Packet-Loss-Geometric-Stream arising from the geometric stream of packet-pair probes that was described informally in Section 1. Section 5 defines Loss episode proto-metrics that summarize the outcomes from a stream metrics as an intermediate step to forming the loss episode metrics; they need not be reported in general. Section 6 defines the final loss episode metrics that are the focus of this memo, the new metrics Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration, the average duration, in seconds, of a loss episode Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency, the average frequency, per second, at which loss episodes start. Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio, which is the average packet loss ratio metric arising from the geometric stream probing methodology Section 7 details applications and relations to existing loss models.

Type-P-One-way-Bi-Packet-Loss

Src, the IP address of a source host Dst, the IP address of a destination host T1, a sending time of the first packet T2, a sending time of the second packet, with T2>T1 F, a selection function defining unambiguously the two packets from the stream selected for the metric. P, the specification of the packet type, over and above the source and destination addresses

A Loss Pair is pair (l1, l2) where each of l1 and l2 is a binary value 0 or 1, where 0 signifies successful transmission of a packet and 1 signifies loss. The metric unit of Type-P-One-way-Bi-Packet-Loss is a Loss Pair.

"The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst, T1, T2, F, P) is (1,1)" means that Src sent the first bit of a Type-P packet to Dst at wire-time T1 and the first bit of a Type-P packet to Dst at wire-time T2>T1, and that neither packet was received at Dst. The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst, T1, T2, F, P) is (1,0)" means that Src sent the first bit of a Type-P packet to Dst at wire-time T1 and the first bit of a Type-P packet to Dst at wire-time T2>T1, and that the first packet was not received at Dst, and the second packet was received at Dst The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst, T1, T2, F, P) is (0,1)" means that Src sent the first bit of a Type-P packet to Dst at wire-time T1 and the first bit of a Type-P packet to Dst at wire-time T2>T1, and that the first packet was received at Dst, and the second packet was not received at Dst The Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst, T1, T2, F, P) is (0,0)" means that Src sent the first bit of a Type-P packet to Dst at wire-time T1 and the first bit of a Type-P packet to Dst at wire-time T2>T1, and that both packets were received at Dst.

The purpose of the selection function is to specify exactly which packets are to be used for measurement. The notion is taken from Section 2.5 of , where examples are discussed.

The methodologies related to the Type-P-One-way-Packet-Loss metric in Section 2.6 of are similar for the Type-P-One-way-Bi-Packet-Loss metric described above. In particular, the methodologies described in RFC 2680 apply to both packets of the pair.

Sources of error for the Type-P-One-way-Packet-Loss metric in Section 2.7 of apply to each packet of the pair for the Type-P-One-way-Bi-Packet-Loss metric.

Refer to Section 2.8 of .

Given the singleton metric for Type-P-One-way-Bi-Packet-Loss, we now define examples of samples of singletons. The basic idea is as follows. We first specify a set of times T1 < T2 <...<Tn, each of which acts as the first time of a packet pair for a single Type-P-One-way-Bi-Packet-Loss measurement. This results is a set of n metric values of Type-P-One-way-Bi-Packet-Loss.

Type-P-One-way-Bi-Packet-Loss-Stream

Src, the IP address of a source host Dst, the IP address of a destination host (T11,T12), (T21,T22)....,(Tn1,Tn2) a set of n times of sending times for packet pairs, with T11 < T12 <= T21 < T22 <=...<= Tn1 < Tn2 F, a selection function defining unambiguously the two packets from the stream selected for the metric. P, the specification of the packet type, over and above the source and destination address

A set L1,L2,...,Ln of loss pairs

Each loss pair Li for i-1,....n is the Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst, Ti1, Ti2, Fi, P) where Fi is the restriction of the selection function F to the packet pair at time Ti1, Ti2.

The metric definition of Type-P-One-way-Bi-Packet-Loss-Stream is sufficiently general to describe the case where packets are sampled from a pre-existing stream. This is useful in the case that there is a general purpose measurement stream setup between two hosts, and we wish to select a substream from it for the purposes of loss episode measurement. Packet pairs selected as bi-packet loss probes need not be consecutive within such a stream. In the next section we specialize this somewhat to more concretely describe a purpose built packet stream for loss episode measurement.

The methodologies related to the Type-P-One-way-Packet-Loss metric in Section 2.6 of are similar for the Type-P-One-way-Bi-Packet-Loss-Stream metric described above. In particular, the methodologies described in RFC 2680 apply to both packets of each pair.

Sources of error for the Type-P-One-way-Packet-Loss metric in Section 2.7 of apply to each packet of each pair for the Type-P-One-way-Bi-Packet-Loss-Stream metric.

Refer to Section 2.8 of .

This section specializes the preceding section for an active probing methodology. The basic idea is a follows. We set up a sequence of evenly spaced times T1 < T2 < ... < Tn. Each time Ti is potentially the first packet time for a packet pair measurement. We make an independent random decision at each time, whether to initiate such a measurement. Hence the interval count between successive times at which a pair is initiated follows a geometric distribution. We also specify that the spacing between successive times Ti is the same as the spacing between packets in a given pair. Thus if pairs happen to be launched at the successive times Ti T(i+1), the second packet of the first pair is actually used as the first packet of the second pair.

Type-P-One-way-Bi-Packet-Loss-Geometric-Stream

Src, the IP address of a source host Dst, the IP address of a destination host T0, the randomly selected starting time for periodic launch opportunities d, the time spacing between potential launch times, Ti and Ti+1 n, a count of potential measurement instants q, a launch probability F, a selection function defining unambiguously the two packets from the stream selected for the metric. P, the specification of the packet type, over and above the source and destination address

A set of Loss Pairs L1, L2, ..., Lm for some m <= n

for each i = 0, 1, ..., n-1 we form the potential measurement time Ti = T + i * d. With probability q, a packet pair measurement is launched at Ti, resulting in a Type-P-One-way-Bi-Packet-Loss with parameters (Src, Dst, Ti, Ti+1, Fi, P) where Fi is the restriction of the selection function F to the packet pair at times Ti, Ti+1. L1, L2,...Lm are the resulting Loss Pairs; m can be less than n since not all time Ti have an associated measurement.

The above definition of Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is equivalent to using Type-P-One-way-Bi-Packet-Loss-Stream with an appropriate statistical definition of the selection function F. The number m of loss pairs in the metric can be less than the number of potential measurement instants because not all instants may generate a probe when the launch probability q is strictly less than 1.

The methodologies follow from: the specific time T0, from which all successive Ti follow, and the specific time spacing, and the methodologies discussion given above for the singleton Type-P-One-way-Bi-Packet-Loss metric. The issue of choosing an appropriate time spacing (e.g., one that is matched to expected characteristics of loss episodes) is outside the scope of this document. Note that as with any active measurement methodology, consideration must be made to handle out-of-order arrival of packets; see also Section 3.6. of .

In addition to sources of errors and uncertainties related to methodologies for measuring the singleton Type-P-One-way-Bi-Packet-Loss metric, a key source of error when emitting packets for Bi-Packet Loss relates to resource limits on the host used to send the packets. In particular, the choice of T0, the choice of the time spacing, and the choice of the launch probability results in a schedule for sending packets. Insufficient CPU resources on the sending host may result in an inability to send packets according to schedule. Note that the choice of time spacing directly affects the ability of the host CPU to meet the required schedule (e.g., consider a 100 microsecond spacing versus a 100 millisecond spacing). For other considerations, refer to Section 3.7. .

Refer to Section 3.8. of .

This section describes four generic proto-metric quantities associated with an arbitrary set of loss pairs. These are the Loss-Pair-Counts, Bi-Packet-Loss-Ratio, Bi-Packet-Loss-Episode-Duration-Number, Bi-Packet-Loss-Episode-Frequency-Number. Specific loss episode metrics can then be constructed when these proto metrics take as their input, sets of loss pairs samples generated by the Type-P-One-way-Bi-Packet-Loss-Stream and Type-P-One-way-Bi-Packet-Loss-Geometric Stream. The second of these is described in . It is not expected that these proto-metrics would be reported themselves. Rather they are intermediate quantities in the production of the final metrics of Section 6 below, and could be rolled up into them in implementations. The metrics report loss episode durations and frequencies in terms of packet counts, since they do not depend on the actual time between probe packets. The final metrics of Section 6 incorporate timescales and yield durations in seconds, and frequencies as per second.

Loss-Pair-Counts are the absolute frequencies of the 4 types of loss pair outcome in a sample. More precisely, the Loss-Pair-Counts associated with a set of loss pairs L1,,,,Ln are the numbers N(i,j) of such loss pairs that take each possible value (i,j) in the set ( (0,0), (0,1), (1,0), (1,1)).

The Bi-Packet-loss-ratio associated with a set of n loss pairs L1,,,,Ln is defined in terms of their Loss-Pair-Counts by the quantity (N(1,0) +N(1,1))/n. Note this is formally equivalent to the loss metric Type-P-One-way-Packet-Loss-Average from since it averages single packet losses.

The Bi-Packet-Loss-Episode-Duration-Number associated with a set of n loss pairs L1,,,,Ln is defined in terms of their Loss-Pair-Counts in the following cases: 2*(N(0,1) + N(1,0) + N(1,1)/ (N(0,1)+N(1,0)) - 1 if N(0,1) + N(1,0) >1 0 if N(0,1) + N(1,0) + N(1,1) = 0 (no probe packets lost) Undefined if N(0,1) + N(1,0) + N(0,0) = 0 (all probe packets lost) Note N(0,1) + N(1,0) is zero if there are no transitions between loss and no-loss outcomes.

The Bi-Packet-Loss-Episode-Frequency-Number associated with a set of n loss pairs L1,,,,Ln is defined in terms of their Loss-Pair-Counts as Bi-Packet-Loss-Ratio / Bi-Packet-Loss-Episode-Duration-Number, when this can be defined, specifically, it is: (N(1,0)+N(1,1)) * (N(0,1)+N(1,0)) / (2*N(1,1)+N(0,1)+N(1,0) ) / n if N(0,1)+N(0,1) > 0 0 if N(0,1)+N(1,0) +N(1,1) = 0 (no probe packets lost) 1 if N(0,1) +N(1,0) +N(0,0) = 0 (all probe packets lost)

Metrics for the time frequency and time duration of loss episodes are now defined as functions of set of n loss pairs L1,....,Ln. Although a loss episode is defined as a maximal set of successive lost packets, the loss episode metrics are not defined directly in terms of the sequential patterns of packet loss exhibited by loss pairs. This is because samples, including Type-P-One-way-Bi-Packet-Loss-Geometric-Stream, generally do not report all lost packets in each episode. Instead, the metrics are defined as functions of the Loss-Pair-Counts of the sample, for reasons that are now described. Consider an idealized Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample in which the launch probability q =1. It is shown in that the average number of packets in a loss episode of this ideal sample is exactly the Bi-Packet-Loss-Episode-Duration derived from its set of loss pairs. Note this computation makes no reference to the position of lost packet in the sequence of probes. A general Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with launch probability q < 1, independently samples, with probability q, each loss pair of an idealized sample. On average, the Loss-Pair-Counts (if normalized by the total number of pairs) will be the same as in the idealized sample. The loss episode metrics in the general case are thus estimators of those for the idealized case; the statistical properties of this estimation, including a derivation of the estimation variance, is provided in .

Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio

A decimal number in the interval [0,1]

The result obtained by computing the Bi-Packet-Loss-Ratio over a Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with the metric parameters.

Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio estimates the fraction of packets lost from the geometric stream of Bi-Packet probes.

Refer to

Because Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is sampled in general (when the launch probability q <1) the metrics described in this Section can be regarded as statistical estimators of the corresponding idealized version corresponding to q = 1. Estimation variance as it applies to Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Loss-Ratio is described in . For other issues refer to

Refer to

Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration

A non-negative number of seconds.

The result obtained by computing the Bi-Packet-Loss-Episode-Duration-Number over a Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with the metric parameters, then multiplying the result by the launch spacing parameter d.

Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration estimates the average duration of a loss episode, measured in seconds. The duration measured in packets is obtained by dividing the metric value by the packet launch spacing parameter d.

Refer to

Because Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is sampled in general (when the launch probability q <1) the metrics described in this Section can be regarded as statistical estimators of the corresponding idealized version corresponding to q = 1. Estimation variance as it applies to Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration is described in . For other issues refer to

Refer to

Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency

A positive number.

The result obtained by computing the Bi-Packet-Loss-Episode-Frequency over a Type-P-One-way-Bi-Packet-Loss-Geometric-Stream sample with the metric parameters, then dividing the result by the launch spacing parameter d.

Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency estimates the average frequency per unit time with which loss episodes start (or finish). The frequency relative to the count of potential probe launches is obtained by multiplying the metric value by the packet launch spacing parameter d.

Refer to

Because Type-P-One-way-Bi-Packet-Loss-Geometric-Stream is sampled in general (when the launch probability q <1) the metrics described in this Section can be regarded as statistical estimators of the corresponding idealized version corresponding to q = 1. Estimation variance as it applies to Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Frequency is described in . For other issues refer to

Refer to

The general Gilbert-Elliot model is a discrete time Markov chain over two states, Good (g) and Bad (b), each with its own independent packet loss rate. In the simplest case, the Good loss rate is 0 while the Bad loss rate is 1. Correspondingly, there are two independent parameters, the Markov transition probabilities P(g|b) = 1- P(b|b) and P(b|g) = 1- P(g|g), where P(i|j) is the probability to transition from state j and step n to state i at step n+1. With these parameters, the fraction of steps spent in the bad state is P(b|g)/(P(b|g) + P(g|b)) while the average duration of a sojourn in the bad state is 1/P(g|b) steps. Now identify the steps of the Markov chain with the possible sending times of packets for a Type-P-One-way-Bi-Packet-Loss-Geometric-Stream with launch spacing d. Suppose the loss episode metrics Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Ratio and Type-P-One-way-Bi-Packet-Loss-Geometric-Stream-Episode-Duration take the values r and m respectively. Then from the discussion in Section 6.2.5 the following can be equated: r = P(b|g)/(P(b|g) + P(g|b)) and m/d = 1/P(g|b). These relationships can be inverted in order to recover the Gilbert model parameters: P(g|b) = d/m and P(b|g)=d/m/(1/r - 1)

An IPR disclosure concerning some of the material covered in this document has been made to the IETF: see https://datatracker.ietf.org/ipr/1354/

Conducting Internet measurements raises both security and privacy concerns. This memo does not specify an implementation of the metrics, so it does not directly affect the security of the Internet nor of applications which run on the Internet. However,implementations of these metrics must be mindful of security and privacy concerns. There are two types of security concerns: potential harm caused by the measurements, and potential harm to the measurements. The measurements could cause harm because they are active, and inject packets into the network. The measurement parameters MUST be carefully selected so that the measurements inject trivial amounts of additional traffic into the networks they measure. If they inject "too much" traffic, they can skew the results of the measurement, and in extreme cases cause congestion and denial of service. The measurements themselves could be harmed by routers giving measurement traffic a different priority than "normal" traffic, or by an attacker injecting artificial measurement traffic. If routers can recognize measurement traffic and treat it separately, the measurements may not reflect actual user traffic. If an attacker injects artificial traffic that is accepted as legitimate, the loss rate will be artificially lowered. Therefore, the measurement methodologies SHOULD include appropriate techniques to reduce the probability that measurement traffic can be distinguished from "normal" traffic. Authentication techniques, such as digital signatures, may be used where appropriate to guard against injected traffic attacks. The privacy concerns of network measurement are limited by the active measurements described in this memo: they involve no release of user data.

This document requests no actions from IANA.