I. INTRODUCTION

The ADEC project presents unique challenges and opportunities in fulfilling its mission to bring distance education and Internet connectivity to both remote and financially disenfranchised sites. Bandwidth allocations, at first gleaning appear to be sufficient to support general Internet applications such as Web browsing, email, and file transfers. However, University of Nebraska network staff has reported IMAP performance problems. This may be partially an application related issue, however it raises serious questions about actual performance and general traffic behaviors in not only a bandwidth-constrained environment; it also has particular significance for real time applications.

UNL’s previous tests, has produced data in regards to application simulation. Our plan is to build upon it and provide very low level testing and analysis, focusing on isolation of packets by size and transmission rates, inter-packet delay variation. We are also going to begin with H.323 as a primer and move on to other applications such as the GRID, streaming video, VoIP, etc. Additionally, we also plan to collect the net-flow data from the ADEC router and perform statistical analysis on it.

Tachyon Net, the satellite services vendor, has already expressed an interest in collaborative QoS testing. It is important to define Quality of Service (QoS). Unfortunately, the term appears frequently in trade publications and often is defined as DiffServ or IntServ. It is important to note that these models are QoS solutions and fail to define QoS in terms of hard performance bounds, in fact DiffServ is expressed as strictly a Per Hop Behavior, so it literally is a relative treatment of traffic [1]. For example, the simple expression D(i) -E(i)<=S*R/MTU is used to describe a PHB [2]. D(i) is the departure rate and E(i) is the ingress rate. Stated simply, no queuing will occur as long as the departure rate is greater than the arrival rate. DiffServ suggests a variety of queuing strategies but focuses on using bit marking as the mechanism to allow a network device to treat packet forwarding in a particular way. So it is literally true that a given packet could meet the specification in degenerate cases, and still fail to meet an application’s performance requirements.

The Y.1541 draft standard offers as an alternative expected hard performance bounds for servicing various classes of traffic [3]. For example, Class 0 approximates the Expedited Forwarding class in RFC 2598 and gives a UBR like definition for unconstrained best effort traffic.

II. Definitions

For purposes of clarity the following definitions will be adopted from the ITU Y.1541 draft Standard.

IPTD - ip packet transfer delay commonly referred to as latency.

IPDV- ip packet delay variation.

IPLR – ip packet loss ratio.

IPER – ip packet error ratio.

IPPT – ip packet through put.

IPOT – Octet based IP packet throughput.

And the general convention of BPS meaning Bits Per Second.

III. Bounds

The following table from Y.1541 permits general assumptions about traffic requirements and the network’s ability to meet them. Testing using various application traffic models will help verify or revise those assumptions.

Note: blue is for total of network sections.

IV. Goals

The testing program proposed here has the following goals and hopes to definitively answer at least some of the following questions.

1. What is the total effective throughput of the system? Does it reasonably meet the stated bandwidth?
2. What are the inherent performance metrics for IPTD, IPDV, IPLR, and IPER under various loads?
3. In light of the previous two questions is it reasonable to assume the network can support real time or near real time applications?
4. To improve the performance of real time applications such as H.323 voice and video what are the strategies that will enhance the probability of success? Are technical solutions such as CBWFQ, PQ, or WRR appropriate or will voluntary compliance be required? The latter assumes a resource poor environment in hardware, software and technically proficient personnel at many CPEs.
5. In light of question #4 can Tachyon use traffic prioritization mechanisms to over come CPE limitations? If these can be implemented is DiffServ or some of its primitives the appropriate solution?

The test protocol is a simple duplex approach. First, evaluate at the network layer (layers 1through 3 of the ISO model) if the performance of the service is acceptable to assume the usability a particular application. Secondly, test the application. The purpose of this approach is to separate performance issues into the appropriate layer for evaluation and trouble shooting in the context of a known network performance base line.

V. Scope

Clearly, as a matter of practicality the scope of this testing cannot extend beyond the demarcation bounds that ADEC has immediate control over. The tests are proposed to take place between ITEC-OHIO and the San Diego Super Computer using Smartbits testing apparati. The test apparatus on the ITEC side will be directly connected to a LAN Ethernet switch attached to the CPE Linux platform. The San Diego side can be attached minimally to the Cisco 7140 and ideally to the Tachyon handoff switch. Additionally, both points could be used if time and significant information might be obtained. See diagram 1.

VI. Tests

A: Up Link Down Link Through put test

A:1 Test Objective

Find the maximum bandwidth performance bounds.

A:2 Entry Criteria:

None.

A:3 Test Setup:

1. Synchronize GPS units (only necessary for data correlation in this test).
2. Set Smartbits at SDSC to send a 2 Mbps stream to ITEC-Ohio CPE. Set Smartbits at ITEC-Ohio to send 512Kbps stream.

A:4 Test Execution:

1. Send the 2 Mbps from SDSC to Ohio for 60 seconds.
2. After completion send 512 Kbps stream from Ohio to SDSC.
3. Repeat test with both streams simultaneously.
4. Repeat both tests by stepping down traffic sent by approximately 1% increments until a reliable bound is found.
5. Record IPTD after bound is established.

A:5 Expected results.

It is expected that there will be a reasonably good efficiency coefficient. For example, 100 Mbps ethernet cannot be any more than 96% under ideal circumstances. It is expected that the satellite service will provide efficiency equal to or greater than 90%.

B: IPDV under various loads test.

B:1 Test objective:

Find IPDV characteristics under various loads and packet size.

B:2 Entry Criteria:

Test A has been completed.

B:3 Test Setup.

1. Same hardware and GPS configuration as (A).
2. Setup Smartbits in SDSC to send the following loads 64, 128, 256, 512, 1024, and

maximum bound found in A. Each load should be configured in each test to use 64, 128, 256, 512, 1024 and 1518 byte packets.

3. Ohio should set up to send the following loads 64, 128, 256 and highest maximum

load found in A. Each load should be sent with same packet sizes as SDSC.

B:4 Test Execution:

1. Send each load from SDSC to Ohio with at least one of each packet size for 60 seconds.
2. Send each load from Ohio to SDSC with least one of each packet size for 60 seconds.
3. Repeat with bi-directional streams.
4. Note the following for each category: IPTD, IPDV, IPLR, IPER and ( IPPT or IPOT or BPS or MBPS).

B:5 Expected Results.

IPTD <= 400ms and IPDV <= 50 ms. No other assumptions regarding outcomes is made.

C: Real Time Performance Test Under Load.

C:1 Test Objective:

The objective of this test is to see if Tachyon can provide a preferential treatment for real time applications while the link is under load.

C:2 Entry Criteria:

Tests A and B have completed. The results of test B indicate there is a reasonable probability of success for a Real Time application to function.

C:3 Test Setup:

1. Configure Smartbits at SDSC and Ohio to send multiple streams. One stream is the preferred stream. In each iteration, there will be either 64 or 384 Kbps streams.

1. All packets in the preferred stream will have the DSCP field marked to decimal 46.
2. All packets in the best effort stream will have the DSCP field marked to decimal 0.
3. Tachyon will configure satellite links to show preference to marked packets using mechanisms yet to be determined.

C:4 Test Execution

1. Send streams uni-directionally and repeat with bi-directional streams.
2. Perform tests with multiple packet sizes 64, 128, 256, 512, 1024, and 1518 byte packets.
3. Note IPTD, IPDV, IPLR and IPER for each stream and packet size.

C:5 Expected results

It is expected that preferred packets will have a lower IPTD, IPLR and IPER than BE packets. However, it is expected that IPDV may be worse for preferred traffic if a strict priority queuing mechanism is used. If a CBWFQ mechanism is used it is expected that small packets will be similar to PQ behaviors but 512 byte packets and above will exhibit a more homogenous IPDV as noted in Ferarri’s research [4].

VII. Conclusion

As stated earlier, the overall objective of this test plan is to help establish network performance base lines to help network engineers make decisions regarding appropriate applications for use in ADEC. Additionally, the information gained may help in trouble shooting in terms of examining all facets of a performance problem. For example, if an application basically meets the criteria for a particular class of service one may want to look at other factors than the network.

To take full advantage of this plan it should be followed up with testing applications that should be suitable for use in ADEC.

[1] V. Jacobson, K. Nichols, and K. Poduri. RFC 2598: An expedited forwarding PBH,

June 1999.

[2] G. Armitage, A. Casati, J. Crowcroft, J. Halpern, B. Kumar, J. Schnizlein. <draft-ietf-diffserv-efresolve-00.txt>: A revised expression of the Expedited Forwarding PHB. November 2000.

[3] ITU Study Group 13, Revised draft Recommendation Y. 1541 'Internet protocol

communication service - IP Performance and Availability Objectives and Allocations', November 2000.

[4] Tizana Ferrari, End-to-end performance analysis with traffic aggregation.

Computer Networks, Volume 34 Number 6, December 2000. pp 905-914. Elesevier, Amsterdam.

"This material is based upon work supported by the National Science Foundation under Grant No. 0073240. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation."