Wi-Fi metrics

The IEEE 802.11 Wireless Local Area Network (WLAN) technology, commonly known as Wi-Fi, has been steadily gaining popularity, keeping users on the go productively connected at airports, in hotels, and even in front of living-room TV sets. Most of us who have experienced the freedom and flexibility of wireless connectivity have no wish to return to tethered networking.

So far, wireless networking has been largely confined to the airport, hotel, coffee-shop, and small-office home-office (SOHO) markets, where performance is less important than cost. The new target for the wireless industry is the lucrative enterprise market, but those of us who yearn for 802.11 to take hold in the enterprise have to wait until the technology proves itself sufficiently robust and ready to carry demanding mission-critical applications. In enterprises, network performance impacts productivity, and IT executives won't make the decision to deploy 802.11 lightly.

Formal, standardized test procedures could help convince IT skeptics that 802.11 equipment can deliver the throughput, client capacity, security, and fault tolerance that corporate environments require. Recognizing that the definition of test methods and metrics is a timely and important undertaking, the IEEE 802 committee in July formed the IEEE 802.11T Task Group, which is charged with developing a test-specification document titled, "Recommended Practice for the Evaluation of 802.11 Wireless Performance."

Proper testing in accordance with this document will provide end users with an objective means of evaluating functionality and performance of 802.11 products, and it will help guide the Wi-Fi industry toward making 802.11 products faster, stronger, and more resilient to adverse network conditions—in short, ready to face the challenges and demands of the enterprise.

Originally touted as "wireless Ethernet," IEEE 802.11 (Ref. 1) evolved to be more elaborate than its IEEE 802.3 wireline counterpart. The inherent mobility and erratic transmission environment of Wi-Fi require new media access control (MAC) layer protocols that do not exist in the wired world. These protocols have added significantly to the number of test metrics that are needed to qualify a product's performance and behavior. A rough count shows that wireless-LAN metrics outnumber traditional Ethernet metrics 5 to 1.

Figure 1. Three measurements performed an open air (top) give diverging results, while the same measurements performed in a controlled and shielded RF environment (bottom) give identical results. A controlled RF environment can save time and facilitate results correlation among different test labs.

The testing of 802.11 devices and systems for performance and stability is a challenge for the industry. The 802.11 protocol's complexity brings with it a corresponding test complexity that is further compounded by the prevalence of RF interference and the inherent mobility of the wireless devices.

RF interference makes it difficult to obtain repeatable results that can be correlated among multiple laboratories. Interference, originating from sources including phones, microwave ovens, and adjacent wireless networks, can force the devices to retransmit or to continuously vary the data rate, thereby producing random results. Figure 1 provides examples of throughput measurements on data forwarded by an 802.11 access point (AP) onto the Ethernet interface.

You could combat interference by performing measurements in walk-in shielded EMI chambers, but such chambers are impractical for 802.11 roaming tests. Tests of mobility-related protocols, such as roaming and rate adaptation, must be performed while the device is in motion, often rolling on a cart in uncontrolled, interference-prone settings.

So how do you deal with mobility and interference when testing wireless systems? The 802.11T committee is considering several solutions. One is based on a cabled environment that uses programmable RF attenuators to emulate variable distances between devices.

In such an environment, each device in a test setup is placed in a shielded chamber for isolation. RF cables

Figure 2. RF cables connect to the 50-V antenna ports of the devices-under-test with antennas removed. Programmable RF attentuator settings emulate physical distances between devices.

Shielding and filtering protect the test setup from outside interference and achieve device-to-device isolation. Devices can connect through a network of attenuators and combiners to emulate complicated multi-BSS topologies. (A BSS, or Basic Service Set, is a network of one access point and the stations associated with it.)

The goal of the 802.11T project is to provide a set of performance metrics, measurement methodologies, and test conditions to enable manufacturers, test labs, service providers, and users to measure the performance of 802.11 WLAN devices and networks at the component and application level. The document developed by this group will supply IT managers with the same solid set of standard benchmarks that they have for Ethernet today—making it more likely that they will accept wireless connectivity for their enterprise networks.

The 802.11T committee plans to model its work after the IETF Requests for Comment (RFCs) 2285, 2544, and 2889 (Ref. 2), specifying metrics and methods for evaluating the performance of Ethernet switches. The author of two of these RFCs, Bob Mandeville of Iometrix, is actively working with the committee. Measurements such as throughput, packet loss, delay, and jitter can be based on the RFCs, but 802.11T will have to define new wireless-specific metrics.

Table 1 summarizes the wireless test metrics (such as packet forwarding rate, jitter, and rate vs. range) and the numerous test configurations (ranging from number of clients to physical layer rate selection) for each metric currently being studied by 802.11T. Adding mobility and interference considerations to this list will compound the difficulty of the group's task.

Active participants in the 802.11T committee include representatives of more than 16 companies, including Broadcom, Intel, Microsoft, Symbol, Texas Instruments, and the University of New Hampshire Interoperability Lab. The last face-to-face meeting of the 802.11T committee was September 12 in Berlin, Germany; the next will be held November 14–19 in San Antonio, TX.

The group also holds weekly teleconference sessions. Those wishing to join the teleconferences should send an e-mail request to Charles Wright, 802.11T committee chair and Azimuth Systems chief scientist: charles_wright@azimuth.net.