Network Test :: Benchmarking Services Network Test Methodologies::Results Contact Information About Network Test

Network World Lab Test: Wireless LAN Switches

Scheduled for Publication in September 2003

Test Methodology

 

v. 2.0 Copyright 2003 by Network Test. Vendors are encouraged to comment on this document and any other aspect of test methodology. Network Test reserves the right to change the parameters of this test at any time

 

Please send comments to David Newman of Network Test (dnewman at networktest.com)

 

1         Executive Summary

This document describes the test procedures we use to assess the emerging class of wireless LAN switches.

 

We plan tests of device capabilities on four areas:

 

 

For the provisioning and management tests, we ask vendors to complete a ďmini-RFP.Ē Written responses and a walk-through of system features will help determining scoring in this area.

 

1.1      Organization of this document

This document is organized as follows. This section introduces the project. Section 2 describes product requirements. Section 3 describes test procedures. Section 4 logs changes to this document.

 

2         Product requirements

Participating vendors must supply the following:

 

One wireless LAN switch

At least four 802.11b access points

Provisioning and management software, if necessary

 

3         Test procedures

 

3.1      Provisioning and management

 

We ask each participating vendor to prepare a written response to the following mini-RFP. We also ask each vendor to walk us through the salient features of its proposal.

 

While we do use subjective scoring in this section, we also gather a large amount of data to compare system feature sets. Preparing a product taxonomy is a major goal of this project.

 

3.1.1      Mini-RFP

Ohshutup.com is a leader in cell-phone jamming technology with more than 20,000 employees around the world. The company is opening a small branch location in an office park in Westlake Village, CA. This 1,200-square-foot office suite will accommodate members of the companyís research, marketing, and finance teams.

 

Each of the three groups requires aggregate bandwidth of at least 5 Mbit/s. For security policy reasons, Ohshutup.com requires that different groupsí traffic run on physically diverse networks wherever possible. Separate access point(s) for each group may be desirable.

 

This policy of separation of workgroups extends to the companyís wired infrastructure as well. Branch sites (as well as departments or floors at headquarters) use layer-2 workgroup switches with 802.1q VLAN tagging to identify workgroups. The VLAN tags in turn are mapped to different IP subnets on aggregation switch/routers, with different IP subnets for each division.

 

The company currently uses internal firewalls between its various departments. If possible, traffic from wireless users should be subject to the same firewall rules as other users.

 

The company also uses IPSec gateways to authenticate and encrypt traffic from telecommuters and road warriors. IPSec VPN access for wireless users is highly desirable, if available. In addition, the company currently uses a Windows 2000 Radius server for layer-2 authentication of users, and this server can easily be upgraded to allow for 802.1x PEAP authentication.

 

Whether these security features are implemented with a new or existing firewall and/or VPN gateway is unimportant.

 

Unauthorized access is another security concern. Ohshutup.comís employees may attempt to deploy unauthorized access points, and other companies in the office park may have their own WLAN equipment. For both reasons, rogue access point detection and adaptation are both highly desirable.

 

Although the company uses its network only for data traffic today, it is seriously considering adaptation of voice-over-IP telephony. The WLAN infrastructure should demonstrate the ability to prioritize voice traffic and, if possible, enforce latency and jitter boundaries.

 

Ohshutup.com relies on Openview to manage virtually any device that supports SNMP. It expects to be able to do the same with all WLAN infrastructure, including clients, access points, and switches.

 

Ohshutup.comís current fleet of notebook computers and PDAs are equipped with 802.11b radios. The company is very interested in migrating to 802.11a and/or 802.11g within one to two years. However, depreciation schedules for existing equipment dictate that 802.11b connectivity is a must for the near term.

 

Please demonstrate how your product will address each of these requirements and provide an itemized cost breakout for the total solution.

 

3.2      Performance testing

We assess performance with measurements of forwarding, delay, jitter, and failover delay.

 

3.2.1      Forwarding rates

We use Netwarrior, a Linux-based traffic generator/analyzer from QOSmetrix, to send traffic from one server on a wired segment to client(s) on a wireless segment.

 

The Netwarrior machines are equipped with GPS and special software to ensure precise measurements of delay and jitter. Timestamp resolution is accurate to within 40 nanoseconds.

 

The Netwarrior systems run Red Hat Linux (2.4.20 kernel). They are equipped with Netgear MA311 cards and the linux-wlan-ng driver.

 

We test performance in four configurations:

 

3.2.1.1       1 client, 1 AP

We run four tests: UDP unidirectional and bidirectional, and TCP unidirectional and bidirectional. In all cases, the test duration is 300 seconds and the IP packet length is 40 bytes. We report forwarding rates for the client station.

 

3.2.1.2       3 clients, 3 APs

We repeat the same tests from the single AP scenario, this time using 3 APs in relatively close proximity. The APs should use different channels (e.g., 1, 6, and 11).

 

3.2.1.3       4 clients, 4 APs

We repeat the same tests from the single AP scenario but this time using 4 APs in relatively close proximity. The first two APs use different channels but the third and fourth APs will have to share a channel. We report forwarding rates for all stations. (Note: All 4 APs will operate for at least 300 seconds before we offer data traffic to give the systems under test a chance to adjust signal levels.)

 

3.2.1.4       3 clients, 4 APs (1 rogue)

We repeat the same tests from the single AP scenario, this time using 3 authorized APs in relatively close proximity. The authorized APs should use different channels (e.g., 1, 6, and 11), but one of them must contend for bandwidth with a rogue AP. We report forwarding rates for all stations.

 

(Note: All 4 APs will operate for at least 300 seconds before we offer data traffic to give the systems under test a chance to adjust signal levels.)

 

3.3      Delay tests

 

3.3.1      Association

Using the Airopeek NX and RF Grabber analyzers from Wildpackets Inc., we measure authentication and association times for a single client using 802.1x PEAP authentication. A single CA/Radius server running Windows 2000 Advanced Server authenticates users. This machine also acts as a DHCP server.

 

To isolate the processing time of the WLAN switch itself, we measure only downstream delays. This ensures any processing time for authentication on the server is not charged to the system under test.

 

3.3.2      Failover

A client associates to one of three APs attached to a switch. (In this scenario, the system under test must be configured to allow clients to roam from one AP to another). We use Netperf to offer a unidirectional stream of UDP packets to the clientís IP address.

 

Once traffic flow begins, we physically disconnect the AP to which the client is associated from the switch. The client should re-associate with a different AP after some interval. We derive the length of the interval from packet loss statistics. We also determine what the SUTís management tools say about user roaming and AP status.

 

3.4      Security

To assess system security, we use FakeAP, an open-source denial-of-service attack tool that mimics tens of thousands of rogue access points. More information about FakeAP is available here:

 

http://www.blackalchemy.to/project/fakeap/

 

Note that FakeAP requires Linux, the hostap driver, and a card with a Prism 2, 2.5, or 3.0 chipset. For this test we use SMCís SMC2532W-B card, which is an OEMíd version of the Zcomax XI-325.

 

4         Change history

Version: 2.0

Date: 27 July 2003

Added reference to QOSmetrix Netwarrior

Deleted Netperf as traffic generator

 

Version: 1.1

Date: 30 June 2003

Initial vendor release

 

Section 3.4: Added card and driver info for hostap

 

Version: 1.0

Date: 16 June 2003

Initial public release to Network World

 

 

Network Test Footer. Privacy Policy. Contact us to learn more.