Soekris Throughput Performance Measurements

Bhaskaran Raman 06 Dec 2004

System description

• Performance measurement in terms of network throughput between a laptop and a soekris board.
• Laptop: Dell Latitude D600, 600MHz, 1MB cache, 1GB memory
• Soekris: net4521, 512MB compact flash
• Laptop software: Fedora Core 2, Linux 2.6.5-1.358
• Soekris software: Linux 2.4.26, pebble

Software used for throughput measurement

Two different kinds of traffic were used: TCP and UDP

• TCP:
  • packets are sent from user-level code at max possible speed;
  • a specified number of bytes are sent and throughput is measured at the end of the connection; an "appropriate" number of bytes were used (beyond a number, the throughput does not vary with the number of bytes); we used 1Mbyte transfers in some cases and 100Kbyte transfers in others (does not seem to affect the results).
  • user-level code sends 4Kbytes to kernel in each "send" system call, and the receiving side tries to receive 4Kbytes in one "recv" system call.
  • command-line option specifies whether traffic is bidirectional on the TCP connection, or unidirectional.
• UDP:
  • packets are sent from user-level code at a specified inter-packet gap -- unlike with TCP, note that the kernel does not do any flow-control for the user-level code which is sending packets; we used an "appropriate" inter-packet interval (when inter-packet time drops below a certain number, the throughput remains the same largely -- the kernel starts dropping packets at the sending side); we used 1.5ms in some cases and 2ms in some other cases (does not seem to affect the results).
  • Size of UDP packets used is 1430 bytes.
  • command-line option specifies whether traffic is bidirectional or unidirectional;
  • throughput is measured at the receiver every 5 seconds

Performance Measurements on Ethernet (eth0)

Interface eth0 was used on the soekris net4521. And the built-in ethernet interface (uses the tg3 driver) was used on the laptop.

Performance Measurements on Wireless Atheros-based Card (ath0)

Laptop uses built-in Intel PRO/Wireless 2100 802.11b, and the corresponding linux driver version 0.54. The soekris board uses atheros-based EnGenius/Senao NL-5354 MP PLUS Aries2 in 802.11b mode. It uses the madwifi driver version 0.9.4.2. Since the driver has problems with the ad-hoc mode, the soekris board was used in Master mode (802.11b AP) and the laptop in Managed mode (802.11b client).

Performance Measurements on Wireless Intersil-based Card (wlan0)

Both the laptop and the soekris used the same type of card (Intersil-based SL-2511CD Plus EXT2). We used the HostAP 0.2.4 driver. All measurements were in Ad-Hoc mode. And all measurements are modulo the all-pervasive RF pollution -- hopefully the effect of this would have been minimal since the two cards were in close proximity (5-10cm).

Note: It was quite difficult to get the two cards to be an exclusive ad-hoc network by themselves due to the RF pollution. Probably the driver can be fixed to force the cards into an exclusive ad-hoc network by themselves.

The performance results are here: [sxc] [html]

Discussion and Explanation of Results

• The UDP results for eth0 indicate that the bottleneck is the system call (and perhaps the associated memory copy) overhead on the soekris. The difference between the UDP and TCP results for eth0 substantiates this -- TCP achieves the same amount of byte transfer with lesser system calls since we use 4Kbyte transfer per send/recv call for TCP.
• The comparison between the three TCP results suggest that the sending overhead is higher than the receiving overhead in the soekris system.
• When using a 3ms inter-pkt interval with UDP transmissions from the soekris board using the atheros-card (ath0), the madwifi driver repeatedly prints the message "ath_hardstart: discard, no xmit buf". This should probably be edited out of the driver code.
• The UDP results for unidirectional traffic for ath0 match that of eth0. The bidirectional traffic throughput is likely bottlenecked by the available wireless bandwidth (11Mbps raw, usually 6-7Mbps effective).
• The TCP results for ath0 also follow along the same lines as for eth0, although now bottlenecked by the wireless bandwidth again. In the case of ath0, the receiving overhead seems a little higher than sending overhead (can't explain this fully -- perhaps just an experimental error?).
• The UDP results for wlan0 do not match those of eth0/ath0 when the soekris is sending. This is very likely because the HostAP driver (and the Intersil card) do not support DMA transfer, and hence the overhead is higher (perhaps the driver can be rewritten to achieve better parallelism since things seem fine on the receiving direction in the soekris).
• Again for the case of TCP in wlan0, the per-system call overhead seems to be lesser than for UDP. The throughput seems to be lesser when soekris is sending, again likely due to lack of DMA. The throughput seems higher than for ath0 when soekris is receiving -- can't explain this too fully yet.