Example script to look at delays handling network RX packets

[William Cohen <wcohen at redhat dot com>]

Hi All,

This past week I was talking with Andy Gospodarek about a script that could help diagnose why received packets were being dropped by the kernel for a gigabit ethernet card. The network card uses a circular ring buffer to hold information for the received packets. If the packets are small, excessive latency by the kernel could lead to overwriting some of the older entries in the circular ring buffer and losing some rx packets. This issue can be observed by looking at hardware counters on the network hardware. If they are tcp packets the application are not going to notices this because things are automatically resent.

The attached script, napi_poll_rx.stp, attempts to provides some more information about the latencies between different places in the code. It provides statistics between the time that a the rx operation is scheduled and the time that start napi_polling, the time between napi_poll operations, and the number of napi_poll operations per rx operation.

I would like to incorporate this into the systemtap examples. Some questions related to the current script:

-Would it be possible to have a trace_point for __netif_rx_schedule to reduce overhead and eliminate debuginfo
-Would it make sense to remove cpu() from the associative arrays and just key off the $napi pointer to allow
	netif_rx_schedul/napi_poll on different processors?
	Or could multiple processors be using the same $napi pointer at the same time?
-Are there other things that the script could be measuring to give a better understanding of latency issues?

Any feedback on the script would be appreciated.

-Will

Attachment: napi_poll_rx.stp
Description: Text document