This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: [RFC] Ofed tapset
- From: "K.Prasad" <prasad at linux dot vnet dot ibm dot com>
- To: "David J. Wilder" <dwilder at us dot ibm dot com>
- Cc: systemtap at sources dot redhat dot com, xma at us dot ibm dot com, predeep at us dot ibm dot com, wilder at us dot ibm dot com
- Date: Tue, 16 Sep 2008 00:42:27 +0530
- Subject: Re: [RFC] Ofed tapset
- References: <1221158104.15193.25.camel@wilder.ibm.com>
- Reply-to: prasad at linux dot vnet dot ibm dot com
Hi Dave,
Thanks for the tapset. The probe points, the helper functions
and the overall scheme used for deriving information using the tapsets
seem to have been designed after much thought!
Although the location of the probes, the information extracted in the
probe aliases require good knowledge of the IPoIB driver for reviewing,
here are a few consolidated suggestions which are mostly generic and
are concerned with usability (in no particular order).
1. Can the name of the function dprint_on() be more specific to denote
its functionality? Like say ipoib_debug_on() or something like that?
2. Can the various printing formats - eventually used by dprint_on() be
named (like PRINT_FORMAT_INPUT, PRINT_FORMAT_OUTPUT, etc) rather than
numbering them for better readability?
3. Again, some of the macros have very generic names, thereby increasing
the possibility of namespace clashes - like get(). Can they be given
specific names?
Looking forward for use-cases in the form of example scripts using this
tapset and testcases for the same.
Thanks,
K.Prasad
On Thu, Sep 11, 2008 at 11:35:04AM -0700, David J. Wilder wrote:
> Hi-
>
> This is a tapset for the ofed infiniband stack.
> Please take a look and tell me what you think.
>
> SLES and RHEL are using slightly different ofed stacks. I
> developed this tapset using SLES10 SP2. I plan to add support
> for RHEL. I will also provide a test for this tapset.
>
>
> I am not on the ml so please cc me on any reply or discussions.
>
> // OFED tapset
> // Copyright (C) 2008 IBM Corp.
> //
> // This file is part of systemtap, and is free software. You can
> // redistribute it and/or modify it under the terms of the GNU General
> // Public License (GPL); either version 2, or (at your option) any
> // later version.
> //
> // Initial version: wilder@us.ibm.com 2008/09/09
> //
> // ----------start of documentation------------
> // Description:
> // This family of probe points is used to probe the ofed infiniband stack.
> // A "probe data output service" (dprint) has been included in the tapset.
> // When enabled the dprint service provides a infiniband protocol and
> // function trace.
> //
> // Probe points:
> // Probe points aliases are assigned using the following format:
> //
> // <module name>.[UD|CM].[ofed function name]
> //
> // module name: The name of the kernel module containing the probe.
> // Supported modules are ib_ipoib, ib_cm.
> //
> // UD, CM: Used only with the ib_ipoib module. Effectively
> // splits the probe points of the ib_ipoib module
> // into two groups. Use UD to specify probes
> // related to datagram mode or CM to specify
> // connection mode probes.
> //
> // function: Name of the function containing the probe point.
> // Most probes are placed at the function entry.
> //
> // Enumerations:
> // See:/usr/share/systemtap/tapset/ofed.stp to identify probe point names.
> //
> // Helper functions:
> // These helper functions are available for use by tap scripts.
> //
> // null_packet_filter_on(): Call this function to turn on the
> // null packet filter. Variable "filtered"
> // will be set true if the infiniband data packet
> // processed at the probe point contains no user
> // data.
> //
> // null_packet_filter_off():Call this function to turn off the
> // null packet filter.
> //
> // dprint_on(): Turn on the data print feature.
> //
> // dprint_off(): Turn off the data print feature.
> //
> // null_packet_filter():Returns true (1) if dprint has been switched on.
> //
> // hexdump (buff:long, len:long):Prints "len" bytes in hex that are
> // pointed to by "buff".
> //
> // _hexdump:string (buff:long, _len:long):Returns a string of "_len"
> // bytes in hex that are pointed to by "buff"(*).
> //
> // skbdump(skb:long):Prints in hex the buffer of a struct sk_buff.
> // Input variable skb is a pointer to an sk_buff.(*)
> //
> // _skbdump:string(skb:long):Returns a string of hex bytes found in the
> // buffer of a struct sk_buff.(*)
> //
> // (*) - The maximum number of bytes processed is about MAXSTRING/3.
> //
> // cm_event_num2str:string (event:long) |
> // cm_event_str2num:long (event:string) | Converts event and state
> // ib_event_num2str:string | values to strings or
> // ib_event_str2num:long (event:string) | integers.
> // cm_state_num2str:string (state:long) |
> // cm_state_str2num:long (state:string) |
> // cm_lap_state_num2str:string (state:long)|
> // cm_lap_state_str2num:long (state:string)|
> //
> // Using the dprint feature:
> // When switched on the dprint feature prints a set of default data
> // for each enabled probe point. To enable dprint, call
> // dprint_on() in your begin probe. The probe body must sets probes
> // for the probe points you wish to generate dprint data. It is
> // not necessary to provide a probe handler for these probe points.
> //
> // The following example tap script demonstrates the use of dprint:
> //
> // # Example of using dprint.
> // probe begin
> // {
> // dprint_on();
> // }
> //
> // # Enable all the ib_ipoib.UD probes
> // probe ib_ipoib.UD.*{}
> //
> // # Enable all the ib_ipoib CM probes
> // probe ib_ipoib.CM.*{}
> //
> // # Enable all ib_cm probes
> // probe ib_cm.*{}
> //
> // # Optional - Add a skb_dump to probe ib_poib.ipoib_start_xmit
> // probe ib_ipoib.ipoib_start_xmit
> // {
> // skbdump($skb);
> // }
> // --------end of documentation----------
> //
> // TODO:
> // Add ib and ip address filter.
> // gid fe80:0000:0000:0000:0005:ad00:0003:05b9 qpn 0x404"/
> // Move documentation to man page.
> // Finnish reject reason processing.
> //
> %{
> #include <linux/netdevice.h>
> #include <linux/workqueue.h>
> #include <rdma/ib_verbs.h>
> #include <rdma/ib_cache.h>
> #include <rdma/ib_cm.h>
> %}
>
>
> /*
> * Macros for safely accessing members of a structure.
> *
> * Example expansion of macro get():
> * --------------------------------------------------------
> * function ib_cm_id__state:long (A:long)
> * %{ THIS->__retvalue = get(ib_cm_id, state, __be32) %}
> * --------------------------------------------------------
> * void function_ib_cm_id__state (A)
> * {
> * THIS->__retvalue = (long)(__be32)kread(&((struct ib_cm_id *)THIS->A)->state);
> * goto end;
> * deref_fault:
> * printk("deref fault in ofed.stp, base pointer = %p\n",
> * (struct ib_cm_id *)THIS->A);
> * THIS->__retvalue = 0xbad;
> * end:;
> * }
> */
> %{
>
> #define getfault(STRUCT, MEMBER) \
> goto end; \
> deref_fault: \
> printk("deref fault in ofed.stp, base pointer = %p\n", \
> (struct STRUCT *)THIS->A); \
> THIS->__retvalue = 0xbad; \
> end: ;
>
> #define __get(STRUCT,MEMBER) kread(&((struct STRUCT *)THIS->A)->MEMBER);
> #define _get(STRUCT,MEMBER) __get(STRUCT,MEMBER); getfault(STRUCT,MEMBER);
> #define get(STRUCT,MEMBER,TYPE) (long)(TYPE)_get(STRUCT, MEMBER);
> %}
>
> /*
> * Functions to toggle features on and off
> */
> global global_use_dprint = 0
> global global_null_packet_filter = 0
>
> function null_packet_filter:long ()
> {
> return global_null_packet_filter;
> }
>
> function null_packet_filter_on ()
> {
> global_null_packet_filter = 1;
> }
>
> function null_packet_filter_off ()
> {
> global_null_packet_filter = 0;
> }
>
> function dprint_on ()
> {
> global_use_dprint = 1;
> }
>
> function dprint_off ()
> {
> global_use_dprint = 0;
> }
>
>
> /*
> * Data formatting functions
> */
> function pname (name:string) {
> if ( strlen(name) )
> return sprintf("%s.%s(%s):",probemod(), probefunc(), name)
> else
> return sprintf("%s.%s:",probemod(), probefunc())
> }
>
> function hexdump (buff:long, len:long) {
> printf("%s\n",_hexdump(buff, len))
> }
>
> function _hexdump:string (buff:long, _len:long)
> %{ /* pure */
> char c;
> uint i=0; // init to stop gcc warning
> char *p = (char *)(long)THIS->buff;
> int len = (int)(long)THIS->_len;
> char *t = (char *) &(THIS->__retvalue);
>
> for (i = 0; i < len; i++) {
> c = (u8)kread(p++);
> t += sprintf(t, "%02x ", c);
> if ((i & 0x0f) == 0x0f)
> t += sprintf(t, "\n");
> if (i >= MAXSTRINGLEN)
> break;
> }
> deref_fault: ;
> %}
>
> function skbdump (skb:long) {
> printf("%s\n",_skbdump(skb))
> }
>
> function _skbdump:string (skb:long)
> %{ /* pure */
> char c;
> uint i=0; // init to stop gcc warning
> char *t = (char *) &(THIS->__retvalue);
> struct sk_buff *skb = (struct sk_buff *)(long)THIS->skb;
>
> char *p = (char *)kread(&skb->data);
> // safe skb_headlen()
> uint len = (uint)kread(&skb->len)-(uint)kread(&skb->data_len);
>
> for (i = 0; i < len; i++) {
> c = (u8)kread(p++);
> t += sprintf(t, "%02x ", c);
> if ((i & 0x0f) == 0x0f)
> t += sprintf(t, "\n");
> if ( i >= MAXSTRINGLEN )
> break;
> }
> deref_fault: ;
> %}
>
>
> /*
> * These functions return the value of each member
> * of a struct cm_id. kread() is used for all de-references.
> *
> * struct ib_cm_id {
> * ib_cm_handler cm_handler;
> * void *context;
> * struct ib_device *device;
> * __be64 service_id;
> * __be64 service_mask;
> * enum ib_cm_state state;
> * enum ib_cm_lap_state lap_state;
> * __be32 local_id;
> * __be32 remote_id;
> * u32 remote_cm_qpn;
> * };
> */
>
> function ib_cm_id__cm_handler:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, cm_handler, void *) %}
>
> function ib_cm_id__device:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, device, void *) %}
>
> function ib_cm_id__service_id:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, service_id, __be64) %}
>
> function ib_cm_id__service_mask:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, service_mask, __be64) %}
>
> function ib_cm_id__state:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, state, __be32) %}
>
> function ib_cm_id__lap_state:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, lap_state, __be32) %}
>
> function ib_cm_id__local_id:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, local_id, __be32) %}
>
> function ib_cm_id__remote_id:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, remote_id, __be32) %}
>
> function ib_cm_id__remote_cm_qpn:long (A:long)
> %{ THIS->__retvalue = get(ib_cm_id, remote_cm_qpn, __be32) %}
>
> /*
> * These functions return the value of some members
> * of a struct cm_work. kread() is used for all de-references.
> *
> * struct cm_work {
> * struct delayed_work work;
> * struct list_head list;
> * struct cm_port *port;
> * struct ib_mad_recv_wc *mad_recv_wc;
> * __be32 local_id;
> * __be32 remote_id;
> * struct ib_cm_event cm_event;
> * struct ib_sa_path_rec path[0];
> * };
> */
>
>
> %{
> // taken from cm.c
> enum ipoib_cm_state {
> IPOIB_CM_RX_LIVE,
> IPOIB_CM_RX_ERROR, /* Ignored by stale task */
> IPOIB_CM_RX_FLUSH /* Last WQE Reached event observed */
> };
>
>
> struct ipoib_cm_rx {
> struct ib_cm_id *id;
> struct ib_qp *qp;
> struct ipoib_cm_rx_buf *rx_ring;
> struct list_head list;
> struct net_device *dev;
> unsigned long jiffies;
> enum ipoib_cm_state state;
> int index;
> int recv_count;
> };
>
> struct delayed_work {
> struct work_struct work;
> };
>
> struct cm_work {
> struct delayed_work work;
> struct list_head list;
> struct cm_port *port;
> struct ib_mad_recv_wc *mad_recv_wc;
> __be32 local_id;
> __be32 remote_id;
> struct ib_cm_event cm_event;
> struct ib_sa_path_rec path[0];
> };
>
> %}
>
> function work__cm_work:long (work:long)
> %{ THIS->__retvalue = (long) container_of((struct work_struct *)THIS->work, struct cm_work, work.work); %}
>
> function cm_work__local_id:long (A:long)
> %{ THIS->__retvalue = get(cm_work, local_id, __be32); %}
>
> function cm_work__remote_id:long (A:long)
> %{ THIS->__retvalue = get(cm_work, remote_id, __be32); %}
>
> function cm_work__event:long (A:long)
> %{ THIS->__retvalue = get(cm_work, cm_event.event, __be32) %}
>
> function cm_work__mad_recv_wc:long (A:long)
> %{ THIS->__retvalue = get(cm_work, mad_recv_wc, void *) %}
>
> # not done XXXX
> # function mad_recv_wc__cm_rej_msg:long (A:long)
> # %{ THIS->__retvalue = get(, recv_buf.mad, void *) %}
>
> # function get_rej_reason:long (_work:long)
> # %{ XXX broken !!
> # int ret;
> # struct cm_rej_msg *rej_msg;
> # struct work_struct *_work = (struct work_struct *)(long)THIS->_work;
> # struct cm_work *work = container_of(_work, struct cm_work, work.work);
> # rej_msg = (struct cm_rej_msg *)work->mad_recv_wc->recv_buf.mad;
> # THIS->__retvalue = __be16_to_cpu(rej_msg->reason);
> # %}
>
> #
> # Data printing function for ib_ipoib.
> # dprint_format values: 0 output, 1 input, 2 event, 3 name only.
> #
>
> function ipoib_dprint (name:string,
> cm_id:long,
> device_str:string,
> qp_num:long,
> wr_id:long,
> event:long,
> dprint_format:long)
> {
> if ( !global_use_dprint )
> return 0;
>
> module = probemod();
> if (module != "ib_ipoib") {
> printf("%s: module %s is not supported by ipoib_dprint\n",
> name, module)
> return 0
> }
>
> if ( dprint_format == 0 ) { // output
> printf("%s(%s):\t\t\t\t0x%x(%d)--->\n",
> name, device_str, qp_num, wr_id)
>
> } else if ( dprint_format == 1 ) { // input
> printf("%s(%s):\t\t\t\t<---0x%x(%d)\n",
> name, device_str, qp_num, wr_id)
>
> } else if ( dprint_format == 2 ) { // event
> printf("%s(%s):%s cm_id=%p state=%s lap_state=%s\n",
> name, device_str,cm_event_num2str(event),
> cm_id, cm_state_num2str(ib_cm_id__state(cm_id)),
> cm_lap_state_num2str(ib_cm_id__lap_state(cm_id)));
>
> } else if ( dprint_format == 3 ) { // name only
> printf("%s\n",name);
>
> } else {
> printf("%s(%s): Bad print format\n",
> name, device_str);
> }
> }
>
> #
> # Data printing function for ib_cm.
> # dprint_format values: 0 output, 1 input, 2 event, 3 name only.
> #
> function cm_dprint (name:string,
> cm_id:long,
> event:long,
> dprint_format:long)
> {
> if ( !global_use_dprint )
> return 0;
>
> module = probemod();
> if (module != "ib_cm") {
> printf("%s: module %s is not supported by cm_dprint\n",
> name, module)
> return 0
> }
> printf("%s ",name) // XXX Should I check if name is a valid string?
>
> if (event != -1)
> if ( dprint_format == 1 ) // reject reason is passed
> // as event.
> printf("reason = %d ",event)
> else
> printf("%s ",cm_event_num2str(event));
>
> if (cm_id != -1) {
> printf("cm_id=%p state=%s lap_state=%s ",
> cm_id, cm_state_num2str(ib_cm_id__state(cm_id)),
> cm_lap_state_num2str(ib_cm_id__lap_state(cm_id)));
>
> /* debug
> printf(" cm_handler=%p device=%p service_id=0x%x service_mask=0x%x local_id=0x%x remote_id=0x%x remote_cm_qpn=0x%x \n",
> ib_cm_id__cm_handler(cm_id),
> ib_cm_id__device(cm_id),
> ib_cm_id__service_id(cm_id),
> ib_cm_id__service_mask(cm_id),
> ib_cm_id__local_id(cm_id),
> ib_cm_id__remote_id(cm_id),
> ib_cm_id__remote_cm_qpn(cm_id));
> */
> }
> printf("\n");
> }
>
>
> /*
> * Event and state helper functions
> */
>
> /*
> * cm_event_num2str
> * Given an event number, return a string representation.
> */
> function cm_event_num2str:string (event:long)
> {
> return (event in _cm_event_num2str ? _cm_event_num2str[event] : "UNDEF")
> }
>
> /*
> * cm_event_str2num
> * Given a event name (string), return the corresponding event number.
> */
> function cm_event_str2num:long (event:string)
> {
> return (event in _cm_event_str2num ? _cm_event_str2num[event] : -1)
> }
>
> /*
> * ib_event_num2str
> * Given an event number, return a string representation.
> */
> function ib_event_num2str:string (event:long)
> {
> return (event in _ib_event_num2str ? _ib_event_num2str[event] : "UNDEF")
> }
>
> /*
> * ib_event_str2num
> * Given a event name (string), return the corresponding event number.
> */
> function ib_event_str2num:long (event:string)
> {
> return (event in _ib_event_str2num ? _ib_event_str2num[event] : -1)
> }
>
> /*
> * cm_state_num2str
> */
> function cm_state_num2str:string (state:long)
> {
> return (state in _ib_cm_state_num2str ? _ib_cm_state_num2str[state] : "UNDEF")
> }
>
> /*
> * cm_state_str2num
> */
> function cm_state_str2num:long (state:string)
> {
> return (state in _ib_cm_state_str2num ? _ib_cm_state_str2num[state] : -1)
> }
>
> /*
> * cm_lap_state_num2str
> */
> function cm_lap_state_num2str:string (state:long)
> {
> return (state in _ib_cm_lap_state_num2str ? _ib_cm_lap_state_num2str[state] : "UNDEF")
> }
>
> /*
> * cm_lap_state_str2num
> */
> function cm_lap_state_str2num:long (state:string)
> {
> return (state in _ib_cm_lap_state_str2num ? _ib_cm_lap_state_str2num[state] : -1)
> }
>
> /*
> * Internal mapping arrays
> */
> global _cm_event_num2str[18]
> global _cm_event_str2num[18]
> global _ib_event_num2str[18]
> global _ib_event_str2num[18]
> global _ib_cm_state_num2str[16]
> global _ib_cm_state_str2num[16]
> global _ib_cm_lap_state_num2str[6]
> global _ib_cm_lap_state_str2num[6]
>
> /*
> * Initialize the state and event tables.
> */
> probe begin(-1001)
> {
> /* From rdma/ib_cm.h
> * Mapping of ib_cm_event_type enumeration
> */
> _cm_event_num2str[0] = "IB_CM_REQ_ERROR"
> _cm_event_num2str[1] = "IB_CM_REQ_RECEIVED"
> _cm_event_num2str[2] = "IB_CM_REP_ERROR"
> _cm_event_num2str[3] = "IB_CM_REP_RECEIVED"
> _cm_event_num2str[4] = "IB_CM_RTU_RECEIVED"
> _cm_event_num2str[5] = "IB_CM_USER_ESTABLISHED"
> _cm_event_num2str[6] = "IB_CM_DREQ_ERROR"
> _cm_event_num2str[7] = "IB_CM_DREQ_RECEIVED"
> _cm_event_num2str[8] = "IB_CM_DREP_RECEIVED"
> _cm_event_num2str[9] = "IB_CM_TIMEWAIT_EXIT"
> _cm_event_num2str[10] = "IB_CM_MRA_RECEIVED"
> _cm_event_num2str[11] = "IB_CM_REJ_RECEIVED"
> _cm_event_num2str[12] = "IB_CM_LAP_ERROR"
> _cm_event_num2str[13] = "IB_CM_LAP_RECEIVED"
> _cm_event_num2str[14] = "IB_CM_APR_RECEIVED"
> _cm_event_num2str[15] = "IB_CM_SIDR_REQ_ERROR"
> _cm_event_num2str[16] = "IB_CM_SIDR_REQ_RECEIVED"
> _cm_event_num2str[17] = "IB_CM_SIDR_REP_RECEIVED"
>
> _cm_event_str2num["IB_CM_REQ_ERROR"] = 0
> _cm_event_str2num["IB_CM_REQ_RECEIVED"] = 1
> _cm_event_str2num["IB_CM_REP_ERROR"] = 2
> _cm_event_str2num["IB_CM_REP_RECEIVED"] = 3
> _cm_event_str2num["IB_CM_RTU_RECEIVED"] = 4
> _cm_event_str2num["IB_CM_USER_ESTABLISHED"] = 5
> _cm_event_str2num["IB_CM_DREQ_ERROR"] = 6
> _cm_event_str2num["IB_CM_DREQ_RECEIVED"] = 7
> _cm_event_str2num["IB_CM_DREP_RECEIVED"] = 8
> _cm_event_str2num["IB_CM_TIMEWAIT_EXIT"] = 9
> _cm_event_str2num["IB_CM_MRA_RECEIVED"] = 10
> _cm_event_str2num["IB_CM_REJ_RECEIVED"] = 11
> _cm_event_str2num["IB_CM_LAP_ERROR"] = 12
> _cm_event_str2num["IB_CM_LAP_RECEIVED"] = 13
> _cm_event_str2num["IB_CM_APR_RECEIVED"] = 14
> _cm_event_str2num["IB_CM_SIDR_REQ_ERROR"] = 15
> _cm_event_str2num["IB_CM_SIDR_REQ_RECEIVED"] = 16
> _cm_event_str2num["IB_CM_SIDR_REP_RECEIVED"] = 17
>
> /* From rdma/ib_verbs.h
> * Mapping of ib_event_type enumeration
> */
> _ib_event_num2str[0] = "IB_EVENT_CQ_ERR"
> _ib_event_num2str[1] = "IB_EVENT_QP_FATAL"
> _ib_event_num2str[2] = "IB_EVENT_QP_REQ_ERR"
> _ib_event_num2str[3] = "IB_EVENT_QP_ACCESS_ERR"
> _ib_event_num2str[4] = "IB_EVENT_COMM_EST"
> _ib_event_num2str[5] = "IB_EVENT_SQ_DRAINED"
> _ib_event_num2str[6] = "IB_EVENT_PATH_MIG"
> _ib_event_num2str[7] = "IB_EVENT_PATH_MIG_ERR"
> _ib_event_num2str[8] = "IB_EVENT_DEVICE_FATAL"
> _ib_event_num2str[9] = "IB_EVENT_PORT_ACTIVE"
> _ib_event_num2str[10] = "IB_EVENT_PORT_ERR"
> _ib_event_num2str[11] = "IB_EVENT_LID_CHANGE"
> _ib_event_num2str[12] = "IB_EVENT_PKEY_CHANGE"
> _ib_event_num2str[13] = "IB_EVENT_SM_CHANGE"
> _ib_event_num2str[14] = "IB_EVENT_SRQ_ERR"
> _ib_event_num2str[15] = "IB_EVENT_SRQ_LIMIT_REACHED"
> _ib_event_num2str[16] = "IB_EVENT_QP_LAST_WQE_REACHED"
> _ib_event_num2str[17] = "IB_EVENT_CLIENT_REREGISTER"
>
> _ib_event_str2num["IB_EVENT_CQ_ERR"] = 0
> _ib_event_str2num["IB_EVENT_QP_FATAL"] = 1
> _ib_event_str2num["IB_EVENT_QP_REQ_ERR"] = 2
> _ib_event_str2num["IB_EVENT_QP_ACCESS_ERR"] = 3
> _ib_event_str2num["IB_EVENT_COMM_EST"] = 4
> _ib_event_str2num["IB_EVENT_SQ_DRAINED"] = 5
> _ib_event_str2num["IB_EVENT_PATH_MIG"] = 6
> _ib_event_str2num["IB_EVENT_PATH_MIG_ERR"] = 7
> _ib_event_str2num["IB_EVENT_DEVICE_FATAL"] = 8
> _ib_event_str2num["IB_EVENT_PORT_ACTIVE"] = 9
> _ib_event_str2num["IB_EVENT_PORT_ERR"] = 10
> _ib_event_str2num["IB_EVENT_LID_CHANGE"] = 11
> _ib_event_str2num["IB_EVENT_PKEY_CHANGE"] = 12
> _ib_event_str2num["IB_EVENT_SM_CHANGE"] = 13
> _ib_event_str2num["IB_EVENT_SRQ_ERR"] = 14
> _ib_event_str2num["IB_EVENT_SRQ_LIMIT_REACHED"] = 15
> _ib_event_str2num["IB_EVENT_QP_LAST_WQE_REACHED"] = 16
> _ib_event_str2num["IB_EVENT_CLIENT_REREGISTER"] = 17
>
> /* from ib_cm.h */
> _ib_cm_state_num2str[0] = "IB_CM_IDLE"
> _ib_cm_state_num2str[1] = "IB_CM_LISTEN"
> _ib_cm_state_num2str[2] = "IB_CM_REQ_SENT"
> _ib_cm_state_num2str[3] = "IB_CM_REQ_RCVD"
> _ib_cm_state_num2str[4] = "IB_CM_MRA_REQ_SENT"
> _ib_cm_state_num2str[5] = "IB_CM_MRA_REQ_RCVD"
> _ib_cm_state_num2str[6] = "IB_CM_REP_SENT"
> _ib_cm_state_num2str[7] = "IB_CM_REP_RCVD"
> _ib_cm_state_num2str[8] = "IB_CM_MRA_REP_SENT"
> _ib_cm_state_num2str[9] = "IB_CM_MRA_REP_RCVD"
> _ib_cm_state_num2str[10] = "IB_CM_ESTABLISHED"
> _ib_cm_state_num2str[11] = "IB_CM_DREQ_SENT"
> _ib_cm_state_num2str[12] = "IB_CM_DREQ_RCVD"
> _ib_cm_state_num2str[13] = "IB_CM_TIMEWAIT"
> _ib_cm_state_num2str[14] = "IB_CM_SIDR_REQ_SENT"
> _ib_cm_state_num2str[15] = "IB_CM_SIDR_REQ_RCVD"
>
> _ib_cm_state_str2num["IB_CM_IDLE"] = 0
> _ib_cm_state_str2num["IB_CM_LISTEN"] = 1
> _ib_cm_state_str2num["IB_CM_REQ_SENT"] = 2
> _ib_cm_state_str2num["IB_CM_REQ_RCVD"] = 3
> _ib_cm_state_str2num["IB_CM_MRA_REQ_SENT"] = 4
> _ib_cm_state_str2num["IB_CM_MRA_REQ_RCVD"] = 5
> _ib_cm_state_str2num["IB_CM_REP_SENT"] = 6
> _ib_cm_state_str2num["IB_CM_REP_RCVD"] = 7
> _ib_cm_state_str2num["IB_CM_MRA_REP_SENT"] = 8
> _ib_cm_state_str2num["IB_CM_MRA_REP_RCVD"] = 9
> _ib_cm_state_str2num["IB_CM_ESTABLISHED"] = 10
> _ib_cm_state_str2num["IB_CM_DREQ_SENT"] = 11
> _ib_cm_state_str2num["IB_CM_DREQ_RCVD"] = 12
> _ib_cm_state_str2num["IB_CM_TIMEWAIT"] = 13
> _ib_cm_state_str2num["IB_CM_SIDR_REQ_SENT"] = 14
> _ib_cm_state_str2num["IB_CM_SIDR_REQ_RCVD"] = 15
>
> /* from ib_cm.h */
> _ib_cm_lap_state_num2str[0] = "IB_CM_LAP_UNINIT"
> _ib_cm_lap_state_num2str[1] = "IB_CM_LAP_IDLE"
> _ib_cm_lap_state_num2str[2] = "IB_CM_LAP_SENT"
> _ib_cm_lap_state_num2str[3] = "IB_CM_LAP_RCVD"
> _ib_cm_lap_state_num2str[4] = "IB_CM_MRA_LAP_SENT"
> _ib_cm_lap_state_num2str[5] = "IB_CM_MRA_LAP_RCVD"
>
> _ib_cm_lap_state_str2num["IB_CM_LAP_UNINIT"] = 0
> _ib_cm_lap_state_str2num["IB_CM_LAP_IDLE"] = 1
> _ib_cm_lap_state_str2num["IB_CM_LAP_SENT"] = 2
> _ib_cm_lap_state_str2num["IB_CM_LAP_RCVD"] = 3
> _ib_cm_lap_state_str2num["IB_CM_MRA_LAP_SENT"] = 4
> _ib_cm_lap_state_str2num["IB_CM_MRA_LAP_RCVD"] = 5
> }
>
> /*
> * Start of ib_ipoib probe definitions
> */
> #
> # int ipoib_start_xmit(struct sk_buff *skb, struct net_device *dev)
> #
> probe ib_ipoib.ipoib_start_xmit = module("ib_ipoib").function("ipoib_start_xmit")
> {
> name = pname("start transmit")
> device_str = kernel_string($dev->name);
> cm_id = -1
> qp_num = -1
> wr_id = -1
> skb = $skb
> event = -1
> state = -1
> dprint_format = 3
> filtered = 0
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id,
> event, dprint_format);
> }
>
> #
> # void ipoib_path_lookup(struct sk_buff *skb, struct net_device *dev)
> #
> probe ib_ipoib.ipoib_path_lookup = module("ib_ipoib").function("ipoib_path_lookup")
> {
> name = pname("path lookup")
> device_str = kernel_string($dev->name);
> cm_id = -1
> qp_num = -1
> wr_id = -1
> skb = $skb
> event = -1
> state = -1
> dprint_format = 3
> filtered = 0
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id, event,
> dprint_format);
> }
>
> #
> # ipoib_send(struct net_device *dev, struct sk_buff *skb,
> # struct ipoib_ah *address, u32 qpn)
> #
> probe ib_ipoib.UD.ipoib_send = module("ib_ipoib").function("ipoib_send")
> {
> name = pname("ud send")
> device_str = kernel_string($dev->name);
> cm_id = -1
> qp_num = $qpn
> wr_id = -1
> skb = $skb
> event = -1
> state = -1
> dprint_format = 3
> filtered = 0
> ah = $address
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id,
> event, dprint_format);
> }
>
>
> #
> # ipoib_cm_send(struct net_device *dev, struct sk_buff *skb,
> # struct ipoib_cm_tx *tx)
> #
> probe ib_ipoib.CM.ipoib_cm_send = module("ib_ipoib").function("ipoib_cm_send"){
> name = pname("cm send")
> device_str = kernel_string($dev->name);
> cm_id = $tx->id
> qp_num = $tx->qp->qp_num
> skb = $skb
> filtered = 0
> }
>
> #
> # static void ipoib_ib_handle_tx_wc(struct net_device *dev, struct ib_wc *wc)
> #
> probe ib_ipoib.UD.ipoib_ib_handle_tx_wc = module("ib_ipoib").function("ipoib_ib_handle_tx_wc")
> {
> IPOIB_OP_RECV = 1<<31;
> IPOIB_OP_CM = 1<<30;
> name = pname("ud send completion")
> device_str = kernel_string($dev->name);
> cm_id = -1
> qp_num = $wc->qp->qp_num;
> wr_id = $wc->wr_id & ~(IPOIB_OP_RECV|IPOIB_OP_CM);
> event = -1
> state = $wc->status;
> dprint_format = 1
>
> filtered = 0
> byte_len = $wc->byte_len;
> if ( byte_len == 0 && null_packet_filter() ) {
> filtered = 1;
> }else{
> ipoib_dprint(name, cm_id, device_str, qp_num,
> wr_id, event, dprint_format);
> }
> }
>
> #
> # void ipoib_cm_handle_tx_wc(struct net_device *dev, struct ib_wc *wc)
> #
> probe ib_ipoib.CM.ipoib_cm_handle_tx_wc = module("ib_ipoib").function("ipoib_cm_handle_tx_wc")
> {
> IPOIB_OP_CM = 1<<30;
> name = pname("cm send completion")
> device_str = kernel_string($dev->name);
> cm_id = -1
> qp = $wc->qp;
> qp_num = $wc->qp->qp_num;
> wr_id = $wc->wr_id & ~IPOIB_OP_CM;
> event = -1;
> state = $wc->status;
> dprint_format = 1;
> filtered = 0;
> wc_flags = $wc_flags;
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id,
> event, dprint_format);
> }
>
> #
> # static void ipoib_ib_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
> #
> probe ib_ipoib.UD.ipoib_ib_handle_rx_wc = module("ib_ipoib").function("ipoib_ib_handle_rx_wc")
> {
> IPOIB_OP_RECV = 1<<31;
> IPOIB_OP_CM = 1<<30;
> name = pname("ud receive completion")
> device_str = kernel_string($dev->name);
> cm_id = -1
> qp_num = $wc->qp->qp_num;
> wr_id = ($wc->wr_id) & ~(IPOIB_OP_CM | IPOIB_OP_RECV);
> event = -1
> state = $wc->status
> dprint_format = 1
> qp = $wc->qp;
> wc_flags = $wc_flags;
>
> filtered=0;
> byte_len = $wc->byte_len;
> if ( (byte_len == 40) && null_packet_filter() ) {
> filtered = 1;
> }else{
> ipoib_dprint(name, cm_id, device_str, qp_num,
> wr_id, event, dprint_format);
> }
> }
>
> #
> # void ipoib_cm_handle_rx_wc(struct net_device *dev, struct ib_wc *wc)
> #
> probe ib_ipoib.CM.ipoib_cm_handle_rx_wc = module("ib_ipoib").function("ipoib_cm_handle_rx_wc")
> {
> IPOIB_OP_RECV = 1<<31;
> IPOIB_OP_CM = 1<<30;
> name = pname("cm receive completion")
> device_str = kernel_string($dev->name);
> cm_id = -1
> qp = $wc->qp
> qp_num = $wc->qp->qp_num;
> wr_id = ($wc->wr_id) & ~(IPOIB_OP_CM | IPOIB_OP_RECV);
> event = -1
> state = $wc->status;
> dprint_format = 1 # (0=output, 1=input)
> filtered = 0
>
> wc_flags = $wc_flags;
>
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id, event,
> dprint_format);
> }
>
> #
> # static inline int post_send(struct ipoib_dev_priv *priv,
> # struct ipoib_cm_tx *tx,
> # unsigned int wr_id,
> # u64 addr, int len)
> #
> probe ib_ipoib.CM.post_send = module("ib_ipoib").inline("post_send@/usr/src/debug/ofa_kernel-1.3/obj/ppc64/drivers/infiniband/ulp/ipoib/ipoib_cm.c")
> {
> name = pname("cm post send")
> device_str = kernel_string($dev->name);
> qp_num = $tx->qp->qp_num;
> wr_id = $wr_id;
> filtered = 0
> ipoib_dprint(name, -1, device_str, qp_num, wr_id, -1, 0);
> }
>
> #
> # static inline int post_send(struct ipoib_dev_priv *priv,
> # unsigned int wr_id,
> # struct ib_ah *address, u32 qpn,
> # struct ipoib_tx_buf *tx_req,
> # void *head, int hlen)
> #
> probe ib_ipoib.UD.post_send = module("ib_ipoib").inline("post_send@/usr/src/debug/ofa_kernel-1.3/obj/ppc64/drivers/infiniband/ulp/ipoib/ipoib_ib.c")
> {
> IPOIB_OP_RECV = 1<<31
> name = pname("ud post send")
> device_str = kernel_string($dev->name);
> cm_id = -1
> // qp_num = $qp_num; whats the correct way to get qpn? - wilder
> qp_num = $priv->qp->qp_num;
> wr_id = $wr_id & ~IPOIB_OP_RECV;
> event = -1
> dprint_format = 0
> filtered = 0
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id,
> event, dprint_format);
> }
> #
> # int ipoib_cm_tx_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> #
> probe ib_ipoib.CM.ipoib_cm_tx_handler =module("ib_ipoib").function("ipoib_cm_tx_handler"){
> name = pname("cm_tx_handler")
> device_str = "--" // XXX can find from cm_id but need pure.
> cm_id = $cm_id
> qp_num = -1
> wr_id = -1
> event = $event->event
> dprint_format = 2
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id,
> event, dprint_format);
> }
>
>
> #
> # int ipoib_cm_rx_handler(struct ib_cm_id *cm_id,struct ib_cm_event *event)
> #
> probe ib_ipoib.CM.cm_rx_handler =module("ib_ipoib").function("ipoib_cm_rx_handler")
> {
> name = pname("cm_rx_handler")
> device_str = "--" // XXX, get from cm id?
> cm_id = $cm_id;
> qp_num = -1 // XXX
> wr_id = -1
> event = $event->event
> dprint_format = 2
> filtered = 0
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id,
> event, dprint_format);
> }
>
> #
> # static int ipoib_cm_send_rep(struct net_device *dev, struct ib_cm_id *cm_id,
> # struct ib_qp *qp, struct ib_cm_req_event_param *req,
> # unsigned psn)
> #
> # XXX - stap can not find probe point ipoib_cm_send_rep.
> #
> probe XXXXXipoib.CM.send_rep = module("ib_ipoib").function("ipoib_cm_send_rep")
> {
> name = pname("REP-->")
> device_str = kernel_string($dev->name);
> cm_id = $cm_id
> qp = $qp
> qp_num = $qp->qp_num
> wr_id = $psn->starting_psn // XXX Not tested
> event = -1
> state = -1
> dprint_format = 0 # (0=output, 1=input)
> filtered = 0
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id, event,
> dprint_format);
> }
>
> #
> # static int ipoib_cm_send_req(struct net_device *dev,
> # struct ib_cm_id *id, struct ib_qp *qp,
> # u32 qpn,
> # struct ib_sa_path_rec *pathrec)
> #
> probe ib_ipoib.CM_send_req = module("ib_ipoib").function("ipoib_cm_send_req")
> {
> name = pname("REQ-->")
> device_str = kernel_string($dev->name);
> cm_id = $cm_id
> qp_num = $qpn
> wr_id = -1
> skb = -1
> event = -1
> state = -1
> dprint_format = 0 # (0=output, 1=input)
> filtered = 0
> dprint(name, device_str, cm_id, qp_num, wr_id, skb, event,
> state, dprint_format);
> }
>
> #
> # static void ipoib_cm_rx_event_handler(struct ib_event *event, void *ctx)
> #
> probe ib_ipoib.CM.rx_event = module("ib_ipoib").function("ipoib_cm_rx_event_handler")
> {
> name = pname("RX event")
> device = "--"
> cm_id = -1
> qp_num = -1
> wr_id = -1
> skb = -1
> event = $event->event
> state = -1 // XXX struct ipoib_cm_rx *p = ctx;
> // p->state = IPOIB_CM_RX_FLUSH;
> dprint_format = 2 # (0=output, 1=input 3=event)
> ipoib_dprint(name, cm_id, device_str, qp_num, wr_id, event, dprint_format);
> }
>
> /*
> * Start of probe definitions for module ib_cm
> */
>
> #
> # static void cm_work_handler(struct work_struct *_work)
> #
> probe ib_cm.cm_work_handler = module("ib_cm").function("cm_work_handler")
> {
> name = pname("work handler")
> event = cm_work__event(work__cm_work($_work));
> // XXX get_rej_reason is broken
> // if ( event == cm_event_str2num("IB_CM_REJ_RECEIVED"))
> // printf("cm handler got a REJ reason=%d \n",
> // get_rej_reason($_work));
> cm_dprint(name,-1,event,0)
> }
>
> # XXX
> # static int cm_rej_handler(struct cm_work *work)
> #
> # probe ib_cm.rej_handler = module("ib_cm").function("cm_rej_handler")
> # Stap error: no match for probe point while resolving probe point module
> # ("ib_cm").function("cm_rej_handler")
> #{
> # name = pname("REJ handler")
> # rej_msg = (struct cm_rej_msg *)work->mad_recv_wc->recv_buf.mad;
> # event = rej_msg->reason
> # Need a way to get the cm_id. .
> # cm_id_priv = cm_acquire_rejected_id(rej_msg);
> # cm_dprint(name, -1, -1, 0);
> #}
>
> #
> # int ib_send_cm_rep(struct ib_cm_id *cm_id,
> # struct ib_cm_rep_param *param)
> #
> probe ib_cm.ib_send_cm_rep = module("ib_cm").function("ib_send_cm_rep")
> {
> name = pname("send REP")
> cm_id = $cm_id
> event = -1 // Todo: return parm.
> cm_dprint(name,cm_id,event,0);
> }
>
>
> #
> # int ib_send_cm_req(struct ib_cm_id *cm_id,
> # struct ib_cm_req_param *param)
> #
> probe ib_cm.ib_send_cm_req = module("ib_cm").function("ib_send_cm_req")
> {
> name = pname("send REQ")
> cm_id = $cm_id
> event = -1
> dprint_format = 0
> cm_dprint(name, cm_id, event, dprint_format);
> }
>
> #
> # int ib_send_cm_rtu(struct ib_cm_id *cm_id,
> # const void *private_data,
> # u8 private_data_len)
> #
> probe ib_cm.ib_send_cm_rtu = module("ib_cm").function("ib_send_cm_rtu")
> {
> name = pname("send RTU")
> cm_id = $cm_id
> cm_dprint(name, cm_id, -1,0);
> }
>
> #
> # int ib_send_cm_dreq(struct ib_cm_id *cm_id,
> # const void *private_data,
> # u8 private_data_len)
>
> probe ib_cm.ib_send_cm_dreq = module("ib_cm").function("ib_send_cm_dreq")
> {
> name = pname("send DREQ")
> cm_id = $cm_id
> cm_dprint(name,cm_id,-1,0);
> }
>
> #
> # int ib_send_cm_drep(struct ib_cm_id *cm_id,
> # const void *private_data,
> # u8 private_data_len)
> #
> probe ib_cm.ib_send_cm_drep = module("ib_cm").function("ib_send_cm_drep")
> {
> name = pname("send DREP")
> cm_id = $cm_id
> cm_dprint(name,cm_id,-1,0)
> }
>
> #
> # int ib_send_cm_mra(struct ib_cm_id *cm_id,
> # u8 service_timeout,
> # const void *private_data,
> # u8 private_data_len)
> probe ib_cm.ib_send_cm_mra = module("ib_cm").function("ib_send_cm_mra")
> {
> name = pname("send MRA")
> cm_id = $cm_id
> cm_dprint(name,cm_id,-1,0)
> }
>
> #
> # int ib_send_cm_lap(struct ib_cm_id *cm_id,
> # struct ib_sa_path_rec *alternate_path,
> # const void *private_data,
> # u8 private_data_len)
> #
> probe ib_cm.ib_send_cm_lap = module("ib_cm").function("ib_send_cm_lap")
> {
> name = pname("send LAP")
> cm_id = $cm_id
> cm_dprint(name,cm_id,-1,0)
> }
>
> #
> # int ib_send_cm_apr(struct ib_cm_id *cm_id,
> # enum ib_cm_apr_status status,
> # void *info,
> # u8 info_length,
> # const void *private_data,
> # u8 private_data_len)
> #
> probe ib_cm.ib_send_cm_apr = module("ib_cm").function("ib_send_cm_apr")
> {
> name = pname("send APR")
> cm_id = $cm_id
> cm_dprint(name,cm_id,-1,0)
> }
>
> #
> # int ib_send_cm_sidr_req(struct ib_cm_id *cm_id,
> # struct ib_cm_sidr_req_param *param)
> #
> probe ib_cm.ib_send_cm_sidr_req = module("ib_cm").function("ib_send_cm_sidr_req")
> {
> name = pname("send SIDR-REQ")
> cm_id = $cm_id
> cm_dprint(name,cm_id,-1,0)
> }
>
> #
> # int ib_send_cm_sidr_rep(struct ib_cm_id *cm_id,
> # struct ib_cm_sidr_rep_param *param)
> #
> probe ib_cm.ib_send_cm_sidr_rep=module("ib_cm").function("ib_send_cm_sidr_rep")
> {
> name = pname("send SIDR-REP")
> cm_id = $cm_id
> cm_dprint(name,cm_id,-1,0)
> }
>
> #
> # ib_send_cm_rej(struct ib_cm_id *cm_id,
> # enum ib_cm_rej_reason reason,
> # void *ari,
> # u8 ari_length,
> # const void *private_data,
> # u8 private_data_len)
> #
> probe ib_cm.send_rej = module("ib_cm").function("ib_send_cm_rej")
> {
> name = pname("send REJ")
> cm_id = $cm_id
> state = $cm_id->state
> // send reject reason as the event with dprint_format=1
> cm_dprint(name,cm_id,$reason,1)
> }
>
> #
> # struct ib_cm_id *ib_create_cm_id(struct ib_device *device,
> # ib_cm_handler cm_handler,
> # void *context)
> #
> probe ib_cm.ib_create_cm_id = module("ib_cm").function("ib_create_cm_id").return
> {
> name = pname("ib_create_cm_id")
> cm_id = $return
> cm_dprint(name, cm_id, -1, 0);
> }
>
>