It looks like uname's "_func_name" is also just an implementation detail.
I'd guess single underscores can be filtered too.

(In reply to Martin Cermak from comment #0)
> Created attachment 8522 [details]
> consistency checker
> 
> I've been checking for inconsistencies in the syscall and nd_syscall tapset.
> The goal was to identify probes whose list of syscall convenience variables
> differs from list of respective nd_syscall convenience variables.
> Convenience variables prefixed with "__" are being ignored. Results:
> 
> -------
>       i386                  syscall.uname ['argstr', 'name']
>       i386               nd_syscall.uname ['_func_name', 'argstr', 'name']
>     x86_64                  syscall.uname ['argstr', 'name']
>     x86_64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>    powerpc                  syscall.uname ['argstr', 'name']
>    powerpc               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       s390                  syscall.uname ['argstr', 'name']
>       s390               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       ia64                  syscall.uname ['argstr', 'name']
>       ia64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>        arm                  syscall.uname ['argstr', 'name']
>        arm               nd_syscall.uname ['_func_name', 'argstr', 'name']
>      arm64                  syscall.uname ['argstr', 'name']
>      arm64               nd_syscall.uname ['_func_name', 'argstr', 'name']
> -------
>       i386                  syscall.uname ['argstr', 'name']
>       i386               nd_syscall.uname ['_func_name', 'argstr', 'name']
>     x86_64                  syscall.uname ['argstr', 'name']
>     x86_64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>    powerpc                  syscall.uname ['argstr', 'name']
>    powerpc               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       s390                  syscall.uname ['argstr', 'name']
>       s390               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       ia64                  syscall.uname ['argstr', 'name']
>       ia64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>        arm                  syscall.uname ['argstr', 'name']
>        arm               nd_syscall.uname ['_func_name', 'argstr', 'name']
>      arm64                  syscall.uname ['argstr', 'name']
>      arm64               nd_syscall.uname ['_func_name', 'argstr', 'name']
> -------
>       i386                  syscall.uname ['argstr', 'name']
>       i386               nd_syscall.uname ['_func_name', 'argstr', 'name']
>     x86_64                  syscall.uname ['argstr', 'name']
>     x86_64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>    powerpc                  syscall.uname ['argstr', 'name']
>    powerpc               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       s390                  syscall.uname ['argstr', 'name']
>       s390               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       ia64                  syscall.uname ['argstr', 'name']
>       ia64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>        arm                  syscall.uname ['argstr', 'name']
>        arm               nd_syscall.uname ['_func_name', 'argstr', 'name']
>      arm64                  syscall.uname ['argstr', 'name']
>      arm64               nd_syscall.uname ['_func_name', 'argstr', 'name']
> -------
>       i386                  syscall.uname ['argstr', 'name']
>       i386               nd_syscall.uname ['_func_name', 'argstr', 'name']
>     x86_64                  syscall.uname ['argstr', 'name']
>     x86_64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>    powerpc                  syscall.uname ['argstr', 'name']
>    powerpc               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       s390                  syscall.uname ['argstr', 'name']
>       s390               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       ia64                  syscall.uname ['argstr', 'name']
>       ia64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>        arm                  syscall.uname ['argstr', 'name']
>        arm               nd_syscall.uname ['_func_name', 'argstr', 'name']
>      arm64                  syscall.uname ['argstr', 'name']
>      arm64               nd_syscall.uname ['_func_name', 'argstr', 'name']
> -------
>       i386                  syscall.uname ['argstr', 'name']
>       i386               nd_syscall.uname ['_func_name', 'argstr', 'name']
>     x86_64                  syscall.uname ['argstr', 'name']
>     x86_64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>    powerpc                  syscall.uname ['argstr', 'name']
>    powerpc               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       s390                  syscall.uname ['argstr', 'name']
>       s390               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       ia64                  syscall.uname ['argstr', 'name']
>       ia64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>        arm                  syscall.uname ['argstr', 'name']
>        arm               nd_syscall.uname ['_func_name', 'argstr', 'name']
>      arm64                  syscall.uname ['argstr', 'name']
>      arm64               nd_syscall.uname ['_func_name', 'argstr', 'name']
> -------
>       i386                  syscall.uname ['argstr', 'name']
>       i386               nd_syscall.uname ['_func_name', 'argstr', 'name']
>     x86_64                  syscall.uname ['argstr', 'name']
>     x86_64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>    powerpc                  syscall.uname ['argstr', 'name']
>    powerpc               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       s390                  syscall.uname ['argstr', 'name']
>       s390               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       ia64                  syscall.uname ['argstr', 'name']
>       ia64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>        arm                  syscall.uname ['argstr', 'name']
>        arm               nd_syscall.uname ['_func_name', 'argstr', 'name']
>      arm64                  syscall.uname ['argstr', 'name']
>      arm64               nd_syscall.uname ['_func_name', 'argstr', 'name']
> -------
>       i386                  syscall.uname ['argstr', 'name']
>       i386               nd_syscall.uname ['_func_name', 'argstr', 'name']
>     x86_64                  syscall.uname ['argstr', 'name']
>     x86_64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>    powerpc                  syscall.uname ['argstr', 'name']
>    powerpc               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       s390                  syscall.uname ['argstr', 'name']
>       s390               nd_syscall.uname ['_func_name', 'argstr', 'name']
>       ia64                  syscall.uname ['argstr', 'name']
>       ia64               nd_syscall.uname ['_func_name', 'argstr', 'name']
>        arm                  syscall.uname ['argstr', 'name']
>        arm               nd_syscall.uname ['_func_name', 'argstr', 'name']
>      arm64                  syscall.uname ['argstr', 'name']
>      arm64               nd_syscall.uname ['_func_name', 'argstr', 'name']

As Josh said, the '_func_name' variable is an internal implementation detail. Please filter it out (or I guess we could rename it to '__func_name').

> -------
>       i386       syscall.compat_fallocate ---
>       i386    nd_syscall.compat_fallocate ---
>     x86_64       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>     x86_64    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>    powerpc       syscall.compat_fallocate ---
>    powerpc    nd_syscall.compat_fallocate ---
>       s390       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'mode_str', 'name', 'offset']
>       s390    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>       ia64       syscall.compat_fallocate ---
>       ia64    nd_syscall.compat_fallocate ---
>        arm       syscall.compat_fallocate ---
>        arm    nd_syscall.compat_fallocate ---
>      arm64       syscall.compat_fallocate ---
>      arm64    nd_syscall.compat_fallocate ---
> -------
>       i386       syscall.compat_fallocate ---
>       i386    nd_syscall.compat_fallocate ---
>     x86_64       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>     x86_64    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>    powerpc       syscall.compat_fallocate ---
>    powerpc    nd_syscall.compat_fallocate ---
>       s390       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'mode_str', 'name', 'offset']
>       s390    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>       ia64       syscall.compat_fallocate ---
>       ia64    nd_syscall.compat_fallocate ---
>        arm       syscall.compat_fallocate ---
>        arm    nd_syscall.compat_fallocate ---
>      arm64       syscall.compat_fallocate ---
>      arm64    nd_syscall.compat_fallocate ---
> -------
>       i386       syscall.compat_fallocate ---
>       i386    nd_syscall.compat_fallocate ---
>     x86_64       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>     x86_64    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>    powerpc       syscall.compat_fallocate ---
>    powerpc    nd_syscall.compat_fallocate ---
>       s390       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'mode_str', 'name', 'offset']
>       s390    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>       ia64       syscall.compat_fallocate ---
>       ia64    nd_syscall.compat_fallocate ---
>        arm       syscall.compat_fallocate ---
>        arm    nd_syscall.compat_fallocate ---
>      arm64       syscall.compat_fallocate ---
>      arm64    nd_syscall.compat_fallocate ---
> -------
>       i386       syscall.compat_fallocate ---
>       i386    nd_syscall.compat_fallocate ---
>     x86_64       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>     x86_64    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>    powerpc       syscall.compat_fallocate ---
>    powerpc    nd_syscall.compat_fallocate ---
>       s390       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'mode_str', 'name', 'offset']
>       s390    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> 'name', 'offset']
>       ia64       syscall.compat_fallocate ---
>       ia64    nd_syscall.compat_fallocate ---
>        arm       syscall.compat_fallocate ---
>        arm    nd_syscall.compat_fallocate ---
>      arm64       syscall.compat_fallocate ---
>      arm64    nd_syscall.compat_fallocate ---
> -------

Is the compat_fallocate report duplicated 3 extra times or am I missing something?

In the case of compat_fallocate (and really any other syscall), it is OK that it doesn't exist on some kernels. When it does exist, the variables should match, which it appears they do. 

Notice that with the exception of powerpc, the other failing platforms for compat_fallocate don't have compat functions since they are either 32-bit (i386 and arm) or are 64-bit but CONFIG_COMPAT isn't defined (ia64 and arm64).

I'm not sure why ppc64 wouldn't need a compat fallocate probe. Does the fallocate syscall test pass there (and does it get run in 32-bit mode)?

(In reply to David Smith from comment #2)

... stuff deleted ...

> > -------
> >       i386       syscall.compat_fallocate ---
> >       i386    nd_syscall.compat_fallocate ---
> >     x86_64       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> > 'name', 'offset']
> >     x86_64    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> > 'name', 'offset']
> >    powerpc       syscall.compat_fallocate ---
> >    powerpc    nd_syscall.compat_fallocate ---
> >       s390       syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> > 'mode_str', 'name', 'offset']
> >       s390    nd_syscall.compat_fallocate ['argstr', 'fd', 'len', 'mode',
> > 'name', 'offset']
> >       ia64       syscall.compat_fallocate ---
> >       ia64    nd_syscall.compat_fallocate ---
> >        arm       syscall.compat_fallocate ---
> >        arm    nd_syscall.compat_fallocate ---
> >      arm64       syscall.compat_fallocate ---
> >      arm64    nd_syscall.compat_fallocate ---
> > -------
> 

... stuff deleted ...

> 
> Notice that with the exception of powerpc, the other failing platforms for
> compat_fallocate don't have compat functions since they are either 32-bit
> (i386 and arm) or are 64-bit but CONFIG_COMPAT isn't defined (ia64 and
> arm64).
> 
> I'm not sure why ppc64 wouldn't need a compat fallocate probe.

It's because the "cross compilation" doesn't work as I expected. It seems to take into account properties of currently installed kernel (x86_64) although it's architecture doesn't match the `-a` requested (powerpc). I checked that my fedora x86_64 kernel doesn't have kprobe.function("compat_sys_fallocate") but it does have kprobe.function("sys_fallocate"). 

=======
fc22 x86_64 # stap -l 'kprobe.function("compat_sys_fallocate")'
fc22 x86_64 # stap -l 'kprobe.function("sys_fallocate")'
kprobe.function("sys_fallocate")
fc22 x86_64 # 
=======

This has following (surprising for me) effect for the powerpc/nd_syscalls tapset:

=======
fc22 x86_64 # grep 'compat_fallocate\ ' powerpc/nd_syscalls.stp 
probe nd_syscall.compat_fallocate = kprobe.function("compat_sys_fallocate") ?
fc22 x86_64 # stap -a powerpc -L 'nd_syscall.*fallocate*' 2>/dev/null
nd_syscall.fallocate name:unknown fd:unknown mode:unknown mode_str:unknown offset:unknown len:unknown argstr:unknown
fc22 x86_64 # 
fc22 x86_64 # vim powerpc/nd_syscalls.stp # edit the compat_fallocate probe
fc22 x86_64 # 
fc22 x86_64 # grep 'compat_fallocate\ ' powerpc/nd_syscalls.stp 
probe nd_syscall.compat_fallocate = kprobe.function("sys_fallocate") ?
fc22 x86_64 # stap -a powerpc -L 'nd_syscall.*fallocate*' 2>/dev/null
nd_syscall.compat_fallocate name:unknown fd:unknown mode:unknown offset:unknown len:unknown argstr:unknown
nd_syscall.fallocate name:unknown fd:unknown mode:unknown mode_str:unknown offset:unknown len:unknown argstr:unknown
fc22 x86_64 #
=======

So clearly the `-a` switch has *some* effect when running `stap -L`, but not a "clear cross-compilation" happens here.

> Does the fallocate syscall test pass there (and does it get run in 32-bit mode)?

Yes, it does pass on the powerpc architecture both in 64 and in 32 bit mode.

Created attachment 8531 [details]
proposed testcase (and a few trivial tapset fixes)

I rewrote the testcase to dejagnu and attached here as a proposal. This way it'll only get run on a "native" arch; No "cross compilation" happens, and thus the issue from previous comment shouldn't happen here.

The testcase checks if
  - all syscalls have a nd_syscall counterpart (and vice-versa)
  - list of locals (convenience variables) of each syscall match
    respective list of respective nd_syscall (and vice versa)

I found a few issues using it. Proposed fixes for trivial el[67] ones are included as a part of attached patch. Fix for following mmap/mmap2/mmap32 issue is not included in the attached patch:

6.7 S s390x:
=======
FAIL: unmatched locals for mmap:
syscall.mmap:  argstr fd flags len name offset prot start
nd_syscall.mmap:  argstr name
FAIL: unmatched locals for mmap2:
syscall.mmap2:  argstr fd flags length name pgoffset prot start
nd_syscall.mmap2:  argstr name
=======

6.7 S x86_64:
=======
FAIL: unmatched locals for mmap32:
syscall.mmap32:  argstr fd flags len name offset prot start
nd_syscall.mmap32:  argstr name
FAIL: unmatched locals for mmap2:
syscall.mmap2:  argstr fd flags length name pgoffset prot start
nd_syscall.mmap2:  argstr name
=======

I plan to open a separate PR for this.

(In reply to Josh Stone from comment #1)
> It looks like uname's "_func_name" is also just an implementation detail.
> I'd guess single underscores can be filtered too.

Note that the above patch also includes SystemTap_Tapset_Reference/syscalls.py update that hides "_func_name" too in docs.

(In reply to Martin Cermak from comment #4)
> Created attachment 8531 [details]
> proposed testcase (and a few trivial tapset fixes)
> 
> I rewrote the testcase to dejagnu and attached here as a proposal. This way
> it'll only get run on a "native" arch; No "cross compilation" happens, and
> thus the issue from previous comment shouldn't happen here.
> 
> The testcase checks if
>   - all syscalls have a nd_syscall counterpart (and vice-versa)
>   - list of locals (convenience variables) of each syscall match
>     respective list of respective nd_syscall (and vice versa)

The testcase needs some comments explaining what it is doing, especially the bit about copying the tapset and changing 'kernel.function' to 'kprobe.function'. Why is that necessary?

Created attachment 8540 [details]
updated patch

(In reply to Martin Cermak from comment #7)
> Created attachment 8540 [details]
> updated patch

Thanks for the comments in the test case. I don't really think you need the bit about copying the tapset and changing 'kernel.function' to 'kprobe.function'. Yes, the results will depend on debuginfo-quality (and systemtap's ability to parse it), but so do the rest of the testsuite results.

Created attachment 8566 [details]
updated patch

(In reply to David Smith from comment #8)
> Thanks for the comments in the test case. I don't really think you need the
> bit about copying the tapset and changing 'kernel.function' to
> 'kprobe.function'. Yes, the results will depend on debuginfo-quality (and
> systemtap's ability to parse it), but so do the rest of the testsuite
> results.

Updated the testcase to reflect the above. Additionally fixed one '.return' -> '.call' issue in 'syscall.mq_timedsend' probe. Does the patch look good now?

(In reply to Martin Cermak from comment #9)
> Created attachment 8566 [details]
> updated patch
> 
> (In reply to David Smith from comment #8)
> > Thanks for the comments in the test case. I don't really think you need the
> > bit about copying the tapset and changing 'kernel.function' to
> > 'kprobe.function'. Yes, the results will depend on debuginfo-quality (and
> > systemtap's ability to parse it), but so do the rest of the testsuite
> > results.
> 
> Updated the testcase to reflect the above. Additionally fixed one '.return'
> -> '.call' issue in 'syscall.mq_timedsend' probe. Does the patch look good
> now?

The new test looks much simpler now. Looks good to me.

Fixed in commit eb73b74a4abcb9c85c3fc4c1e411ff19ff5d1460