Bug 2111

Summary: document syscalls tapset
Product: systemtap Reporter: Frank Ch. Eigler <fche>
Component: tapsetsAssignee: Unassigned <systemtap>
Status: RESOLVED FIXED    
Severity: normal CC: dsmith, jistone, lberk, mcermak
Priority: P1    
Version: unspecified   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:
Bug Depends on: 2060    
Bug Blocks:    
Attachments: proposed patch
A pdf output acquired using aforementioned patch on f22/x86_64.
updated patch
Outputs acquired using patch from comment 15

Description Frank Ch. Eigler 2006-01-05 20:11:07 UTC
 
Comment 1 Frank Ch. Eigler 2006-01-18 20:00:42 UTC
If you're still interested, I'll work on optimizing out unused $target
assignments, which should enable you to produce a more portable/general
syscalls.stp tapset.
Comment 2 Martin Hunt 2006-01-18 20:10:53 UTC
Good. I've already started.  Once your work is checked in, I can test with older
kernel/compilers and get the tapset checked in.
Comment 3 Frank Ch. Eigler 2006-01-28 19:07:42 UTC
*** Bug 907 has been marked as a duplicate of this bug. ***
Comment 4 Frank Ch. Eigler 2006-04-21 21:02:43 UTC
Good progress has been made, though the work is still incomplete.
For example, there are still many "kernel.syscall.*" aliases as well as
"syscall.*" ones.

Plus there appears to be no test case yet that forces a reference to every
auxiliary embedded-C function, which would let us remove the "-u" flag from
the buildok/syscall.stp test.
Comment 5 Frank Ch. Eigler 2006-05-09 15:09:14 UTC
Another important element of finishing this work is to document the syscalls
tapset in the stapprobes.5 man page.  There is probably no need to elaborately
enumerate all couple of hundred in the man page, but the general conventions
should be there:
- what general variables are available ("name", "argstr", ...)
- what is done for user-space strings, decomposed struct fields
Comment 6 Frank Ch. Eigler 2006-06-02 14:21:05 UTC
Regarding the growing test suite in systemtap.syscalls, consider
reworking it so that you don't need so many little .c programs to
exercise the system.  A single C program could do the job just
fine, if it is parametrized ("syscall-test NAME") and thus reused
by many tests.
Comment 7 Martin Hunt 2006-06-02 16:48:11 UTC
Subject: Re:  reorganize syscalls tapset

On Fri, 2006-06-02 at 14:21 +0000, fche at redhat dot com wrote:
> ------- Additional Comments From fche at redhat dot com  2006-06-02 14:21 -------
> Regarding the growing test suite in systemtap.syscalls, consider
> reworking it so that you don't need so many little .c programs to
> exercise the system.  A single C program could do the job just
> fine, if it is parametrized ("syscall-test NAME") and thus reused
> by many tests.

It is not clear to me what the benefit of doing this would be. It could
be done, but small self-contained test programs are easier to manage. 


Comment 8 Frank Ch. Eigler 2010-04-08 19:12:44 UTC
It may be sufficient for now to excise the section for the syscall
tapset from stapprobes.3stap and put them into a separate tapset::syscall
manpage.
Comment 9 Lukas Berk 2011-08-15 18:21:00 UTC
As mentioned in PR11775, at some point aux_syscalls.stp, nd_syscalls.stp, nd_syscalls2.stp, syscalls.stp, syscalls2.stp, and syscalls_cfg_trunc.stp should probably be mentioned/added to doc/SystemTap_Tapset_Reference/tapsets.tmpl
Comment 10 Martin Cermak 2015-07-29 18:18:50 UTC
(In reply to Frank Ch. Eigler from comment #5)
> Another important element of finishing this work is to document the syscalls
> tapset in the stapprobes.5 man page.  There is probably no need to
> elaborately
> enumerate all couple of hundred in the man page, but the general conventions
> should be there:
> - what general variables are available ("name", "argstr", ...)
> - what is done for user-space strings, decomposed struct fields

On  Tue  2015-07-21  14:40 , David Smith wrote:                                    
> We've about polished the [nd_]syscall tapsets to a fairly brilliant           
> shine as far as the code goes, but those tapsets aren't documented -             
> PR2111: <https://sourceware.org/bugzilla/show_bug.cgi?id=2111>. Although         
> this sounds a bit boring, there are actually some thinking that needs            
> doing there - like how to handle the nd_syscall probes vs. the syscall           
> probes. We might want to combine the documentation into one set instead          
> of two, but how do you do that cleanly?                                          
>                                                                                  
                                                                                   
So this needs parser. Either stap's one, or some external one.                     
So firstly I'll think about how to resuse the former. What's                       
somewhat close to what we need is `stap -L 'syscall.*,nd_syscall.*'`.              
I see two drawbacks here:                                                          
                                                                                   
1) This includes conditionals evaluation and thus output is                        
  architecture (and otherwise) specific. I think disabling
  conditionals evaluation might be doable.                                                      
                                                                                   
2) We don't have correct datatype information. For dwarf                           
   based probes, types do not match due to syscall wrappers, for                   
   non-dwarf based syscalls we don't know types at all. Tapset                     
   source could be manually extended to contain type information                   
   say, using comments, and then documentation could be                             
   autogenerated. But probably not using stap's internal parser.                   
   At least not without significant tweaks.                                        
                                                                                   
So using stap parser with conditionals evaluation disabled, collecting
syscalls list, matching syscalls against nd_syscalls and collecting 
convenience variables list per syscall probe (without type information)
might be doable. Thoughts?
Comment 11 Martin Cermak 2015-08-05 11:38:44 UTC
When 'kernel.function' gets replaced by 'kprobe.function' in whole the tapset, I seem to be able to get syscalls list per architecture using cross-compilation (without having respective kernel-debuginfo):

=======
fc20 x86_64 # cat syscalls_per_arch 
MYARCH=$1

ORIG=/usr/share/systemtap/tapset
TEMP=`mktemp -d`

cp -r ${ORIG}/* ${TEMP}/
find $TEMP -type f -exec sed -i 's/kernel\.function/kprobe\.function/g' {} \;

export SYSTEMTAP_TAPSET=$TEMP
stap -a $MYARCH -L 'syscall.*' 2> /dev/null | perl -anle 'print shift @F, " @{[sort @F]}"' | sort

rm -rf $TEMP



fc20 x86_64 # ./syscalls_per_arch s390 | tail
syscall.utimensat argstr:unknown dfd:unknown dfd_str:unknown filename:unknown filename_uaddr:unknown flags:unknown flags_str:unknown name:unknown tsp_uaddr:unknown
syscall.utimes argstr:unknown filename:unknown filename_uaddr:unknown name:unknown tvp_uaddr:unknown
syscall.vfork argstr:unknown name:unknown
syscall.vhangup argstr:unknown name:unknown
syscall.vmsplice argstr:unknown name:unknown
syscall.wait4 argstr:unknown name:unknown options:unknown options_str:unknown pid:unknown rusage_uaddr:unknown status_uaddr:unknown
syscall.waitid argstr:unknown infop_uaddr:unknown name:unknown options:unknown options_str:unknown pid:unknown rusage_uaddr:unknown which:unknown which_str:unknown
syscall.waitpid argstr:unknown name:unknown options:unknown options_str:unknown pid:unknown status_uaddr:unknown
syscall.write argstr:unknown buf_uaddr:unknown count:unknown fd:unknown name:unknown
syscall.writev argstr:unknown count:unknown fd:unknown name:unknown vector_uaddr:unknown
fc20 x86_64 # 
=======

With nd_syscalls this is even more straightforward. I wonder if this is worth of some elaboration.
Comment 12 Martin Cermak 2015-08-12 13:21:39 UTC
Created attachment 8508 [details]
proposed patch

Attached patch generates a table of [nd_]syscalls and list of their convenience variables for me.
Comment 13 Martin Cermak 2015-08-12 13:23:01 UTC
Created attachment 8509 [details]
A pdf output acquired using aforementioned patch on f22/x86_64.
Comment 14 Josh Stone 2015-08-13 20:09:05 UTC
1. The table would be a lot more useful if it were sorted by syscall name.

2. The "argstr" and "name" should be common to all syscall probes.  Let's mention those in the intro section and filter them from the rest of the table.  Might also mention "retstr" for syscall return probes.

3. Underscored names are most likely "private" locals, like the "__nr" that frequently appears.  Let's filter them.

4. I think any differences between syscall and nd_syscall locals should be considered bugs, not documented features.  But AFAICS the only differences right now are underscored names -- see point 3.  So maybe we can report any additional differences with a build-time warning, or even an error, but I don't think they need any mention in the document.

5. (off-topic) That 15-page Table of Contents is kind of ridiculous IMO.  I'm glad your syscall addition didn't get fully enumerated there too!  I think we should probably reduce the TOC to just major section headings, and maybe add a fuller index at the end of the document.
Comment 15 Martin Cermak 2015-08-14 11:12:31 UTC
Created attachment 8520 [details]
updated patch

Attached patch tries to address aforementioned comments plus it adds a 3stap syscall man page. Using 'make rpm' I seem to get expected outputs.
Comment 16 Martin Cermak 2015-08-14 11:13:32 UTC
Created attachment 8521 [details]
Outputs acquired using patch from comment 15
Comment 17 Martin Cermak 2015-08-14 12:54:55 UTC
(In reply to Josh Stone from comment #14)
> 4. I think any differences between syscall and nd_syscall locals should be
> considered bugs, not documented features.  But AFAICS the only differences
> right now are underscored names -- see point 3.  So maybe we can report any
> additional differences with a build-time warning, or even an error, but I
> don't think they need any mention in the document.

Will try to fix this within PR18827.
Comment 18 Martin Cermak 2015-08-14 14:14:47 UTC
Fixed in commit 86204b9b663525f5a9d42aabf9989b59c50ef0c1