Re: Regarding systemtap support for AArch64

[William Cohen <wcohen at redhat dot com>]

On 10/24/2013 12:19 AM, Sandeepa Prabhu wrote:
> On 24 October 2013 07:20, William Cohen <wcohen@redhat.com> wrote:
>> On 10/02/2013 12:17 AM, Sandeepa Prabhu wrote:
>>> Hi all,
>>>
>>> I have uploaded ARM64 kprobes work on Linaro public git:
>>> git://git.linaro.org/people/sandeepa.prabhu/linux-aarch64.git  Branch:
>>> kprobes_devel_v8.  Patches are published on LAKML too.  This is based
>>> on v8 upstream kernel (3.12-rc1) right now, and works with linaro
>>> boot-wrapper and fast model setup, though, not sure what it takes to
>>> build for fedora.
>>>
>>> Will,
>>>
>>> Is aarch64 fc19 port  public? I am interested in using fc on v8 fast
>>> model, are there instructions about how to get the packages and
>>> build/run them?
>>>
>>> Thanks,
>>> Sandeepa
>>
>> Hi Sandeepa,
>>
>> I finally got a locally built aarch64 kernel with the kprobe patches
>> built and running on the simulator.  The SystemTap tapsets.cxx needed to be
>> patched to understand the aarch64 (the attached patch). With that patch the
>> SystemTap Beginner's guide smoke test example showed signs of life!
>> However, it took minutes for it to compile and run.
>>
>> [wcohen@localhost systemtap]$ sudo ../install/bin/stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
>> [sudo] password for wcohen:
>> Pass 1: parsed user script and 92 library script(s) using 140528virt/24952res/2744shr/22868data kb, in 4740usr/160sys/5319real ms.
>> Pass 2: analyzed script: 1 probe(s), 1 function(s), 3 embed(s), 0 global(s) using 326804virt/106856res/3568shr/103752data kb, in 33860usr/14520sys/52942real ms.
>> Pass 3: translated to C into "/tmp/stap6Cbu2d/stap_3a0ef010f6cdfb2bdc3cac691a0f3c0e_1393_src.c" using 326804virt/109236res/5948shr/103752data kb, in 120usr/30sys/163real ms.
>> Pass 4: compiled C into "stap_3a0ef010f6cdfb2bdc3cac691a0f3c0e_1393.ko" in 117620usr/10440sys/141829real ms.
>> Pass 5: starting run.
>> read performed
>> Pass 5: run completed in 100usr/90sys/573real ms.
> Hi Will,
> 
> Great to know this!, Thanks for the patch. Hope you have taken v2
> version of kprobe patches, found here:
> https://git.linaro.org/gitweb?p=people/sandeepa.prabhu/linux-aarch64.git;a=shortlog;h=refs/heads/arm64-kprobes-v2
> v2 is cleaner, and implements recursive kprobes (like probing printk)
> and also few fixes in missed-kprobe handling.
> So, do you think we can measure kprobes performance running on v8 fast
> models (slow model though :))? the milliseconds it shows for
> completion will be accurate only if have real hardware right?
> 
> From this weekend thru next week, we are mostly under travel to Linaro
> connect @Santa Clara, so may not spend more time on this, but we would
> like to start validating the kprobes using systemtap and may get more
> hands within Linaro to work with systemtap community. If anyone is
> attending LCU this time, ping me and we can meet.
> 
> Cheers,
> Sandeepa

Hi Sandeepa,

The run above was with the old armv8 kprobes patches.  I just switched my kernel sources to the arm64-kprobes-v2 branch this morning and building a new kernel.  It is building with the minimal modules needed for the armv8 simulator, but it will take a while because I am building it on the armv8 simulator, so that all the kernel-devel support is available for systemtap.

The fedora 19 image I am using uses uefi to boot up.  There are additional patches that I needed to pull in from  https://github.com/mosalter/linux the armv8-uefi-latest branch to make a kernel that boot on uefi.  

It does appear to be able to read parameter values, but the $return isn't available:

[wcohen@localhost systemtap]$ sudo ../install/bin/stap -v -e 'probe vfs.read {printf("read performed %s\n", $$parms$); exit()}'
Pass 1: parsed user script and 92 library script(s) using 140532virt/24956res/2744shr/22872data kb, in 4790usr/110sys/5315real ms.
Pass 2: analyzed script: 1 probe(s), 21 function(s), 3 embed(s), 0 global(s) using 326840virt/106908res/3580shr/103788data kb, in 34110usr/8180sys/45930real ms.
Pass 3: translated to C into "/tmp/stapso4oWX/stap_165197dc0524f0f70a7a3b68cb1cb26f_12311_src.c" using 326840virt/109304res/5976shr/103788data kb, in 180usr/10sys/200real ms.
Pass 4: compiled C into "stap_165197dc0524f0f70a7a3b68cb1cb26f_12311.ko" in 49890usr/3230sys/58657real ms.
Pass 5: starting run.
read performed file={.f_u={...}, .f_path={...}, .f_inode=0xffffffc04f5ff600, .f_op=0xffffffbffc0c05b8, .f_lock={...}, .f_sb_list_cpu=0, .f_count={...}, .f_flags=133122, .f_mode=31, .f_pos=0, .f_owner={...}, .f_cred=0xffffffc05113d9c0, .f_ra={...}, .f_version=0, .f_security=0xffffffc051158900, .private_data=0x0, .f_ep_links={...}, .f_tfile_llink={...}, .f_mapping=0xffffffc04f5ff750} buf="" count=8196 pos=-273515643216
Pass 5: run completed in 120usr/70sys/508real ms.

However, $return doesn't work:

[wcohen@localhost systemtap]$ sudo ../install/bin/stap -v -e 'probe vfs.read.return {printf("read performed %d\n", $return); exit()}'
[sudo] password for wcohen: 
Pass 1: parsed user script and 92 library script(s) using 140532virt/24956res/2744shr/22872data kb, in 4830usr/70sys/5316real ms.
semantic error: failed to retrieve return value location for vfs_read [man error::dwarf] (fs/read_write.c): identifier '$return' at <input>:1:54
        source: probe vfs.read.return {printf("read performed %d\n", $return); exit()}
                                                                     ^

Pass 2: analyzed script: 2 probe(s), 6 function(s), 6 embed(s), 0 global(s) using 326832virt/106912res/3584shr/103780data kb, in 34190usr/8060sys/45880real ms.
Pass 2: analysis failed.  [man error::pass2]

-Will



> 
>> [wcohen@localhost systemtap]$ uname -a
>> Linux localhost 3.12.0-rc5+ #3 SMP Wed Oct 23 15:03:27 EDT 2013 aarch64 aarch64 aarch64 GNU/Linux
>>
>> -Will