Re: Regarding systemtap support for AArch64

[William Cohen <wcohen at redhat dot com>]

On 09/26/2013 10:57 AM, Sandeepa Prabhu wrote:
> On 26 September 2013 20:05, William Cohen <wcohen@redhat.com> wrote:
>> On 09/25/2013 11:13 PM, Sandeepa Prabhu wrote:
>>> Hi Will, Masami,
>>>
>>> Nice to hear from you, I am using ARM fast model/Foundation model with
>>> ARM v8 upstream kernel and a Linaro minimal busybox based ramdisk,
>>> testing with loadable modules for now (so don't have dependency on
>>> elfutils or GCC autoconf etc)
>>>
>>> Would you be interested to use Linaro kprobe work as a base for
>>> development and validation of systemtap-aarch64?  We are happy to
>>> share a public git repo where 'upstream' kernel can be built with
>>> kprobes support, which systemtap team can use for verification.  I can
>>> do this soon as I have most things working locally, except for
>>> kretprobes and 'boosting' support(systemtap can be run without these I
>>> believe).
>>>
>>> Looking forward for a collaboration :-)
>>>
>>> Cheers,
>>> Sandeepa
>>
>> Hi Sandeepa,
>>
>> It is very nice to hear that Linaro has been working on aarch64 kprobes.  Is there a git repository or kernel available that has the aarch64 patches?  Also is there a recommend config file and instructions on how to get a new kernel running on the simulator?
>>
>> We would like to get systemtap working on aarch64.  There are a number architecture specific things that need to be adjusted in systemtap to support aarch64.  A few of the changes have been pushed into the systemtap git repo, but more are needed in the runtime and some more in the tapset.
>>
>> Locally I have a ARM fast model/Foundation model set up with Fedora 19.  It can build systemtap from source, but it is pretty slow. Running the systemtap testsuite will probably require access to actual hardware. Otherwise tests will timeout and just fail.
> 
> Hi Will,
> 
> I would create a linaro git repository and upload all the patches and
> config file that are needed to build the kernel with kprobes support,
> doing it sometimes early next week and share it on systemtap mailing
> list.

Hi Sandeepa,

I went though and made a minimal patch (aarch64_changes.patch) that allows systemtap to translate a simple hello world script into c code.  The kernel on my software armv8 simulator is a cross compiled version and some of the scripts programs in there don't work because they are ARM-64 executable so it failed when doing the actual compile of module:  

sudo ../install/bin/stap  -v -m hello -r 3.12.0-0.rc0.git20.1.x2.fc19.aarch64 -p4 -k -e 'probe begin {printf("hello world\n")}'
Pass 1: parsed user script and 92 library script(s) using 38784virt/30016res/550
4shr/23360data kb, in 4700usr/90sys/4805real ms.
Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0 global(s) usin
g 39296virt/31936res/5952shr/23872data kb, in 90usr/0sys/81real ms.
Pass 3: translated to C into "/tmp/stapy7iaug/hello_src.c" using 39296virt/31936
res/5952shr/23872data kb, in 0usr/0sys/7real ms.
/bin/sh: scripts/basic/fixdep: cannot execute binary file
make[1]: *** [/tmp/stapy7iaug/hello_src.o] Error 126
make: *** [_module_/tmp/stapy7iaug] Error 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "hello.ko" in 110730usr/3270sys/115966real ms.
Pass 4: compilation failed.  [man error::pass4]
Keeping temporary directory "/tmp/stapy7iaug"

-Will

> As I said, I am using minimal setup (busybox based) and using only
> sample kernel modules to verify, have not used sophisticated rootfs
> like fedora or ubuntu on the models since any GUI will be run very
> slow.  Alternately, do you consider using lightweight command-line
> system like LAMP stack (available with linaro public builds) and
> cross-compile the systemtap and dependent tools on open-embedded
> environment. I understand that minimal system is far away from real
> environment where systemtap will be used (like fedora, enterprise
> servers) but it should serve as temporary setup until the real
> hardware is available.
> 
> Thanks,
> Sandeepa

diff --git a/runtime/linux/copy.c b/runtime/linux/copy.c
index fc730d2..2bdb52a 100644
--- a/runtime/linux/copy.c
+++ b/runtime/linux/copy.c
@@ -24,9 +24,6 @@
  * @{
  */
 
-
-static long __stp_strncpy_from_user(char *dst, const char __user *src, long count);
-
 #ifdef CONFIG_GENERIC_STRNCPY_FROM_USER
 #define __stp_strncpy_from_user(dst,src,count,res) \
 	do { res = strncpy_from_user(dst, src, count); } while(0)
@@ -91,7 +88,7 @@ do {									   \
 #define __stp_strncpy_from_user(dst,src,count,res) \
 	do { res = __strncpy_from_user(dst, src, count); } while(0)
 
-#elif defined (__s390__) || defined (__s390x__)
+#elif defined (__s390__) || defined (__s390x__)|| defined (__aarch64__)
 #define __stp_strncpy_from_user(dst,src,count,res) \
 	do { res = strncpy_from_user(dst, src, count); } while(0)
 #elif defined (__ia64__)
diff --git a/runtime/linux/loc2c-runtime.h b/runtime/linux/loc2c-runtime.h
index e01b7f4..060b8c3 100644
--- a/runtime/linux/loc2c-runtime.h
+++ b/runtime/linux/loc2c-runtime.h
@@ -563,6 +563,56 @@ extern void __store_deref_bad(void);
       STORE_DEREF_FAULT(addr);						      \
   })
 
+#elif defined (__aarch64__)
+
+#define _stp_deref(size, addr, seg)                                           \
+  ({									      \
+    int _bad = 0;							      \
+    intptr_t _v = 0;							      \
+    mm_segment_t _oldfs = get_fs();                                           \
+    set_fs(seg);                                                              \
+    pagefault_disable();                                                      \
+    if (lookup_bad_addr((unsigned long)addr, size))			      \
+      _bad = 1;                                                               \
+    else                                                                      \
+      switch (size)                                                           \
+        {                                                                     \
+	case 1: __get_user_asm("ldrb", "%w", _v, (unsigned long)addr, _bad); break;\
+	case 2: __get_user_asm("ldrh", "%w",_v, (unsigned long)addr, _bad); break;\
+	case 4: __get_user_asm("ldr", "%w",_v,  (unsigned long)addr, _bad); break;\
+	case 8: __get_user_asm("ldr", "%",_v,  (unsigned long)addr, _bad); break;\
+        default: BUILD_BUG();			                              \
+        }                                                                     \
+    pagefault_enable();                                                       \
+    set_fs(_oldfs);                                                           \
+    if (_bad)								      \
+      DEREF_FAULT(addr);						      \
+    _v;									      \
+  })
+
+#define _stp_store_deref(size, addr, value, seg)                              \
+  ({									      \
+    int _bad = 0;							      \
+    mm_segment_t _oldfs = get_fs();                                           \
+    set_fs(seg);                                                              \
+    pagefault_disable();                                                      \
+    if (lookup_bad_addr((unsigned long)addr, size))			      \
+      _bad = 1;                                                               \
+    else                                                                      \
+      switch (size)                                                           \
+        {                                                                     \
+	case 1: __put_user_asm("strb", "%w", ((u8)(value)), addr, _bad); break;\
+	case 2: __put_user_asm("strh", "%w", ((u16)(value)), addr, _bad); break;\
+	case 4: __put_user_asm("str", "%w", ((u32)(value)), addr, _bad); break;\
+	case 8: __put_user_asm("str", "%", ((u64)(value)), addr, _bad); break;\
+        default: BUILD_BUG();                                                 \
+        }                                                                     \
+    pagefault_enable();                                                       \
+    set_fs(_oldfs);                                                           \
+    if (_bad)								      \
+      STORE_DEREF_FAULT(addr);						      \
+  })
+
 #elif defined (__arm__)
 
 /* Macros for ARM lifted from 2.6.21.1's linux/include/asm-arm/uaccess.h
diff --git a/runtime/loc2c-runtime.h b/runtime/loc2c-runtime.h
index ca9ad11..bd13341 100644
--- a/runtime/loc2c-runtime.h
+++ b/runtime/loc2c-runtime.h
@@ -154,6 +154,15 @@
 #define pt_regs_store_register(pt_regs,regno,value) \
   (pt_regs->gpr[regno] = (value))
 
+#elif defined (__aarch64__)
+
+#undef pt_regs_fetch_register
+#undef pt_regs_store_register
+#define pt_regs_fetch_register(pt_regs,regno) \
+  ((long) pt_regs->regs[regno])
+#define pt_regs_store_register(pt_regs,regno,value) \
+  (pt_regs->regs[regno] = (value))
+
 #elif defined (__arm__)
 
 #undef pt_regs_fetch_register
diff --git a/runtime/regs.h b/runtime/regs.h
index dbe41ab..2f027ad 100644
--- a/runtime/regs.h
+++ b/runtime/regs.h
@@ -42,6 +42,12 @@
 #define REG_SP(regs) regs->gpr[1]
 #define REG_LINK(regs) regs->link
 
+#elif defined (__aarch64__)
+
+#define REG_IP(regs) regs->pc
+#define REG_SP(regs) regs->sp
+#define REG_LINK(regs) regs->regs[30]
+
 #elif defined (__arm__)
 
 #define REG_IP(regs) regs->ARM_pc
diff --git a/runtime/time.c b/runtime/time.c
index 068db0a..7ae07af 100644
--- a/runtime/time.c
+++ b/runtime/time.c
@@ -75,7 +75,7 @@ __stp_get_freq(void)
     // If we can get the actual frequency of HW counter, we use it.
 #if defined (__ia64__)
     return local_cpu_data->itc_freq / 1000;
-#elif defined (__s390__) || defined (__s390x__) || defined (__arm__)
+#elif defined (__s390__) || defined (__s390x__) || defined (__arm__) || defined (__aarch64__)
     // We don't need to find the cpu freq on s390 as the 
     // TOD clock is always a fix freq. (see: POO pg 4-36.)
     return 0;
@@ -83,7 +83,7 @@ __stp_get_freq(void)
     return tsc_khz;
 #elif (defined (__i386__) || defined (__x86_64__)) && defined(STAPCONF_CPU_KHZ)
     return cpu_khz;
-#else /* __i386__ || __x86_64__ || __ia64__ || __s390__ || __s390x__ || __arm__*/
+#else /* __i386__ || __x86_64__ */
     // If we don't know the actual frequency, we estimate it.
     cycles_t beg, mid, end;
     beg = get_cycles(); barrier();