SystemTap's reduced memory usage

[Abe Jakop <ajakop at redhat dot com>]

Hello everyone,

Work has been done to reduce the amount of memory SystemTap consumes.

A need for this optimization was brought up by the case where there is a
large amount of outside tapsets, such that stap's excessive memory
usage shows.

To optimize SystemTap's memory usage, struct interned_string was introduced
as a wrapper on Boost's string_ref. These interned_strings are meant to
replace a handful of instances of std::strings. 

The new interned_strings are meant to act like pointers to existing
strings. With this change, instances where new duplicate strings were
created are now replaced with a sort of pointer to the original string's
location, reducing the number of new strings that are generated.

These changes are already merged into the master branch. To try out the
changes, see the master branch of the git repo:

https://sourceware.org/git/gitweb.cgi?p=systemtap.git

Notable changes would be the inclusion of files stringtable.h and
stringtable.cxx, and the new peculiarities sections in INTERNALS.

____
NOTE

This optimization is reliant on having Boost version >=1.53, such that
boost::string_ref is available. To see if boost string_ref is available
for use by stap, check for BOOST_STRING_REF in the list of enabled
features from "stap -V".

______
IMPACT

With some outside tapsets installed to increase how much has to be parsed,
the tapset folder ended up being >11M. 


For a simple script, before the optimization:
$ valgrind --tool=massif stap -e "probe begin{exit()}"
peak cost: "203.9 MiB"  heap "58.8 MiB"  heap extra "0 B"  stacks
$ time stap -e "probe begin{exit()}"
real    0m1.382s
user    0m1.032s
sys     0m0.076s

After:
$ valgrind --tool=massif stap -e "probe begin{exit()}"
peak cost: "176.9 MiB"  heap "49.7 MiB"  heap extra "0 B"  stacks
$ time stap -e "probe begin{exit()}"
real    0m1.398s
user    0m1.051s
sys     0m0.074s

The peak memory usage has a decrease of ~15 % and a minimal change
in runtime (hard to tell when it's that small and fluctuates a bit).


On the other hand, there is stap -L "**" which is what brought the
need for this optimization to our attention.

Before:
$ time stap -L "**" > /dev/null
real    3m47.497s
user    3m35.774s
sys     0m8.728s
$ valgrind --tool=massif stap -L "**"
[...]
peak cost: "753.8 MiB"  heap "185.2 MiB"  heap extra "0 B"  stacks


After:
$ time stap -L "**" > /dev/null
real    3m27.391s
user    3m18.245s
sys     0m8.628s
$ valgrind --tool=massif stap -L "**"
[...]
peak cost: "710.2 MiB"  heap "175.2 MiB"  heap extra "0 B"  stacks


The peak memory usage has a decrease of ~6 % and the runtime saw a
decrease of ~9 %.


More work can be done to take advantage of interned_strings and reduce
the memory consumption even further.

Comments and feedback are always welcome.

Abegail Jakop