This is the mail archive of the
gdb@sourceware.org
mailing list for the GDB project.
Re: Re: gdb and fission scheme (gcc: -gsplit-dwarf, gnu ld gold :--gdb-index )
- From: "ISHIKAWA,chiaki" <ishikawa at yk dot rim dot or dot jp>
- To: gdb at sourceware dot org
- Cc: "ISHIKAWA,chiaki" <ishikawa at yk dot rim dot or dot jp>
- Date: Sun, 22 Sep 2013 11:29:23 +0900
- Subject: Re: Re: gdb and fission scheme (gcc: -gsplit-dwarf, gnu ld gold :--gdb-index )
- Authentication-results: sourceware.org; auth=none
- References: <523CC176 dot 9010309 at yk dot rim dot or dot jp>
I have investigated a little further.
Now it has dawned on me that using "-gsplit-dwarf" binaries for
debugigng is not a simple plug-and-play in contrast to ordinary dwarf
binaries where debug information for a source module is in the single
.o file and linker consolidates the information into a single final
binary.
After looking at this error message line below, I have come to the
conclusion that gdb is complaining that
- it has found a CU dwarf section (or whatever),
- this CU section refers to a DWO section (or file) with ID
77955146488e868b,
- but it has no knowledge of such DWO section and its whereabout,
thus "unknown DWO".
--- quote ---
Reading symbols from
/REF-OBJ-DIR/objdir-tb3/mozilla/dist/bin/thunderbird...Dwarf Error: CU at
offset 0x0 references
unknown DWO with ID 77955146488e868b [in module
/REF-OBJ-DIR/objdir-tb3/mozilla/dist/bin/thunderbird]
(no debugging symbols found)...done.
--- end quote ---
So I think I need to tell that relevant .dwo files are in the object directory of thunderbird.
Now that debug information is split, I have to do this. OK, fair enough.
But how can I tell gdb that all the .dwo files and dynamically linked
.so files are under certain directories? Do I have to tell gdb.
Also, do I have to tell gdb in advance, all the *.dwo files that are referenced by thunderbird, and
how?
After reading through gdb documentation and not finding much about
split dwarf feature itself , I now realize that it looks to me that I have to
- create file location dwarf information for gdb to locate
information in scattered *.dwo files,
B ut again, how?
- One possibility: this is done by using objcopy by extracting certain debug
information from ALL *.dwo files and merging into a certain dwarf
data structure (list of known DWO sections with its unique IDs and
the file pathname?),
- AND maybe physically attaching this merged dwarf information to
|thunderbird| and |thunderbird-bin| binary. Basically extracting all
the sections of |thunderbird| and merge them with the list of known
DWO sections with the IDs and file pathnames.) and creating a new
binary.
- This recreated binary will be used so that gdb will notice all the
indirect references to the debug information in all *.dwo files when
|thunderbird| and |thunderbird-bin| is loaded?
Is this what one is supposed to do when -gsplit-dwarf is used
and gdb debugging is attempted?
Oh, I now realize another possibility of creating consolidated
symbolic information for gdb debugging.
maybe I can read all the *.dwo files one by one from within gdb
(of course, I will use gdb script to read them in one sweep),
and write out the symbolic information for next time by |save gdb-index|
This may be the way to go, but not sure
I appreciate any helpful tips and guidance.
TIA
cf. *.dwo files and *.so files created and used
by mozilla thunderbird
I have re-compiled the tree under /REF-COMM-CENTRAL/comm-central
using the ordinary dwarf debug information (without -gsplit-dwarf and -Wl,--gdb-index)
so I am referring to a different tree which was compiled using -gsplit-dwarf.
The top-most directory of the source file hierarchy is /COMM-CENTRAL/comm-central, and
the top-most directory of the object file hierarchy is /TEST-MAIL-OBJ/objdir-tb3.
The final thunderbird binary is in the following directory.
/TEST-MAIL-DIR/objdir-tb3 is the top of the object directory.
There are many *.dwo files scattered in the object tree.
(The structure of the object tree reflects the original source tree more or less.)
The accompanying *.o files are under the same tree.
Also, there are *.so files scattered in the object tree. They are
consolidated into several directories and the number of directories were smaller than in the case of
*.dwo files.
ishikawa@vm-debian-amd64:/TEST-MAIL-DIR/objdir-tb3$ ls -l mozilla/dist/bin/thunderbird*
-rwxr-xr-x 1 ishikawa ishikawa 267277 Sep 20 12:19 mozilla/dist/bin/thunderbird*
-rwxr-xr-x 1 ishikawa ishikawa 267277 Sep 20 08:42 mozilla/dist/bin/thunderbird-bin*
[*.dwo files]
ishikawa@vm-debian-amd64:/TEST-MAIL-DIR/objdir-tb3$ find . -name "*.dwo" -print
./ldap/xpcom/src/nsLDAPProtocolModule.dwo
[..omission..]
./ldap/xpcom/src/nsLDAPServer.dwo
./db/mork/src/morkStore.dwo
./db/mork/src/morkPortTableCursor.dwo
[...omission...]
./db/mork/src/morkWriter.dwo
./db/mork/build/nsMorkFactory.dwo
./mozilla/parser/htmlparser/src/nsHTMLEntities.dwo
./mozilla/parser/htmlparser/src/nsHTMLTags.dwo
[...omission...]
./mozilla/parser/htmlparser/src/nsExpatDriver.dwo
./mozilla/parser/xml/src/nsSAXXMLReader.dwo
./mozilla/parser/xml/src/nsSAXLocator.dwo
./mozilla/parser/xml/src/nsSAXAttributes.dwo
./mozilla/parser/expat/lib/xmlparse.dwo
./mozilla/parser/expat/lib/xmltok.dwo
./mozilla/parser/expat/lib/xmlrole.dwo
./mozilla/parser/html/nsHtml5TreeBuilder.dwo
[...omission...]
./mozilla/parser/html/nsHtml5Parser.dwo
./mozilla/xpcom/reflect/xptcall/src/xptcall.dwo
[...omission...]
./mozilla/xpcom/reflect/xptinfo/src/xptiWorkingSet.dwo
./mozilla/xpcom/io/Base64.dwo
[...omission...]
./mozilla/xpcom/io/nsAppFileLocationProvider.dwo
[... lots of lines omitted.]
./mail/components/shell/DirectoryProvider.dwo
[*.so files]
ishikawa@vm-debian-amd64:/TEST-MAIL-DIR/objdir-tb3$ find . -name "*.so" -print
./ldap/sdks/c-sdk/ldap/libraries/libldif/libldif60.so
./ldap/sdks/c-sdk/ldap/libraries/libprldap/libprldap60.so
./ldap/sdks/c-sdk/ldap/libraries/libldap/libldap60.so
./mozilla/xpcom/tests/component_no_aslr/libtestcompnoaslr.so
./mozilla/xpcom/tests/component/libtestcomponent.so
./mozilla/xpcom/tests/bug656331_component/libtest656331.so
./mozilla/dist/plugins/libnptest.so
./mozilla/dist/plugins/libnpsecondtest.so
./mozilla/dist/bin/libmozalloc.so <--- this is the directory where
./mozilla/dist/bin/libnspr4.so <--- the main binary is stored.
./mozilla/dist/bin/libfreebl3.so <--- *.so files are symlinks to other files under the
./mozilla/dist/bin/libssl3.so <--- top-most objdir-tb3
./mozilla/dist/bin/libnssdbm3.so
./mozilla/dist/bin/libplds4.so
./mozilla/dist/bin/libsmime3.so
./mozilla/dist/bin/libmozsqlite3.so
./mozilla/dist/bin/libnssutil3.so
./mozilla/dist/bin/libxul.so <--- a symlink libxul.so -> ../../toolkit/library/libxul.so
./mozilla/dist/bin/libsoftokn3.so
./mozilla/dist/bin/libprldap60.so
./mozilla/dist/bin/libldap60.so
./mozilla/dist/bin/libnssckbi.so
./mozilla/dist/bin/libldif60.so
./mozilla/dist/bin/libnss3.so
./mozilla/dist/bin/libplc4.so
./mozilla/dist/bin/libnpsecondtest.so
./mozilla/dist/bin/components/libdbusservice.so
./mozilla/dist/bin/components/libmozgnome.so
./mozilla/dist/lib/libmozalloc.so
./mozilla/dist/lib/libnspr4.so
./mozilla/dist/lib/libfreebl3.so
./mozilla/dist/lib/libssl3.so
./mozilla/dist/lib/libnssdbm3.so
./mozilla/dist/lib/libplds4.so
./mozilla/dist/lib/libsmime3.so
./mozilla/dist/lib/libmozsqlite3.so
./mozilla/dist/lib/libnssutil3.so
./mozilla/dist/lib/libxul.so <--- symlink to libxul.so -> ../../toolkit/library/libxul.so
./mozilla/dist/lib/libsoftokn3.so
./mozilla/dist/lib/libprldap60.so
./mozilla/dist/lib/libldap60.so
./mozilla/dist/lib/libnssckbi.so
./mozilla/dist/lib/libldif60.so
./mozilla/dist/lib/libnss3.so
./mozilla/dist/lib/libplc4.so
./mozilla/dist/lib/libnpsecondtest.so
./mozilla/dist/sdk/lib/libmozalloc.so
./mozilla/dist/sdk/lib/libnspr4.so
./mozilla/dist/sdk/lib/libssl3.so
./mozilla/dist/sdk/lib/libplds4.so
./mozilla/dist/sdk/lib/libsmime3.so
./mozilla/dist/sdk/lib/libnssutil3.so
./mozilla/dist/sdk/lib/libxul.so <--- libxul.so -> ../../../toolkit/library/libxul.so
./mozilla/dist/sdk/lib/libnss3.so
./mozilla/dist/sdk/lib/libplc4.so
./mozilla/dom/plugins/test/testplugin/secondplugin/libnpsecondtest.so
./mozilla/dom/plugins/test/testplugin/libnptest.so
./mozilla/security/nss/lib/util/libnssutil3.so
./mozilla/security/nss/lib/ckfw/builtins/libnssckbi.so
./mozilla/security/nss/lib/smime/libsmime3.so
./mozilla/security/nss/lib/softoken/legacydb/libnssdbm3.so
./mozilla/security/nss/lib/softoken/libsoftokn3.so
./mozilla/security/nss/lib/freebl/libfreebl3.so
./mozilla/security/nss/lib/nss/libnss3.so
./mozilla/security/nss/lib/ssl/libssl3.so
./mozilla/build/unix/elfhack/test-array.so
./mozilla/build/unix/elfhack/test-ctors.so
./mozilla/nsprpub/pr/src/libnspr4.so
./mozilla/nsprpub/lib/libc/src/libplc4.so
./mozilla/nsprpub/lib/ds/libplds4.so
./mozilla/memory/mozalloc/libmozalloc.so
./mozilla/toolkit/system/dbus/libdbusservice.so
./mozilla/toolkit/system/gnome/libmozgnome.so
./mozilla/toolkit/crashreporter/test/libtestcrasher.so
./mozilla/toolkit/library/libxul.so <--- libary file : size is 283MiB
This is acccompanied by libxul.so-gdb.py in the same directory
""" GDB Python customization auto-loader for libxul """
import os.path
sys.path[0:0] = [os.path.join('/COMM-CENTRAL/comm-central/mozilla', 'js', 'src', 'gdb')]
import mozilla.autoload
mozilla.autoload.register(gdb.current_objfile())
./mozilla/toolkit/components/ctypes/tests/libjsctypes-test.so
./mozilla/db/sqlite3/src/libmozsqlite3.so
./mozilla/js/xpconnect/tests/components/native/libxpctest.so
./mozilla/_tests/testing/mochitest/chrome/libraries/libjsctypes-test.so
./mozilla/_tests/testing/mochitest/chrome/toolkit/components/toolkit/components/ctypes/tests/libjsctypes-test.so
./mozilla/_tests/mozmill-virtualenv/lib/python2.7/site-packages/simplejson/_speedups.so
./mozilla/_tests/mozmill/mozmillprofile/plugins/libnptest.so
./mozilla/_tests/mozmill/mozmillprofile/plugins/libnpsecondtest.so
./mozilla/_tests/xpcshell/xpcom/tests/unit/libtestcompnoaslr.so
./mozilla/_tests/xpcshell/xpcom/tests/unit/libtestcomponent.so
./mozilla/_tests/xpcshell/xpcom/tests/unit/libtest656331.so
./mozilla/_tests/xpcshell/toolkit/crashreporter/test/unit/libtestcrasher.so
./mozilla/_tests/xpcshell/toolkit/crashreporter/test/unit_ipc/libtestcrasher.so
./mozilla/_tests/xpcshell/toolkit/components/ctypes/tests/unit/libjsctypes-test.so
./mozilla/_tests/xpcshell/js/xpconnect/tests/components/native/libxpctest.so
ishikawa@vm-debian-amd64:/TEST-MAIL-DIR/objdir-tb3$
(2013/09/21 6:43), ISHIKAWA,chiaki wrote:
I think the following problem is related to the reading of dynamically linked libxul.so library file.
The problem affects the behavior of gdb in that it fails to access the source files.
So it can not list the source lines.
I checked the operation of gdb against statically linked small program and
all is well.
It can list the source files, and this is the first thing I immediately noticed.
With the setup described in the previous post of mine,
gdb can print the stack trace (sans the argument values), but
it can't print source files. Since the argument values are not shown in the
stack trace. Its value is also diminished.
Help is appreciated.
Yes, libxul.so is rather large. Thunderbird is a large program and so
we may hit into an unknown bug here.
(2013/09/20 14:24), ishikawa wrote:
Hi,
I recently learned of "-gsplit-dwarf" of GCC 4.8 and -Wl,--gdb-index
that is passed to GNU ld.gold linker, and have tried to use the split
debug information scheme for local compilation and link of mozilla
thunderbird mail client.
( http://gcc.gnu.org/wiki/DebugFission )
The compiling and linking produced working thunderbird binary. The
time required for whole build process, especially the time necessary
for creating a large libxul.so file diminished dramatically. I would
like to thank all the developers who have made this possible.
There is a fly in the ointment, though. Unfortunately, somehow gdb is
not reading the symbol information from a created shared library,
libxul.so, correctly. Below is the beginning of a gdb session of
trying to debug a crash of thunderbird: I attach gdb to a dying
thunderbird process. Note the error message marked with **** in the
excerpted lines.
This bug is discussed in the following bug entry and
the gdb session log is also uploaded at the following URLs.
https://bugzilla.mozilla.org/show_bug.cgi?id=918234
https://bug918234.bugzilla.mozilla.org/attachment.cgi?id=807287
Except of gdb session
$ gdb ../../.././mozilla/dist/bin/thunderbird 29680
GNU gdb (GDB) 7.6 (Debian 7.6-5)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
*****
***** The next line states that such file or directory does not exist
***** (in Japanese).
***** Strange: the command line is suggested by mozilla's
***** |make mozmill| test suite. I have never figured out what
***** the relative path is meant to be.
***** This may play an important role when -gsplit-dwarf is used.
***** Anyway as you can see from the full session log in the above URL,
***** libraries and such are correctly accessed and symbols are
***** deciphered except for libxul.so and a few others.
../../.././mozilla/dist/bin/thunderbird: ããããããããããããããããã
ããããã.
Attaching to process 29680
**** PROBLEM! A fly in the ointment!
**** Next line indicates that there is some mismatch of the data
**** layout and gdb's read routine.
Reading symbols from
/REF-OBJ-DIR/objdir-tb3/mozilla/dist/bin/thunderbird...Dwarf Error: CU at
offset 0x0 references
unknown DWO with ID 77955146488e868b [in module
/REF-OBJ-DIR/objdir-tb3/mozilla/dist/bin/thunderbird]
(no debugging symbols found)...done.
**** I am afraid Debian GNU/Linux does not ship symbol information for
**** ld*.so type files: these files seem to be shipped in stripped
**** form.
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols
from
/usr/lib/debug/lib/x86_64-linux-gnu/libpthread-2.17.so...done.
done.
[New LWP 29712]
[New LWP 29708]
[New LWP 29693]
[New LWP 29692]
[New LWP 29687]
[New LWP 29686]
... the rest omitted ...
I wonder if anyone can shed light on the error:
Dwarf Error: CU at offset 0x0 references unknown DWO with ID
77955146488e868b [in module
/REF-OBJ-DIR/objdir-tb3/mozilla/dist/bin/thunderbird]
(no debugging symbols found)...done.
I am using
gcc 4.8
gcc-4.8 --version
gcc-4.8 (Debian 4.8.1-2) 4.8.1
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
ld.gold
ld --version
GNU gold (GNU Binutils for Debian 2.22) 1.11
Copyright 2011 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later
version.
This program has absolutely no warranty.
ishikawa@vm-debian-amd64:/COMM-CENTRAL/comm-central$
gdb
gdb --version
GNU gdb (GDB) 7.6 (Debian 7.6-5)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(I have upgraded objcopy, etc. to support the split-dwarf scheme, also.
Otherwise, the compile and liking with the options mentioned above would fail.)
If this reading failure on startup of gdb
is a known error (in which case, I would like it to be fixed
soon), or if there is a workaround, or if I am using gdb incorrectly,
I would like to hear about it.
TIA