This is the mail archive of the gdb@sourceware.org mailing list for the GDB project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Python enabled gdb on Windows and relocation


On 2011-5-16 0:11, Ruben Van Boxem wrote:
2011/5/15 Ruben Van Boxem<vanboxem.ruben@gmail.com>:
2011/5/15 Ruben Van Boxem<vanboxem.ruben@gmail.com>:
2011/5/14 Doug Evans<dje@google.com>:
On Sat, May 14, 2011 at 2:09 AM, Ruben Van Boxem
<vanboxem.ruben@gmail.com>  wrote:
2011/5/14 Doug Evans<dje@google.com>:
On Thu, May 12, 2011 at 9:19 AM, Ruben Van Boxem
<vanboxem.ruben@gmail.com>  wrote:
(now in plain-text as required by gdb mailing list)

Hi,

I am currently trying to integrate Python support into my toolchain
build (including GDB of course). It is a sysrooted
binutils+GCC+GDB+mingw-w64 toolchain.

I currently have the basic setup working: I can link gdb with my
manually generated import lib to the python dll from the official
Windows install. If there is anything I am missing or a very easy
solution to the problems decsribed below, please just say so. I am
only suggesting what I would like to happen.

Now on to the problems I'd like to discuss:

1. gdb.exe won't start without me having set PYTHONPATH manually.

In a properly configured/built gdb on linux this isn't necessary, even if python is installed in some random place. I'm not sure about windows though. Did you specify --with-python when you configured gdb, and if so did you specify a value? e.g., --with-python=SOME_VALUE

I was cross-compiling a mingw toolchain+gdb from Linux, so I used --with-python without a value (because gdb configure tries to find the Python executabe), and I added -I"/path/to/python/includes" to CFLAGS and -L"/path/to/pythondll/importlib" to LDFLAGS, which built as it should. This is hacky though, and gdb configure should provide --with-python-libs and --with-python-include to make it more streamlined with any other build prerequisite (like gmp/mpfr/mpc/cloog/ppl in GCC for example).

Ah. Cross-compiling gdb with python is in need of improvement. Alas python hasn't been designed with cross-compilation in mind (e.g. build on linux, run on windows). AIUI, the way to get the parameters required for compiling with libpython is to get them from python's "distutils": kinda hard to do in a cross-compile. Done correctly there's no need to run python.

I haven't done anything more to support python in gdb's configure.ac
because it's not clear to me what the right thing to do is: distutils
provides more than just --libs and --includes (btw, we don't use
--libs though, we use --ldflags which includes all of: the directory
in which to find libpython, the -l for libpython, and the -l's for all
the other libraries python needs). [Which isn't to say that someone
else isn't free to tackle this.]

In the meantime, what I've been doing is a hack: write a script that
responds to:
--includes
--ldflags
--exec-prefix
and pass that as --with-python.

E.g.
bash$ cat $HOME/my-python-for-config
#! /bin/sh

if [ $# -ne 2 ]
then
        echo "Bad # args.  Blech!">&2
        exit 1
fi

# The first argument is the path to python-config.py, ignore it.

case "$2" in
--includes) echo "-I/usr/include/python2.6 -I/usr/include/python2.6" ;;
--ldflags) echo "-L/usr/lib/python2.6/config -lpthread -ldl -lutil -lm
-lpython2.6" ;;
--exec-prefix) echo "/usr" ;;
*) echo "Bad arg $2.  Blech!">&2 ; exit 1 ;;
esac

exit 0
bash$ ./configure --with-python=$HOME/my-python-for-config [...]
[...]


Note that --exec-prefix is the runtime location of python. GCC uses this to tell libpython where to find its support files. [grep for Py_SetProgramName in gdb/python/python.c]

OK, I tried your script in a couple of variations. It gets rid of the traceback I had before, but still doesn't help the PYTHONPATH problem. My directory structure is as follows (this is not in root, "/" is just my main build directory, there are several levels below it):

/gdb<-- gdb build dir
/gdb/gdb<-- where the python configuration is done
/python<-- temporary install dir for python files for build,
extracted from the official Windows installer, also location of
libpython2.7.a import library
/python/include/python27<-- python headers, found by gdb in both cases
/mingw64<-- toolchain sysroot prefix
/mingw64/bin<-- install location of gdb and python27.dll
/mingw64/lib/python27<-- install location of all python scripts

First variant:
--includes) echo "-I../../python/include" ;;
--ldflags) echo "-L../../python -lpython2.7" ;;
--exec-prefix) echo "../../mingw64/lib/python27" ;;

Here exec-prefix would be the relative path from where "configure"
does its magic to the final location of the scripts on the build
system.

Second variant:
--includes) echo "-I../../python/include" ;;
--ldflags) echo "-L../../python -lpython2.7" ;;
--exec-prefix) echo "../lib/python27" ;;

I thought the second points gdb to the installed location of the
python scripts, but it still needed PYTHONPATH for that. I used
relative paths in an attempt to "do the right thing", hackwise...
Remember that the whole "/mingw64" directory gets zipped and moved,
then it gets extracted to a random location in the Windows filesystem.

Thanks for the help so far, I think we might just get this worked out together.

Ruben

Wow, I think I have a partial solution. Delving into the Python docs, for example here: http://docs.python.org/using/windows.html#finding-modules, you can see that PYTHONPATH is used first, then the Windows registry, then PYTHONHOME, then some default relative paths. I placed the python scripts all in the directory structure like so:

/bin/gdb
/bin/Lib/<python scripts>
/bin/python27.dll

This works, even without any manual PYTHONPATH intervention. Problem
is though, that as soon as someone has a PYTHONPATH environment
variable from a (incompatible) Python installation (think different
bitness or version 3.x instead of 2.7.1), I cannot predict what will
go wrong. This problem originates in Python's way of filling in the
search path (sys.path). A true solution in the GDB case to prevent
this collision of an incompatible PYTHONPATH would be that GDB sets an
internal PYTHONPATH as directed by configure, uses that to load its
Python internals, and allows the GDB child processes (apps being
debugged) to use the environment PYTHONPATH. For now, I have a
functional installation, but it will break as soon as someone installs
Python on their system.

The part your script plays in this (if I haven't misunderstood the
intention/effects of the "--exec-prefix" part) is allowing a uniform
way of directing the configure script to the right directories for
includes/libs.

Ruben

I am sorry for the repeated messages that no one cares about, but I may have discovered GDB in its current form already allows what I want: I tried to figure out what exact paths the snake in gdb was using to search for its modules, and came up with this: (gdb) python import sys (gdb) python print sys.path ['m:\\development\\mingw64\\share\\gdb/python', 'M:\\Development\\mingw64\\bin\\python27.zip', 'M:\\Development\\mingw64\\bin\\DLLs', 'M:\\Development\\mingw64\\bin\\lib', 'M:\\Development\\mingw64\\bin\\lib\\plat-win', 'M:\\Development\\mingw64\\bin\\lib\\lib-tk', 'M:\\Development\\mingw64\\bin', 'M:\\Development\\mingw64\\bin\\lib\\site-packages']

This means that every python command within gdb searches
<sysroot>/share/gdb/python FIRST (even before an environment's
PYTHONPATH), alleviating any concerns or problems I or anyone would
have with another python installation, as this apparently built-in
path comes up first. All I, or anyone interested in doing this kind of
thing, have to do is copy all the python scripts from the Windows
installation's Lib directory to the<sysroot>/share/gdb/python
directory.

I don't know where this path comes from, but it is quite handy, and
makes this whole discussion  moot for Python people. Only "issue" that
I'll have to work around is the --with-python-includes and
--with-python-libs that are missing, using either manual
CFLAGS/LDFLAGS or a variant of your script.

Thanks for all the help and motivation to keep trying :)

Ruben




I understand the need for this, but as gdb requires Python 2, and users
of my toolchain may have installed Python 3 or a 32-bit version python
they want to use from the same environment (without changing their own
PYTHONPATH), there is no way to run python-enabled gdb.
[...]

Yeah. There is a proposal to add GDB_PYTHONPATH (or some such IIRC) and have gdb use that instead of PYTHONPATH if it exists, but there's been resistance to it. I think(!) what would happen is that gdb would set $PYTHONPATH to the value of $GDB_PYTHONPATH. [Inferiors started by gdb should still get the original value of PYTHONPATH though.]

That way would be almost ideal, but a hardcoded *relative* path to the python scripts (that is standardized within gdb) wouldn't hurt.

See above re: --exec-prefix.


An
extra environment variable would require a lot of explaining for
Windows, and is not "plug-and-play", like the rest of a sysrooted
toolchain is supposed to be like. I think this should work on all
setups:

1. Check hardcoded path; my suggestion would be "<gdb
executable>/../lib/python27"
2. If this fails to find the necessary files/scripts, find it like you
described above in Linux, without PYTHONPATH set.
3. Check PYTHONPATH.

The problem being solved by the proposed GDB_PYTHONPATH is "What if the user has PYTHONPATH set and it points to an incompatible version of python?". Leaving such a value for PYTHONPATH set while gdb's python is running feels wrong (and IIRC has caused some problems).

The problem of telling python where to find itself is already solved
(or at least is intended to be solved) with gdb's calling
Py_SetProgramName with a value derived from the python-provided
--exec-prefix.

I would think only number one would change, and perhaps be only
enabled with a special configure option. Nothing else would have to
change, and Windows users would rejoice :)
Again, this is only my suggestion, if there are problems with it in
way I haven't thought of, please say so, and we can come up with
another solution.




Hi, vanboxem.

what's your finding is quite interesting. In fact, I just copy the Lib(python supporting scripts) under the MinGW/bin/ folder, then every thing works fine.

I never know the details about the python search path.

So, thank you very much.

asmwarrior


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]