Re: Sv: [ECOS] Dynamic linking

[Bart Veer <bartv at redhat dot com>]

>>>>> "Per" == Per Oestergaard <per@k64.dk> writes:

    Per> What I meant by "user programs" is: The user of my product
    Per> shall add his application (limited to a few commands as for
    Per> example like Lego mindstorms upload) to the full design. My
    Per> design/program handles the hardware and the master-controls.

    Per> or as you put it: " load additional applications dynamically
    Per> " !

OK. Right now eCos does not really provide for anything along these
lines, but I can give some ideas about what is involved. Before that,
I should point out that eCos was designed specifically for the deeply
embedded marketplace. The expectation is that there is a single
application, linked directly with eCos, and typically the whole of the
resulting executable will be burned into a ROM and shipped in some
sort of consumer product. In many such products there will be severe
memory constraints in order to keep the cost down, and there may be
performance constraints. Obviously it is possible to use eCos in other
scenarios, but you have to bear in mind the design goals.

If you need dynamic code loading there are a number of approaches.
First you can use a different operating system such as embedded Linux,
which provide greater functionality at the cost of extra memory etc.
That is a perfectly valid solution, we do not expect eCos to meet
everybody's requirements.

A second approach is to incorporate some sort of interpreter in the
target application. This could be something like a Tcl interpreter, or
Java bytecodes, or something custom-designed for the application.
General purpose solutions such as Tcl are likely to be more powerful,
but involve more memory overhead. For something like Lego mindstorms a
simple custom interpreted language for accessing the motors and
sensors might well be appropriate.

If the dynamically loaded applications involve compiled C or C++ code,
there are still quite a few variations. In some cases you need exactly
two applications, one permanently resident in the ROM and one
dynamically loaded. Effectively the first application acts as a ROM
monitor, allowing the second to be downloaded, but it can also provide
additional functionality, e.g. via a vector of function pointers. The
dynamically loaded application can then have stub functions to
indirect via that vector. Obviously care has to be taken when figuring
out which application is responsible for what, e.g. servicing a
particular interrupt. The existing eCos code contains some examples of
this, for example ROM-booting gdb stubs in conjunction with a
RAM-booted application.

If you need to load multiple applications things get much more
complicated. Let's assume that the target hardware is already running
a core eCos application, containing the kernel, the C library, various
device drivers, and so on, and you want to load some additional code.

eCos kernel services are accessed by direct calls, not via some sort
of trap interface, so how is the dynamically loaded code going to
access the kernel or any other part of the system? One approach is
again to have the core application supply a vector of function
pointers at a specific address, and the dynamically loaded code can
access the kernel through that vector. This is less efficient but a
lot easier to implement.

Alternatively, the dynamically loaded code would call the kernel
functions directly which means that a linker has to get involved
somewhere: somehow the linker would know that cyg_thread_create() is
at this address, cyg_mutex_lock() is at that address, and the
dynamically loaded code would get fixed up accordingly.

These fix-ups can happen either on the target side or on the host
side. The former is what happens in systems like Linux, i.e. you have
an ELF linker/loader permanently available. This involves rather
complicated code, see the glibc sources for one implementation. In
addition it requires keeping track of symbol names and corresponding
addresses, and that uses yet more memory. Plus the code you are
actually downloading needs to contain symbol names and relocation
addresses, so there are overheads there (although much of that can be
discarded after the dynamic linking).

A big problem here is the configurable nature of eCos, plus the use of
selective linking. If nothing in the core system uses cyg_mutex_lock()
then that function will not be part of the core image, so dynamically
loaded code had better not try to use the function either. To work
around this you will need to disable selective linking somehow, and
hence the system as a whole will be a lot larger.

The alternative approach is to do the linking on the host side, so
that the additional code that gets downloaded to the target is just a
pre-linked binary. For this to work you would have to tell the linker
about the results of the link for the core application, i.e. that
cyg_thread_create() ended up at this particular address. I am not
quite sure how to set about this, but something should be possible.
You would still run into problems with selective linking, although it
might be possible for additional kernel functions to end up in the
dynamically loaded code.

Unfortunately linking is only part of the problem, you also need to
worry about where in the target system the dynamically loaded code is
going to end up and perform appropriate relocation, either on the host
or the target. If the linking/loading is done on the target then the
memory can be obtained by malloc() or equivalent. If the linking is
done on the host then somehow you need to figure out where the code
should end up, e.g. by having some reserved memory areas. You also
need to worry about static data and how that gets initialized, and how
the dynamically loaded code should start up.

There may well be further complications of which I am unaware, since I
have never actually implemented dynamic linking.


All of this is possible - theoretically - but it is not easy, and it
can be very expensive in terms of memory and other resources. At some
point eCos may provide support for some of the above, if customers
insist on it and are willing to pay, or if appropriate code gets
contributed. But not yet.

Bart Veer // eCos net maintainer