Re: the viability of translators, and Guile itself

[Mikael Djurfeldt <mdj@mdj-pc.nada.kth.se>]

Greg Badros <gjb@cs.washington.edu> writes:

> > >From that point of view, the immediate quality of the implementation
> > isn't so important, as long as there's a consensus to work together on
> > improving it.  That consensus doesn't exist now, but I think we can
> 
> the quality of implementation can go a long way towards building that
> consensus, though!  We need to make it less likely for a person to have
> any second thoughts about using Guile instead of SCM, STk, Elk,
> mzscheme/Dr.Scheme, etc.

I agree very much with both Jim and you, but...

I have a fairly good overview of Guile's internals and I'm not sure
that the naive reader of recent posts on this list with benchmarks
where Guile shows "poor" performance, complaints on the quality of
code, and talk about limitations in the internal architecture of Guile
gets a proper picture.

I feel like adding that many parts of Guile are good.  Those parts of
the implementation which can be considered poor fall mainly into one
of two categories:

1. Unreadable (except for a small elite), monolithic, but very
   efficient code which does exactly what is necessary and nothing
   more (the Jaffer legacy).  (This code is often quite beautiful once
   you have understood it.)

2. Unnecessarily complex, confusing, poorly modularized code.

Luckily, most of the "poor" code falls into category 1.  Category 2
code occurs in some spots, like in the module system, in symbols.c,
in struct.c and the stack translation code in stacks.c.  But these
parts are replacable and *are* successively being replaced.

I have not only used Guile a lot but also been messing around in Guile
internals, modifying and extending, and I have not experienced any big
problems which are due to the quality of the implementation.

I don't think the current state of the implementation scares people
away from Guile.  I *do* think that if we build up a false impression
that Guile has poor performance or is hard to work with due to a poor
implementation, that could scare people.  (I don't mean that we should
stop pointing out what should be fixed.)  And of course, even more
importantly, simple things like the startup time has a large effect on
the popularity of Guile.

I think startup time is an example of a problem with Guile that really
scares people away.  This is irrational in some respects, but it is
understandable why people take it as an indication of poor quality.

It has been very well understood for a long time why Guile has a long
startup time.  If I've interpreted Jost's recent figures comparing his
module system implementation with the current one correctly, about 85%
of the startup time is due to the insane idea to execute several
layers of Scheme code for *every* symbol encountered in the program.

This, and a simple remedy (it is possible to fix this with perhaps one
page of C code without implementing a module system) has been pointed
out several times during the past two years.  I don't know why, but
no-one has felt inclined to fix it.  Personally, I don't care as much
about the startup time as I do about things like threading and object
orientation, so I've let it be.

Another cause (again I'm repeating what has been said over and over
again) for the startup time is that Guile loads too much code at
startup.  Again, the main reason why it hasn't been fixed has to do
with priorities: It seems that the right order of things is first to
implement a real module system, then a packaging system, and then
structure Guile into reasonable modules, and make sure that 95% of the
original environment with little else than R5RS is implemented in C
and doesn't need to be loaded.

My point is that in this example (startup time), the apparent reason
is that Guile is slow due to poor implementation while the *real*
reason has to do with current personal and project priorities and the
development process and has *nothing* to do with the implementation.

Guile's evaluator in itself is fast as lightning and only maybe 10-30%
slower than current SCM, which is one sign that the implementation is
good at least in some respects.  (And since Guile's and SCM's
evaluators are *very* similar, there are no limits in the architecture
which prevents moving the ecache code over to Guile making it as fast
as SCM.)

Unfortunately some of the category 2 code influences benchmarks:
Startup time adds to benchmark evaluation time in some benchmarks.
Even when this time isn't included the large amount of loaded objects
adds to GC time since we don't yet have a generational GC...

Also, it is common that people don't know how to switch off the
debugging evaluator.