Re: binary-io, opposable-thumb, pack/unpack (was Re: binary-io (was Re: rfc 2045 base64 encoding/decoding module))

[Per Bothner <per at bothner dot com>]

Gary Houston <ghouston@arglist.com> writes:

> Yes.  It would be best to give it a name that's an abbreviation for
> "the encoding that maps 8-bit bytes to the first 256 characters in the
> internal character set." rather than just using ISO-Latin-1.  But
> defining this as a completely different port type seems like overkill.

My point is that logically some character ports work by doing
decoding/encoding of bytes that it gets/puts from/to an underlying
byte sequence - i.e. byte port.  You want people to be able to plug in
new byte sequence types, so you *do* have a completely different port
type.  I.e. you have byte-port objects that are instances of specific
kinds of byte-port classes, with methods for things like reading a
byte.  So the question is: Do you want to want to make these
underlying objects available as Scheme values, and if so do you want
to overload the existing character-port functions on byte-port
objects?  You don't have to - it is quite plausible to say the
underlying byte ports are only accessible from the C api.

> This seems reasonable to me.  It only becomes much of an issue when
> dealing with stateful encodings doesn't it? 

I think so, but I'm not sure.  There may also be buffer synchronization
problems, even with stateless encodings, depending on the implementation.

> Would it seem reasonable, when defining "unpack" routines like
> (read-foobar-16 port), to throw an error if the port was not in "the
> encoding that maps 8-bit bytes to the first 256 characters in the
> internal character set."?

Possibly.

> I'm trying to get a grasp on how your "dual port" system actually
> works.  I guessed you meant the character port has a buffer using the
> internal encoding: the conversion from the byte stream into the
> internal encoding is done when the buffer is filled.

That is one way to make things more efficient.  However, there is
also the option to just have a byte buffer, and do encoding/decoding
one character at a time.  That would make it harder to get good
performance, but easier to manage synchronization and changing
encoding.

> Guile is intended primarily to be embeddable into C applications
> I believe.  But in any case it would be inconvenient to need to
> switch to a completely different I/O system when moving between
> Guile and a Scheme intended for systems-programming.

I don't think that is implied by my design (which is of course
not original with me).
-- 
	--Per Bothner
per@bothner.com   http://www.bothner.com/~per/