Re: How big are your /etc/passwd and /etc/group files?

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

On Jan 31 20:00, Max Polk wrote:
> On 1/31/2014 5:03 PM, Corinna Vinschen wrote:
> >On Jan 31 22:40, Frank Fesevur wrote:
> >>2014-01-31 Corinna Vinschen:
> >>>   Is anybody here who's using /etc/passwd and/or group files
> >>>   of more than 16K in size?
> >>The new way to store the stuff would make Cygwin definitely
> >>faster, but it would struggle with... uhm... 2.6 Megs file on
> >>the 32 bit version of Cygwin, Hmm. I'm wondering how to solve
> >>that elegantly. Corinna
> 
> Every process needs to load only the current user's entry up front.
> Somewhere down the road it only *might* have to do things like
> translate from uid/gid into a string for directory listings, in some
> cases only a handful of these translations.  It's essentially a (old
> dbm style unix) database lookup.

Exactly, but that's just not how the existing code for reading
/etc/passwd and /etc/group works.  At the time of writing the original
code (1996 or so), it was considered faster to load the files once into
memory than having to read them every time a passwd or group entry is
requested.  Apparently, the idea that passwd and group files might
become rather big wasn't considered at the time, and later on we sped up
the code by various measures, but never changed the general idea.

> So defer the database lookups to a libgdbm that goes against a (old

The Cygwin DLL can't load other DLLs, except basic Windows DLLs.
Dependency chicken-egg.  But other than that...

> dbm style unix) database, and don't keep that in memory, unless you
> want a small LRU algorithm in there to keep a small fixed number.  I
> bet the tiny delay later on a ls or other unix translation won't be
> very noticeable.

This means, a rewrite of the /etc/passwd, /etc/group reading code from
"all or nothing" to "one at a time when needed".  Maybe that's in fact
the way to go.

In theory, we don't want to keep the files around anyway.  I have
implemented ways to configure user settings from /etc/passwd (like
home dir or login shell) in the local SAM or in AD.  So the admin
can switch to centralized maintainance even for Cygwin stuff.

On the other hand, we'd like to keep passwd and group for home users
which don't feel comfortable to make changes to the SAM, but then
we only need very tiny files with one or two entries anyway.

Reading the files on demand only might be the right thing to do.
Catching two birds with one stone...


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Maintainer                 cygwin AT cygwin DOT com
Red Hat

Attachment: pgppNkuJlBIsw.pgp
Description: PGP signature