This is the mail archive of the
cygwin
mailing list for the Cygwin project.
RE: locate and updatedb
- From: "Buchbinder, Barry (NIH/NIAID) [E]" <BBuchbinder at niaid dot nih dot gov>
- To: "cygwin at cygwin dot com" <cygwin at cygwin dot com>
- Cc: 'Byron Boulton' <daytonb at zoho dot com>
- Date: Wed, 17 Feb 2016 16:49:51 +0000
- Subject: RE: locate and updatedb
- Authentication-results: sourceware.org; auth=none
- References: <56BC940F dot 6070109 at zoho dot com> <56BCD05C dot 2040409 at gmail dot com> <56BCD414 dot 2010304 at zoho dot com> <56BD0D87 dot 6030008 at gmail dot com> <56BF1E4D dot 5000901 at tlinx dot org> <6CF2FC1279D0844C9357664DC5A08BA21BD2FA07 at msgb09 dot nih dot gov> <56C478E0 dot 70904 at zoho dot com> <6CF2FC1279D0844C9357664DC5A08BA21BD31EAE at msgb09 dot nih dot gov> <56C49DFD dot 5060700 at zoho dot com>
Byron Boulton sent the following at Wednesday, February 17, 2016 11:21 AM
>On 2/17/2016 11:00 AM, Buchbinder, Barry (NIH/NIAID) [E] wrote: locate
>> Byron Boulton sent the following at Wednesday, February 17, 2016 8:43
>> AM
>>> On 2/16/2016 5:55 PM, Buchbinder, Barry (NIH/NIAID) [E] wrote:
>>>>
>>>> This is technically OT since this involved a non-cygwin tool.
>>>>
>>>> find is slow compared with a non-Cygwin tool, specifically dir (cmd.exe).
>>>>
>>>> Compare find with cmd.exe's dir. Note that even with the benefit of
>>>> caching (compare the 1st and 3rd times), find takes twice as long as dir.
>>>> Comparing cached times (2nd vs 3rd), dir is 3X faster.
>>>>
>>>> $ time cmd /c dir /s /b 'C:\usr' > /dev/null ; \ time find /c/usr >
>>>> /dev/null ; \ time cmd /c dir /s /b 'C:\usr' > /dev/null
>>>>
>>>> real 0m1.326s
>>>> user 0m0.000s
>>>> sys 0m0.047s
>>>>
>>>> real 0m2.465s
>>>> user 0m0.280s
>>>> sys 0m2.184s
>>>>
>>>> real 0m0.874s
>>>> user 0m0.000s
>>>> sys 0m0.031s
>>>>
>>>> (Note: c:\usr has nothing to do with /usr.)
>>>>
>>>> Here's how I use dir *in the abstract* for drives C: and D:. (Note:
>>>> the
>>>> /a: option of dir lists all files, including hidden ones; /o:n sorts
>>>> by
>>>> name.)
>>>>
>>>> for D in /c /d
>>>> do
>>>> "$(cygpath "${COMSPEC}")" /c dir /s /b /a: /o:n "$(cygpath -w "$D")"
>>>> done | \
>>>> tr -s '\r\n' '\n' | \
>>>> cygpath -u -f - | \
>>>> sed -e '/^$/d' -e 's,/\+,/,g' \
>>>> sort -u \
>>>> /usr/libexec/frcode > /tmp/updatedb.tmp chmod --reference
>>>> /var/locatedb /tmp/updatedb.tmp mv /tmp/updatedb.tmp /var/locatedb
>>>>
>>>> What I actually do (attached) is more complicated. My script
>>>> chooses which directories are scanned, does them in parallel, and
>>>> prints pretty messages. I get error messages for very long paths (>
>>>> ~250 bytes). It works well enough for me; YMMV.
>>>
>>> Are you using dir in some sort of custom way to build the database
>>> used by locate? Or are you saying that rather than ever using the
>>> find command to find files, you use a custom script which uses dir?
>>
>> I use dir only to generate the locate database, because scanning the
>> better part of several disks takes so long. I do not substitute dir
>> for find for other purposes. One could, but usually locate does what
>> I need, and when it doesn't, I use find.
>
>understands how to read this custom database? If I read you updatedb.sh
>script properly, it produces a file which is just a sorted text file
>with one line per file found by updatedb.sh.
Sorry. In the example in the email text I forgot a pipe sign after sort
and feeding into /usr/libexec/frcode, which convert to located format.
That fragment should have been as follows.
sort -u | \
/usr/libexec/frcode > /tmp/updatedb.tmp
It's really been so long since I put updated.sh together that I would need
to study it to make detailed comments. Indeed, my memories of putting it
together are lost in the mists of time.
What I'd advise is to use the script that comes with findutils,
/usr/bin/updated, as your model. Substitute dir for find, adjust start
Points 9drives or directories), convert line endings, etc., and running
through cygpath, and making other necessary changes before running through
frcode.
Sorry that I cannot be of more help.
Good luck.
- Barry
Disclaimer: Statements made herein are not made on behalf of NIAID.
--
Problem reports: http://cygwin.com/problems.html
FAQ: http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple