This is the mail archive of the cygwin-developers mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: support for writing reparse-point symlinks


On Dec 8, 2012, at 7:57 PM, Daniel Colascione <dancol@dancol.org> wrote:

> On 12/8/12 7:28 PM, James Gregurich wrote:
>> Around the beginning of this year, I took it upon myself to modify
>> cygwin's dll so that when one invokes the posix symlink() function it
>> will attempt to use a reparse-point rather than the usual cygwin
>> symlink file. 
> 
> By reparse point, you mean a Windows symbolic link, right? (The
> Windows symbolic link is one kind of reparse point.

yes.


> 
>> I have employed a strategy for handling reparse points
>> in cygwin that works well in practical use. We use this modified
>> version of cygwin to host our git repositories on Windows.
> 
> Why did you do this work? Were you trying to increase performance or
> improve interoperability? If the former, do you have before and after
> performance numbers?

Our repositories liberally use symlinks. We needed those to function correctly with the Windows environment…particularly Visual Studio. Performance was not a concern. However, it works fast enough. Our repositories are huge..the largest is tens of thousands of files and about 13 GB in size. I've made no attempts to measure performance differences. I would expect it to be slower in the case where a reparse point can be created because there is more work done. 




>> I have seen in past emails on this list that there was no interest
>> in implementing support for creating reparse points in cygwin because
>> there were questions about the practical usability. Our experience has
>> been that it works fine for us over the last year. I'd like to raise
>> the question for reconsideration by the cygwin developers. I'd be
>> willing to contribute my and design and code to the project as I would
>> prefer not to maintain our own custom version of cygwin forever.
> 
> As I understand it, there are four major issues that prevent Cygwin
> using Windows symbolic links for its own symlinks:
> 
> 1) Windows symbolic links are "directory" or "file" symbolic links,
> depending on the type of the target, but POSIX has only one kind of
> symbolic link.
> 
> 3) By default, only administrators can create Windows symbolic links.



These are true.


The strategy I employ is to attempt to create the reparse-point IF the target exists. If the attempt fails, then it falls back to the default behavior that already existed.

I created two command line utilities. The first will accept a path name, check if that file is a cygwin symlink, then attempt to change it to a reparse point symlink. The second utility scans a directory tree for symlinks and updates them en-masse. In the git repository, I run the second utility as a post-processing trigger to update all the symlinks once all the changes are completed.

This strategy means that a symlink never fails and it is never invalid in the cygwin environment.  this allows all unix software to work without any knowledge of the implementation details. If a person needs the symlink(s) to work in the Windows environment, then he can use the command line utilities to update it/them  on an as-needed basis. Cygwin doesn't care which format the link is in.

Secondly, I've enabled the feature at runtime by extending the symlink environment variable to have one additional valid setting. If a person is not interested in the links ever being valid on Windows, then he can simply not turn on the functionality. In that case, there should be no performance difference.

Thirdly, I have translated relative paths to relative paths where possible. As you probably know, to exceed about 254 chars in a Windows path name requires using the '\\??\' prefix and an absolute path. So, in some cases, the relative path has to be made an absolute path to work at all.


> 2) Windows symbolic links point to Windows paths, but Cygwin symbolic
> links point to POSIX paths. If the Cygwin mapping table changes,
> Windows symbolic links will begin pointing to the wrong place.

I have not explored this idea. Certainly, if one changes the directory structure, he can break symlinks. AS afar as I know, that condition is true even in the unix environment as the symlinks are just dumb paths.

I would suggest that a user who is interested in making the unix file system fully usable on the Windows side can certainly be educated to the limitations. I would argue that the ability to to enhance Windows compatibility is desirable for a certain class of user even if it has limitations.




> 4) Cygwin still supports Windows XP, and Windows XP does not support
> symbolic links.

This is true.


> Cygwin symbolic links work well enough, and I don't think trying to
> overcome these difficulties is a high priority.


The answer is to simply ignore the functionality on XP. You might have to contrive a macro definition or two to get it compile on XP, but that isn't hard. The check on the Windows version can be made at the point of setting the environment variable and simply disallow enabling the value on XP.



Because the current functionality is not disturbed by the new functionality if the environment variable is not set, only people who are actually interested in the additional interoperability need be concerned with the limitations. People who want native symlinks on Windows will be using Windows Vista or higher already and they will be people who understand the limitations of symlinks on Windows.

I've already overcome the difficulties for all PRACTICAL purposes. One simply does not need a technically complete solution. He simply needs a solution to be good enough to function given the constraints he already has and copes with on Windows.




> That said, someone brought up a few months ago the possibility of
> using a _custom_ reparse point type, paired with a filesystem
> minifilter driver, to store Cygwin symbolic links. The filesystem
> minifilter driver would allow normal Windows programs to work with
> these links. This discussion didn't go anywhere.
> 
> What problem are you really trying to solve here?
> 

As I said...enabling Visual Studio to work with our git repositories. Occasionally, we use scripts in Cygwin to manipulate data and files on the Windows side just because we are unix people who already know how to operate with unix tools. This feature is useful for that use-case too.


-James



Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]