[Python-Dev] Mercurial migration: help needed

[Martin Geisler]

Mark Hammond <skippy.hammond at gmail.com> writes:

> 1) I've stalled on the 'none:' patch I promised to resurrect.  While
> doing this, I re-discovered that the tests for win32text appear to
> check win32 line endings are used by win32text on *all* platforms, not
> just Windows.

I think it is only Patrick Mezard who knows how to run (parts of) the
test suite on Windows.

> I asked for advice from Dirkjan who referred me to the mercurual-devel
> list, but my request of slightly over a week ago remains unanswered
> (http://selenic.com/pipermail/mercurial-devel/2009-August/014873.html)
> - 
> maybe I just need to be more patient...

Oh no, that's usually the wrong tactic :-) I've been too busy for real
Mercurial work the last couple of weeks, but you should not feel bad
about poking us if you don't get a reply. Or come to the IRC channel
(#mercurial on irc.freenode.net) where Dirkjan (djc) and myself (mg)
hang out when it's daytime in Europe.

> Further, Martin's comments in this thread indicate he believes a new
> extension will be necessary rather than 'fixing' win32text.  If this
> is the direction we take, it may mean the none: patch, which targets
> the implementation of win32text, is no longer necessary anyway.

I suggested a new extension for two reasons:

* I'm using Linux, and I mentally skip over all extensions that mention
  "win32"... I guess others do the same, and in this case it's really a
  shame since converting EOL markers is a cross-platform problem: if
  someone creates a repository on Windows, I might find it nice to
  translate the EOL markers into LF on my machine.

  As far as I know, all my tools works correctly with CRLF EOL markers,
  but I can see the usefulness of such an extension when adding new
  files (which would default to LF unless I take care).

* A new extension will not have to deal with backwards compatibility
  issues. That would let us clean up the strange names: I think
  "cleverencode:" and "cleverdecode:" quite poor names that convey
  little meaning (and what's with the colon?). We could instead use the
  same names as Subversion: "native", "CRLF" and "LF".

  The new extension could be named 'convert-eol' or something like that.

> 2) These same recent discussions about an entirely new extension and
> no clear indication of our expectations regarding what the tool
> actually enforces means I'm not sure how to make a start on the more
> general issue.

It would be a folly to require all files in all changesets to use the
right EOL markers -- people will be making mistakes offline. The
important thing is that they fix them before pushing to a public server.

So the extension should do that: either abort commits with the wrong EOL
markers or do as Subversion and automatically convert the file in the
working copy.

> I also fear that should I try to make a start on this, it will still
> wind up fruitless - eg, it seems any work targeting win32text
> specifically would have been wasted, so I'd really like to see a
> consensus on what needs to be done before attempting to start it.

As I understand it, what is lacking is that win32text will read the
encode/decode settings from a versioned file called <repo>/.hgeol. This
means that you can just enable the extension and be done with it,
instead of configuring it in every clone. The <repo>/.hgeol file should
contain two sections:

  [repository]
  native = LF

  [patterns]
  Windows.txt = CRLF
  Unix.txt = LF
  Tools/buildbot/** = CRLF
  **.txt = native
  **.py = native
  **.dsp = CRLF

The [repository] setting controls what native is translated into upon
commit. The [patterns] section can be translated into safe [decode] /
[encode] settings by the extension:

  [encode]
  Windows.txt = to-crlf
  Unix.txt = to-lf
  Tools/buildbot/** = to-crlf
  **.txt = to-lf
  **.py = to-lf
  **.dsp = to-crlf

  [decode]
  Windows.txt = to-crlf
  Unix.txt = to-lf
  Tools/buildbot/** = to-crlf
  **.txt = to-native
  **.py = to-native
  **.dsp = to-crlf

where to-crlf, to-lf, to-native are filters installed by the extension.

I guess your 'none' encode/decode filter patch would be needed if the
Unix.txt file were to be stored unchanged in the repository? Instead I
imagine that the extension will convert a modified Unix.txt to LF EOL
markers automatically (Subversion behaves like that, as far as I can
tell from a bit of testing).

That way the repository will contain most files in the format specified
as native for it, but selected files are stored using whatever EOLs they
like. The result is that someone who has not enabled the extension will
get correct files from a checkout. Had we stored the *.dsp files with LF
EOLs in the repository (like Subversion does), then using the extension
would be mandatory for everybody.

-- 
Martin Geisler

VIFF (Virtual Ideal Functionality Framework) brings easy and efficient
SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090830/508baa7e/attachment.pgp>