Re: PostgreSQL Developer meeting minutes up

[moving this onto -hackers, where I think it belongs]

Tom Lane wrote:
> Huh? The buildfarm will only prove that HEAD of the active branches
> builds. What the concern was was whether we could correctly extract
> past states (particularly, but not solely, the tags corresponding to
> releases) from a converted git repository. The testing I had in mind
> was to check out various tags and diff that tree against actual release
> tarballs.
>
>
>

It appears that our git repo is only picking up the branch tags (e.g.
REL8_0_STABLE) , not all the release tags (e.g. REL8_0_5) . That needs
to be fixed (if possible).

cheers

andrew

Andrew Dunstan wrote:
>
> [moving this onto -hackers, where I think it belongs]
>
> Tom Lane wrote:
>> Huh? The buildfarm will only prove that HEAD of the active branches
>> builds. What the concern was was whether we could correctly extract
>> past states (particularly, but not solely, the tags corresponding to
>> releases) from a converted git repository. The testing I had in mind
>> was to check out various tags and diff that tree against actual release
>> tarballs.
>>
>>
>>
>
> It appears that our git repo is only picking up the branch tags (e.g.
> REL8_0_STABLE) , not all the release tags (e.g. REL8_0_5) . That needs
> to be fixed (if possible).

Hmm. I looked through the source of the import script. It appears to
mention tags here and there, but doesn't seem to do it. There is a
comment that reads:

# Previous CVS versions just added the tag to the current HEAD
# revision and didn't insert a dead revision on the branch with
# the same date, like it is happening now.
# This means history is unclear as we can't reliably determine
# if the tagging happened at the same time as the addition to
# the branch. For now, just assume it did.
#
# XXX can't reproduce for now, disabling, as it breaks some
# things
#

Basically, it comes down to cvs tags not being actual first class
happening, but just metadata on files.

I'm sure we could script the creation of these tags fairly reliably on
*our* repository since we know which files are always updated when a tag
is added. I'm thinking we could just parse the log for configure.in and
grab the tags from there. Thoughts?

--
Magnus Hagander
Self: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Again,

This has been raised and ignored many times before on -hackers... The
reason is because the tags in the CVS repository are "broken" (i.e they
are such that it's impossible to actually create all the tags), so the
git "cvsimport" tools that try to tags all croak on the PG CVS repository.

The tool which doesn't croak doesn't try and import all the tags, just
the sticky "branch tags"...

Scripts to "fix" (actually, remove) the broken tags have also been
posted, along with requests that if somebody is "mucking" with the
actual repository, to make sure it's known about, and access is "denied"
during the mucking period (access being any rsync/anoncvs/mirroring of
the cvs root).

As long as the tags are broken, you aren't going to get the tags
imported.

If you're going to fix the tags, warn everybody (because most people
doing automatic conversions must know - they may need to be very
careful to avoid a full re-import), do it, and let us know when it's
done.

a.

* Andrew Dunstan <andrew(at)dunslane(dot)net> [090526 10:41]:
>
> [moving this onto -hackers, where I think it belongs]
>
> Tom Lane wrote:
>> Huh? The buildfarm will only prove that HEAD of the active branches
>> builds. What the concern was was whether we could correctly extract
>> past states (particularly, but not solely, the tags corresponding to
>> releases) from a converted git repository. The testing I had in mind
>> was to check out various tags and diff that tree against actual release
>> tarballs.
>>
>>
>>
>
> It appears that our git repo is only picking up the branch tags (e.g.
> REL8_0_STABLE) , not all the release tags (e.g. REL8_0_5) . That needs
> to be fixed (if possible).
>
> cheers
>
> andrew
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
> This has been raised and ignored many times before on -hackers... The
> reason is because the tags in the CVS repository are "broken" (i.e they
> are such that it's impossible to actually create all the tags), so the
> git "cvsimport" tools that try to tags all croak on the PG CVS repository.

> The tool which doesn't croak doesn't try and import all the tags, just
> the sticky "branch tags"...

> Scripts to "fix" (actually, remove) the broken tags have also been
> posted, along with requests that if somebody is "mucking" with the
> actual repository, to make sure it's known about, and access is "denied"
> during the mucking period (access being any rsync/anoncvs/mirroring of
> the cvs root).

Up to now I've always been of the opinion that fixing those tags wasn't
worth taking any risk for. But if we are thinking of moving away from
CVS, then this clearly becomes one of the hurdles we have to jump on the
way. Can you refresh our memory about which tags are problematic and
exactly what needs to be done about 'em?

regards, tom lane

* Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> [090526 11:20]:
> Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
> > This has been raised and ignored many times before on -hackers... The
> > reason is because the tags in the CVS repository are "broken" (i.e they
> > are such that it's impossible to actually create all the tags), so the
> > git "cvsimport" tools that try to tags all croak on the PG CVS repository.
>
> > The tool which doesn't croak doesn't try and import all the tags, just
> > the sticky "branch tags"...
>
> > Scripts to "fix" (actually, remove) the broken tags have also been
> > posted, along with requests that if somebody is "mucking" with the
> > actual repository, to make sure it's known about, and access is "denied"
> > during the mucking period (access being any rsync/anoncvs/mirroring of
> > the cvs root).
>
> Up to now I've always been of the opinion that fixing those tags wasn't
> worth taking any risk for. But if we are thinking of moving away from
> CVS, then this clearly becomes one of the hurdles we have to jump on the
> way. Can you refresh our memory about which tags are problematic and
> exactly what needs to be done about 'em?

Specifically, it's 2 tags, and I just remove them:
REL7_1_BETA2
REL7_1_BETA3

Previous threads:
http://news.gmane.org/find-root(dot)php?message_id=20080220225300(dot)GE16099(at)yugib(dot)highrise(dot)ca
http://news.gmane.org/find-root(dot)php?message_id=20081229155140(dot)GP12094(at)yugib(dot)highrise(dot)ca

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Tom Lane wrote:
> Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
>
>> This has been raised and ignored many times before on -hackers... The
>> reason is because the tags in the CVS repository are "broken" (i.e they
>> are such that it's impossible to actually create all the tags), so the
>> git "cvsimport" tools that try to tags all croak on the PG CVS repository.
>>
>
>
>> The tool which doesn't croak doesn't try and import all the tags, just
>> the sticky "branch tags"...
>>
>
>
>> Scripts to "fix" (actually, remove) the broken tags have also been
>> posted, along with requests that if somebody is "mucking" with the
>> actual repository, to make sure it's known about, and access is "denied"
>> during the mucking period (access being any rsync/anoncvs/mirroring of
>> the cvs root).
>>
>
> Up to now I've always been of the opinion that fixing those tags wasn't
> worth taking any risk for. But if we are thinking of moving away from
> CVS, then this clearly becomes one of the hurdles we have to jump on the
> way. Can you refresh our memory about which tags are problematic and
> exactly what needs to be done about 'em?
>
>
>

I think we need just to remove the two tags in question (they have long
been irrelevant). Prudence suggests that we should do that some time
(weeks, I think) after the 8.4 release, when reverting ,if we find any
breakage, won't be too painful.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Tom Lane wrote:
>> Up to now I've always been of the opinion that fixing those tags wasn't
>> worth taking any risk for. But if we are thinking of moving away from
>> CVS, then this clearly becomes one of the hurdles we have to jump on the
>> way.

> I think we need just to remove the two tags in question (they have long
> been irrelevant). Prudence suggests that we should do that some time
> (weeks, I think) after the 8.4 release, when reverting ,if we find any
> breakage, won't be too painful.

I don't see a lot of point in waiting till after 8.4.0. There is no
time, ever, where we are sure there will be no release for weeks ---
a security or data-loss bug could crop up at any time. And not messing
up back branch update releases is even more important than not messing
up 8.4.0, because the back branches are much more likely to get dropped
straight into production.

Obviously we want a solid backup of the pre-modification CVS repository,
and we have to follow Aidan's advice about synchronizing the change with
mirror repositories, but I don't see a strong argument for waiting weeks
to do this. I think we should get it over with, so people can get on
with the work that it's blocking.

regards, tom lane

Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
> Specifically, it's 2 tags, and I just remove them:
> REL7_1_BETA2
> REL7_1_BETA3

> Previous threads:
> http://news.gmane.org/find-root(dot)php?message_id=20080220225300(dot)GE16099(at)yugib(dot)highrise(dot)ca
> http://news.gmane.org/find-root(dot)php?message_id=20081229155140(dot)GP12094(at)yugib(dot)highrise(dot)ca

It looks like the ill-considered commit message mentioned in that first
thread hasn't been dealt with, either.

regards, tom lane

On Tue, 26 May 2009, Aidan Van Dyk wrote:

> * Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> [090526 11:20]:
>> Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
>>> This has been raised and ignored many times before on -hackers... The
>>> reason is because the tags in the CVS repository are "broken" (i.e they
>>> are such that it's impossible to actually create all the tags), so the
>>> git "cvsimport" tools that try to tags all croak on the PG CVS repository.
>>
>>> The tool which doesn't croak doesn't try and import all the tags, just
>>> the sticky "branch tags"...
>>
>>> Scripts to "fix" (actually, remove) the broken tags have also been
>>> posted, along with requests that if somebody is "mucking" with the
>>> actual repository, to make sure it's known about, and access is "denied"
>>> during the mucking period (access being any rsync/anoncvs/mirroring of
>>> the cvs root).
>>
>> Up to now I've always been of the opinion that fixing those tags wasn't
>> worth taking any risk for. But if we are thinking of moving away from
>> CVS, then this clearly becomes one of the hurdles we have to jump on the
>> way. Can you refresh our memory about which tags are problematic and
>> exactly what needs to be done about 'em?
>
> Specifically, it's 2 tags, and I just remove them:
> REL7_1_BETA2
> REL7_1_BETA3

So, you are suggesting:

cvs -q tag -d REL7_1_BETA2 .
cvs -q tag -d REL7_1_BETA3 .

correct?

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
Yahoo . yscrappy Skype: hub.org ICQ . 7615664

On Tuesday 26 May 2009 17:44:59 Magnus Hagander wrote:
> Hmm. I looked through the source of the import script. It appears to
> mention tags here and there, but doesn't seem to do it.

Which is part of the reason we use this script and not one of the other ones,
because some of our tags are broken. Any conversion will either have to drop
or repair some tags; see my previous posts to hackers about this.

Hi,

Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
> This has been raised and ignored many times before on -hackers... The
> reason is because the tags in the CVS repository are "broken"

Please keep in mind that the amount of "brokenness" here depends a lot
on the tool used for the conversion to git. AFAIK 'git cvsimport' is
used for the conversion of the Postgres repository to git.
git-cvsimport uses cvsps, which is known for its deficiencies.

> (i.e they
> are such that it's impossible to actually create all the tags), so the
> git "cvsimport" tools that try to tags all croak on the PG CVS repository.
>
> The tool which doesn't croak doesn't try and import all the tags, just
> the sticky "branch tags"...

I cannot confirm that assertion. I've just tried with cvs2svn, which
converts the Postgres repository just fine, including 170 tags, among
them REL7_1_BETA2 and REL7_1_BETA3. A quick glance at the resulting
checkouts' diff looks pretty good as well.

I consider cvsps to be lacking rather than blaming the Postgres CVS
repository. (Of course that doesn't mean the Postgres CVS repository
is perfectly self-consistent - CVS repositories aren't by definition.
I'm just pointing out that there are tools with better heuristics than
those of cvsps.)

Has anybody ever tried using cvs2git? Being based on cvs2svn, it
should yield better results than cvsps. It's even recommended from the
issues section of the git-cvsimport man page [1]. And git-cvsimport
seems to be able to continue from an initial import with cvs2git.

Regards

Markus Wanner

Hi,

Quoting "Peter Eisentraut" <peter_e(at)gmx(dot)net>:
> .. some of our tags are broken. Any conversion will either have to drop
> or repair some tags; see my previous posts to hackers about this.

Can you please point me to that post? I didn't find it.

In what way do you consider the tags "broken"? (As CVS does not
guarantee any inter-file consistency, I don't think one can speak of
brokenness at all. IMO it's rather just a matter of how to convert to
another VCS's representation. Certainly, it's not always possible to
convert without any kind of information loss. However, as just pointed
out, there are certainly differences between converters).

Regards

Markus Wanner

Markus Wanner wrote:
> Hi,
>
> Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
>> This has been raised and ignored many times before on -hackers... The
>> reason is because the tags in the CVS repository are "broken"
>
> Please keep in mind that the amount of "brokenness" here depends a lot
> on the tool used for the conversion to git. AFAIK 'git cvsimport' is
> used for the conversion of the Postgres repository to git. git-cvsimport
> uses cvsps, which is known for its deficiencies.

No, we use fromcvs, not "git cvsimport".

IIRC that was the only one people could make working with incremental
updates.

--
Magnus Hagander
Self: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:
> Markus Wanner wrote:
>> Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
>>> This has been raised and ignored many times before on -hackers... The
>>> reason is because the tags in the CVS repository are "broken"
>> Please keep in mind that the amount of "brokenness" here depends a lot
>> on the tool used for the conversion to git. AFAIK 'git cvsimport' is
>> used for the conversion of the Postgres repository to git. git-cvsimport
>> uses cvsps, which is known for its deficiencies.
>
> No, we use fromcvs, not "git cvsimport".
>
> IIRC that was the only one people could make working with incremental
> updates.

Right. When I looked at the converters last time, there was others that
produce a better conversion, but they didn't work incrementally. If
we're going to switch over the main repository, we should look at the
alternatives.

OTOH, there's some value in staying with current GIT repository. In
EnterpriseDB, we maintain all the Oracle-compatibility stuff in a GIT
repository that's based on the PostgreSQL mirror. If PostgreSQL switches
to a new GIT repository/mirror, I'll have to rebase all that, and I'm
not sure how well that works with all the merges and stuff. I'm probably
the one with most complex situation, but others who have
work-in-progress patches in local repositories will face the same issue
at a smaller scale.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Hi,

Quoting "Magnus Hagander" <magnus(at)hagander(dot)net>:
> No, we use fromcvs, not "git cvsimport".

Oh, thanks for the correction.

I haven't heard of fromcvs before, but solely judging by lines of
code, it's hardly as elaborate as cvs2svn. So my arguments hold true
for it as well.

> IIRC that was the only one people could make working with incremental
> updates.

Understood.

Regards

Markus Wanner

On Wednesday 27 May 2009 13:25:14 Markus Wanner wrote:
> Hi,
>
> Quoting "Peter Eisentraut" <peter_e(at)gmx(dot)net>:
> > .. some of our tags are broken. Any conversion will either have to drop
> > or repair some tags; see my previous posts to hackers about this.
>
> Can you please point me to that post? I didn't find it.

http://archives.postgresql.org/pgsql-hackers/2008-12/msg01879.php

> In what way do you consider the tags "broken"?

The tag applies to different commits on different files.

> (As CVS does not
> guarantee any inter-file consistency, I don't think one can speak of
> brokenness at all.

Just because CVS doesn't guarantee it, it doesn't mean it's not broken.
Otherwise, any possible random permutation of files would be a non-broken
checkout by your definition.

On Wednesday 27 May 2009 00:54:52 Marc G. Fournier wrote:
> So, you are suggesting:
>
> cvs -q tag -d REL7_1_BETA2 .
> cvs -q tag -d REL7_1_BETA3 .

Note that there are actually two different issues related to tags:

One is, the tags REL7_1_BETA2 and REL7_1_BETA3 cannot be parsed by cvsps. But
no one has analyzed why that is. Nor is there any proof that they are wrong
or broken.

The other is, the tag REL7_1 produces different files than were actually in
the release. cvsps warns about this. I had posted a patch to fix this.

* Markus Wanner <markus(at)bluegap(dot)ch> [090527 06:20]:

> Has anybody ever tried using cvs2git? Being based on cvs2svn, it should
> yield better results than cvsps. It's even recommended from the issues
> section of the git-cvsimport man page [1]. And git-cvsimport seems to be
> able to continue from an initial import with cvs2git.

cvs2svn (and hence cvs2git certainly has *oodles* of code explicitly to
try and deal with "weird" (non-linear) cvs histories (a la type of the
PG repo)... I'm not sure I would take the reference to "the old cvs2git
tool" to be the cvs2git that's currently active and based on cvs2svn...
If anybody can confirm that the incremental git cvsimport can follow a
recent cvs2git conversion, that would definitely be awesome! If I can
come across a few free hours some time, I might even try it myself!

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

* Marc G. Fournier <scrappy(at)hub(dot)org> [090526 18:00]:

>> Specifically, it's 2 tags, and I just remove them:
>> REL7_1_BETA2
>> REL7_1_BETA3
>
> So, you are suggesting:
>
> cvs -q tag -d REL7_1_BETA2 .
> cvs -q tag -d REL7_1_BETA3 .
>
> correct?

Not directly, I claim *no* knowledge of the safety of any CVS commands
;-)

But whatever you do, please, lock us our of the repository first (by us,
I mean any publc access to it, via anoncvs, rsync, mirror, anything),
notify us first, (give us at *least* day warning so we can disable any
automatic conversions), and let us know when it's done, so we can watch
the first run after any conversion carefully.

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

* Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> [090527 07:29]:

> OTOH, there's some value in staying with current GIT repository. In
> EnterpriseDB, we maintain all the Oracle-compatibility stuff in a GIT
> repository that's based on the PostgreSQL mirror. If PostgreSQL switches
> to a new GIT repository/mirror, I'll have to rebase all that, and I'm
> not sure how well that works with all the merges and stuff. I'm probably
> the one with most complex situation, but others who have
> work-in-progress patches in local repositories will face the same issue
> at a smaller scale.

But there are oodles of options in git available to handle a cutover
like that:
- grafts
- filter-branch
- rebase (the new rebase toolset can even attempt to rebase a DAG onto an
existing DAG, not just linear patches))

So I'm whatever becomes the "official" git repo can simply be "grafted"
into your history, your your new development grafted on to the
"official" history...

But, if there is nothing wrong with the current repo (except that it
doesn't have tags), than we can easily add tags to it...

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Aidan Van Dyk wrote:
> * Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> [090527 07:29]:
>
>> OTOH, there's some value in staying with current GIT repository. In
>> EnterpriseDB, we maintain all the Oracle-compatibility stuff in a GIT
>> repository that's based on the PostgreSQL mirror. If PostgreSQL switches
>> to a new GIT repository/mirror, I'll have to rebase all that, and I'm
>> not sure how well that works with all the merges and stuff. I'm probably
>> the one with most complex situation, but others who have
>> work-in-progress patches in local repositories will face the same issue
>> at a smaller scale.
>
> But there are oodles of options in git available to handle a cutover
> like that:
> - grafts
> - filter-branch

Okay, your git-fu is stronger than mine, I had never heard of grafts
before :-).

> - rebase (the new rebase toolset can even attempt to rebase a DAG onto an
> existing DAG, not just linear patches))

That's interesting, I once tested git-rebase on the version I have
installed on a similar scenario and it didn't handle merges. If it does
now, that's great.

> But, if there is nothing wrong with the current repo (except that it
> doesn't have tags), than we can easily add tags to it...

Yep. There's not *that* many tags in the CVS repository, we could just
add them manually.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Hi,

Peter Eisentraut wrote:
> http://archives.postgresql.org/pgsql-hackers/2008-12/msg01879.php

Thanks for the link. I'm assuming you've adjusted the tags to fit a
single commit.

Out of curiosity: do you think (or have evidence that) this is the only
tag that spans multiple commits?

>> In what way do you consider the tags "broken"?
>
> The tag applies to different commits on different files.

That's perfectly valid for CVS (and can be represented in subversion as
well). Such a tag cannot (easily) be converted to git, though (nor
mercurial or monotone), where tags are attached to a single commit.

>> (As CVS does not
>> guarantee any inter-file consistency, I don't think one can speak of
>> brokenness at all.
>
> Just because CVS doesn't guarantee it, it doesn't mean it's not broken.

It depends on your understanding of what a tag is. CVS and subversion
certainly have a different understanding from yours (and sometimes tout
this as a feature): their tags can easily span multiple commits.

You (as well as myself, BTW) seem to think of a tag like something
that's attached to a single commit.

> Otherwise, any possible random permutation of files would be a non-broken
> checkout by your definition.

Note that this is not necessarily my definition, rather CVS's (or that
of subversion). And yes, CVS repositories can be pretty badly screwed.

Regards

Markus Wanner

On Wed, May 27, 2009 at 11:44 AM, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> Peter Eisentraut wrote:
>>> In what way do you consider the tags "broken"?
>> The tag applies to different commits on different files.
> That's perfectly valid for CVS (and can be represented in subversion as
> well). Such a tag cannot (easily) be converted to git, though (nor
> mercurial or monotone), where tags are attached to a single commit.
[...]
> You (as well as myself, BTW) seem to think of a tag like something
> that's attached to a single commit.

I think this is a semantic argument. The problem isn't that we don't
understand how CVS behaves; it's that we find that behavior
undesirable, aka broken. If we really care about having a tag that
contains the exact files that are tagged in CVS, we can create a
branch from one of the commits involved, and then apply a commit to
that branch that places it in the state that matches the contents of
the CVS tag. AIUI, this is not very different from what you'd have to
do in Subversion, where a tag is a branch is a copy.

...Robert

* Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> [090527 07:29]:

> OTOH, there's some value in staying with current GIT repository. In
> EnterpriseDB, we maintain all the Oracle-compatibility stuff in a GIT
> repository that's based on the PostgreSQL mirror. If PostgreSQL switches
> to a new GIT repository/mirror, I'll have to rebase all that, and I'm
> not sure how well that works with all the merges and stuff. I'm probably
> the one with most complex situation, but others who have
> work-in-progress patches in local repositories will face the same issue
> at a smaller scale.

OK, so I took a gander at the git repositories out there, namely the
"official" one (I'll call gpo) and the "original" (I'll call git):

gpo: from git://git.postgresql.org/git/postgresql.git
git: from git://repo.or.cz/PostgreSQL.git

For each branch, I diffed them to a cvs export of that sticky tag they
are on. Diffs are available here:
http://people.ifax.com/~aidan/pg/pg-git-cvs/

Since I run the git version of the conversion, I took a close look at
that one... All the differences in it (excpet for the 7.3 ones I'll
mention in a minute) are $Keyword$ differences, stuff like:
-# My2Pg \$Revision: 1.27 $ \translated dump
+# My2Pg \$Revision$ \translated dump
and
-/* $OpenBSD$ */
+/* $OpenBSD: blf.c,v 1.3 2000/06/17 23:36:22 provos Exp $ */
Here, - is git and + is cvs, so you can see, my conversion has some even
backwards... And some are ugly:
-Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2]) by renoir.op.net (o1/$Revision: 1.2 $) with ESMTP id BAA27265 for <pgman(at)candle(dot)pha(dot)pa(dot)us>; Sat, 10 Jun 2000 01:16:07 -0400 (EDT)
+Received: from sss2.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2]) by renoir.op.net (o1/$Revision$) with ESMTP id BAA 27265 for <pgman(at)candle(dot)pha(dot)pa(dot)us>; Sat, 10 Jun 2000 01:16:07 -0400 (EDT)
My conversion left the $Revision$ alone in the mail message, and the cvs
checkout munged it... Actually, notice how stuff like
src/backend/optimizer/plan/README has keywords in it that CVS munches too,
ugh!)

And the $Log$ Keyword make for some huge differences ;-(

But there seems to be some files "missing" in my REL7_3_STABLE, namely:
pg-git-REL7_3_STABLE.diff-diff --git a/contrib/xml/README.pgxml b/contrib/xml/README.pgxml
pg-git-REL7_3_STABLE.diff-diff --git a/contrib/xml/pgxml.sql.in b/contrib/xml/pgxml.sql.in
pg-git-REL7_3_STABLE.diff-diff --git a/contrib/xml/pgxml_dom.sql.in b/contrib/xml/pgxml_dom.sql.in
pg-git-REL7_3_STABLE.diff-diff --git a/src/bin/pg_controldata/po/zh_CN.po b/src/bin/pg_controldata/po/zh_CN.po
pg-git-REL7_3_STABLE.diff-diff --git a/src/bin/pg_resetxlog/po/zh_CN.po b/src/bin/pg_resetxlog/po/zh_CN.po
pg-git-REL7_3_STABLE.diff-diff --git a/src/port/getopt.c b/src/port/getopt.c
pg-git-REL7_3_STABLE.diff-diff --git a/src/test/regress/expected/geometry-bsd-precision.out b/src/test/regress/expected/geometry-bsd-precision.out
I'm not really sure why, I can look into it if it's important...

But in all, the differneces are small and otherwise all keyword related:
pg-git-MANUAL_DIST.diff 0 files changed
pg-git-PG95-DIST.diff 0 files changed
pg-git-PG95_DIST.diff 2 files changed, 2 insertions(+), 2 deletions(-)
pg-git-REL2_0B.diff 2 files changed, 2 insertions(+), 2 deletions(-)
pg-git-REL6_4.diff 14 files changed, 53 insertions(+), 13 deletions(-)
pg-git-REL6_5_PATCHES.diff 20 files changed, 67 insertions(+), 20 deletions(-)
pg-git-REL7_0_PATCHES.diff 7 files changed, 12 insertions(+), 7 deletions(-)
pg-git-REL7_1_STABLE.diff 24 files changed, 149 insertions(+), 146 deletions(-)
pg-git-REL7_2_STABLE.diff 30 files changed, 153 insertions(+), 150 deletions(-)
pg-git-REL7_3_STABLE.diff 59 files changed, 1457 insertions(+), 167 deletions(-)
pg-git-REL7_4_STABLE.diff 53 files changed, 172 insertions(+), 169 deletions(-)
pg-git-REL8_0_0.diff 26 files changed, 44 insertions(+), 28 deletions(-)
pg-git-REL8_0_STABLE.diff 26 files changed, 44 insertions(+), 28 deletions(-)
pg-git-REL8_1_STABLE.diff 25 files changed, 26 insertions(+), 26 deletions(-)
pg-git-REL8_2_STABLE.diff 24 files changed, 26 insertions(+), 26 deletions(-)
pg-git-REL8_3_STABLE.diff 23 files changed, 25 insertions(+), 25 deletions(-)
pg-git-Release_1_0_3.diff 2 files changed, 2 insertions(+), 2 deletions(-)
pg-git-WIN32_DEV.diff 53 files changed, 172 insertions(+), 169 deletions(-)
pg-git-ecpg_big_bison.diff 0 files changed
pg-git-master.diff 81 files changed, 85 insertions(+), 85 deletions(-)
TOTAL 234 files changed, 2491 insertions(+), 1065 deletions(-)

But the gpo conversion seems to be in pretty bad shape:
aidan(at)db1-dapper:/tmp$ grep 'new file' pg-gpo-* | wc
301 1245 14096

pg-gpo-MANUAL_DIST.diff 0 files changed
pg-gpo-PG95-DIST.diff 0 files changed
pg-gpo-PG95_DIST.diff 2 files changed, 2 insertions(+), 2 deletions(-)
pg-gpo-REL2_0B.diff 2 files changed, 2 insertions(+), 2 deletions(-)
pg-gpo-REL6_4.diff 12 files changed, 51 insertions(+), 11 deletions(-)
pg-gpo-REL6_5_PATCHES.diff 12 files changed, 59 insertions(+), 12 deletions(-)
pg-gpo-REL7_0_PATCHES.diff 3 files changed, 8 insertions(+), 3 deletions(-)
pg-gpo-REL7_1_STABLE.diff 22 files changed, 147 insertions(+), 144 deletions(-)
pg-gpo-REL7_2_STABLE.diff 30 files changed, 153 insertions(+), 150 deletions(-)
pg-gpo-REL7_3_STABLE.diff 58 files changed, 925 insertions(+), 167 deletions(-)
pg-gpo-REL7_4_STABLE.diff 136 files changed, 43302 insertions(+), 169 deletions(-)
pg-gpo-REL8_0_0.diff 31 files changed, 2288 insertions(+), 28 deletions(-)
pg-gpo-REL8_0_STABLE.diff 206 files changed, 16289 insertions(+), 11798 deletions(-)
pg-gpo-REL8_1_STABLE.diff 155 files changed, 24386 insertions(+), 26 deletions(-)
pg-gpo-REL8_2_STABLE.diff 24 files changed, 26 insertions(+), 26 deletions(-)
pg-gpo-REL8_3_STABLE.diff 23 files changed, 25 insertions(+), 25 deletions(-)
pg-gpo-Release_1_0_3.diff 2 files changed, 2 insertions(+), 2 deletions(-)
pg-gpo-WIN32_DEV.diff 119 files changed, 16637 insertions(+), 169 deletions(-)
pg-gpo-ecpg_big_bison.diff 0 files changed
pg-gpo-master.diff 81 files changed, 85 insertions(+), 85 deletions(-)
TOTAL 627 files changed, 104387 insertions(+), 12819 deletions(-)

And actually looking at the history of the gpo repo, the branches are all
messed up with "merges" and stuff that I'm not sure where they are coming
from... 8.2, 8.3, and master(HEAD) are all the same as my gpo repo, but the
back branchs are very bad...

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

- --On Wednesday, May 27, 2009 16:33:28 +0300 Peter Eisentraut <peter_e(at)gmx(dot)net>
wrote:

> On Wednesday 27 May 2009 00:54:52 Marc G. Fournier wrote:
>> So, you are suggesting:
>>
>> cvs -q tag -d REL7_1_BETA2 .
>> cvs -q tag -d REL7_1_BETA3 .
>
> Note that there are actually two different issues related to tags:
>
> One is, the tags REL7_1_BETA2 and REL7_1_BETA3 cannot be parsed by cvsps.
> But no one has analyzed why that is. Nor is there any proof that they are
> wrong or broken.

I'm curious as to what is different about these vs all the other tags I've ever
done, both before, and after ...

> The other is, the tag REL7_1 produces different files than were actually in
> the release. cvsps warns about this. I had posted a patch to fix this.

Please repost ...

- --
Marc G. Fournier Hub.Org Hosting Solutions S.A. (http://www.hub.org)
Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
Yahoo . yscrappy Skype: hub.org ICQ . 7615664
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (FreeBSD)

iEYEARECAAYFAkod5DYACgkQ4QvfyHIvDvPfcgCfTCGz5JG5KmCQrbdx9+37l8sT
nFAAnjcH3oL11J5CIKR5ZIVHRtSe+MVj
=SE8O
-----END PGP SIGNATURE-----

"Marc G. Fournier" <scrappy(at)hub(dot)org> wrote:

> I'm curious as to what is different about these vs all the other
> tags I've ever done, both before, and after ...

Any chance you tagged, changes were committed, and you then tagged
files from such a later commit as part of the release, or moved the
tag to the later commit? Those are perfectly reasonable things to do
under the CVS philosophy, and not in line with the philosophy of some
of the other products.

If there's a chance you did that on a couple beta releases in that
time frame, and on no others, that might explain it.

One product's flexibility is another product's "broken".

-Kevin

On Wed, May 27, 2009 at 5:22 PM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
>    gpo: from git://git.postgresql.org/git/postgresql.git
>    git: from git://repo.or.cz/PostgreSQL.git
[...]
> But the gpo conversion seems to be in pretty bad shape:
>        aidan(at)db1-dapper:/tmp$ grep 'new file' pg-gpo-* | wc
>            301    1245   14096
>
>        pg-gpo-MANUAL_DIST.diff        0 files changed
>        pg-gpo-PG95-DIST.diff          0 files changed
>        pg-gpo-PG95_DIST.diff          2 files changed, 2 insertions(+), 2 deletions(-)
>        pg-gpo-REL2_0B.diff            2 files changed, 2 insertions(+), 2 deletions(-)
>        pg-gpo-REL6_4.diff             12 files changed, 51 insertions(+), 11 deletions(-)
>        pg-gpo-REL6_5_PATCHES.diff     12 files changed, 59 insertions(+), 12 deletions(-)
>        pg-gpo-REL7_0_PATCHES.diff     3 files changed, 8 insertions(+), 3 deletions(-)
>        pg-gpo-REL7_1_STABLE.diff      22 files changed, 147 insertions(+), 144 deletions(-)
>        pg-gpo-REL7_2_STABLE.diff      30 files changed, 153 insertions(+), 150 deletions(-)
>        pg-gpo-REL7_3_STABLE.diff      58 files changed, 925 insertions(+), 167 deletions(-)
>        pg-gpo-REL7_4_STABLE.diff      136 files changed, 43302 insertions(+), 169 deletions(-)
>        pg-gpo-REL8_0_0.diff           31 files changed, 2288 insertions(+), 28 deletions(-)
>        pg-gpo-REL8_0_STABLE.diff      206 files changed, 16289 insertions(+), 11798 deletions(-)
>        pg-gpo-REL8_1_STABLE.diff      155 files changed, 24386 insertions(+), 26 deletions(-)
>        pg-gpo-REL8_2_STABLE.diff      24 files changed, 26 insertions(+), 26 deletions(-)
>        pg-gpo-REL8_3_STABLE.diff      23 files changed, 25 insertions(+), 25 deletions(-)
>        pg-gpo-Release_1_0_3.diff      2 files changed, 2 insertions(+), 2 deletions(-)
>        pg-gpo-WIN32_DEV.diff          119 files changed, 16637 insertions(+), 169 deletions(-)
>        pg-gpo-ecpg_big_bison.diff     0 files changed
>        pg-gpo-master.diff             81 files changed, 85 insertions(+), 85 deletions(-)
>        TOTAL                          627 files changed, 104387 insertions(+), 12819 deletions(-)
>
> And actually looking at the history of the gpo repo, the branches are all
> messed up with "merges" and stuff that I'm not sure where they are coming
> from...  8.2, 8.3, and master(HEAD) are all the same as my gpo repo, but the
> back branchs are very bad...

This is really quite horrible. What is the best way forward here?

...Robert

On Wed, 27 May 2009, Kevin Grittner wrote:

> "Marc G. Fournier" <scrappy(at)hub(dot)org> wrote:
>
>> I'm curious as to what is different about these vs all the other
>> tags I've ever done, both before, and after ...
>
> Any chance you tagged, changes were committed, and you then tagged
> files from such a later commit as part of the release, or moved the
> tag to the later commit? Those are perfectly reasonable things to do
> under the CVS philosophy, and not in line with the philosophy of some
> of the other products.
>
> If there's a chance you did that on a couple beta releases in that
> time frame, and on no others, that might explain it.

Actually, I have done that on at least one of the 8.x tags too, so if that
is it, more then those two tags should be causing issues ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
Yahoo . yscrappy Skype: hub.org ICQ . 7615664

* Robert Haas <robertmhaas(at)gmail(dot)com> [090527 21:30]:

> > And actually looking at the history of the gpo repo, the branches are all
> > messed up with "merges" and stuff that I'm not sure where they are coming
> > from...  8.2, 8.3, and master(HEAD) are all the same as my gpo repo, but the
> > back branchs are very bad...
>
> This is really quite horrible. What is the best way forward here?

That depends entirely on what the project wants.

If you're just developing on HEAD with git to submit patches to official
CVS use, don't do anything... HEAD is in good state in both repos.

If you're using git to rebase patches and changes between branches, in
order to submit patches for official CVS use, use the PostgreSQL.git I
publish on repo.or.cz. It's back branches are in pretty good state too.

If the "project" wants to provide canonical "git" repository as a
cut-over from CVS, and CVS isn't going to be used anymore (other than
served as an anonymous "mirror" of the git repo by git cvsserver, or
have an old CVSROOT available as a public download for purely historical
inspection), then it's probably best to do a single CVS->git conversion
with one of the better tools (like parsecvs) that don't do incremental,
and publish a set of good "graft" point for those using gpo who want to
switch their current development onto the new git repo.

But, since git's a completely Dvcs at its core, and was built by people
thoroughly immersed in the tool-needs of vastly distributed VCS stuff,
no matter what the project deems the "official" git repository, people
will be able to continue to keep their current investment in git
development with stuff like grafts, filter-branch, rebase, etc.

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

On Wed, May 27, 2009 at 10:09 PM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> * Robert Haas <robertmhaas(at)gmail(dot)com> [090527 21:30]:
>
>> > And actually looking at the history of the gpo repo, the branches are all
>> > messed up with "merges" and stuff that I'm not sure where they are coming
>> > from...  8.2, 8.3, and master(HEAD) are all the same as my gpo repo, but the
>> > back branchs are very bad...
>>
>> This is really quite horrible.  What is the best way forward here?
>
> That depends entirely on what the project wants.

I can't speak for anyone else, but what I want is for the git tree on
git.postgresql.org to match CVS.

...Robert

Hi,

Quoting "Robert Haas" <robertmhaas(at)gmail(dot)com>:
> I think this is a semantic argument. The problem isn't that we don't
> understand how CVS behaves; it's that we find that behavior
> undesirable

I fully agree to that and find it undesirable as well.

> aka broken.

Well, for some it's a feature, for others a bug ;-)

My point was that other converters have better support for such
(undesirable, but still existent) tags that span multiple commits. If
that's unwanted anyway, it seems cleaner to fix the CVS repository,
yes. Has that been done now? Or is somebody going to do it? (See
Peter's patch he just linked again upthread).

> If we really care about having a tag that
> contains the exact files that are tagged in CVS, we can create a
> branch from one of the commits involved, and then apply a commit to
> that branch that places it in the state that matches the contents of
> the CVS tag.

Exactly (with the difference that with the branch you preserve the
history of changes, while the variant with the tag does not).

> AIUI, this is not very different from what you'd have to
> do in Subversion, where a tag is a branch is a copy.

I think so, too. I'd even state that subversion doesn't really support
tagging, instead it simulates tags with branches.

Regards

Markus Wanner

Hi,

Quoting "Marc G. Fournier" <scrappy(at)hub(dot)org>:
> Please repost ...

Peter referred to this message here:

http://archives.postgresql.org/pgsql-hackers/2008-12/msg01879.php

However, please be cautious before applying such a patch.

Regards

Markus Wanner

Hi,

Quoting "Marc G. Fournier" <scrappy(at)hub(dot)org>:
> Actually, I have done that on at least one of the 8.x tags too, so
> if that is it, more then those two tags should be causing issues ...

Not *every* such issue causes problems. An example that's perfectly fine:

cvs commit -m "first commit" fileA
cvs tag TEST filA
cvs commit -m "second commit" fileB
cvs tag TEST fileB

In such a situation, a converter can easily "push-down" the tag TEST
to the second commit, because fileA is the same (in that revision) as
after the first commit. After all, the results in the RCS files are
exactly the same as if you did the following:

cvs commit -m "first commit" fileA
cvs commit -m "second commit" fileB
cvs tag TEST fileA fileB

A converter can't possibly distinguish these two.

However, if both files get committed the second time, but only one
gets tagged, it gets problematic (always assuming the commit actually
changes the file):

cvs commit -m "first commit" fileA
cvs tag TEST filA
cvs commit -m "second commit" fileA fileB
cvs tag TEST fileB

That's perfectly valid from CVS's point of view, unwanted for the
Postgres repository and hard to handle for a converter to git (or
mercurial, monotone, etc..), because the tag TEST is on the first
commit for fileA but on the second for fileB, while both of fileA and
fileB differ between the commits.

Regards

Markus Wanner

* Robert Haas <robertmhaas(at)gmail(dot)com> [090527 22:43]:
> On Wed, May 27, 2009 at 10:09 PM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> > * Robert Haas <robertmhaas(at)gmail(dot)com> [090527 21:30]:
> >
> >> > And actually looking at the history of the gpo repo, the branches are all
> >> > messed up with "merges" and stuff that I'm not sure where they are coming
> >> > from...  8.2, 8.3, and master(HEAD) are all the same as my gpo repo, but the
> >> > back branchs are very bad...
> >>
> >> This is really quite horrible.  What is the best way forward here?
> >
> > That depends entirely on what the project wants.
>
> I can't speak for anyone else, but what I want is for the git tree on
> git.postgresql.org to match CVS.

Well, sure, but I think the "way forward" part implied recognition that
the current tree at git.postgresql.org *doesn't* match CVS very closely
(for back branches), and that people currently rely on it and use it.

So, again, the answer to the question really does depend on what the
"canonical" VCS of the project is. As of now, it's *still* CVS, and
those using either git repo can still develop and submit patches to CVS
easily.

When the project switches, there will probably need to be a more
canonical conversion, with one of the tools that doesn't support
incremental imports, and then people will have to adjust their current
repo with any of rebase/graft/filter-branch to adjust their work
history onto the "official" tree...

All that based on the assumption that when the project switches to git,
they actually want all the CVS history in their official tree. Its
certainly not necessary, and possibly not even desirable... PostgreSQL
could just as easily to a "linus" style switch when they switch to git,
and just "import" the latest release in each branch as the starting
point for each branch. The git repository will have no history, and
people can choose which history they want to graft in... CVSROOT can be
made available as a historical download.

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

On Thu, May 28, 2009 at 8:59 AM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> All that based on the assumption that when the project switches to git,
> they actually want all the CVS history in their official tree.  Its
> certainly not necessary, and possibly not even desirable...  PostgreSQL
> could just as easily to a "linus" style switch when they switch to git,
> and just "import" the latest release in each branch as the starting
> point for each branch.  The git repository will have no history, and
> people can choose which history they want to graft in...  CVSROOT can be
> made available as a historical download.

That would suck for me. I use git log a lot to see how things have
changed over time.

...Robert

* Robert Haas <robertmhaas(at)gmail(dot)com> [090528 09:49]:
> On Thu, May 28, 2009 at 8:59 AM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> > All that based on the assumption that when the project switches to git,
> > they actually want all the CVS history in their official tree.  Its
> > certainly not necessary, and possibly not even desirable...  PostgreSQL
> > could just as easily to a "linus" style switch when they switch to git,
> > and just "import" the latest release in each branch as the starting
> > point for each branch.  The git repository will have no history, and
> > people can choose which history they want to graft in...  CVSROOT can be
> > made available as a historical download.
>
> That would suck for me. I use git log a lot to see how things have
> changed over time.

No, the whole point is that you graft whatever history *you* want in...
So if PostgreSQL "offical" git only starts when the offical VCS was in
git, you graft on gpo, or git, or some personal one-time cvs2git or
parsecvs history you want in...

It would be the projects way of saying basically "None of the current
cvs imports are perfect and we recognize that. So we're starting fresh,
use whatever historical cvs import *you* find best for your history and
graft it in". Just the linux kernel has a few "historical" repos
available for people to graft into linus's tree which only started in
2.6.12.

If you have work that requires the history of the current gpo repo, you
keep using it. If you have work requring the current git repo, you keep
using it. If you have no work, but you're a stickler for perfect
imports, you start working on parsecvs and cvs2git, and make a new
history every time you find another quirk...

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Robert Haas wrote:
> On Thu, May 28, 2009 at 8:59 AM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
>
>> All that based on the assumption that when the project switches to git,
>> they actually want all the CVS history in their official tree. Its
>> certainly not necessary, and possibly not even desirable... PostgreSQL
>> could just as easily to a "linus" style switch when they switch to git,
>> and just "import" the latest release in each branch as the starting
>> point for each branch. The git repository will have no history, and
>> people can choose which history they want to graft in... CVSROOT can be
>> made available as a historical download.
>>
>
> That would suck for me. I use git log a lot to see how things have
> changed over time.
>
>
>

Indeed. Losing the history is not an acceptable option.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Robert Haas wrote:
>> That would suck for me. I use git log a lot to see how things have
>> changed over time.

> Indeed. Losing the history is not an acceptable option.

I think the same. If git is not able to maintain our project history
then it is not mature enough to be considered as our official VCS.
This is not a negotiable requirement.

regards, tom lane

On Thu, May 28, 2009 at 10:18 AM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> * Robert Haas <robertmhaas(at)gmail(dot)com> [090528 09:49]:
>> On Thu, May 28, 2009 at 8:59 AM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
>> > All that based on the assumption that when the project switches to git,
>> > they actually want all the CVS history in their official tree.  Its
>> > certainly not necessary, and possibly not even desirable...  PostgreSQL
>> > could just as easily to a "linus" style switch when they switch to git,
>> > and just "import" the latest release in each branch as the starting
>> > point for each branch.  The git repository will have no history, and
>> > people can choose which history they want to graft in...  CVSROOT can be
>> > made available as a historical download.
>>
>> That would suck for me.  I use git log a lot to see how things have
>> changed over time.
>
> No, the whole point is that you graft whatever history *you* want in...
> So if PostgreSQL "offical" git only starts when the offical VCS was in
> git, you graft on gpo, or git, or some personal one-time cvs2git or
> parsecvs history you want in...

I want the project infrastructure to do this for me so I don't have to
do anything except git clone. It's not a big deal for me to port my
WIP over to a new git repo if this one is busted, which it sounds like
it is. But I'm not interested in rolling my own history.

...Robert

On Thu, May 28, 2009 at 3:52 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>> Robert Haas wrote:
>>> That would suck for me.  I use git log a lot to see how things have
>>> changed over time.
>
>> Indeed. Losing the history is not an acceptable option.
>
> I think the same.  If git is not able to maintain our project history
> then it is not mature enough to be considered as our official VCS.
> This is not a negotiable requirement.

I think the idea is that you could choose, for example, the level of
granularity you want to keep. That could be interesting in the future
-- someone who submitted a patch (or anyone who was working in that
area) might want to keep all their intermediate commits and not just
the one big commit for the whole feature.

But it's not like we have a lot of choices for our history. Only a few
patches were maintained in a distributed vc system so far and I don't
think many people followed them. Also given the massive changes
patches have tended to get when being committed keeping the history of
the patch development seems kind of pointless.

--
greg

On Thu, May 28, 2009 at 11:04 AM, Greg Stark <stark(at)enterprisedb(dot)com> wrote:
> On Thu, May 28, 2009 at 3:52 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>>> Robert Haas wrote:
>>>> That would suck for me.  I use git log a lot to see how things have
>>>> changed over time.
>>
>>> Indeed. Losing the history is not an acceptable option.
>>
>> I think the same.  If git is not able to maintain our project history
>> then it is not mature enough to be considered as our official VCS.
>> This is not a negotiable requirement.
>
> I think the idea is that you could choose, for example, the level of
> granularity you want to keep. That could be interesting in the future
> -- someone who submitted a patch (or anyone who was working in that
> area) might want to keep all their intermediate commits and not just
> the one big commit for the whole feature.

I don't think that was the idea - Aidan floated the idea of just
checking the current version of each branch into git, rather than
importing the full history from CVS (and letting indivdual cloners fix
their own history if they were so inclined). I think that's a
non-starter.

I'm still not sure who is going to take responsibility for fixing the
git tree we have now. I don't think it's going to work for us to
leave it broken until we're ready to do "the cutover", and then do one
monolithic move. If the tools we're using to do the import now have
broken our tree, then we need to fix it, and them. Ideally I'd like
to get a bi-directional conversion working, so that committers could
commit via either CVS or GIT during the transition, but I'm not sure
whether that's feasible.

...Robert

Hi,

Quoting "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
> I think the same. If git is not able to maintain our project history
> then it is not mature enough to be considered as our official VCS.

As Aidan pointed out, the question is not *if* git can represent it.
It's rather *how*. Especially WRT changes of historical information in
the CVS repository underneath.

Heikki is considered about having to merge WIP branches in case the
(CVS and git repository) history changes, so he'd like to maintain the
old history as well as the changed one. OTOH Robert doesn't want to
fiddle with multiple histories and expects to have just exactly one
history. Obviously one can't have both. Either one has to rebase/merge
his changes onto the new history, or continue with multiple histories.

Being a monotone fan, I have to admit that git definitely provides the
most options on *how* to handle these cases, see Aidan's mail upthread.

Knowing most of the corruptions of CVS in use in the wild (by fiddling
with cvs_import for monotone) I now consider git (and svn, hg, bzr,
mtn..) to be more mature than CVS, certainly much more consistent. So
if maturity (not age) is your major concern, I'd rather flee from CVS
now than tomorrow.

Regards

Markus Wanner

On Thu, May 28, 2009 at 11:40 AM, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> Quoting "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
>> I think the same.  If git is not able to maintain our project history
>> then it is not mature enough to be considered as our official VCS.
>
> As Aidan pointed out, the question is not *if* git can represent it. It's
> rather *how*. Especially WRT changes of historical information in the CVS
> repository underneath.
>
> Heikki is considered about having to merge WIP branches in case the (CVS and
> git repository) history changes, so he'd like to maintain the old history as
> well as the changed one. OTOH Robert doesn't want to fiddle with multiple
> histories and expects to have just exactly one history. Obviously one can't
> have both. Either one has to rebase/merge his changes onto the new history,
> or continue with multiple histories.

My understanding is that the histories of some of the branches we have
now are flat-out wrong. I don't have a problem keeping those
alongside the corrected history for ease of rebasing and porting
commits, but I don't want to punt the problem of figuring out what the
one, true, and correct history is to the user. The canonical
repository needs to provide that, and if it provides other alternative
timelines (a la Star Trek) for the convenience of people in Heikki's
situation, that's OK too, as long as they are clearly labeled as such.
I think ideally we'd phase those out and garbage collect them
eventually, but we can certainly keep them for a while.

...Robert

Hi,

Quoting "Robert Haas" <robertmhaas(at)gmail(dot)com>:
> I don't think that was the idea - Aidan floated the idea of just
> checking the current version of each branch into git, rather than
> importing the full history from CVS (and letting indivdual cloners fix
> their own history if they were so inclined). I think that's a
> non-starter.

I'd say it depends on how hard it is to "fix" one's history. If it's
just a config option instructing git to fetch everything before
revision X from repository Y...

OTOH, it would certainly be nicer to have a "default" history, where
only people who require another history would need such a config
option. I'm not quite sure what's possible there.

> I don't think it's going to work for us to
> leave it broken until we're ready to do "the cutover", and then do one
> monolithic move.

Agreed. However, I'm pretty certain this won't be the last time we
have to "fix" the git repository. Conversion from a bunch of RCS files
is just way too ambiguous.

> If the tools we're using to do the import now have
> broken our tree, then we need to fix it, and them.

..and the CVS repository.

Regards

Markus Wanner

Hi,

Quoting "Robert Haas" <robertmhaas(at)gmail(dot)com>:
> My understanding is that the histories of some of the branches we have
> now are flat-out wrong.

AFAIU only the latest revisions of the branches have been compared.
Keeping history and future in mind, that's not telling much, IMO. In
my experience, there's much more wrong with converted CVS repositories
- the latest revisions are often just the tip of the iceberg.
Depending on your definition of "wrong", of course.

> I don't have a problem keeping those
> alongside the corrected history for ease of rebasing and porting
> commits, but I don't want to punt the problem of figuring out what the
> one, true, and correct history is to the user.

Understood and agreed. (In a distributed VCS, you cannot "delete"
history by definition, because every user is free to keep his version).

However, I'm pretty certain this is not the last "flat-out wrong"
thing we find in the CVS or in the converted git repository. Going to
fix and rebase every time might be pretty annoying and time consuming.
Thus alternatives like those mentioned by Aidan sound interesting to me.

Regards

Markus Wanner

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> I'm still not sure who is going to take responsibility for fixing the
> git tree we have now. I don't think it's going to work for us to
> leave it broken until we're ready to do "the cutover", and then do one
> monolithic move. If the tools we're using to do the import now have
> broken our tree, then we need to fix it, and them. Ideally I'd like
> to get a bi-directional conversion working, so that committers could
> commit via either CVS or GIT during the transition, but I'm not sure
> whether that's feasible.

I fear the latter is probably pie in the sky, unfortunately --- to take
just one minor point, which commit timestamp is authoritative? I think
we will have to make a clean cutover from "CVS is authoritative" to
"CVS is dead and git is authoritative", and do a fresh repository
conversion at that instant. What we should be doing to get prepared for
that is testing various conversion tools to see which one gives us the
best conversion. And fixing anything in the CVS repository that is
preventing getting a sane conversion.

The existing git mirror is an unofficial service and is not going to be
the basis of the future authoritative repository. Folks who have cloned
it will have to re-clone. Sorry about that, but maintaining continuity
with that repository is just too far down the list of priorities
... especially when we already know it's broken.

I am hoping that git's cvs server emulation is complete enough that you
can commit through it --- anybody know? But that will be just a
stopgap.

BTW, can anyone comment on whether and how we can maintain the current
split between master repository (that's not even accessible to
non-committers) and a public mirror? If only from a standpoint of
security paranoia, I'd rather like to preserve that split, but I don't
know how well git will play with it.

regards, tom lane

On Thu, May 28, 2009 at 12:10 PM, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> Hi,
>
> Quoting "Robert Haas" <robertmhaas(at)gmail(dot)com>:
>>
>> My understanding is that the histories of some of the branches we have
>> now are flat-out wrong.
>
> AFAIU only the latest revisions of the branches have been compared. Keeping
> history and future in mind, that's not telling much, IMO. In my experience,
> there's much more wrong with converted CVS repositories - the latest
> revisions are often just the tip of the iceberg. Depending on your
> definition of "wrong", of course.

That's not the best news I've had today...

>> I don't have a problem keeping those
>> alongside the corrected history for ease of rebasing and porting
>> commits, but I don't want to punt the problem of figuring out what the
>> one, true, and correct history is to the user.
>
> Understood and agreed. (In a distributed VCS, you cannot "delete" history by
> definition, because every user is free to keep his version).
>
> However, I'm pretty certain this is not the last "flat-out wrong" thing we
> find in the CVS or in the converted git repository. Going to fix and rebase
> every time might be pretty annoying and time consuming. Thus alternatives
> like those mentioned by Aidan sound interesting to me.

To me they sound complex and inconvenient. I guess I'm kind of
mystified by why we can't make this work reliably. Other than the
"broken tags" issue we've discussed, it seems like the only real issue
should be how to group changes to different files into a single
commit. Once you do that, you should be able to construct a
well-defined, total function f : <cvs-file, cvs-revision> -> <git
commit> which is surjective on the space of git commits. In fact it
might be a good idea to explicitly construct this mapping and drop it
into a database table somewhere so that people can sanity check it as
much as they wish. Why is this harder than I think it is?

...Robert

On Thu, May 28, 2009 at 12:19 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> I'm still not sure who is going to take responsibility for fixing the
>> git tree we have now.  I don't think it's going to work for us to
>> leave it broken until we're ready to do "the cutover", and then do one
>> monolithic move.  If the tools we're using to do the import now have
>> broken our tree, then we need to fix it, and them.  Ideally I'd like
>> to get a bi-directional conversion working, so that committers could
>> commit via either CVS or GIT during the transition, but I'm not sure
>> whether that's feasible.
>
> I fear the latter is probably pie in the sky, unfortunately --- to take
> just one minor point, which commit timestamp is authoritative?

That's just a question of deciding on a date when git becomes
authoritative and CVS ceases to be.

> I think
> we will have to make a clean cutover from "CVS is authoritative" to
> "CVS is dead and git is authoritative", and do a fresh repository
> conversion at that instant.  What we should be doing to get prepared for
> that is testing various conversion tools to see which one gives us the
> best conversion.  And fixing anything in the CVS repository that is
> preventing getting a sane conversion.

That might work, but then we better be pretty darn confident that that
"fresh conversion" is actually correct. I'd rather have them going
side-by-side so that we can verify everything before shutting the old
system off.

> The existing git mirror is an unofficial service and is not going to be
> the basis of the future authoritative repository.  Folks who have cloned
> it will have to re-clone.  Sorry about that, but maintaining continuity
> with that repository is just too far down the list of priorities
> ... especially when we already know it's broken.
>
> I am hoping that git's cvs server emulation is complete enough that you
> can commit through it --- anybody know?  But that will be just a
> stopgap.
>
> BTW, can anyone comment on whether and how we can maintain the current
> split between master repository (that's not even accessible to
> non-committers) and a public mirror?  If only from a standpoint of
> security paranoia, I'd rather like to preserve that split, but I don't
> know how well git will play with it.

You can set up one repository to mirror another.

...Robert

On Thu, 28 May 2009, Robert Haas wrote:

> My understanding is that the histories of some of the branches we have
> now are flat-out wrong. I don't have a problem keeping those
> alongside the corrected history for ease of rebasing and porting
> commits, but I don't want to punt the problem of figuring out what the
> one, true, and correct history is to the user.

Right. There has to be "one true repo" for the history here, and if it
takes another repo conversion to do it that's unfortunate for people
already using the existing repo, but as pointed out there are tools
available to help them out. You can't prioritize users of this early test
repo ahead of the long-term goals here, and making it easier for new
people to quickly start hacking on the codebase is very much a motivating
factor behind the conversion.

Because the mapping of CVS commits into git ones has a bit of fuzziness to
it, it's possible to turn fine-tuning the repo history into an endless
project. Wandering down that road helps no one.

The best way to control the scope creep here is to avoid doing that, and
instead focus on what you really need from the repo conversion. In this
case, it's a hard requirement that current and back branches that are
still maintained must produce a checked out result that is identical to if
you were to check that version out of CVS. There's already been some spot
checking of that already, it may make sense to write up an official QA
spec here.

Reconversion of the old history needs to happen as many times as necessary
until that goal is reached for git to be adopted by the project one day.
Because I think that's going to require an iterative process
(convert/test/fix/repeat) I'm not sure what value there is to the better
conversion tools that can't be used incrementally here.

If the goalposts are moved to "every ancient tag/release ever must build
perfectly and have sane history no matter how nasty its CVS history was",
history conversion is doomed. I don't think it's unrealistic to plan
reaching a point where you can say "we've confirmed every release build
from 7.4 forward builds identically from git; older releases, betas, and
similarly early builds should instead be built from the deprecated CVS
repo". If the scope of the conversion has higher standards than that, and
I can't imagine why it should, there's going to be an enormous amount of
time wasted playing around with tags that results in no benefit to users
of the software.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> On Thu, May 28, 2009 at 12:19 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I think
>> we will have to make a clean cutover from "CVS is authoritative" to
>> "CVS is dead and git is authoritative", and do a fresh repository
>> conversion at that instant. What we should be doing to get prepared for
>> that is testing various conversion tools to see which one gives us the
>> best conversion. And fixing anything in the CVS repository that is
>> preventing getting a sane conversion.

> That might work, but then we better be pretty darn confident that that
> "fresh conversion" is actually correct.

Well, yeah, which is one of several reasons why this isn't happening
tomorrow ;-). Whatever tool we use should have survived a good deal
of advance testing.

regards, tom lane

Robert Haas escribió:

> To me they sound complex and inconvenient. I guess I'm kind of
> mystified by why we can't make this work reliably. Other than the
> "broken tags" issue we've discussed, it seems like the only real issue
> should be how to group changes to different files into a single
> commit.

There's another issue which is that of the $Id$ and similar tags. We
have to decide what we want to do with them. If we're not going to have
them in the Git repository, then they are only causing trouble right now
and it would be better to get rid of them completely for the conversion,
to avoid the noise that they will invariably cause.

We could, for example, say that a conversion process is supposed to
un-expand them (say sed -e 's/$Revision:[^$]*$/$Revision$/' and so on;
obviously it's a lot more complex for $Log$) *before* attempting to
analyze any revision. I think that would make further munging a lot
simpler.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> The best way to control the scope creep here is to avoid doing that, and
> instead focus on what you really need from the repo conversion. [...]
> If the goalposts are moved to "every ancient tag/release ever must build
> perfectly and have sane history no matter how nasty its CVS history was",
> history conversion is doomed.

Right. Shall we try to spec out exactly what our conversion
requirements are? Here's a shot:

* Head of each active branch must check out the same as it does from CVS
(modulo $PostgreSQL$ and similar tags, which we've already agreed we can
abandon).

* Each released minor version tag must check out the same as from CVS,
at least back to some specified point (perhaps 7.4.0). I'd really
prefer to insist on that all the way back.

* Each commit message in the CVS history must be retrievable from the
git history, and should correspond to the same file changes. However,
we are okay with git sometimes treating "one" CVS commit as two or more
events with similar messages. (I'm basing this on the behavior of
cvs2cl, which sometimes does that depending on how time-extended the
individual file updates were.) Also, we won't be too picky about
whether the "same" commits on different branches are treated as one
event or multiple events.

Comments? Other considerations?

regards, tom lane

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> There's another issue which is that of the $Id$ and similar tags. We
> have to decide what we want to do with them. If we're not going to have
> them in the Git repository, then they are only causing trouble right now
> and it would be better to get rid of them completely for the conversion,
> to avoid the noise that they will invariably cause.

What was in the back of my mind was that we'd go around and mass-remove
$PostgreSQL$ (and any other lurking tags), but only from HEAD and only
after the repo conversion. Although just before it would be okay too.
The stickier part of this is what to do about back branches;
particularly whether we are okay with checked-out versions of past
releases not matching the actual shipped tarballs on this point.

regards, tom lane

On Thu, May 28, 2009 at 12:51 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
>> There's another issue which is that of the $Id$ and similar tags.  We
>> have to decide what we want to do with them.  If we're not going to have
>> them in the Git repository, then they are only causing trouble right now
>> and it would be better to get rid of them completely for the conversion,
>> to avoid the noise that they will invariably cause.
>
> What was in the back of my mind was that we'd go around and mass-remove
> $PostgreSQL$ (and any other lurking tags), but only from HEAD and only
> after the repo conversion.  Although just before it would be okay too.
> The stickier part of this is what to do about back branches;
> particularly whether we are okay with checked-out versions of past
> releases not matching the actual shipped tarballs on this point.

Mass-deleting these tags from HEAD and the current head of each
back-branch seems like a good place to start.

...Robert

* Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> Right. Shall we try to spec out exactly what our conversion
> requirements are? Here's a shot:
[...]
> Comments? Other considerations?

Certainly sounds reasonable to me. I'd be really suprised if that's
really all that hard to accomplish. I'd be happy to help with some
testing too if we feel that the current git repo is in reasonable shape
to do that testing against (or someone has another).

+1

Thanks,

Stephen

Tom Lane escribió:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > There's another issue which is that of the $Id$ and similar tags. We
> > have to decide what we want to do with them. If we're not going to have
> > them in the Git repository, then they are only causing trouble right now
> > and it would be better to get rid of them completely for the conversion,
> > to avoid the noise that they will invariably cause.
>
> What was in the back of my mind was that we'd go around and mass-remove
> $PostgreSQL$ (and any other lurking tags), but only from HEAD and only
> after the repo conversion. Although just before it would be okay too.
> The stickier part of this is what to do about back branches;
> particularly whether we are okay with checked-out versions of past
> releases not matching the actual shipped tarballs on this point.

You mean we would remove them from CVS? I don't think that's
necessarily a good idea; it'd be massive changes for no good reason. My
idea was to remove them from the repository that would be used for the
conversion (I think that means editing the ,v files), and not put that
change back to the "real" CVS repo. Then the conversion to Git gets a
lot simpler; and the checking of this modified repo against copies
checked out from Git would be simpler.

Since this change is supposed to be scriptable, the script should be
available so potential testers of the conversion can get a converted
repository too. (Or maybe we should just provide access to the modified
copy of the repo).

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Tom Lane escribi:
>> What was in the back of my mind was that we'd go around and mass-remove
>> $PostgreSQL$ (and any other lurking tags), but only from HEAD and only
>> after the repo conversion. Although just before it would be okay too.

> You mean we would remove them from CVS? I don't think that's
> necessarily a good idea; it'd be massive changes for no good reason.

Uh, how is it different from any other mass edit, such as our annual
copyright-year updates, or pgindent runs?

> My idea was to remove them from the repository that would be used for the
> conversion (I think that means editing the ,v files),

Ick ... I'm willing to tolerate a few small manual ,v edits if we have
to do it to make tags consistent or something like that. I don't think
we should be doing massive edits of that kind.

But anyway, that's not the interesting point. The interesting point is
what about the historical aspect of it, not whether we want to dispense
with the tags going forward. Should our repo conversion try to
represent the historical states of the files including the tag strings?

regards, tom lane

On Thu, 28 May 2009, Tom Lane wrote:

> Each released minor version tag must check out the same as from CVS, at
> least back to some specified point (perhaps 7.4.0). I'd really prefer
> to insist on that all the way back.

We'd all like to hope that conversion process that works for everything
back to 7.4.0 would would also give useful results for all the old ones,
too. And it's worth testing as far back as possible. I think it's just
unrealistic to set the bar too high in the off chance that one of these
old releases has something that's harder to fix than producing that
version is worth. That might be the case for some of the 7.1 stuff
mentioned upthread for example. If there are only a few stragglers that
won't play nice, it might be easier to just publish a "git errata" list of
those releases and move on.

In related news, I wanted to make it a bit easier to track followup on the
whole "Action Item" list from the meeting. I converted those to the
standard format we were already using on the ToDo list, which provides a
way to check off items that are done. It may be worth breaking those out
from the rest of the minutes, so that it's easier to extend them with
things like these fleshed out git requirements. Example:
http://wiki.postgresql.org/wiki/PgCon_2009_Developer_Meeting#Source_Code_Management

Thoughts?

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

* Aidan Van Dyk <aidan(at)highrise(dot)ca> [090527 17:22]:

> And actually looking at the history of the gpo repo, the branches are all
> messed up with "merges" and stuff that I'm not sure where they are coming
> from... 8.2, 8.3, and master(HEAD) are all the same as my gpo repo, but the
> back branchs are very bad...

Ok, so seeing the interest in having a "good conversion", I took a stab at
parsecvs this afternoon, probably what I consider the leading "static"
conversion tool. I put the following patch into it to teach it about
$PostgreSQL$ and make the date formats match what my cvs export had:
aidan(at)db1-dapper:~/test/parsecvs$ git diff
diff --git a/rcs2git.c b/rcs2git.c
index c13c1f4..de6841d 100644
--- a/rcs2git.c
+++ b/rcs2git.c
@@ -52,7 +52,7 @@ struct diffcmd {
const int initial_out_buffer_size = 1024;
char const ciklog[] = "checked in with -k by ";

-#define KEYLENGTH 8 /* max length of any of the above keywords */
+#define KEYLENGTH 10 /* max length of any of the above keywords */
#define KDELIM '$' /* keyword delimiter */
#define VDELIM ':' /* separates keywords from values */
#define SDELIM '@' /* string delimiter */
@@ -61,10 +61,10 @@ char const ciklog[] = "checked in with -k by ";

char const *const Keyword[] = {
0, "Author", "Date", "Header", "Id", "Locker", "Log",
- "Name", "RCSfile", "Revision", "Source", "State"
+ "Name", "RCSfile", "Revision", "Source", "State", "PostgreSQL"
};
enum markers { Nomatch, Author, Date, Header, Id, Locker, Log,
- Name, RCSfile, Revision, Source, State };
+ Name, RCSfile, Revision, Source, State, PostgreSQL };
enum stringwork {ENTER, EDIT};

enum expand_mode {EXPANDKKV, EXPANDKKVL, EXPANDKK, EXPANDKV, EXPANDKO, EXPANDKB};
@@ -492,7 +492,7 @@ static void keyreplace(enum markers marker)
char const *sp = Keyword[(int)marker];

strftime(date_string, 25,
- "%Y/%m/%d %H:%M:%S", localtime(&Gversion->date));
+ "%Y-%m-%d %H:%M:%S", localtime(&Gversion->date));

if (exp != EXPANDKV)
out_printf("%c%s", KDELIM, sp);

It takes about 10 minutes to run my old xeon.

And a comparison between it's conversions and my cvs checkouts:
pg-parsecvs-MANUAL_DIST.diff 0 files changed
pg-parsecvs-REL2_0B.diff 0 files changed
pg-parsecvs-REL6_4.diff 0 files changed
pg-parsecvs-REL6_5_PATCHES.diff 0 files changed
pg-parsecvs-REL7_0_PATCHES.diff 0 files changed
pg-parsecvs-REL7_1_STABLE.diff 0 files changed
pg-parsecvs-REL7_2_STABLE.diff 0 files changed
pg-parsecvs-REL7_3_STABLE.diff 0 files changed
pg-parsecvs-REL7_4_STABLE.diff 0 files changed
pg-parsecvs-REL8_0_0.diff 4 files changed, 5053 insertions(+), 35326 deletions(-)
pg-parsecvs-REL8_0_STABLE.diff 0 files changed
pg-parsecvs-REL8_1_STABLE.diff 0 files changed
pg-parsecvs-REL8_2_STABLE.diff 1 file changed, 1 insertion(+), 1 deletion(-)
pg-parsecvs-REL8_3_STABLE.diff 1 file changed, 1 insertion(+), 1 deletion(-)
pg-parsecvs-Release_1_0_3.diff 0 files changed
pg-parsecvs-WIN32_DEV.diff 0 files changed
pg-parsecvs-ecpg_big_bison.diff 0 files changed
pg-parsecvs-master.diff 1 file changed, 1 insertion(+), 1 deletion(-)

Much better!!!

The REL8_0_0 branch seem funny yet:
src/backend/po/ru.po | 8416 ++++++++++------
src/backend/parser/gram.c |12088 ------------------------
src/interfaces/ecpg/preproc/pgc.c | 2887 -----
src/interfaces/ecpg/preproc/preproc.c |16988 ----------------------------------
4 files changed, 5053 insertions(+), 35326 deletions(-)

3 files are in the REL8_0_0 conversion but not on the cvs branch anymore (but
looking at the CVS ,v files, I'm partial to thinking there was probably some
CVS file hackery around those versions), and it's missing an update to ru.po.

The other difference is the same line for REL8_2_STABLE/REL8_3_STABLE/master:
aidan(at)db1-dapper:~/test/pg$ cat /tmp/pg-parsecvs-master.diff
diff --git a/src/tools/backend/index.html b/src/tools/backend/index.html
index ec2dcb8..846f002 100644
--- a/src/tools/backend/index.html
+++ b/src/tools/backend/index.html
@@ -1,4 +1,4 @@
-<!-- $PostgreSQL: pgsql/src/tools/backend/index.html,v 1.35 2006/03/11 04:38:41 momjian Exp $ -->
+<!-- $PostgreSQL$ -->
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">

Inspecting $CVSROOT/pgsql/src/tools/backend/index.html,v shows it's actually
got a strange $PostgreSQL$ tag:

The tag came into existence here:
1.35
log
@Add CVS tag lines to files that were lacking them.
@
text
@d1 1
a1 1
<!-- $PostgreSQL: pgsql/src/backend/utils/misc/guc.c,v 1.314 2006/03/07 02:54:23 momjian Exp $ -->
Note the bogus file name and version.

And then it was updated once since then:
1.36
log
@Improve backend flowchart to show more detail.
@
text
@<!-- $PostgreSQL: pgsql/src/tools/backend/index.html,v 1.35 2006/03/11 04:38:41 momjian Exp $ -->

But, you have all your branches and tags:
REL2_0B 01038eb Here's a little patch to keep the compiler quiet when compiling PostgreSQL V6.0 on the SPARC Solaris2 platform.
REL6_4 218f738 Retrofit hashtable and shared-mem-size-estimation bug fixes into REL6_4.
REL6_5_PATCHES 07af5e4 Back-patch critical fixes for NUMERIC values in plpgsql functions.
REL7_0_PATCHES ddb6d9f Back-patch password leak fix for Vaschenko.
REL7_1_STABLE c1e5c6e Remove stray semicolons in old ecpg preproc grammar ... modern bison versions won't compile it at all with those there. Probably of only academic interest now, but ...
REL7_2_STABLE 695a260 Remove registration message in all the supported back branches; we had decided to drop it for 7.4, and no one misses it.
REL7_3_STABLE 9ff59c7 Stamp release 7.3.21.
REL7_4_STABLE 4257497 Split the release notes into a separate file for each (active) major branch, as per my recent proposal. release.sgml itself is now just a stub that should change rarely; ideally, only once per major release to add a new include line. Most editing work will occur in the release-N.N.sgml files. To update a back branch for a minor release, just copy the appropriate release-N.N.sgml file(s) into the back branch.
REL8_0_0 1df3a89 its that time ... tag it for release
REL8_0_STABLE a52c648 Split the release notes into a separate file for each (active) major branch, as per my recent proposal. release.sgml itself is now just a stub that should change rarely; ideally, only once per major release to add a new include line. Most editing work will occur in the release-N.N.sgml files. To update a back branch for a minor release, just copy the appropriate release-N.N.sgml file(s) into the back branch.
REL8_1_STABLE 04b339b Update relpages and reltuples estimates in stand-alone ANALYZE, even if there's no analyzable attributes or indexes. We also used to report 0 live and dead tuples for such tables, which messed with autovacuum threshold calculations.
REL8_2_STABLE e2fee50 Update relpages and reltuples estimates in stand-alone ANALYZE, even if there's no analyzable attributes or indexes. We also used to report 0 live and dead tuples for such tables, which messed with autovacuum threshold calculations.
REL8_3_STABLE 18b7ff5 Fix LIKE's special-case code for % followed by _. I'm not entirely sure that this case is worth a special code path, but a special code path that gets the boundary condition wrong is definitely no good. Per bug #4821 from Andrew Gierth.
Release_1_0_3 21f8ea0 A small patch from Andrew for the linux port in v1.09
WIN32_DEV 0f1714d Change Win32 rename/unlink timeout to 3 seconds.
ecpg_big_bison 3cb2aa3 Synced yet again. Deactivated backend prepare/execute/deallocate for the time being.
master fd02d25 Fix compiler warnings on Sun Studio of the sort
master-UNNAMED-BRANCH 4cc264d Make the world at least somewhat safe for zero-column tables, and remove the special case in ALTER DROP COLUMN to prohibit dropping a table's last column.

tags/MANUAL_1_0 0f52f25 Import of PostgreSQL User Manual
tags/PG95-1_01 4d809f0 Postgres95 1.01 Distribution - Virgin Sources
tags/REL2_0 401afd3 Remove include of libpq-fe.h. This file has nothing to do with libpq.
tags/REL6_5 744f5e3 Update TODO list.
tags/REL7_0 ab6f4fd Change HISTORY to show outer joins in 7.1 or 7.2.
tags/REL7_1 4aa3f7b Remove as-of from HISTORY file.
tags/REL7_1_BETA 63fbd7a Fix bogus makefiles ... these didn't build on platforms that are sticky about being given accurate references to referenced libraries ...
tags/REL7_1_BETA2 a233e6c tag configure as beta2 ..
tags/REL7_1_BETA3 9ef1bcb jump version to beta3 ... beta2 was created and pulled due to a couple of large-ish bugs that Tom and Vadim were able to fix, but to avoid any confusion, beta2 was removed ... and for tag'ng purposes, beta3 is being created ...
tags/REL7_1_2 7b7fbe9 Correct recently-broken avg(interval) definition. We can't force an initdb to fix this in 7.1 installations, but it seems better to be shipping a correct entry than a wrong one.

tags/REL7_2_BETA1 74644d4 Code cleanup.
tags/REL7_2_BETA2 f0d9d23 Update for latest version of horology test.
tags/REL7_2_BETA3 c122aae Some minor tweaks of REINDEX processing: grab exclusive lock a little earlier, make error checks more uniform.
tags/REL7_2_BETA4 374d3a2 tag it as b4, with all the changes that have gone on ...
tags/REL7_2_BETA5 ebb23f8 tag as beta 5 for *hopefully* a very very short beta cycle on this one?
tags/REL7_2_RC1 30f1836 okay, sorry for delay all ... here is the tag for RC1 ...
tags/REL7_2_RC2 7291790 let's roll up rc2 ..
tags/REL7_2 b8e3d8f Stamp configure/configure.in for 7.2, already did register.txt and bug.template.
tags/REL7_2_3 8ddb964 Brand 7.2.3.
tags/REL7_2_4 bfb3ddc Brand 7.2.4.
tags/REL7_2_5 473091d Update 7.2 regression tests to match what you get when using a modern version of Bison.
tags/REL7_2_6 26c477e Stamp release 7.2.6.
tags/REL7_2_7 cdb721b Recommend security(at)postgresql(dot)org as the contact point for security-related bugs.
tags/REL7_2_8 21f3e3a Update release notes for upcoming re-releases.

tags/REL7_3_2 652e3c8 Add mention of CURRENT_SCHEMA for object creation.
tags/REL7_3_4 9a46450 Fix timestamp_date for HAVE_INT64_TIMESTAMP case.
tags/REL7_3_5 ebeb0c0 Brand 7.3.5.
tags/REL7_3_6 718dda1 Brand 7.3.6.
tags/REL7_3_7 e73b50d Wups, seem to have used an ungood version of lynx to generate this.
tags/REL7_3_8 2291a3f Stamp release 7.3.8.
tags/REL7_3_9 cdb721b Recommend security(at)postgresql(dot)org as the contact point for security-related bugs.
tags/REL7_3_10 21f3e3a Update release notes for upcoming re-releases.
tags/REL7_3_11 10c4262 Stamp release 7.3.11.
tags/REL7_3_12 d65fbbd Stamp 7.3.12.
tags/REL7_3_13 92ee1dc Release-note updates and copy editing.
tags/REL7_3_14 bbfa98c Stamp 7.3.14.
tags/REL7_3_15 b954d6d Stamp release 7.3.15.
tags/REL7_3_16 7ec7706 Stamp 7.3.16.
tags/REL7_3_17 ee65483 Fix markup because older releases couldn't like to refernce pages.
tags/REL7_3_18 2edf36f Stamp release 7.3.18.
tags/REL7_3_19 552a435 Update configure.in for release
tags/REL7_3_20 f3ab989 Update release notes for last-minute fix.
tags/REL7_3_21 9ff59c7 Stamp release 7.3.21.

tags/REL7_4_BETA1 9060870 can't mix and match .gz and .bz2 in here ... won't build
tags/REL7_4_BETA2 f2e55e5 update to beta2
tags/REL7_4_BETA3 c2481d7 tag her for beta3, as announced on Friday ...
tags/REL7_4_BETA4 1df1740 brand her beta4
tags/REL7_4_BETA5 6803ce7 up configure to beta5
tags/REL7_4_RC1 5715d2b tag it Release Candidate 1, as previously discussed
tags/REL7_4_RC2 441a921 autoconf
tags/REL7_4 b1f8cc4 k, tag the release
tags/REL7_4_1 9d947b0 Update HISTORY for 7.4.1 release.
tags/REL7_4_2 3462f95 Some editorial work on 7.4.2 release notes.
tags/REL7_4_3 fc86d4b Remove README.CVS when making a distribution.
tags/REL7_4_4 a9771b2 Stamp 7.4.4.
tags/REL7_4_5 6ff4540 Brand 7.4.5 ... now that was our shortest-lived release ever ...
tags/REL7_4_6 0833822 Stamp release 7.4.6.
tags/REL7_4_7 cdb721b Recommend security(at)postgresql(dot)org as the contact point for security-related bugs.
tags/REL7_4_8 21f3e3a Update release notes for upcoming re-releases.
tags/REL7_4_9 959dacc COPY's test for read-only transaction was backward; it prohibited COPY TO where it should prohibit COPY FROM. Found by Alon Goldshuv.
tags/REL7_4_10 b250647 Translation updates
tags/REL7_4_11 92ee1dc Release-note updates and copy editing.
tags/REL7_4_12 7fb4b1d Stamp 7.4.12.
tags/REL7_4_13 d6d136f Stamp release 7.4.13.
tags/REL7_4_14 fedacf2 Stamp 7.4.14.
tags/REL7_4_15 dd03a59 commit before tag ...
tags/REL7_4_16 c06c90d Stamp release 7.4.16.
tags/REL7_4_17 61fc168 Update configure in for new release
tags/REL7_4_18 f3ab989 Update release notes for last-minute fix.
tags/REL7_4_19 01d0a31 Stamp release 7.4.19.
tags/REL7_4_20 f17483e Remove link that pre-8.2 doc tools don't support.
tags/REL7_4_21 7814f48 tag 7.4.21
tags/REL7_4_22 8bde44d tag for 7.4.22
tags/REL7_4_23 0b96d5e tag 7.4.23
tags/REL7_4_24 204cde4 tag 7.4.24
tags/REL7_4_25 3a5e48a tag 7.4.25

tags/REL8_0_0BETA1 6eb9cb3 Fix Win32 pg_dumpall check.
tags/REL8_0_0BETA2 9b01845 tag configure beta2
tags/REL8_0_0BETA3 3be70cd update for beta3, and update Copyright date to 2004
tags/REL8_0_0BETA4 4e175ff make sure we tag configure.in as beta4 as well ...
tags/REL8_0_0BETA5 2cdd35b update us to beta5
tags/REL8_0_0RC1 da9f52d tag configure for rc1 ..
tags/REL8_0_0RC2 19e8edf tag files for rc2
tags/REL8_0_0RC3 be040da forgot to autoconf after tag'ng configure.in with rc3
tags/REL8_0_0RC4 7da46c4 upgrade tags to rc4
tags/REL8_0_0RC5 81f6bcb up release to rc5
tags/REL8_0_1 cdb721b Recommend security(at)postgresql(dot)org as the contact point for security-related bugs.
tags/REL8_0_2 61f3737 Stamp 8.0.2.
tags/REL8_0_3 04b942e Rename encryption section.
tags/REL8_0_4 959dacc COPY's test for read-only transaction was backward; it prohibited COPY TO where it should prohibit COPY FROM. Found by Alon Goldshuv.
tags/REL8_0_5 b250647 Translation updates
tags/REL8_0_6 92ee1dc Release-note updates and copy editing.
tags/REL8_0_7 65edf6e Stamp 8.0.7.
tags/REL8_0_8 991d699 Stamp release 8.0.8.
tags/REL8_0_9 0d007fb Stamp 8.0.9.
tags/REL8_0_10 f9dba8f tag it
tags/REL8_0_11 12f2780 Stamp release 8.0.11.
tags/REL8_0_12 9988796 Stamp releases notes for 8.2.3, 8.1.8, 8.0.12.
tags/REL8_0_13 5032c16 Update configure for release
tags/REL8_0_14 f3ab989 Update release notes for last-minute fix.
tags/REL8_0_15 89453bc Stamp release 8.0.15.
tags/REL8_0_16 f17483e Remove link that pre-8.2 doc tools don't support.
tags/REL8_0_17 ba9d7d7 tag 8.0.17
tags/REL8_0_18 8611315 tag for 8.0.18
tags/REL8_0_19 e077640 tag for 8.0.19
tags/REL8_0_20 9f80110 commit first then tag 8.0.20
tags/REL8_0_21 13651cb tag 8.0.21

tags/REL8_1_0BETA1 1762d42 fix up a few references to 8.1devel -> 8.1beta1
tags/REL8_1_0BETA2 6c005da tag it all beta2 ...
tags/REL8_1_0BETA3 972df1d must commit *after* autoconf, not before
tags/REL8_1_0BETA4 7b100fc update configure and bugtemplate for beta 4 ...
tags/REL8_1_0RC1 8dd55b3 tag it for rc1
tags/REL8_1_0 cdecffb Tag everything for 8.1.0 ... Finally, a relesae on scheduale!!
tags/REL8_1_1 5f6994f Remove incorrect increment of lineno, per David Fetter. Sync HEAD and 8.1 branches of pgbench.
tags/REL8_1_2 92ee1dc Release-note updates and copy editing.
tags/REL8_1_3 da0a737 Stamp 8.1.3.
tags/REL8_1_4 d17b9c1 Stamp release 8.1.4.
tags/REL8_1_5 21eeb6a Stamp 8.1.5.
tags/REL8_1_6 5558874 Links to GUC variables from HISTORY don't work in back branches...
tags/REL8_1_7 0349a82 Stamp release 8.1.7.
tags/REL8_1_8 9988796 Stamp releases notes for 8.2.3, 8.1.8, 8.0.12.
tags/REL8_1_9 2f934c8 Update configure.in for release
tags/REL8_1_10 f3ab989 Update release notes for last-minute fix.
tags/REL8_1_11 9ae878a Stamp release 8.1.11.
tags/REL8_1_12 f17483e Remove link that pre-8.2 doc tools don't support.
tags/REL8_1_13 b2b4697 tag 8.1.13
tags/REL8_1_14 7cb6c06 tag for 8.1.14
tags/REL8_1_15 1123c54 tag 8.1.15
tags/REL8_1_16 d59f5d8 tagging 8.1.16
tags/REL8_1_17 9ad74fc tag 8.1.17

tags/REL8_2_BETA1 d1511a2 Tag us Beta1
tags/REL8_2_BETA2 2630594 Stamp 8.2beta2.
tags/REL8_2_BETA3 f7ea773 Tag as Beta3 ... two outstanding *known* bugs before RC1 ...
tags/REL8_2_RC1 33061a5 update for rc1
tags/REL8_2_0 7d3db29 v8.2.0 is now released ...
tags/REL8_2_1 b878fc4 tag configure
tags/REL8_2_2 538ed7c Stamp release 8.2.2.
tags/REL8_2_3 9988796 Stamp releases notes for 8.2.3, 8.1.8, 8.0.12.
tags/REL8_2_4 96acc85 Fix markup.
tags/REL8_2_5 f3ab989 Update release notes for last-minute fix.
tags/REL8_2_6 019e5ea Stamp release 8.2.6.
tags/REL8_2_7 08a055a Translation updates
tags/REL8_2_8 34b0b9f tag 8.2.8
tags/REL8_2_9 b117db0 tag 8.2.9
tags/REL8_2_10 25afb91 tag for 8.2.10
tags/REL8_2_11 aedcad2 tag 8.2.11
tags/REL8_2_12 37a9110 tag 8.2.12
tags/REL8_2_13 9f8fa50 tag 8.2.13
tags/REL8_3_BETA1 0d74383 tag it 8.3beta1 ... the beta cycle begins
tags/REL8_3_BETA2 28343c6 Stamp 8.3beta2.
tags/REL8_3_BETA3 0b5494d Fix markup that doesn't work in HISTORY generation.
tags/REL8_3_BETA4 60becb8 Stamp 8.3beta4.
tags/REL8_3_RC1 b28079b Stamp release 8.3RC1.
tags/REL8_3_RC2 17a82cf must commit after autoconf ... and yes, I used the right autoconf
tags/REL8_3_0 0126093 configure tag'd 8.3.0 and built witih autoconf 2.59
tags/REL8_3_1 7e8f8b2 Fix inappropriately-timed memory context switch in autovacuum_do_vac_analyze. This accidentally failed to failbefore 8.3, because the context we were switching back to was long-lived anyway; but it sure looks risky as can be now. Well spotted by Pavan Deolasee.
tags/REL8_3_2 2343441 tag for 8.3.2
tags/REL8_3_3 84d7f08 tag 8.3.3
tags/REL8_3_4 08f7e02 tag for 8.3.4
tags/REL8_3_5 aed4eac commit for 8.3.5
tags/REL8_3_6 8e9babe Fix plpgsql to not treat INSERT INTO as an INTO-variables clause anywhere in the string, not just at the start. Per bug #4629 from Martin Blazek.
tags/REL8_3_7 98b774c tag 8.3.7

tags/REL8_4_BETA1 6c313e8 commit and tag beta1
tags/REL8_4_BETA2 6e61436 commit for BETA2

tags/Release-1-6-0 7e5f5ea creation for postgresql-6.1
tags/Release_1_0_2 f720353 Okay...*last* commit, now to create a release...
tags/Release_2_0 c081f4a | |Here is a fix for the psql alignment problem. It turns out that libpq |was trying to determine if the column contained only numeric values so |it could right justify it. The 'e' values were taked as exponient |values and all columns were considerednumeric. | |The patch excludes 'e' and 'E' as being valid first-column numeric |values. |
tags/Release_2_0_0 b185bb0 changed missed err() change to err_out()
tags/SUPPORT 2ffc02d Support Docs & Contrib
tags/creation 7e5f5ea creation for postgresql-6.1
tags/release-6-3 5b4a2e7 One last change to configure for 'non-gcc' compiler

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Hi,

On 05/28/2009 06:19 PM, Tom Lane wrote:
> I am hoping that git's cvs server emulation is complete enough that you
> can commit through it --- anybody know? But that will be just a
> stopgap.
Comitting is no problem - you can't tag, branch or merge through it
though (Not really surprisingly I think).

> BTW, can anyone comment on whether and how we can maintain the current
> split between master repository (that's not even accessible to
> non-committers) and a public mirror? If only from a standpoint of
> security paranoia, I'd rather like to preserve that split, but I don't
> know how well git will play with it.
Absolutely not a problem (Doing such things is one of the strengths of git).

Andres

* Andres Freund <andres(at)anarazel(dot)de> [090528 16:07]:

>> BTW, can anyone comment on whether and how we can maintain the current
>> split between master repository (that's not even accessible to
>> non-committers) and a public mirror? If only from a standpoint of
>> security paranoia, I'd rather like to preserve that split, but I don't
>> know how well git will play with it.

> Absolutely not a problem (Doing such things is one of the strengths of git).

In fact, that's generally the standing procedure for git... The
"master" is "pushed" to by a select few, controlled by SSH key access to
accounts with write access to the files/directories of the git repo
(much like I'm assuming the current CVS master is). And that's pushed
out to any number of other places, either from cron, or post-receive
hooks.

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

* Aidan Van Dyk <aidan(at)highrise(dot)ca> [090528 15:56]:

> Ok, so seeing the interest in having a "good conversion", I took a stab at
> parsecvs this afternoon, probably what I consider the leading "static"
> conversion tool.

> It takes about 10 minutes to run my old xeon.
>
> And a comparison between it's conversions and my cvs checkouts:
> pg-parsecvs-MANUAL_DIST.diff 0 files changed
> pg-parsecvs-REL2_0B.diff 0 files changed
> pg-parsecvs-REL6_4.diff 0 files changed
> pg-parsecvs-REL6_5_PATCHES.diff 0 files changed
> pg-parsecvs-REL7_0_PATCHES.diff 0 files changed
> pg-parsecvs-REL7_1_STABLE.diff 0 files changed
> pg-parsecvs-REL7_2_STABLE.diff 0 files changed
> pg-parsecvs-REL7_3_STABLE.diff 0 files changed
> pg-parsecvs-REL7_4_STABLE.diff 0 files changed
> pg-parsecvs-REL8_0_0.diff 4 files changed, 5053 insertions(+), 35326 deletions(-)
> pg-parsecvs-REL8_0_STABLE.diff 0 files changed
> pg-parsecvs-REL8_1_STABLE.diff 0 files changed
> pg-parsecvs-REL8_2_STABLE.diff 1 file changed, 1 insertion(+), 1 deletion(-)
> pg-parsecvs-REL8_3_STABLE.diff 1 file changed, 1 insertion(+), 1 deletion(-)
> pg-parsecvs-Release_1_0_3.diff 0 files changed
> pg-parsecvs-WIN32_DEV.diff 0 files changed
> pg-parsecvs-ecpg_big_bison.diff 0 files changed
> pg-parsecvs-master.diff 1 file changed, 1 insertion(+), 1 deletion(-)
>
> Much better!!!

And for those interested in looking at that repo:
git clone git://code.highrise.ca/~mountie/pg-static.git

It won't be around forever, and it's *not* incremental.

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

On Thu, May 28, 2009 at 3:56 PM, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> Ok, so seeing the interest in having a "good conversion", I took a stab at

Awesome!

> Much better!!!
>
> The REL8_0_0 branch seem funny yet:
>         src/backend/po/ru.po                  | 8416 ++++++++++------
>         src/backend/parser/gram.c             |12088 ------------------------
>         src/interfaces/ecpg/preproc/pgc.c     | 2887 -----
>         src/interfaces/ecpg/preproc/preproc.c |16988 ----------------------------------
>         4 files changed, 5053 insertions(+), 35326 deletions(-)
>
> 3 files are in the REL8_0_0 conversion but not on the cvs branch anymore (but
> looking at the CVS ,v files, I'm partial to thinking there was probably some
> CVS file hackery around those versions), and it's missing an update to ru.po.

So we should probably look into fixing this...

> The other difference is the same line for REL8_2_STABLE/REL8_3_STABLE/master:
>        aidan(at)db1-dapper:~/test/pg$ cat /tmp/pg-parsecvs-master.diff
>        diff --git a/src/tools/backend/index.html b/src/tools/backend/index.html
>        index ec2dcb8..846f002 100644
>        --- a/src/tools/backend/index.html
>        +++ b/src/tools/backend/index.html
>        @@ -1,4 +1,4 @@
>        -<!-- $PostgreSQL: pgsql/src/tools/backend/index.html,v 1.35 2006/03/11 04:38:41 momjian Exp $ -->
>        +<!-- $PostgreSQL$ -->
>         <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
>             "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
>         <html xmlns="http://www.w3.org/1999/xhtml">
>
> Inspecting $CVSROOT/pgsql/src/tools/backend/index.html,v shows it's actually
> got a strange $PostgreSQL$ tag:
>
> The tag came into existence here:
>        1.35
>        log
>        @Add CVS tag lines to files that were lacking them.
>        @
>        text
>        @d1 1
>        a1 1
>        <!-- $PostgreSQL: pgsql/src/backend/utils/misc/guc.c,v 1.314 2006/03/07 02:54:23 momjian Exp $ -->
> Note the bogus file name and version.
>
> And then it was updated once since then:
>        1.36
>        log
>        @Improve backend flowchart to show more detail.
>        @
>        text
>        @<!-- $PostgreSQL: pgsql/src/tools/backend/index.html,v 1.35 2006/03/11 04:38:41 momjian Exp $ -->

...and this.

...Robert

Hi,

Quoting "Robert Haas" <robertmhaas(at)gmail(dot)com>:
> That might work, but then we better be pretty darn confident that that
> "fresh conversion" is actually correct. I'd rather have them going
> side-by-side so that we can verify everything before shutting the old
> system off.

I agree, as long as you take non-incremental converters into account
as well. Otherwise, we'd mostly test functionality we don't need later
on (incremental updates).

>> BTW, can anyone comment on whether and how we can maintain the current
>> split between master repository (that's not even accessible to
>> non-committers) and a public mirror?  If only from a standpoint of
>> security paranoia, I'd rather like to preserve that split, but I don't
>> know how well git will play with it.
>
> You can set up one repository to mirror another.

Yes, that's the point of a distributed VCS. The good thing about it is
that everybody is free to work (including committing) *on his own
copy* of the branch and then provide a patch (or patches) for
committers (or gain commit rights and upload his work later on). That
fits pretty well with the Postgres development process, AFAICT.

Regards

Markus Wanner

Hi,

Quoting "Robert Haas" <robertmhaas(at)gmail(dot)com>:
> That's not the best news I've had today...

Sorry :-(

> To me they sound complex and inconvenient. I guess I'm kind of
> mystified by why we can't make this work reliably. Other than the
> "broken tags" issue we've discussed, it seems like the only real issue
> should be how to group changes to different files into a single
> commit. Once you do that, you should be able to construct a
> well-defined, total function f : <cvs-file, cvs-revision> -> <git
> commit> which is surjective on the space of git commits. In fact it
> might be a good idea to explicitly construct this mapping and drop it
> into a database table somewhere so that people can sanity check it as
> much as they wish. Why is this harder than I think it is?

Well, as CVS doesn't guarantee any consistency between files, you end
up with silly situations more often than you think. One of the
simplest possible example is something like:

commit 1: fileA @ 1.1, fileB @ 1.2
commit 2: fileA @ 1.2, fileB @ 1.1

Seen from fileA, it's obvious that commit 1 (@1.1) comes before commit
2 (@1.2), but seen from fileB it's the exact opposite. The most
promising approach to solve these problems seems to be based on Graph
Theory, where you work with a graph of dependencies from fileA @ 1.1
to fileA @ 1.2.

To resolve the above situation, you'd have "split" a blob of
single-file commits into two end-result commits (for monotone / git).
In the above example, you'd have two options to resolve the conflict:

commit 1a: fileA @ 1.1
commit 2: fileA @ 1.2, fileB @ 1.1
commit 1b: fileA @ 1.2

Or:

commit 2a: fileB @ 1.1
commit 1: fileA @ 1.1, fileB @ 1.2
commit 2b: fileB @ 1.2

(Note that often enough, these have actually been separate commits in
CVS as well, there's just no way to represent that. And no, timestamps
are simply not reliable enough).

Now add tags, branches and cyclic dependencies involving many files
and many 100 commits to the example above and you start to get an idea
of the complexity of the problem in general.

See my description and diagrams of the steps used for cvs_import in
monotone at [1] or follow descriptions of how cvs2svn works internally.

A few numbers about a conversion I'm trying for testing my algorithm
and heuristics. It's converting a pretty recent snapshot of the
Postgres repository:

* running at 100% CPU time since: April, 17
* Total number of files involved: 6'847
* total number of blobs (before splitting): 28'010
* blobs split due to cyclic dependencies: 12'801

Admittedly, my algorithm isn't optimized at all. However, I'm focusing
on good results rather than speed of conversion.

Also note, that monotone uses SQLite, so it actually stores the
results of this conversion in an SQL database, as you proposed.
Recently, a git_export command has been added, so that's definitely
worth a try for converting CVS to git. However, I fear cvs2git is more
mature.

Regards

Markus Wanner

[1]: a description of the various steps in conversion from CVS to monotone:
http://www.monotone.ca/wiki/CvsImport/

On Thursday 28 May 2009 20:03:38 Stephen Frost wrote:
> * Tom Lane (tgl(at)sss(dot)pgh(dot)pa(dot)us) wrote:
> > Right. Shall we try to spec out exactly what our conversion
> > requirements are? Here's a shot:
>
> [...]
>
> > Comments? Other considerations?
>
> Certainly sounds reasonable to me. I'd be really suprised if that's
> really all that hard to accomplish. I'd be happy to help with some
> testing too if we feel that the current git repo is in reasonable shape
> to do that testing against (or someone has another).

Sounds like writing a comprehensive test suite against Tom's spec would be the
first step. And then this test suite can be run against various conversion
tools and configurations thereof.

On Fri, May 29, 2009 at 2:41 AM, Markus Wanner <markus(at)bluegap(dot)ch> wrot> Hi,
> Quoting "Robert Haas" <robertmhaas(at)gmail(dot)com>:
>> Why is this harder than I think it is?
>
> One of the simplest possible example is something like:

Thanks for the explanation, I understand it better now. I'm still
dismayed, but at least I know why I'm dismayed.

...Robert

Hi,

Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
>> Ok, so seeing the interest in having a "good conversion", I took a stab at
>> parsecvs this afternoon, probably what I consider the leading "static"
>> conversion tool.

Here are some results from a conversion with cvs2git.

>> It takes about 10 minutes to run my old xeon.

The conversion with cvs2git certainly took a bit longer, however, I
don't think that matters at all. Everything below a day or two is good
enough, IMO. What counts is the result.

The first step is running cvs2git itself:

cvs2svn Statistics:
------------------
Total CVS Files: 6873
Total CVS Revisions: 140191
Total CVS Branches: 36057
Total CVS Tags: 457515
Total Unique Tags: 171
Total Unique Branches: 21
CVS Repos Size in KB: 377337
Total SVN Commits: 32889
First Revision Date: Tue Jul 9 08:21:07 1996
Last Revision Date: Thu May 28 22:02:10 2009

(number of files matches pretty well with my own algorithm, however,
total svn commits is a bit lower, compared to the ~ 40'000 blobs I got).

The output of cvs2git can then be imported with git fast-import:

git-fast-import statistics:
---------------------------------------------------------------------
Alloc'd objects: 350000
Total objects: 349405 ( 19563 duplicates )
blobs : 132672 ( 3255 duplicates 119032 deltas)
trees : 183967 ( 16308 duplicates 165582 deltas)
commits: 32766 ( 0 duplicates 0 deltas)
tags : 0 ( 0 duplicates 0 deltas)
Total branches: 194 ( 664 loads )
marks: 1073741824 ( 168693 unique )
atoms: 5280
Memory total: 16532 KiB
pools: 2860 KiB
objects: 13671 KiB
---------------------------------------------------------------------
pack_report: getpagesize() = 4096
pack_report: core.packedGitWindowSize = 1073741824
pack_report: core.packedGitLimit = 8589934592
pack_report: pack_used_ctr = 124414
pack_report: pack_mmap_calls = 3674
pack_report: pack_open_windows = 1 / 1
pack_report: pack_mapped = 199500913 / 199500913
---------------------------------------------------------------------

The resulting repository contains the following branches. The
unlabeled ones contain only 1-2 files and seem rather irrelevant. In a
next try, I'd disable their creation completely, just wanted to check.

REL2_0B
REL6_4
REL6_5_PATCHES
REL7_0_PATCHES
REL7_1_STABLE
REL7_2_STABLE
REL7_3_STABLE
REL7_4_STABLE
REL8_0_0
REL8_0_STABLE
REL8_1_STABLE
REL8_2_STABLE
REL8_3_STABLE
Release_1_0_3
WIN32_DEV
ecpg_big_bison
* master
unlabeled-1.44.2 -> from src/backend/commands/tablecmds.c
unlabeled-1.51.2 -> from src/test/regress/expected/alter_table.out
unlabeled-1.59.2 -> from src/backend/executor/execTuples.c
unlabeled-1.87.2 -> from src/backend/executor/nodeAgg.c
unlabeled-1.90.2 -> from src/backend/parser/parse_target.c and
src/backend/access/common/tupdesc.c

Comparison of the head of each branch between git and CVS (modulo CVS
keyword expansion, which I've filtered out):

ecpg_big_bison.diff: 0 files changed
master.diff: 0 files changed
REL2_0B.diff: 0 files changed
REL6_4.diff: 0 files changed
REL6_5_PATCHES.diff: 0 files changed
REL7_0_PATCHES.diff: 0 files changed
REL7_1_STABLE.diff: 0 files changed
REL7_2_STABLE.diff: 0 files changed
REL7_3_STABLE.diff: 0 files changed
REL7_4_STABLE.diff: 0 files changed
REL8_0_0.diff: 0 files changed
REL8_0_STABLE.diff: 0 files changed
REL8_1_STABLE.diff: 0 files changed
REL8_2_STABLE.diff: 0 files changed
REL8_3_STABLE.diff: 0 files changed
Release_1_0_3.diff: 0 files changed
WIN32_DEV.diff: 0 files changed

I plan to compare the tags as well and test what branch they are in,
but so far cvs2git seems to hold its promises. I'll report back again
within the next few days.

Regards

Markus Wanner

* Markus Wanner <markus(at)bluegap(dot)ch> [090529 11:06]:
> Hi,

> Comparison of the head of each branch between git and CVS (modulo CVS
> keyword expansion, which I've filtered out):

How did you filter it out, and without the filtering out, how does it
do?

> I plan to compare the tags as well and test what branch they are in, but
> so far cvs2git seems to hold its promises. I'll report back again within
> the next few days.

It definitely seems to have figured out the REL8_0_0 confusing that
tripped up parsecvs. If I'm stuck on another windows project some time
in the near future, I'll try and look into why parsecvs trips up on
those 3 files from REL8_0_0 branch ;-)

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Hi,

Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
> * Markus Wanner <markus(at)bluegap(dot)ch> [090529 11:06]:
>> Comparison of the head of each branch between git and CVS (modulo CVS
>> keyword expansion, which I've filtered out):
>
> How did you filter it out

With perl some regexes.

> and without the filtering out, how does it do?

Uh.. why is that of interest? With content hashing, these keywords do
more harm than good.

I'd have to check again, but there certainly are differences here and there.

Regards

Markus Wanner

* Markus Wanner <markus(at)bluegap(dot)ch> [090529 11:18]:
> Hi,
>
> Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
>> * Markus Wanner <markus(at)bluegap(dot)ch> [090529 11:06]:
>>> Comparison of the head of each branch between git and CVS (modulo CVS
>>> keyword expansion, which I've filtered out):
>>
>> How did you filter it out
>
> With perl some regexes.
>
>> and without the filtering out, how does it do?
>
> Uh.. why is that of interest? With content hashing, these keywords do
> more harm than good.

Yes, but the point is you want an exact replica of CVS right? You're
git repo should have $PostgreSQL$ and the cvs export/checkout (you do
use -kk right) should also have $PostgreSQL$.

The 3 parsecvs errors were that it *didn't* recognoze the strange
$PostgreSQL ... Exp $ expansion that cvs did.

But it's important, because on *some* files you *do* want expanded
"keywords" (like the $OpenBSD ... Exp $. One of the reasons pg CVS went
to the $PostgreSQL$ keyword (I'm guessing) was so they could explictly
de-couple them from other keywords that they didn't want munging on.

So, I wouldn't consider any conversion good unless it had all these:
parsecvs-master:contrib/pgcrypto/crypt-des.c: * $FreeBSD: src/secure/lib/libcrypt/crypt-des.c,v 1.12 1999/09/20 12:39:20 markm Exp $
parsecvs-master:contrib/pgcrypto/crypt-md5.c: * $FreeBSD: src/lib/libcrypt/crypt-md5.c,v 1.5 1999/12/17 20:21:45 peter Exp $
parsecvs-master:contrib/pgcrypto/md5.c:/* $KAME: md5.c,v 1.3 2000/02/22 14:01:17 itojun Exp $ */
parsecvs-master:contrib/pgcrypto/md5.h:/* $KAME: md5.h,v 1.3 2000/02/22 14:01:18 itojun Exp $ */
parsecvs-master:contrib/pgcrypto/rijndael.c:/* $OpenBSD: rijndael.c,v 1.6 2000/12/09 18:51:34 markus Exp $ */
parsecvs-master:contrib/pgcrypto/rijndael.h: * $OpenBSD: rijndael.h,v 1.3 2001/05/09 23:01:32 markus Exp $ */
parsecvs-master:contrib/pgcrypto/sha1.c:/* $KAME: sha1.c,v 1.3 2000/02/22 14:01:18 itojun Exp $ */
parsecvs-master:contrib/pgcrypto/sha1.h:/* $KAME: sha1.h,v 1.4 2000/02/22 14:01:18 itojun Exp $ */
parsecvs-master:contrib/pgcrypto/sha2.c:/* $OpenBSD: sha2.c,v 1.6 2004/05/03 02:57:36 millert Exp $ */
parsecvs-master:contrib/pgcrypto/sha2.h:/* $OpenBSD: sha2.h,v 1.2 2004/04/28 23:11:57 millert Exp $ */
parsecvs-master:src/backend/port/darwin/system.c: * $FreeBSD: src/lib/libc/stdlib/system.c,v 1.6 2000/03/16 02:14:41 jasone Exp $
parsecvs-master:src/port/crypt.c:/* $NetBSD: crypt.c,v 1.18 2001/03/01 14:37:35 wiz Exp $ */
parsecvs-master:src/port/crypt.c:__RCSID("$NetBSD: crypt.c,v 1.18 2001/03/01 14:37:35 wiz Exp $");
parsecvs-master:src/port/qsort.c:/* $NetBSD: qsort.c,v 1.13 2003/08/07 16:43:42 agc Exp $ */
parsecvs-master:src/port/qsort_arg.c:/* $NetBSD: qsort.c,v 1.13 2003/08/07 16:43:42 agc Exp $ */
parsecvs-master:src/port/strlcat.c: * $OpenBSD: strlcat.c,v 1.13 2005/08/08 08:05:37 espie Exp $ */
parsecvs-master:src/port/strlcpy.c:/* $OpenBSD: strlcpy.c,v 1.11 2006/05/05 15:27:38 millert Exp $ */

As well as stuff like:
parsecvs-master:src/backend/access/index/genam.c: * $PostgreSQL$
parsecvs-master:src/backend/access/index/indexam.c: * $PostgreSQL$
parsecvs-master:src/backend/access/nbtree/Makefile:# $PostgreSQL$
parsecvs-master:src/backend/access/nbtree/README:$PostgreSQL$
parsecvs-master:src/backend/access/nbtree/nbtcompare.c: * $PostgreSQL$
parsecvs-master:src/backend/access/nbtree/nbtinsert.c: * $PostgreSQL$
parsecvs-master:src/backend/access/nbtree/nbtpage.c: * $PostgreSQL$
parsecvs-master:src/backend/access/nbtree/nbtree.c: * $PostgreSQL$
parsecvs-master:src/backend/access/nbtree/nbtsearch.c: * $PostgreSQL$

Basically, identical what to a cvs export/checkout/update gives you with
a "-kk".

But I'm picky ;-)

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Aidan Van Dyk wrote:

> Yes, but the point is you want an exact replica of CVS right? You're
> git repo should have $PostgreSQL$ and the cvs export/checkout (you do
> use -kk right) should also have $PostgreSQL$.
>
> The 3 parsecvs errors were that it *didn't* recognoze the strange
> $PostgreSQL ... Exp $ expansion that cvs did.

Huh, no -- I agree that $OpenBSD$ etc should remain (we don't munge them
anyway), but $PostgreSQL$, $Id$, $Revision$ etc tags are best gone
because, as Markus says, their expansion interferes with content hashing.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Tom Lane escribió:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > Tom Lane escribi:
> >> What was in the back of my mind was that we'd go around and mass-remove
> >> $PostgreSQL$ (and any other lurking tags), but only from HEAD and only
> >> after the repo conversion. Although just before it would be okay too.
>
> > You mean we would remove them from CVS? I don't think that's
> > necessarily a good idea; it'd be massive changes for no good reason.
>
> Uh, how is it different from any other mass edit, such as our annual
> copyright-year updates, or pgindent runs?

Well, the other mass edits have a purpose. This one would be only to
help the migration.

> > My idea was to remove them from the repository that would be used for the
> > conversion (I think that means editing the ,v files),
>
> Ick ... I'm willing to tolerate a few small manual ,v edits if we have
> to do it to make tags consistent or something like that. I don't think
> we should be doing massive edits of that kind.

Yeah, that idea wasn't all that great after all.

> But anyway, that's not the interesting point. The interesting point is
> what about the historical aspect of it, not whether we want to dispense
> with the tags going forward. Should our repo conversion try to
> represent the historical states of the files including the tag strings?

Since we're going to lose them functionally after the conversion, it
doesn't seem that they serve any purpose. After all, they will not
represent anything on the new repository.

The problem is that they are a problem for the conversion. Are they
expanded before or after the commit? Because the very expansion causes
the file to change identity, files being identified by the SHA1 sum of
their contents.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

* Alvaro Herrera <alvherre(at)commandprompt(dot)com> [090529 11:45]:
> Aidan Van Dyk wrote:
>
> > Yes, but the point is you want an exact replica of CVS right? You're
> > git repo should have $PostgreSQL$ and the cvs export/checkout (you do
> > use -kk right) should also have $PostgreSQL$.
> >
> > The 3 parsecvs errors were that it *didn't* recognoze the strange
> > $PostgreSQL ... Exp $ expansion that cvs did.
>
> Huh, no -- I agree that $OpenBSD$ etc should remain (we don't munge them
> anyway), but $PostgreSQL$, $Id$, $Revision$ etc tags are best gone
> because, as Markus says, their expansion interferes with content hashing.

I *think* you're actually agreeing with me. *Hiding* the diffs that
include munching of keywords is not what we want. We want the
conversion to *not* munge "keyword-like" things (No, $OpenBSD$ is *not*
a keyword in the PostgreSQL CVS repository. But $PostgreSQL$ *is*.

So we want the conversion to be identical to:
cvs export -kk -r $tag

That will have *keywords* be unexpanded; namely these specific ones:
Author
Date
Header
Id
Locker
Log
Name
RCSfile
Revision
Source
State
PostgreSQL
but *not* "keyword-like" entries, like:
$ NetBSD ... Exp $
$ FreeBSD ... Exp $
$ OpenBSD ... Exp $
$ KAME ... Exp $
which are *not* CVS keywords in the PostgreSQL repository.

i.e. Just like I said, "identical to cvs checkout/export -kk.

Now, and intersting question, do you want the "perfect" conversion to
contain *other* keyword un-expansion possiblities that would have happened
on any commits on Nov 29/30 2003 when CVSROOT/options contained:
+tagexpand=iPostgreSQL
If you had checked out something on that day, even with a -kk, $Log$
would have been expanded, because for that day, $Log$ was *not* an
eligable keyword on the PostgreSQL CVS repository.

Whooee... Fun with CVS history....

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Hi,

a newish conversion with cvs2git is available to check here:

git://www.bluegap.ch/

(it's not incremental and will only stay for a few days)

For everybody interested, please check the committer names and emails.
I'm missing the names and email addresses for these committers:

'barry' : ('barry??', ''),
'dennis' : ('Dennis??', ''),
'inoue' : ('inoue??', ''),
'jurka' : ('jurka??', ''),
'pjw' : ('pjw??', ''),

And I'm guessing that 'peter' is the same as 'petere':

'peter' : ('Peter Eisentraut (?)', 'peter_e(at)gmx(dot)net'),

I've compared all branch heads and all tags with a cvs checkout. The
only differences are keyword expansion errors. Most commonly the RCS
version "1.1" is used in the resulting git repository, instead of
version "1.1.1.1". This also leads to getting dates wrong ($Date keyword).

I'm unsure on how to test Tom's requirement that every commit and its
log message is included in the resulting git repository. Feel free to
clone and inspect the mentioned git repository and propose improvements
on the cvs2git options used.

Aidan Van Dyk wrote:
> Yes, but the point is you want an exact replica of CVS right? You're
> git repo should have $PostgreSQL$ and the cvs export/checkout (you do
> use -kk right) should also have $PostgreSQL$.

No, I'm testing against cvs checkout, as that's what everybody is used to.

> But it's important, because on *some* files you *do* want expanded
> "keywords" (like the $OpenBSD ... Exp $. One of the reasons pg CVS went
> to the $PostgreSQL$ keyword (I'm guessing) was so they could explictly
> de-couple them from other keywords that they didn't want munging on.

I don't care half as much about the keyword expansion stuff - that's
doomed to disappear anyway.

What I'm much more interested in is correctness WRT historic contents,
i.e. that git log, git blame, etc.. deliver correct results. That's
certainly harder to check.

In my experience, cvs2svn (or cvs2git) does a pretty decent job at that,
even in case of some corruptions. Plus it offers lots of options to fine
tune the conversion, see the attached configuration I've used.

> So, I wouldn't consider any conversion good unless it had all these:
>
> As well as stuff like:
> parsecvs-master:src/backend/access/index/genam.c: * $PostgreSQL$

I disagree here and find it more convenient for the git repository to
keep the "old" RCS versions - as in the source tarballs that got (and
still get) shipped. Just before switching over to git one can (and
should, IMO) remove these tags to avoid confusion.

Regards

Markus Wanner

Markus Wanner <markus(at)bluegap(dot)ch> writes:
> I'm missing the names and email addresses for these committers:

> 'barry' : ('barry??', ''),

Barry Lind, formerly one of the JDBC bunch, been inactive for awhile

> 'dennis' : ('Dennis??', ''),

I suppose this must be Dennis Bjrklund, but I didn't realize he
used to be a committer.

> 'inoue' : ('inoue??', ''),

Hiroshi Inoue, still active, but ODBC is not part of core anymore

> 'jurka' : ('jurka??', ''),

Kris Jurka, still active, but JDBC is not part of core anymore

> 'pjw' : ('pjw??', ''),

Philip Warner, inactive (still reads the lists though)

> And I'm guessing that 'peter' is the same as 'petere':

> 'peter' : ('Peter Eisentraut (?)', 'peter_e(at)gmx(dot)net'),

No, that would be Peter Mount, also a retired JDBC hacker.

regards, tom lane

Tom Lane wrote:
> Markus Wanner <markus(at)bluegap(dot)ch> writes:

> > 'dennis' : ('Dennis??', ''),
>
> I suppose this must be Dennis Bjrklund, but I didn't realize he
> used to be a committer.

IIRC he was given commit privs for translation files.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Tom Lane wrote:
>> I suppose this must be Dennis Bjrklund, but I didn't realize he
>> used to be a committer.

> IIRC he was given commit privs for translation files.

Ah, right, that does ring a bell now.

BTW, Markus: you do realize "thomas" is not me but Tom Lockhart?

regards, tom lane

On 6/1/09, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> a newish conversion with cvs2git is available to check here:
>
> git://www.bluegap.ch/
>
> (it's not incremental and will only stay for a few days)

+1 for the idea of replacing CVS usernames with full names.

The knowledge about CVS usernames will be increasingly obscure.

Also worth mentioning is that there is no need to assign absolutely
up-to-date email addresses, it's enough if they uniquely identify
person.

> Aidan Van Dyk wrote:
> > Yes, but the point is you want an exact replica of CVS right? You're
> > git repo should have $PostgreSQL$ and the cvs export/checkout (you do
> > use -kk right) should also have $PostgreSQL$.
>
>
> No, I'm testing against cvs checkout, as that's what everybody is used to.
>
>
> > But it's important, because on *some* files you *do* want expanded
> > "keywords" (like the $OpenBSD ... Exp $. One of the reasons pg CVS went
> > to the $PostgreSQL$ keyword (I'm guessing) was so they could explictly
> > de-couple them from other keywords that they didn't want munging on.
>
>
> I don't care half as much about the keyword expansion stuff - that's
> doomed to disappear anyway.

But this is one aspect we need to get right for the conversion.

So preferably we test it sooner not later.

I think Aidan got it right - expand $PostgreSQL$ and others that are
actually expanded on current repo, but not $OpenBSD$ and others
coming from external sources.

> What I'm much more interested in is correctness WRT historic contents,
> i.e. that git log, git blame, etc.. deliver correct results. That's
> certainly harder to check.
>
> In my experience, cvs2svn (or cvs2git) does a pretty decent job at that,
> even in case of some corruptions. Plus it offers lots of options to fine
> tune the conversion, see the attached configuration I've used.
>
>
> > So, I wouldn't consider any conversion good unless it had all these:
> >
>
> > As well as stuff like:
> > parsecvs-master:src/backend/access/index/genam.c: * $PostgreSQL$
>
>
> I disagree here and find it more convenient for the git repository to
> keep the "old" RCS versions - as in the source tarballs that got (and
> still get) shipped. Just before switching over to git one can (and
> should, IMO) remove these tags to avoid confusion.

I'd prefer we immediately test full conversion and not leave some
steps to last moment.

--
marko

On 6/2/09, Marko Kreen <markokr(at)gmail(dot)com> wrote:
> On 6/1/09, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> > a newish conversion with cvs2git is available to check here:
> >
> > git://www.bluegap.ch/
> >
> > (it's not incremental and will only stay for a few days)

Btw this conversion seems broken as it contains random merge commits.

parsecvs managed to do it without them.

--
marko

Hi,

Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
>> I don't care half as much about the keyword expansion stuff - that's
>> doomed to disappear anyway.
>
> But this is one aspect we need to get right for the conversion.

What's your definition of "right"? I personally prefer the keyword
expansion to match a cvs checkout as closely as possible.

> So preferably we test it sooner not later.

I actually *am* testing against that. As mentioned, the only
differences are insignificant, IMO. For example having "1.1.1.1"
instead of "1.1" (or vice versa, I don't remember).

> I think Aidan got it right - expand $PostgreSQL$ and others that are
> actually expanded on current repo, but not $OpenBSD$ and others
> coming from external sources.

AFAIU Aidan proposed the exact opposite.

I'm proposing to leave both expanded, as in a CVS checkout and as
shipped in the source release tarballs.

> I'd prefer we immediately test full conversion and not leave some
> steps to last moment.

IMO that would equal to changing history, so that a checkout from git
doesn't match a released tarball as good as possible.

What you call "leave(ing) some steps to last moment" is IMO not part
of the conversion. It's rather a conscious decision to drop these
keywords as soon as we switch to git. This step should be represented
in history as a separate commit, IMO.

What do others think?

Regards

Markus Wanner

On 6/2/09, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> > > I don't care half as much about the keyword expansion stuff - that's
> > > doomed to disappear anyway.
> > >
> >
> > But this is one aspect we need to get right for the conversion.
> >
>
> What's your definition of "right"? I personally prefer the keyword
> expansion to match a cvs checkout as closely as possible.

This is Definitely Wrong (tm). You seem to be thinking that comparing
GIT checkout to random parallel CVS checkout (eg. from .tgz.) is the
main use-case. It is not. Browsing history and looking and diffs
between versions is. And expanded CVS keywords would be total PITA
for that.

> > So preferably we test it sooner not later.
> >
>
> I actually *am* testing against that. As mentioned, the only differences
> are insignificant, IMO. For example having "1.1.1.1" instead of "1.1" (or
> vice versa, I don't remember).

Why have those at all...

> > I think Aidan got it right - expand $PostgreSQL$ and others that are
> > actually expanded on current repo, but not $OpenBSD$ and others
> > coming from external sources.
> >
>
> AFAIU Aidan proposed the exact opposite.

Ah, sorry, my thinko. s/expanded/stripped/. Take Aidan's description
as authoritative.. :)

> I'm proposing to leave both expanded, as in a CVS checkout and as shipped
> in the source release tarballs.

No, the noise they add to history would seriously hurt usability.

> > I'd prefer we immediately test full conversion and not leave some
> > steps to last moment.
> >
>
> IMO that would equal to changing history, so that a checkout from git
> doesn't match a released tarball as good as possible.

We need to compare against tarballs only when checking the conversion.
And only then. Writing few scripts for that should not be a problem.

> What you call "leave(ing) some steps to last moment" is IMO not part of the
> conversion. It's rather a conscious decision to drop these keywords as soon
> as we switch to git. This step should be represented in history as a
> separate commit, IMO.

The question is how they should appear in historical commits.

I have no strong opinion whether to edit them out or not in the future.
Doing it during the periodic reindent would be good moment tho'.

--
marko

Hi,

Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> Btw this conversion seems broken as it contains random merge commits.

Well, that's a feature, not a bug ;-)

When a commit adds a file to the master *and* then to the branch as
well, cvs2git prefers to represent this as a merge from the master
branch, instead of adding the file twice, once on the master and once
on the branch.

This way the target VCS knows it's the *same* file, originating from
one single commit. This may be important for later merges - otherwise
you may suddenly end up with duplicated files after a merge, because
the VCS doesn't know they are in fact the same.

(Okay, git assumes two files to have the same origin/history as long
as they have the same filename. But just rename one of the two, and
you are have the same troubles, again).

Also note that these situations occur rather frequently in the
Postgres CVS repository. Every back-patch which adds files ends up as
a merge. (One could even argue that in the perfect conversion *all*
back-patches should be represented as merges, rather than as separate
commits).

> parsecvs managed to do it without them.

Now, I'm not calling it broken, but cvs2git's output is arguably
better in that regard.

As you certainly see by now, conversion from CVS is neither simple nor
unambiguous.

Regards

Markus Wanner

Hi,

Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> This is Definitely Wrong (tm). You seem to be thinking that comparing
> GIT checkout to random parallel CVS checkout (eg. from .tgz.) is the
> main use-case. It is not. Browsing history and looking and diffs
> between versions is. And expanded CVS keywords would be total PITA
> for that.

That's an agrument. Point taken. I'll check if cvs2git supports that as well.

Regards

Markus Wanner

On 6/2/09, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> > Btw this conversion seems broken as it contains random merge commits.
> >
>
> Well, that's a feature, not a bug ;-)
>
> When a commit adds a file to the master *and* then to the branch as well,
> cvs2git prefers to represent this as a merge from the master branch, instead
> of adding the file twice, once on the master and once on the branch.
>
> This way the target VCS knows it's the *same* file, originating from one
> single commit. This may be important for later merges - otherwise you may
> suddenly end up with duplicated files after a merge, because the VCS doesn't
> know they are in fact the same.
>
> (Okay, git assumes two files to have the same origin/history as long as
> they have the same filename. But just rename one of the two, and you are
> have the same troubles, again).

Not a problem for git I think - it assumes they are same if they have
same contents...

> Also note that these situations occur rather frequently in the Postgres CVS
> repository. Every back-patch which adds files ends up as a merge. (One could
> even argue that in the perfect conversion *all* back-patches should be
> represented as merges, rather than as separate commits).

Well, such behaviour may be a feature for some repo with complex CVS
usage, but currently we should aim for simple and clear conversion.

The question is - do such merges make any sense to human looking at
history - and the answer is no, as no VCS level merge was happening,
just some copying around (if your description is correct). And
we don't need to add noise for the benefit of GIT as it works fine
without any fake merges.

Our target should be each branch having simple linear history,
without any fake merges. This will result in minimal confusion
to both humans looking history and also GIT itself.

So please turn the merge logic off. If this cannot be turned off,
cvs2git is not usable for conversion.

> > parsecvs managed to do it without them.
> >
>
> Now, I'm not calling it broken, but cvs2git's output is arguably better in
> that regard.

Seems it contains more complex logic to handle more complex CVS usage
cases, but seems like overkill for us if it creates a mess of history.

> As you certainly see by now, conversion from CVS is neither simple nor
> unambiguous.

I know, thats why I'm discussing the tradeoffs. Simple+clear vs.
complex+messy. :)

--
marko

* Markus Wanner <markus(at)bluegap(dot)ch> [090602 07:08]:
> Hi,
>
> Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
>>> I don't care half as much about the keyword expansion stuff - that's
>>> doomed to disappear anyway.
>>
>> But this is one aspect we need to get right for the conversion.
>
> What's your definition of "right"? I personally prefer the keyword
> expansion to match a cvs checkout as closely as possible.

> AFAIU Aidan proposed the exact opposite.
>
> I'm proposing to leave both expanded, as in a CVS checkout and as
> shipped in the source release tarballs.

Well, since I have -kk set in my .cvsrc, "mine" matches exactly the CVS
checkout l-)

Basically, I want the git to be identical to the cvs checkout. If you
use -kk, that means the "PostgreSQL CVS repository keywords" *aren't*
expanded. If you like -kv, that means they are.

Pick your poison (after all, it's CVS), either way, I think the 2 of
*us* are going to disagree which is best here ;-)

But, which ever way (exact to -kk or exact to -kv), the conversion
should be exact, and there should be no reason to "filter out
keyword-like stuff" in the diffs.

> What you call "leave(ing) some steps to last moment" is IMO not part of
> the conversion. It's rather a conscious decision to drop these keywords
> as soon as we switch to git. This step should be represented in history
> as a separate commit, IMO.
>
> What do others think?

I'm assuming they will get removed from the source eventually too - but
that step is *outside* the conversion. Somebody could do it now in CVS
before the conversion, or afterwards, but it's still outside the
conversion.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Hi,

Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
> Pick your poison (after all, it's CVS), either way, I think the 2 of
> *us* are going to disagree which is best here ;-)

Marko already convinced me of -kk, I'm trying that with cvs2git.

> But, which ever way (exact to -kk or exact to -kv), the conversion
> should be exact, and there should be no reason to "filter out
> keyword-like stuff" in the diffs.

I just really didn't want to care about keyword expansion. Besides
lacking consistency, it's one of the worst misfeatures of CVS, IMNSHO.
;-)

I'll let you know how cvs2git behaves WRT -kk.

Regards

Markus Wanner

* Markus Wanner <markus(at)bluegap(dot)ch> [090602 09:37]:

> Marko already convinced me of -kk, I'm trying that with cvs2git.

Good ;-)

> I just really didn't want to care about keyword expansion. Besides
> lacking consistency, it's one of the worst misfeatures of CVS, IMNSHO.
> ;-)

Absolutely... And one of the reasons I've had -kk in my .cvsrc for
years, even before I started with git.

> I'll let you know how cvs2git behaves WRT -kk.

Cool..

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Hi,

Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> Not a problem for git I think

Knowing that git doesn't track files as "hard" as monotone, I
certainly doubt that.

> - it assumes they are same if they have
> same contents...

Why do you assume they have the same contents? Obviously these are
different branches, where files can (and will!) have different contents.

> Well, such behaviour may be a feature for some repo with complex CVS
> usage, but currently we should aim for simple and clear conversion.

First of all, we should aim for a correct one.

> The question is - do such merges make any sense to human looking at
> history - and the answer is no, as no VCS level merge was happening,
> just some copying around (if your description is correct). And
> we don't need to add noise for the benefit of GIT as it works fine
> without any fake merges.

For low expectations of "it works", maybe yes. However if you don't
tell git, it has no chance of knowing that two (different) files
should actually be the same.

Try the following:

git init
echo "base" > basefile
git add basefile
git commit -m "base commit"
git checkout -b branch
echo "hello, world" > testfile
git add testfile
git commit testfile -m "addition on branch"
git checkout master
echo "hello world" > testfile
git add testfile
git commit testfile -m "addition on master"

# here we are a similar point like after a lacking conversion, having two
# distinct, i.e. historically independent files called "testfile"

git mv testfile movedfile
git commit -m "file moved"
git checkout branch
git merge master
ls

# Bang, you suddenly have 'testfile' and 'movedfile', go figure!

I leave it as an exercise for the reader to try the same with a single
historic origin of the file, as cvs2git does the conversion.

> Our target should be each branch having simple linear history,
> without any fake merges. This will result in minimal confusion
> to both humans looking history and also GIT itself.

I don't consider the above a "minimal confusion". And concerning
humans... you get used to merge commits pretty quickly. I for one am
more confused by a linear history which in fact is not.

As mentioned before, I'd personally favor *all* of the back-ports to
actually be merges of some sort, because that's what they effectively
are. However, that also bring up the question of how we are going to
do back-patches in the future with git.

> So please turn the merge logic off. If this cannot be turned off,
> cvs2git is not usable for conversion.

As far as I know, it cannot be turned off. Use parsecvs if you want to
get silly side effects later on in history. ;-)

> Seems it contains more complex logic to handle more complex CVS usage
> cases, but seems like overkill for us if it creates a mess of history.

You consider it a mess, I consider it a better and more valid
representation of the mess that CVS is.

Regards

Markus Wanner

* Markus Wanner <markus(at)bluegap(dot)ch> [090602 10:23]:

> # Bang, you suddenly have 'testfile' and 'movedfile', go figure!
>
> I leave it as an exercise for the reader to try the same with a single
> historic origin of the file, as cvs2git does the conversion.

Sure, and we can all construct example where that move is both right and
wrong... But the point is that in PostgreSQL, (and that may be mainly
because we're using CVS), merges *aren't* something that happens.
Patches are written against HEAD (master) and then back-patched...

If you want to turn PostgreSQL devellopment on it's head, then we can
switch this around, so that patches are always done on the oldest
branch, and fixes always merged "forward"...

I'm not going to be the one that pushes that though ;-)

> I don't consider the above a "minimal confusion". And concerning
> humans... you get used to merge commits pretty quickly. I for one am
> more confused by a linear history which in fact is not.

But the fact is, everyone using CVS wants a "linear history"..... All
they care about is "cvs update...wait...cvs update ... time ... cvs
update ..."... Everything *was* linear to them. Any "merge" type things
certaily wasn't intentional in CVS...

> As mentioned before, I'd personally favor *all* of the back-ports to
> actually be merges of some sort, because that's what they effectively
> are. However, that also bring up the question of how we are going to do
> back-patches in the future with git.

Well, if people get comfortable with it, I expect that "backports" don't
happenen.. Bugs are fixed where they happen, and "merged" forward into
all affected "later development" based on the bugged area.

> As far as I know, it cannot be turned off. Use parsecvs if you want to
> get silly side effects later on in history. ;-)

Ya, that's one of the reasons I considered parsecvs the leading
candidate... And why I went thouth, and showed that with the exception
of the one REL_8_0_0 tip, it *was* and exact copy of the current CVS
repository (minus the 1 messed up tag in the repository).

> You consider it a mess, I consider it a better and more valid
> representation of the mess that CVS is.

So much better that it makes the history as useless as CVS... I think
one of the reasons people are wanting tomove from CVS to git is that it
makes things *better*... The "exact" history will *always* be
available, right in CVS if people need it. I thin the goal is to make
the "git" history as close to CVS as possible, such that it's useful. I
mean, if we want it to be a "more valid" representation, then really, we
should be doing every file change in a single commit, and "merging" that
file commit into the branch *every* *single* *time*... I don't think
anybody wants our conversion to be that much "better and move valid
representation of the mess that CVS is"...

It's a balance... We're moving because we want *better* tools and
access, not the same "mess that CVS is".

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Aidan Van Dyk escribió:
> * Markus Wanner <markus(at)bluegap(dot)ch> [090602 10:23]:
>
> > # Bang, you suddenly have 'testfile' and 'movedfile', go figure!
> >
> > I leave it as an exercise for the reader to try the same with a single
> > historic origin of the file, as cvs2git does the conversion.
>
> Sure, and we can all construct example where that move is both right and
> wrong... But the point is that in PostgreSQL, (and that may be mainly
> because we're using CVS), merges *aren't* something that happens.
> Patches are written against HEAD (master) and then back-patched...
>
> If you want to turn PostgreSQL devellopment on it's head, then we can
> switch this around, so that patches are always done on the oldest
> branch, and fixes always merged "forward"...

The Monotone folk call this "daggy fixes" and it seems a clean way to
handle things.

http://www.monotone.ca/wiki/DaggyFixes/

However,

> I'm not going to be the one that pushes that though ;-)

I'm not either. Maybe someday we'll be familiar enough with the tools
to make things this way, but I think just after the migration we'll
mainly want to be able to press on with development and not waste too
much time learning the new toys.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
> * Markus Wanner <markus(at)bluegap(dot)ch> [090602 10:23]:
>> You consider it a mess, I consider it a better and more valid
>> representation of the mess that CVS is.

> So much better that it makes the history as useless as CVS... I think
> one of the reasons people are wanting tomove from CVS to git is that it
> makes things *better*...

FWIW, the tool that I customarily use (cvs2cl) considers commits on
different branches to be "the same" if they have the same commit message
and occur sufficiently close together (within a few minutes). My
committing habits have been designed around that behavior for years,
and I believe other PG committers have been doing likewise.

I would consider a git conversion to be less useful to me, not more,
if it insists on showing me such cases as separate commits --- and if
it then adds useless "merge" messages on top of that, I'd start to get
seriously annoyed.

What we want here is a readable equivalent of the CVS history, not
necessarily something that is theoretically an exact equivalent.

regards, tom lane

Hi,

Quoting "Aidan Van Dyk" <aidan(at)highrise(dot)ca>:
> Sure, and we can all construct example where that move is both right and
> wrong...

Huh? The problem is the file duplication. The move is an action of a
committer - it's neither right nor wrong in this example.

I cannot see any use case for seemingly random files poping up out of
nowhere, just because git doesn't know how to merge two files after a
mv and a merge.

> But the point is that in PostgreSQL, (and that may be mainly
> because we're using CVS), merges *aren't* something that happens.
> Patches are written against HEAD (master) and then back-patched...

..which can (and better is) represented as a merge in git (for the
sake of comfortable automated merging).

> If you want to turn PostgreSQL devellopment on it's head, then we can
> switch this around, so that patches are always done on the oldest
> branch, and fixes always merged "forward"...

I'd consider that good use of tools, yes. However, I realize that this
probably is pipe-dreaming...

> But the fact is, everyone using CVS wants a "linear history"..... All
> they care about is "cvs update...wait...cvs update ... time ... cvs
> update ..."... Everything *was* linear to them. Any "merge" type things
> certaily wasn't intentional in CVS...

..no, it just wasn't possible in CVS. Switching to git, people soon
want "merge type things". Heck, it's probably *the* reason for
switching to git.

> So much better that it makes the history as useless as CVS... I think
> one of the reasons people are wanting tomove from CVS to git is that it
> makes things *better*...

Yes, especially merging. Please don't cripple that ability just
because CVS once upon a time enforced a linear history.

> The "exact" history will *always* be
> available, right in CVS if people need it.

Agreed. Please note that I mostly talk about a more correct
representation *of history*, as it happened. This has nothing to do
with single commits per file.

> It's a balance... We're moving because we want *better* tools and
> access, not the same "mess that CVS is".

Agreed. And please cut as many of its burdens of the past, like
linearity. History is not linear and has never been. But I'm stopping
now before getting overly philosophic...

Regards

Markus Wanner

On Tue, Jun 2, 2009 at 4:02 PM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
>
>
> The Monotone folk call this "daggy fixes" and it seems a clean way to
> handle things.
>
> http://www.monotone.ca/wiki/DaggyFixes/

Is this like what git calls an octopus? I've been wondering what the
point of such things were.

Or maybe not. I thought an octopus was two patches with the same
parent -- ie, two patches that could independently be applied in any
order.

--
greg

On 6/2/09, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
> > * Markus Wanner <markus(at)bluegap(dot)ch> [090602 10:23]:
>
> >> You consider it a mess, I consider it a better and more valid
> >> representation of the mess that CVS is.
>
> > So much better that it makes the history as useless as CVS... I think
> > one of the reasons people are wanting tomove from CVS to git is that it
> > makes things *better*...
>
>
> FWIW, the tool that I customarily use (cvs2cl) considers commits on
> different branches to be "the same" if they have the same commit message
> and occur sufficiently close together (within a few minutes). My
> committing habits have been designed around that behavior for years,
> and I believe other PG committers have been doing likewise.
>
> I would consider a git conversion to be less useful to me, not more,
> if it insists on showing me such cases as separate commits --- and if
> it then adds useless "merge" messages on top of that, I'd start to get
> seriously annoyed.

They cannot be same commits in GIT as the resulting tree is different.
You could tie them with some sort of merge commits, but doubt the
result would be worth the noise.

Also I doubt there is tool grokking such commits anyway, the merge
discussion above was for full files with exact contents appearing
in several branches.

> What we want here is a readable equivalent of the CVS history, not
> necessarily something that is theoretically an exact equivalent.

I suggest setting the goal to be simple and clear representation
of CVS history that we can make sense later, instead of revising
CVS history to look like we used some better VCS system...

--
marko

On 6/2/09, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> [academic nitpicking]

Sorry, not going there. Just look at the state of VCS systems
that have prioritized academic issues insead of practicality...
(arch/darcs/monotone/etc..)

> > So please turn the merge logic off. If this cannot be turned off,
> > cvs2git is not usable for conversion.
> >
>
> As far as I know, it cannot be turned off. Use parsecvs if you want to get
> silly side effects later on in history. ;-)

--no-cross-branch-commits seems sort of that direction?

And what silly side effects are you talking about? I see only cvs2git
doing silly things...

(I'm talking about only in context of Postgres CVS repo, not in general.)

> > Seems it contains more complex logic to handle more complex CVS usage
> > cases, but seems like overkill for us if it creates a mess of history.
> >
>
> You consider it a mess, I consider it a better and more valid
> representation of the mess that CVS is.

Note that merge is no file-level but tree level. Also note we don't
use branches for feature developement but for major version maintenance.

So how can single file appearing in 2 branches means merge of 2 trees?
How can that be valid?

--
marko

On Tue, Jun 2, 2009 at 11:08 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Aidan Van Dyk <aidan(at)highrise(dot)ca> writes:
>> * Markus Wanner <markus(at)bluegap(dot)ch> [090602 10:23]:
>>> You consider it a mess, I consider it a better and more valid
>>> representation of the mess that CVS is.
>
>> So much better that it makes the history as useless as CVS... I think
>> one of the reasons people are wanting tomove from CVS to git is that it
>> makes things *better*...
>
> FWIW, the tool that I customarily use (cvs2cl) considers commits on
> different branches to be "the same" if they have the same commit message
> and occur sufficiently close together (within a few minutes).  My
> committing habits have been designed around that behavior for years,
> and I believe other PG committers have been doing likewise.

Interesting. I was wondering why all your commit messages always show
up simultaneously for all the back branches.

> I would consider a git conversion to be less useful to me, not more,
> if it insists on showing me such cases as separate commits --- and if
> it then adds useless "merge" messages on top of that, I'd start to get
> seriously annoyed.

There's no help for them being separate commits, but I agree that
useless merge commits are a bad thing. There are plenty of ways to
avoid that, though; I've been using git cherry-pick a lot recently,
and I think git rebase --onto also has some potential.

...Robert

Hi,

Quoting "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>:
> FWIW, the tool that I customarily use (cvs2cl) considers commits on
> different branches to be "the same" if they have the same commit message
> and occur sufficiently close together (within a few minutes). My
> committing habits have been designed around that behavior for years,
> and I believe other PG committers have been doing likewise.

Yeah, that's how I see things as well.

> I would consider a git conversion to be less useful to me, not more,
> if it insists on showing me such cases as separate commits --- and if
> it then adds useless "merge" messages on top of that, I'd start to get
> seriously annoyed.

Hm.. well, in git, there's no such thing as a commit that spans
multiple branches. So it's impossible to fulfill both of your wishes
here.

parsecvs creates multiple independent commits in such a case.

cvs2git creates a single commit and propagates this to the back
branches with merge commits (however, only if new files are added,
otherwise it does the same as parsecvs).

> What we want here is a readable equivalent of the CVS history, not
> necessarily something that is theoretically an exact equivalent.

Understood. However, readability depends on the user's habits. But
failing to merge due to a lacking conversion potentially hurts
everybody who wants to merge.

Having used merging (in combination with renaming) often enough, I'd
certainly be pretty annoyed if merges suddenly begin to bring up
spurious file duplicates.

Regards

Markus Wanner

Hi,

Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> Sorry, not going there. Just look at the state of VCS systems
> that have prioritized academic issues insead of practicality...
> (arch/darcs/monotone/etc..)

I already am there. And I don't want to go back, thanks. But my bias
for monotone certainly shines through, yes ;-)

> --no-cross-branch-commits seems sort of that direction?

Yes, that could lead to the same defect. Uhm.. thank you for pointing
that out, I'm not gonna try it, sorry.

> And what silly side effects are you talking about?

I'm talking about spurious file duplicates popping up after a rename
and a merge, see my example in this thread.

>> You consider it a mess, I consider it a better and more valid
>> representation of the mess that CVS is.
>
> Note that merge is no file-level but tree level.

Depends on your point of view. Each file gets merged pretty
indivitually, but the result ends up in a single commit, yes.

> Also note we don't
> use branches for feature developement but for major version maintenance.

So? You think you are never going to merge?

> So how can single file appearing in 2 branches means merge of 2 trees?
> How can that be valid?

I'm not sure what you are questioning here.

I find it perfectly reasonable to build something on top of
REL8_3_STABLE and later on wanting to merge to REL8_4_STABLE. And I
don't want to manually merge my changes, just because of a rename in
8.4 and a bad decision during the migration to git.

(And no, I don't think any of the other git tools will help with this,
due to the academic-nitpick-reasons above).

Regards

Markus Wanner

On 6/2/09, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> > And what silly side effects are you talking about?
> >
>
> I'm talking about spurious file duplicates popping up after a rename and a
> merge, see my example in this thread.

The example was not actual case from Postgres CVS history,
but hypotetical situation without checking if it already works
with GIT.

> > Also note we don't
> > use branches for feature developement but for major version maintenance.
> >
>
> So? You think you are never going to merge?
>
>
> > So how can single file appearing in 2 branches means merge of 2 trees?
> > How can that be valid?
> >
>
> I'm not sure what you are questioning here.
>
> I find it perfectly reasonable to build something on top of REL8_3_STABLE
> and later on wanting to merge to REL8_4_STABLE. And I don't want to manually
> merge my changes, just because of a rename in 8.4 and a bad decision during
> the migration to git.
>
> (And no, I don't think any of the other git tools will help with this, due
> to the academic-nitpick-reasons above).

Merging between branches with GIT is fine workflow in the future.

But we are currently discussing how to convert CVS history to GIT.
My point is that we should avoid fake merges, to avoid obfuscating
history.

--
marko

Aidan Van Dyk wrote:
> * Markus Wanner <markus(at)bluegap(dot)ch> [090602 10:23]:
>> As mentioned before, I'd personally favor *all* of the back-ports to
>> actually be merges of some sort, because that's what they effectively
>> are. However, that also bring up the question of how we are going to do
>> back-patches in the future with git.
>
> Well, if people get comfortable with it, I expect that "backports" don't
> happenen.. Bugs are fixed where they happen, and "merged" forward into
> all affected "later development" based on the bugged area.

I imagine the closest thing to existing practices would be that people
would to use "git-cherry-pick -x -n" to backport only the commits they
wanted from the current branch into the back branches.

AFAICT, this doesn't record a merge in the GIT history, but looks a lot
like the linear history from CVS - with the exception that the comment
added by "-x" explicitly refers to the exact commit from the main branch.

Hi,

Quoting "Marko Kreen" <markokr(at)gmail(dot)com>:
> The example was not actual case from Postgres CVS history,
> but hypotetical situation without checking if it already works
> with GIT.

Of course it is a simplified example, but it resembles what could
happen i.e. to the file doc/src/sgml/generate_history.pl, which got
added from a backported patch after forking off REL8_3_STABLE.

If you create separate commits during the conversion, rename that file
on the master branch and then - for whatever reason - try to merge the
two branches, you will end up having that file twice. That's what I'm
warning about. Changes on either or both sides of the merge make the
situation worse.

> Merging between branches with GIT is fine workflow in the future.

Do you consider the above scenario a fine merge?

> My point is that we should avoid fake merges, to avoid obfuscating
> history.

Understood. It looks like I'm pretty much the only one who cares more
about merge capability than nice looking history :-(

Attached is my current options file for cvs2git, it includes requested
changes by Alvaro and additional names and emails as given by Tom
(thanks again). A current conversion with cvs2git (and with the
merges) results in a repository with exactly 0 differences against any
branch or tag symbol compared to cvs checkout -kk.

Regards

Markus Wanner

On Wed, Jun 3, 2009 at 12:10 PM, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> If you create separate commits during the conversion, rename that file on
> the master branch

This is all completely irrelevant to the CVS import. I don't think
we've ever renamed files because CVS can't handle it cleanly.

It does sound to me like we really ought to have merge commits marking
the bug fixes in old releases as merged in the equivalent commits to
later branches based on Tom's commit messages.

That would make the git history match Tom's "same commit message"
implicit CVS history that cvs2pcl was giving him. I find git-log's
output including merge commits kind of strange and annoying myself but
having them at least gives us a chance to have a tool that understands
them output something like cvs2pcl. Throwing away that information
because we don't like the clutter in the tool output seems like a
short-sighted plan.

That said, the commit log message isn't being lost. We could always
import the history linearly and add the merge commits later if we
decide having them would help some tool implement cvs2pcl summaries.

--
greg

Hi,

On 06/03/2009 02:08 PM, Greg Stark wrote:
> On Wed, Jun 3, 2009 at 12:10 PM, Markus Wanner<markus(at)bluegap(dot)ch> wrote:
> That would make the git history match Tom's "same commit message"
> implicit CVS history that cvs2pcl was giving him. I find git-log's
> output including merge commits kind of strange and annoying myself but
> having them at least gives us a chance to have a tool that understands
> them output something like cvs2pcl.
"git log --no-merges" hides the actual merge commits if that is what you
want.

Andres

On Wed, Jun 3, 2009 at 1:19 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> "git log --no-merges" hides the actual merge commits if that is what you
> want.

Ooh! Life seems so much sweeter now!

Given that we don't have to see them then I'm all for marking bug fix
patches which were applied to multiple branches as merges. That seems
like it would make it easier for tools like gitk or to show useful
information analogous to the cvs2pcl info.

Given that Tom's been intentionally marking the commits with identical
commit messages we ought to be able to find *all* of them and mark
them properly. That would be way better than only finding patches that
are absolutely identical.

I'm not sure whether we should mark the old branches getting merges
down or the new branches getting merged up. I suspect I'm missing
something but I don't see any reason one is better than the other.

--
greg

Greg Stark wrote:
> On Wed, Jun 3, 2009 at 1:19 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
>> "git log --no-merges" hides the actual merge commits if that is what you
>> want.
>
> Ooh! Life seems so much sweeter now!
>
> Given that we don't have to see them then I'm all for marking bug fix
> patches which were applied to multiple branches as merges. That seems
> like it would make it easier for tools like gitk or to show useful
> information analogous to the cvs2pcl info.

Right, if it adds additional metadata that lets the tools do their magic
better, and it's still easy to filter out, I don't see a downside.

> Given that Tom's been intentionally marking the commits with identical
> commit messages we ought to be able to find *all* of them and mark
> them properly. That would be way better than only finding patches that
> are absolutely identical.

Just to be clear, not just Tom. All committers. I was told to do that
right after my first backpatch which *didn't* do it :-)

So it's an established project practice. That has other advantages as
well, of course..

> I'm not sure whether we should mark the old branches getting merges
> down or the new branches getting merged up. I suspect I'm missing
> something but I don't see any reason one is better than the other.

If you go from older to newer, the automatic merge algorithms have a
better chance of doing something smart since they can track previous
changes. At least I think that's how it works.

But I think for most of the changes it wouldn't make a huge difference,
though - manual merging would be needed anyway.

//Magnus

On Wed, Jun 3, 2009 at 10:13 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> Greg Stark wrote:
>> On Wed, Jun 3, 2009 at 1:19 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
>>> "git log --no-merges" hides the actual merge commits if that is what you
>>> want.
>>
>> Ooh! Life seems so much sweeter now!
>>
>> Given that we don't have to see them then I'm all for marking bug fix
>> patches which were applied to multiple branches as merges. That seems
>> like it would make it easier for tools like gitk or to show useful
>> information analogous to the cvs2pcl info.
>
> Right, if it adds additional metadata that lets the tools do their magic
> better, and it's still easy to filter out, I don't see a downside.
>
>> I'm not sure whether we should mark the old branches getting merges
>> down or the new branches getting merged up. I suspect I'm missing
>> something but I don't see any reason one is better than the other.
>
> If you go from older to newer, the automatic merge algorithms have a
> better chance of doing something smart since they can track previous
> changes. At least I think that's how it works.
>
> But I think for most of the changes it wouldn't make a huge difference,
> though - manual merging would be needed anyway.

In practice, isn't it more likely that you would develop the change on
the newest branch and then try to back-port it? However you do the
import, you're going to want to do subsequent things the same way.

...Robert

Robert Haas wrote:
> On Wed, Jun 3, 2009 at 10:13 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>> I'm not sure whether we should mark the old branches getting merges
>>> down or the new branches getting merged up. I suspect I'm missing
>>> something but I don't see any reason one is better than the other.
>> If you go from older to newer, the automatic merge algorithms have a
>> better chance of doing something smart since they can track previous
>> changes. At least I think that's how it works.
>>
>> But I think for most of the changes it wouldn't make a huge difference,
>> though - manual merging would be needed anyway.
>
> In practice, isn't it more likely that you would develop the change on
> the newest branch and then try to back-port it? However you do the
> import, you're going to want to do subsequent things the same way.

That's definitely the order in which *I* work, and I think that's how
most others do it as well.

--
Magnus Hagander
Self: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 6/3/09, Greg Stark <stark(at)enterprisedb(dot)com> wrote:
> On Wed, Jun 3, 2009 at 1:19 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > "git log --no-merges" hides the actual merge commits if that is what you
> > want.
>
>
> Ooh! Life seems so much sweeter now!
>
> Given that we don't have to see them then I'm all for marking bug fix
> patches which were applied to multiple branches as merges. That seems
> like it would make it easier for tools like gitk or to show useful
> information analogous to the cvs2pcl info.
>
> Given that Tom's been intentionally marking the commits with identical
> commit messages we ought to be able to find *all* of them and mark
> them properly. That would be way better than only finding patches that
> are absolutely identical.
>
> I'm not sure whether we should mark the old branches getting merges
> down or the new branches getting merged up. I suspect I'm missing
> something but I don't see any reason one is better than the other.

Although "mark Tom's back-branch fixes as merges" makes much more
sense than "mark new files as merges", it is quite a step up from
"do tags match official releases".

It seems to require noticeable development effort to get a importer
to a level it can do it. Will this be a requirement for import?
Or just a good thing to have? Also how to check if all such merges
are sensible?

And note that such effort will affect only old imported history,
it will not make easier to handle back-branch fixes in the future...

Various scenarios with git cherry-pick and similar tools would still
result in duplicate commits, so we would need a git log post-processor
anyway if we want to somehow group them together for eg. weekly commit
summary. And such post-processor would work on old history too.

Maybe that's better direction to work on, than to potentially risk in
messy history in GIT?

--
marko

On 6/3/09, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> Robert Haas wrote:
> > On Wed, Jun 3, 2009 at 10:13 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>
> >>> I'm not sure whether we should mark the old branches getting merges
> >>> down or the new branches getting merged up. I suspect I'm missing
> >>> something but I don't see any reason one is better than the other.
> >> If you go from older to newer, the automatic merge algorithms have a
> >> better chance of doing something smart since they can track previous
> >> changes. At least I think that's how it works.
> >>
> >> But I think for most of the changes it wouldn't make a huge difference,
> >> though - manual merging would be needed anyway.
> >
> > In practice, isn't it more likely that you would develop the change on
> > the newest branch and then try to back-port it? However you do the
> > import, you're going to want to do subsequent things the same way.
>
>
> That's definitely the order in which *I* work, and I think that's how
> most others do it as well.

Thats true, but it's not representable in VCS, unless you use cherry-pick,
which is just UI around patch transport. But considering separate
local trees (with can optionally contain local per-fix branches),
it is possible to separate the fix-developement from final representation.

--
marko

* Magnus Hagander <magnus(at)hagander(dot)net> [090603 10:13]:
>
> Right, if it adds additional metadata that lets the tools do their magic
> better, and it's still easy to filter out, I don't see a downside.

Note, that it could (and likely will) have a downside when you get to
doing real merge-based development... A "merge" means that *all* changes
in *both* parents have been combined in *this* commit. And all merge
tools depend on this. That's the directed part of the "DAG" in git. So
if you want to be working in a way that the merge tools work, you
*don't* have master/HEAD merged into REL8_2_STABLE. You can have
REL8_2_STABLE merged into master/head.

I'll concede that in GIT, it's flexible (some say arbitrary) enough that
you can *construct* the DAG otherwise, but then you've done something in
such a fashion that the DAG has no bearing on "real merging", and thus
you loose all the power of DAGs "merge tracking" when working on new
real merging....

a.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

* Marko Kreen <markokr(at)gmail(dot)com> [090603 10:26]:

> Thats true, but it's not representable in VCS, unless you use cherry-pick,
> which is just UI around patch transport. But considering separate
> local trees (with can optionally contain local per-fix branches),
> it is possible to separate the fix-developement from final representation.

I'll note that in git, cherry-pick is *more* than just "patch
transport". I would more call it "patch commute". It does actually
look at the history between the "pick"ed patch, and the current
tree, any merge/fork points, and the differences on each path that lead
to the changes in the current tree and the picked patch.

a.
--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

On Wed, Jun 3, 2009 at 10:20 AM, Marko Kreen <markokr(at)gmail(dot)com> wrote:
> Various scenarios with git cherry-pick and similar tools would still
> result in duplicate commits, so we would need a git log post-processor
> anyway if we want to somehow group them together for eg. weekly commit
> summary.  And such post-processor would work on old history too.
>
> Maybe that's better direction to work on, than to potentially risk in
> messy history in GIT?

I think it is. cherry-picking seems like a much better way of
back-patching than merging, so putting a lot of effort into making
merges "work" doesn't seem like a good expenditure of effort.

It seems pretty clear that searching through the histories of each
branch for duplicate commit messages and producing a unified report is
pretty straightforward if we assume that the commit messages are
byte-for-byte identical (or even modulo whitespace changes). But I
wonder if it would make more sense to include some kind of metadata in
the commit message (or some other property of the commit? does git
support that?) to make it not depend on that. I suppose Tom et. al.
like the way they do it now, so maybe we should just stick with text
comparison, but it seems a bit awkward to me.

...Robert

On 6/3/09, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> * Marko Kreen <markokr(at)gmail(dot)com> [090603 10:26]:
> > Thats true, but it's not representable in VCS, unless you use cherry-pick,
> > which is just UI around patch transport. But considering separate
> > local trees (with can optionally contain local per-fix branches),
> > it is possible to separate the fix-developement from final representation.
>
>
> I'll note that in git, cherry-pick is *more* than just "patch
> transport". I would more call it "patch commute". It does actually
> look at the history between the "pick"ed patch, and the current
> tree, any merge/fork points, and the differences on each path that lead
> to the changes in the current tree and the picked patch.

Well, thats good to know, but this also seems to mean it's rather bad
tool for back-patching, as you risk including random unwanted commits
too that happened in the HEAD meantime. But also, it's very good
tool for forward-patching.

But my point was not about that - rather I was pointing out that
this "patch-commute" will result in duplicate commits, that have
no ties in DAG.

--
marko

* Marko Kreen <markokr(at)gmail(dot)com> [090603 11:12]:

> Well, thats good to know, but this also seems to mean it's rather bad
> tool for back-patching, as you risk including random unwanted commits
> too that happened in the HEAD meantime. But also, it's very good
> tool for forward-patching.

It doesn't "pull in commits" in the sense that darcs does... But rather,
its more like "the patch changes $XXX in $file, but that $file was
really $old_file at the common point between the 2 commits, and
$old_file is still $old file in the commit I'm trying to apply the patch
to".

It looks at the history of the changes to figure out why (or why
not) they apply, and see if they should still be applied to the same
file, or another file (in case of a rename/moved file in 1 branch), or
if the changed area has been moved drastically in the file in one
branch, and the change should be applied there instead.

> But my point was not about that - rather I was pointing out that
> this "patch-commute" will result in duplicate commits, that have
> no ties in DAG.

Yes. That's a cherry-pick, if you want a merge, you merge ;-) But
merge carries the baggage of expectation that *all* changes in both
parents have been combined.

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

On 6/3/09, Aidan Van Dyk <aidan(at)highrise(dot)ca> wrote:
> * Marko Kreen <markokr(at)gmail(dot)com> [090603 11:12]:
> > Well, thats good to know, but this also seems to mean it's rather bad
> > tool for back-patching, as you risk including random unwanted commits
> > too that happened in the HEAD meantime. But also, it's very good
> > tool for forward-patching.
>
> It doesn't "pull in commits" in the sense that darcs does... But rather,
> its more like "the patch changes $XXX in $file, but that $file was
> really $old_file at the common point between the 2 commits, and
> $old_file is still $old file in the commit I'm trying to apply the patch
> to".
>
> It looks at the history of the changes to figure out why (or why
> not) they apply, and see if they should still be applied to the same
> file, or another file (in case of a rename/moved file in 1 branch), or
> if the changed area has been moved drastically in the file in one
> branch, and the change should be applied there instead.

I'm not certain, but I remember using cherry pick and seeing
several commits in result. This seems to be a point that needs
to be checked.

> > But my point was not about that - rather I was pointing out that
> > this "patch-commute" will result in duplicate commits, that have
> > no ties in DAG.
>
>
> Yes. That's a cherry-pick, if you want a merge, you merge ;-) But
> merge carries the baggage of expectation that *all* changes in both
> parents have been combined.

But in forward-merge case it's true.

--
marko