Re: Git conversion progress report and call for testing assistance

Here's a status update on the git conversion, as well as a call for some help
mainly in testing.

After testing a bunch of tools, I've found that using cvs2git is by far the
best option when keeping keywords. It's the one that gives only the issues
that I posted about a couple of days ago.

So I've proceeded based off this one to something that would be how we create
the real repository once we go. This means I've scripted the removal of the
$PostgreSQL$ tags from the tip of the active branches as one big commit after
the migration.

I've also set up the git server and the scripts around it, that we can
eventually use. This includes commit email sending, commit policy enforcement
(no merge commits, correct author/committer tag etc) and proper access control
(a modified version of the one on git.postgresql.org - since we definitely
don't want any external dependencies for the main repository).

This is all available for testing now.

Marc has set up a mailinglist at pgsql-committers-test(at)postgresql(dot)org where
commit messages from the new system is sent. If you care about what they look
like, subscribe there and wait for one to show up :-) Subscription is done
the usual way.

Anonymous users can view the repository at git.postgresql.org using gitweb
or the git:// protocol, under the name postgresql-migration.
DISCLAIMER: DO NOT BASE ANY WORK OFF THIS REPOSITORY. IT *WILL* BE RECREATED
SEVERAL TIMES AND MAY CHANGE COMPLETELY!

Existing committers have been set up to access the new repository at
ssh://git(at)gitmaster(dot)postgresql(dot)org/postgresql.git.
Robert Haas has written some instructoins for how to use this - please read
and review those.

And in general, a call to committers: please test this! Now is the time, not
after we've migrated ;) Just throw in some random commits, and some non-random
ones, both to get yourself familiar with the workflow and to iron out the bugs
in the scripts (I'm sure they're there).

For those interested in what's done, the scripts running this are all up on
github at http://github.com/mhagander/pg_githooks. The root contains the
scripts for commit messages, policy enforcement and access control.

There's also a temporary directory called "migration" that contains the
scripts and configuration files that are in use for the version of the
repository that is up there now. There's some minor plumbing around these
that isn't up there yet, but in general it is all that's used. And note that
if you want to play with it, the script uses around 8Gb of temp disk space
when running, so make sure you have enough space if you do it in a VM...

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

On 21/07/10 23:40, Magnus Hagander wrote:
> I've also set up the git server and the scripts around it, that we can
> eventually use. This includes commit email sending, commit policy enforcement
> (no merge commits, correct author/committer tag etc) and proper access control
> (a modified version of the one on git.postgresql.org - since we definitely
> don't want any external dependencies for the main repository).
>
> This is all available for testing now.

Seems to work great. I've got a local clone of that set up now, with
workdirs for backbranches (per the "Committing Using a Single Clone and
multiple workdirs" instructions I added to the Committing_with_Git wiki
page).

I also tested with a bunch of funny commits, including merge commits,
non-fast-forward commits and a commit with bogus Committer, and the
post-commit hook correctly rejected all those.

> Marc has set up a mailinglist at pgsql-committers-test(at)postgresql(dot)org where
> commit messages from the new system is sent. If you care about what they look
> like, subscribe there and wait for one to show up :-) Subscription is done
> the usual way.

My posts to that lists are been stalled, but looking at the commit mail
in the "stalled post" reply it looks OK.

Good work! Thanks for spending the time on this, I very much prefer
working with git nowadays.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Wed, Aug 11, 2010 at 23:10, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> On 21/07/10 23:40, Magnus Hagander wrote:
>>
>> I've also set up the git server and the scripts around it, that we can
>> eventually use. This includes commit email sending, commit policy
>> enforcement
>> (no merge commits, correct author/committer tag etc) and proper access
>> control
>> (a modified version of the one on git.postgresql.org - since we definitely
>> don't want any external dependencies for the main repository).
>>
>> This is all available for testing now.
>
> Seems to work great. I've got a local clone of that set up now, with
> workdirs for backbranches (per the "Committing Using a Single Clone and
> multiple workdirs" instructions I added to the Committing_with_Git wiki
> page).
>
> I also tested with a bunch of funny commits, including merge commits,
> non-fast-forward commits and a commit with bogus Committer, and the
> post-commit hook correctly rejected all those.

Great, thanks for testing that. I assume this means that the commits
that *should* go through did so? ;)

>> Marc has set up a mailinglist at pgsql-committers-test(at)postgresql(dot)org
>> where
>> commit messages from the new system is sent. If you care about what they
>> look
>> like, subscribe there and wait for one to show up :-) Subscription is done
>> the usual way.
>
> My posts to that lists are been stalled, but looking at the commit mail in
> the "stalled post" reply it looks OK.

Hmm. Were you subscribed to the list? With the same address as your
commits are coming in from?

If not, that would explain it. If you did, I need to figure out why ;)

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> ... This is all available for testing now.

> Marc has set up a mailinglist at pgsql-committers-test(at)postgresql(dot)org where
> commit messages from the new system is sent. If you care about what they look
> like, subscribe there and wait for one to show up :-) Subscription is done
> the usual way.

Hm, is the pgsql-committers-test thing actually working? I did a test
push to ssh://git(at)gitmaster(dot)postgresql(dot)org/postgresql.git, and I haven't
seen any resulting email.

regards, tom lane

On Fri, Aug 27, 2010 at 20:29, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> ... This is all available for testing now.
>
>> Marc has set up a mailinglist at pgsql-committers-test(at)postgresql(dot)org where
>> commit messages from the new system is sent. If you care about what they look
>> like, subscribe there and wait for one to show up :-) Subscription is done
>> the usual way.
>
> Hm, is the pgsql-committers-test thing actually working?  I did a test
> push to ssh://git(at)gitmaster(dot)postgresql(dot)org/postgresql.git, and I haven't
> seen any resulting email.

Um, it seems the list is broken somehow. I see an attempt to deliver
them, but then:

2010-08-27 18:04:57 1Op3IN-00084v-KZ **
pgsql-committers-test(at)postgresql(dot)org R=dnslookup T=remote_smtp: SMTP
error from remote mail server after RCPT
TO:<pgsql-committers-test(at)postgresql(dot)org>: host mx1.hub.org
[200.46.208.106]: 550 5.1.1 <pgsql-committers-test(at)postgresql(dot)org>:
Recipient address rejected: User unknown in relay recipient table

Marc?

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

looking into it ...

On Fri, 27 Aug 2010, Magnus Hagander wrote:

> On Fri, Aug 27, 2010 at 20:29, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> ... This is all available for testing now.
>>
>>> Marc has set up a mailinglist at pgsql-committers-test(at)postgresql(dot)org where
>>> commit messages from the new system is sent. If you care about what they look
>>> like, subscribe there and wait for one to show up :-) Subscription is done
>>> the usual way.
>>
>> Hm, is the pgsql-committers-test thing actually working?  I did a test
>> push to ssh://git(at)gitmaster(dot)postgresql(dot)org/postgresql.git, and I haven't
>> seen any resulting email.
>
> Um, it seems the list is broken somehow. I see an attempt to deliver
> them, but then:
>
> 2010-08-27 18:04:57 1Op3IN-00084v-KZ **
> pgsql-committers-test(at)postgresql(dot)org R=dnslookup T=remote_smtp: SMTP
> error from remote mail server after RCPT
> TO:<pgsql-committers-test(at)postgresql(dot)org>: host mx1.hub.org
> [200.46.208.106]: 550 5.1.1 <pgsql-committers-test(at)postgresql(dot)org>:
> Recipient address rejected: User unknown in relay recipient table
>
>
> Marc?
>
>
> --
>  Magnus Hagander
>  Me: http://www.hagander.net/
>  Work: http://www.redpill-linpro.com/
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

----
Marc G. Fournier Hub.Org Hosting Solutions S.A.
scrappy(at)hub(dot)org http://www.hub.org

Yahoo:yscrappy Skype: hub.org ICQ:7615664 MSN:scrappy(at)hub(dot)org

should be fixed ...

On Fri, 27 Aug 2010, Magnus Hagander wrote:

> On Fri, Aug 27, 2010 at 20:29, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> ... This is all available for testing now.
>>
>>> Marc has set up a mailinglist at pgsql-committers-test(at)postgresql(dot)org where
>>> commit messages from the new system is sent. If you care about what they look
>>> like, subscribe there and wait for one to show up :-) Subscription is done
>>> the usual way.
>>
>> Hm, is the pgsql-committers-test thing actually working?  I did a test
>> push to ssh://git(at)gitmaster(dot)postgresql(dot)org/postgresql.git, and I haven't
>> seen any resulting email.
>
> Um, it seems the list is broken somehow. I see an attempt to deliver
> them, but then:
>
> 2010-08-27 18:04:57 1Op3IN-00084v-KZ **
> pgsql-committers-test(at)postgresql(dot)org R=dnslookup T=remote_smtp: SMTP
> error from remote mail server after RCPT
> TO:<pgsql-committers-test(at)postgresql(dot)org>: host mx1.hub.org
> [200.46.208.106]: 550 5.1.1 <pgsql-committers-test(at)postgresql(dot)org>:
> Recipient address rejected: User unknown in relay recipient table
>
>
> Marc?
>
>
> --
>  Magnus Hagander
>  Me: http://www.hagander.net/
>  Work: http://www.redpill-linpro.com/
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

----
Marc G. Fournier Hub.Org Hosting Solutions S.A.
scrappy(at)hub(dot)org http://www.hub.org

Yahoo:yscrappy Skype: hub.org ICQ:7615664 MSN:scrappy(at)hub(dot)org

"Marc G. Fournier" <scrappy(at)hub(dot)org> writes:
> should be fixed ...

Ah, now I got something. Thanks.

regards, tom lane

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Fri, Aug 27, 2010 at 20:29, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> Hm, is the pgsql-committers-test thing actually working?

> Um, it seems the list is broken somehow.

OK, the list works now, but the commitdiff links in the messages do
not. Looks like they are pointing at the wrong repository.

regards, tom lane

On Fri, Aug 27, 2010 at 21:30, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Fri, Aug 27, 2010 at 20:29, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> Hm, is the pgsql-committers-test thing actually working?
>
>> Um, it seems the list is broken somehow.
>
> OK, the list works now, but the commitdiff links in the messages do
> not.  Looks like they are pointing at the wrong repository.

Well, actually, they point to the right one, but the mirroring off to
that one is turned off. There are just too many different versions
right now to keep that thing pointing the right way :-)

That part has been verified and is working, so I didn't bother keeping
it up to date.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Fri, Aug 27, 2010 at 21:30, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> OK, the list works now, but the commitdiff links in the messages do
>> not. Looks like they are pointing at the wrong repository.

> Well, actually, they point to the right one, but the mirroring off to
> that one is turned off. There are just too many different versions
> right now to keep that thing pointing the right way :-)

Oh, OK.

In any case, you asked for feedback on the message format, so here is
some: I don't like having the commitdiff link first. I think the log
message is the most important thing and should be first. Possible
format (based on the example I just received):

Log Message
-----------
Another experimental commit.
Undo hacking other people previously did on README.

Branch
------
REL9_0_STABLE

Details
-------
http://git.postgresql.org/gitweb?p=postgresql-migration.git;a=commitdiff;h=1add27be08978f20edb67ca5854f6701f0f6ea86

Summary
-------
README | 4 ----
1 files changed, 0 insertions(+), 4 deletions(-)

You could argue it either way about whether to put the commitdiff link
before or after the change summary, but offhand I think before will be
more useful. If there are a lot of files touched, people won't want
to have to scroll to the bottom to find the link.

regards, tom lane

On Fri, Aug 27, 2010 at 22:02, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> On Fri, Aug 27, 2010 at 21:30, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>> OK, the list works now, but the commitdiff links in the messages do
>>> not.  Looks like they are pointing at the wrong repository.
>
>> Well, actually, they point to the right one, but the mirroring off to
>> that one is turned off. There are just too many different versions
>> right now to keep that thing pointing the right way :-)
>
> Oh, OK.
>
> In any case, you asked for feedback on the message format, so here is
> some: I don't like having the commitdiff link first.  I think the log
> message is the most important thing and should be first.  Possible
> format (based on the example I just received):

Ok. I see the point of that - particularly for MUAs that show a short
preview and don't filter the link out of it.

Should we consider actually removing the Log Message header and just
put the message right at the start? If there's no link first, that
would be fairly obvious, I think..

> Log Message
> -----------
> Another experimental commit.
> Undo hacking other people previously did on README.
>
> Branch
> ------
> REL9_0_STABLE
>
> Details
> -------
> http://git.postgresql.org/gitweb?p=postgresql-migration.git;a=commitdiff;h=1add27be08978f20edb67ca5854f6701f0f6ea86
>
> Summary
> -------
> README |    4 ----
> 1 files changed, 0 insertions(+), 4 deletions(-)
>
>
> You could argue it either way about whether to put the commitdiff link
> before or after the change summary, but offhand I think before will be
> more useful.  If there are a lot of files touched, people won't want
> to have to scroll to the bottom to find the link.

I definitely think it should stay before - that way it will almost
always stay "above the fold" for readers. Keeps it very easy for those
who prefer to just click that link to get the full diff right away.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> Should we consider actually removing the Log Message header and just
> put the message right at the start? If there's no link first, that
> would be fairly obvious, I think..

Either way (header or not) is OK by me.

regards, tom lane

On Sat, Aug 28, 2010 at 16:30, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>> Should we consider actually removing the Log Message header and just
>> put the message right at the start? If there's no link first, that
>> would be fairly obvious, I think..
>
> Either way (header or not) is OK by me.

Updated, with the removal of the header to help MUA previews.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/