Re: [SPAM] Re: GIT mirror not updating

Looks like the GIT mirror at git.postgresql.org hasn't been updated
since Saturday.

?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Tue, Dec 16, 2008 at 4:09 PM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> Looks like the GIT mirror at git.postgresql.org hasn't been updated since
> Saturday.

I upgraded most of postgresql.org on Sunday (yeah, I have no life),
and it seems the new version of git in FreeBSD ports has dropped the
convenience binaries such as git-pull. I've update the update-repos
script to use 'git pull' instead, and rerun it.

If there are other repos, not updated by that script, I'll need more
info on them to go fix.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com

Dave Page wrote:
> On Tue, Dec 16, 2008 at 4:09 PM, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> > Looks like the GIT mirror at git.postgresql.org hasn't been updated since
> > Saturday.
>
> I upgraded most of postgresql.org on Sunday (yeah, I have no life),
> and it seems the new version of git in FreeBSD ports has dropped the
> convenience binaries such as git-pull.

It wasn't FreeBSD's fault -- git upstream did it.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Tuesday 16 December 2008 18:42:43 Dave Page wrote:
> On Tue, Dec 16, 2008 at 4:09 PM, Heikki Linnakangas
>
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> > Looks like the GIT mirror at git.postgresql.org hasn't been updated since
> > Saturday.
>
> I upgraded most of postgresql.org on Sunday (yeah, I have no life),
> and it seems the new version of git in FreeBSD ports has dropped the
> convenience binaries such as git-pull. I've update the update-repos
> script to use 'git pull' instead, and rerun it.
>
> If there are other repos, not updated by that script, I'll need more
> info on them to go fix.

I also needed to upgrade the fromcvs ruby scripts analogously. It's all back
to working now.

Peter Eisentraut wrote:
> I also needed to upgrade the fromcvs ruby scripts analogously. It's all back
> to working now.

Oh wow, did you just make the back-branches work too?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas wrote:
> Peter Eisentraut wrote:
>> I also needed to upgrade the fromcvs ruby scripts analogously. It's
>> all back to working now.
>
> Oh wow, did you just make the back-branches work too?

Hmm, apparently not. But there was this commit:

commit 90bf6f50d2324131b2868ac97563c73aa36a9974
Author: repo-copy <repo-copy>
Date: Tue Dec 16 19:30:44 2008 +0000

Repo copy files

to the REL8_3_STABLE branch, and similar commits to other branches.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas wrote:
> Heikki Linnakangas wrote:
>> Peter Eisentraut wrote:
>>> I also needed to upgrade the fromcvs ruby scripts analogously. It's
>>> all back to working now.
>>
>> Oh wow, did you just make the back-branches work too?
>
> Hmm, apparently not. But there was this commit:
>
> commit 90bf6f50d2324131b2868ac97563c73aa36a9974
> Author: repo-copy <repo-copy>
> Date: Tue Dec 16 19:30:44 2008 +0000
>
> Repo copy files
>
> to the REL8_3_STABLE branch, and similar commits to other branches.

Something is wrong with the repository. There seems to be multiple
copies of all commits are in master branch. For example:

$ git-log --grep="Do not try to change" origin/master
commit be4e5c7b547c41d01e7c74dbb0fd805c6b3b6a8f
Author: Michael Meskes <meskes(at)postgresql(dot)org>
Date: Mon Dec 15 15:34:07 2008 +0000

Do not try to change a const variable.

commit 09fce45b965adffa885f5bf9271d74c255e51202
Author: Michael Meskes <meskes(at)postgresql(dot)org>
Date: Mon Dec 15 15:34:07 2008 +0000

Do not try to change a const variable.

commit e2da9d8a752101d0b2d4c0ff4e82c9a2e82e4533
Author: Michael Meskes <meskes(at)postgresql(dot)org>
Date: Mon Dec 15 15:34:07 2008 +0000

Do not try to change a const variable.

Perhaps now is a good time to give up on these scripts, rebuild the
repository using git-cvsimport?

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Wed, 17 Dec 2008, Heikki Linnakangas wrote:

> Perhaps now is a good time to give up on these scripts, rebuild the
> repository using git-cvsimport?
>

My experience with git-cvsimport has been poor. For me it often puts
commits on the wrong branches. It uses the first commit to the branch as
the branch start time, so a branch will show commits to head as being on
the branch until the first commit to the branch.

The only cvs->git converter that's really worked for me (does all
branches, tags, keyword removal, ...) has been parsecvs [1] which isn't
terribly user friendly and is not incremental.

[1] http://git.or.cz/gitwiki/InterfacesFrontendsAndTools#head-769e9aa96370a5b3d0356c918f310c177c0799fe

Kris Jurka

Kris Jurka wrote:
>
>
> On Wed, 17 Dec 2008, Heikki Linnakangas wrote:
>
>> Perhaps now is a good time to give up on these scripts, rebuild the
>> repository using git-cvsimport?
>>
>
> My experience with git-cvsimport has been poor. For me it often puts
> commits on the wrong branches. It uses the first commit to the branch
> as the branch start time, so a branch will show commits to head as being
> on the branch until the first commit to the branch.

I just tried running git-cvsimport on the PG repository, and I can see
what you mean. You're right, that's not acceptable.

> The only cvs->git converter that's really worked for me (does all
> branches, tags, keyword removal, ...) has been parsecvs [1] which isn't
> terribly user friendly and is not incremental.
>
> [1]
> http://git.or.cz/gitwiki/InterfacesFrontendsAndTools#head-769e9aa96370a5b3d0356c918f310c177c0799fe

"Not incremental" rules that one out, unfortunately.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas wrote:
> Kris Jurka wrote:
>> On Wed, 17 Dec 2008, Heikki Linnakangas wrote:
>>
>>> Perhaps now is a good time to give up on these scripts, rebuild the
>>> repository using git-cvsimport?
>>
>> My experience with git-cvsimport has been poor. For me it often puts
>> commits on the wrong branches. It uses the first commit to the branch
>> as the branch start time, so a branch will show commits to head as
>> being on the branch until the first commit to the branch.
>
> I just tried running git-cvsimport on the PG repository, and I can see
> what you mean. You're right, that's not acceptable.

I hacked cvsps a little bit, to move the first commit on the branch
right after the correct branch point, before writing the output. This is
a hack, but seems to work at least for the PG repository.

I'm not sure if we want to run a hacked version of cvsps. But then
again, the current conversion isn't exactly error-free either, and we're
missing tags...

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote:
> I hacked cvsps a little bit, to move the first commit on the branch
> right after the correct branch point, before writing the output. This is
> a hack, but seems to work at least for the PG repository.
>
> I'm not sure if we want to run a hacked version of cvsps. But then
> again, the current conversion isn't exactly error-free either, and we're
> missing tags...

Do you think this hack (or at least a problem report) would be accepted
upstream by the git community?

Regards,
Jeff Davis

On Thursday 25 December 2008 01:47:49 Jeff Davis wrote:
> On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote:
> > I hacked cvsps a little bit, to move the first commit on the branch
> > right after the correct branch point, before writing the output. This is
> > a hack, but seems to work at least for the PG repository.
> >
> > I'm not sure if we want to run a hacked version of cvsps. But then
> > again, the current conversion isn't exactly error-free either, and we're
> > missing tags...
>
> Do you think this hack (or at least a problem report) would be accepted
> upstream by the git community?

cvsps is not part of git. It's a separate package, that has been unmaintained
for a great while, and there are already some forks out there. Which is part
of the problem, overall ...

Peter Eisentraut wrote:
> On Thursday 25 December 2008 01:47:49 Jeff Davis wrote:
>> On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote:
>>> I hacked cvsps a little bit, to move the first commit on the branch
>>> right after the correct branch point, before writing the output. This is
>>> a hack, but seems to work at least for the PG repository.
>>>
>>> I'm not sure if we want to run a hacked version of cvsps. But then
>>> again, the current conversion isn't exactly error-free either, and we're
>>> missing tags...
>> Do you think this hack (or at least a problem report) would be accepted
>> upstream by the git community?
>
> cvsps is not part of git. It's a separate package, that has been unmaintained
> for a great while, and there are already some forks out there. Which is part
> of the problem, overall ...

Yep. The git people are certainly aware of the problem, there's plenty
of discussion about it on the mailing lists. I don't know if this
particular hack has been proposed before, I don't think I saw anything
identical in the mailing lists, though people have posted cvsps hacks to
fix it before.

Another problem with git-cvsimport is that only branches that have had
any commits to them show up in the GIT repository. This hack won't fix
that. If cvsps injected a dummy "branch commit" at the right place,
instead of the reordering that my hack does, that might be fixed too.

What are we going to do about this? I have the feeling that I have the
greatest itch to get the repository fixed. I propose that I will try
that dummy "branch commit" hack in cvsps, and if it works out, create a
new GIT repository using that. That would replace the current
repository, which means that people working against that repository will
need to use "git-fetch --force" to update their clones, and rebase their
own branches. It would also be good to contribute the hack into upstream
cvsps (whatever fork is considered upstream).

I'd like to move to git-cvsimport because:
- it converts tags
- I'm more familiar with it than fromcvs/togit.

However, if anyone else is willing to pick up the ball, I'll just shut
up and be happy that something gets done.

One option is to just leave it as it is, but it's not nice that the
history is repeated multiple times. The repository is big enough without
that..

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

On Monday 29 December 2008 11:40:41 Heikki Linnakangas wrote:
> What are we going to do about this? I have the feeling that I have the
> greatest itch to get the repository fixed. I propose that I will try
> that dummy "branch commit" hack in cvsps, and if it works out, create a
> new GIT repository using that. That would replace the current
> repository, which means that people working against that repository will
> need to use "git-fetch --force" to update their clones, and rebase their
> own branches. It would also be good to contribute the hack into upstream
> cvsps (whatever fork is considered upstream).

Does your version of cvsps not run into infinite loops on our CVS repository?
This is one of the main reasons we are not using it.

Heikki Linnakangas wrote:
> Peter Eisentraut wrote:
>> On Thursday 25 December 2008 01:47:49 Jeff Davis wrote:
>>> On Mon, 2008-12-22 at 19:40 +0200, Heikki Linnakangas wrote:
>>>> I hacked cvsps a little bit, to move the first commit on the branch
>>>> right after the correct branch point, before writing the output.
>>>> This is
>>>> a hack, but seems to work at least for the PG repository.
>>>>
>>>> I'm not sure if we want to run a hacked version of cvsps. But then
>>>> again, the current conversion isn't exactly error-free either, and
>>>> we're
>>>> missing tags...
>>> Do you think this hack (or at least a problem report) would be accepted
>>> upstream by the git community?
>>
>> cvsps is not part of git. It's a separate package, that has been
>> unmaintained for a great while, and there are already some forks out
>> there. Which is part of the problem, overall ...
>
> Yep. The git people are certainly aware of the problem, there's plenty
> of discussion about it on the mailing lists. I don't know if this
> particular hack has been proposed before, I don't think I saw anything
> identical in the mailing lists, though people have posted cvsps hacks to
> fix it before.
>
> Another problem with git-cvsimport is that only branches that have had
> any commits to them show up in the GIT repository. This hack won't fix
> that. If cvsps injected a dummy "branch commit" at the right place,
> instead of the reordering that my hack does, that might be fixed too.

Do we *have* any branches that have had no commits? And in this case, do
we *care* that they don't show up? I assume they will show up as soon as
we get the first commit?

> What are we going to do about this? I have the feeling that I have the
> greatest itch to get the repository fixed. I propose that I will try
> that dummy "branch commit" hack in cvsps, and if it works out, create a
> new GIT repository using that. That would replace the current
> repository, which means that people working against that repository will
> need to use "git-fetch --force" to update their clones, and rebase their
> own branches. It would also be good to contribute the hack into upstream
> cvsps (whatever fork is considered upstream).

Sounds like a reasonable plan to me. We should be careful to test out
all parts before we do it, though, so users don't have to rebase more
than once!

> I'd like to move to git-cvsimport because:
> - it converts tags
> - I'm more familiar with it than fromcvs/togit.

Can't comment on this.

> However, if anyone else is willing to pick up the ball, I'll just shut
> up and be happy that something gets done.
>
> One option is to just leave it as it is, but it's not nice that the
> history is repeated multiple times. The repository is big enough without
> that..

I'd really like to see us do something about it. I don't think just
leaving it as-is is a good solution.

//Magnus

Peter Eisentraut wrote:
> Does your version of cvsps not run into infinite loops on our CVS repository?
> This is one of the main reasons we are not using it.

Ah, no, I worked around that by removing REL7_1_BETA tags before the
conversion. Sorry, forgot to mention. I haven't investigated why those
are causing problems, because I don't care about the 7.1 beta tags much.

So the procedure is to:

1. rsync PostgreSQL main repository.
2. Remove offending tags:

cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql
cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql
cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql

3. Run git-cvsimport.

BTW, if we ever convert to git altogether as a project, we'd probably
use something else again for the conversion. Many people recommend
cvs2git on the git mailing lists. We can't use it for the mirror because
it doesn't support incremental updates, but for a one-off conversion
it's probably the best option.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Magnus Hagander wrote:
> Heikki Linnakangas wrote:
>> Another problem with git-cvsimport is that only branches that have had
>> any commits to them show up in the GIT repository. This hack won't fix
>> that. If cvsps injected a dummy "branch commit" at the right place,
>> instead of the reordering that my hack does, that might be fixed too.
>
> Do we *have* any branches that have had no commits?

No, but we will as soon as we branch again.

> And in this case, do
> we *care* that they don't show up? I assume they will show up as soon as
> we get the first commit?

I guess we could live with them not showing up until first commit.

>> What are we going to do about this? I have the feeling that I have the
>> greatest itch to get the repository fixed. I propose that I will try
>> that dummy "branch commit" hack in cvsps, and if it works out, create a
>> new GIT repository using that. That would replace the current
>> repository, which means that people working against that repository will
>> need to use "git-fetch --force" to update their clones, and rebase their
>> own branches. It would also be good to contribute the hack into upstream
>> cvsps (whatever fork is considered upstream).
>
> Sounds like a reasonable plan to me. We should be careful to test out
> all parts before we do it, though, so users don't have to rebase more
> than once!

Yep. Need to test what happens when new branches are created, at least.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas wrote:

> So the procedure is to:
>
> 1. rsync PostgreSQL main repository.
> 2. Remove offending tags:
>
> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql
> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql
> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql

Can we permanently hack our repo? It doesn't seem like those tags have
a lot of value, after all. (If somebody thinks they do, perhaps we can
re-tag.)

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

On Mon, Dec 29, 2008 at 01:14:19PM +0200, Heikki Linnakangas wrote:
> Peter Eisentraut wrote:
>
> BTW, if we ever convert to git altogether as a project,

Say, around the time we branch to 8.5? :)

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Alvaro Herrera wrote:
> Heikki Linnakangas wrote:
>
>> So the procedure is to:
>>
>> 1. rsync PostgreSQL main repository.
>> 2. Remove offending tags:
>>
>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql
>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql
>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql
>
> Can we permanently hack our repo? It doesn't seem like those tags have
> a lot of value, after all. (If somebody thinks they do, perhaps we can
> re-tag.)

I don't know. It's not a big deal, either way.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas wrote:
> Alvaro Herrera wrote:
>> Heikki Linnakangas wrote:
>>
>>> So the procedure is to:
>>>
>>> 1. rsync PostgreSQL main repository.
>>> 2. Remove offending tags:
>>>
>>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql
>>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql
>>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql
>>
>> Can we permanently hack our repo? It doesn't seem like those tags have
>> a lot of value, after all. (If somebody thinks they do, perhaps we can
>> re-tag.)
>
> I don't know. It's not a big deal, either way.

Or if we can figure out *what* is wrong with them, perhaps we can fix
it? it is some level of historical information that we'd loose, after
all. Granted not very important historical information, but if it's
fixable there's no reason not to do that instead.

//Magnus

On Monday 29 December 2008 15:25:47 Magnus Hagander wrote:
> Heikki Linnakangas wrote:
> > Alvaro Herrera wrote:
> >> Heikki Linnakangas wrote:
> >>> So the procedure is to:
> >>>
> >>> 1. rsync PostgreSQL main repository.
> >>> 2. Remove offending tags:
> >>>
> >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA2 pgsql
> >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA3 pgsql
> >>> cvs -d ~/pgcvsrepository-hacked/ rtag -d REL7_1_BETA pgsql
> >>
> >> Can we permanently hack our repo? It doesn't seem like those tags have
> >> a lot of value, after all. (If somebody thinks they do, perhaps we can
> >> re-tag.)
> >
> > I don't know. It's not a big deal, either way.
>
> Or if we can figure out *what* is wrong with them, perhaps we can fix
> it? it is some level of historical information that we'd loose, after
> all. Granted not very important historical information, but if it's
> fixable there's no reason not to do that instead.

I analyzed this back in April; my analysis is in the archives. The short
version is that it is probably a bug in cvsps.

See also my message to hackers just now about a repository fixup that we *can*
do.

On Mon, Dec 29, 2008 at 3:14 AM, Heikki Linnakangas
<heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
> Peter Eisentraut wrote:
> BTW, if we ever convert to git altogether as a project, we'd probably use
> something else again for the conversion. Many people recommend cvs2git on
> the git mailing lists. We can't use it for the mirror because it doesn't
> support incremental updates, but for a one-off conversion it's probably the
> best option.
>

Do you have any sentiments on how good git-cvsserver is?

Synopsis: basically git can run as a CVS server, allowing for both
read and write access, in addition to being a git repository. Some of
the more advanced functionality (tagging, branching) may have to be
done with a git client, though.

By its own description in the man pages, git cvsserver is 'highly
functional'. I don't know what that means, but since nobody is trying
to sell me anything it seems like it could be true.

One interesting exercise would be to run git with cvsserver and try
porting commits via script from Postgres CVS to git cvsserver. If that
worked convincingly for a while, perhaps the committers wouldn't mind
switching the CVS backend since daily life would not have to change
drastically at any one point in time.

fdr

On Monday 29 December 2008 19:09:23 Daniel Farina wrote:
> One interesting exercise would be to run git with cvsserver and try
> porting commits via script from Postgres CVS to git cvsserver.

But to do that you would be back at cvsps to extract the actual commits from
the CVS repository, which is a major part of the current problem in the first
place. And by the time you're done, you'd likely have re-implemented
git-cvsimport with an extra level of indirection.

On Mon, Dec 29, 2008 at 10:05 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On Monday 29 December 2008 19:09:23 Daniel Farina wrote:
>> One interesting exercise would be to run git with cvsserver and try
>> porting commits via script from Postgres CVS to git cvsserver.
>
> But to do that you would be back at cvsps to extract the actual commits from
> the CVS repository, which is a major part of the current problem in the first
> place. And by the time you're done, you'd likely have re-implemented
> git-cvsimport with an extra level of indirection.
>

This is true as long as one is using some extra machinery to convert from
vanilla CVS to Git (and thus served through cvsserver). The goal would be to
shed those eventually and have CVS users just commit to Git's cvsserver. I only
suggested exercising through cvsserver to make sure it works 'as advertised,'
for Postgres' purposes, making a convincing argument that anyone who wanted to
operate via CVS for a while would not be adversely affected.

There's also a benefit for the committers who would *like* to use Git being
able to take advantage of its features in that scenario. As-is just fixing
cvsps (and cvsimport) doesn't provide an incremental strategy for switching to
Git for the purposes of writes to the repository.

fdr

Daniel Farina wrote:
> There's also a benefit for the committers who would *like* to use Git being
> able to take advantage of its features in that scenario. As-is just fixing
> cvsps (and cvsimport) doesn't provide an incremental strategy for switching to
> Git for the purposes of writes to the repository.

That's hardly worth worrying about. There's only a handful of active
committers. Coordinating a big-bang switch wouldn't be hard after the
decision to convert was made.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
> Daniel Farina wrote:
>> There's also a benefit for the committers who would *like* to use Git being
>> able to take advantage of its features in that scenario. As-is just fixing
>> cvsps (and cvsimport) doesn't provide an incremental strategy for switching to
>> Git for the purposes of writes to the repository.

> That's hardly worth worrying about. There's only a handful of active
> committers. Coordinating a big-bang switch wouldn't be hard after the
> decision to convert was made.

The problem is to convince those active committers that they want to switch.

regards, tom lane

On Mon, 2008-12-29 at 20:05 +0200, Peter Eisentraut wrote:
> On Monday 29 December 2008 19:09:23 Daniel Farina wrote:
> > One interesting exercise would be to run git with cvsserver and try
> > porting commits via script from Postgres CVS to git cvsserver.
>
> But to do that you would be back at cvsps to extract the actual commits from
> the CVS repository, which is a major part of the current problem in the first
> place. And by the time you're done, you'd likely have re-implemented
> git-cvsimport with an extra level of indirection.

How well does CVS -> SVN work? Maybe we could do CVS -> SVN -> Git.

CommandPrompt's SVN repo seems like it's in good shape (has tags, etc.),
so maybe this is possible?

Regards,
Jeff Davis

On Mon, 2008-12-29 at 11:31 -0800, Jeff Davis wrote:
> On Mon, 2008-12-29 at 20:05 +0200, Peter Eisentraut wrote:
> > On Monday 29 December 2008 19:09:23 Daniel Farina wrote:
> > > One interesting exercise would be to run git with cvsserver and try
> > > porting commits via script from Postgres CVS to git cvsserver.
> >
> > But to do that you would be back at cvsps to extract the actual commits from
> > the CVS repository, which is a major part of the current problem in the first
> > place. And by the time you're done, you'd likely have re-implemented
> > git-cvsimport with an extra level of indirection.
>
> How well does CVS -> SVN work? Maybe we could do CVS -> SVN -> Git.
>
> CommandPrompt's SVN repo seems like it's in good shape (has tags, etc.),
> so maybe this is possible?

CVS to SVN is easy and accurate but slow. There is a bug in our
transformation script (which has nothing to do with the cvs-svn part)
but that is why ours is out of date.

I have been meaning to look at tailor to see if it is possible to
convert single commits. Right now, we have to transform the entire repo
everytime we run.

Joshua D. Drake

--
PostgreSQL
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

On Mon, 2008-12-29 at 11:33 -0800, Joshua D. Drake wrote:
> I have been meaning to look at tailor to see if it is possible to
> convert single commits. Right now, we have to transform the entire repo
> everytime we run.
>

Oh, nevermind. Incremental updates were already stated as a requirement
elsewhere in this thread.

Regards,
Jeff Davis

On Mon, 2008-12-29 at 11:41 -0800, Jeff Davis wrote:
> On Mon, 2008-12-29 at 11:33 -0800, Joshua D. Drake wrote:
> > I have been meaning to look at tailor to see if it is possible to
> > convert single commits. Right now, we have to transform the entire repo
> > everytime we run.
> >
>
> Oh, nevermind. Incremental updates were already stated as a requirement
> elsewhere in this thread.

Right that is why I have been meaning to look at Tailor. Supposedly it
can do it proper.

http://progetti.arstecnica.it/tailor/wiki/

Joshua D. Drake

>
> Regards,
> Jeff Davis
>
>
--
PostgreSQL
Consulting, Development, Support, Training
503-667-4564 - http://www.commandprompt.com/
The PostgreSQL Company, serving since 1997

-----BEGIN PGP SIGNED MESSAGE-----
Hash: RIPEMD160

> Right that is why I have been meaning to look at Tailor. Supposedly it
> can do it proper.

FWIW, I looked into Tailor some time ago (> 6 months), as well as other conversion
tools, including all the ones mentioned in this thread, but none of
them worked well, and those that sorta worked (like our current ruby-based one)
were *incredibly* slow. I seem to recall it takes > 12 hours for the initial
import. I really like the idea of removing those old no-longer-needed tags
that were causing some of the problems, and then seeing if we can find
a fast, reliable conversion tool. The git community is very helpful as well,
we should feel free to enlist their help. By "we" I mean someone else, as
I'm on vacation from tomorrow until the 7th. ;)

- --
Greg Sabino Mullane greg(at)turnstep(dot)com
PGP Key: 0x14964AC8 200812291833
http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8
-----BEGIN PGP SIGNATURE-----

iEYEAREDAAYFAklZXqIACgkQvJuQZxSWSsgiTgCfbt/9eJKbx6FJt0nJPBjgzgYt
2pgAn1ANRPolreroAltsEK4FosYjpNiq
=qrQL
-----END PGP SIGNATURE-----

Heikki Linnakangas wrote:
> I propose that I will try
> that dummy "branch commit" hack in cvsps, and if it works out, create a
> new GIT repository using that. That would replace the current
> repository, which means that people working against that repository will
> need to use "git-fetch --force" to update their clones, and rebase their
> own branches. It would also be good to contribute the hack into upstream
> cvsps (whatever fork is considered upstream).
>
> I'd like to move to git-cvsimport because:
> - it converts tags
> - I'm more familiar with it than fromcvs/togit.

I tried that, but gave up after a while. I hacked cvsps to create the
dummy commits, but it choked on some files that exist some branches but
not others. It might be fixable, but I'm now convinced that
fromcvs/togit is the simplest solution after all. The lack of tags is
slightly annoying, but not a show-stopper. And we can add them manually
if we really want them, we don't tag that often.

togit normally looks at all the branches, and retrieves all commits from
CVS with date later than the latest commit on any branch in the GIT
repository. I think that's why the back-branches stopped updating after
there was even a single commit on master with later date. I modified it
so that each branch is treated separately. I'm not sure how we ended up
with duplicated history, though.

I have a recipe to fix the repository, and we don't even need to
recreate it from scratch. I'd appreciate that, because I have some
pretty complex stuff in my GIT repository with merges and everything
that could be a lot of work to rebase. This fixes the back-branches, too.

0. Backup

1. Apply the attached patch to togit.

2. Use git-reset to rewind the branches back to the state they were
before breakage. Attached rewind-branches.sh script does that for all
the branches.

3. Run togit.

After that, pulling from a repository that was last updated before Dec
13th (when it was screwed up), you get a nice fast-forward as usual. If
you have already pulled the duplicated history, you have to use
"git-pull --force" or git-pull will refuse to update because "history
has changed".

It should be safe, but not necessaery, to always use the modified togit
script. After fixing the repository, incremental updates should work
with unmodified togit as well, I think. The patched version is slower,
because it skips already-applied commits at a later stage, but most of
the time is spent in parsing the RCS logs anyway, so it might not be
important.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com