PostgreSQL 8.4 development plan

Hackers,

As you know we've finally released PostgreSQL 8.3, after a development
cycle that lasted well over a year despite our original plans for a 6
month cycle. The core team are aware that there are a number of
factors that contributed to this slippage:

- Lack of prompt and early review of patches.
- A significant rise in the number and complexity of patches submitted.
- Prioritising completion of incomplete patches over meeting the timetable.

In the 8.4 development cycle we would like to try a new style of
development, designed to keep the patch queue to a limited size and to
provide timely feedback to developers on the work they submit. To do
this we will replace the traditional 'feature freeze' with a series of
'commit fests' throughout the development cycle. The idea of commit
fests was discussed last October in -hackers, and it seemed to meet
with general approval. Whenever a commit fest is in progress, the
focus will shift from development to review, feedback and commit of
patches. Each fest will continue until all patches in the queue have
either been committed to the CVS repository, returned to the author
for additional work, or rejected outright, and until that has
happened, no new patches will be considered. Of course, individual
developers are free to continue working on their
patches throughout the fest, but we encourage everyone to do what they
can to help work through the patch queue. We feel that this idea can
only be successful if the whole development community is willing to
focus on patch review during the commit fests, in the same way that
everyone is expected to focus on testing during beta period.

The proposed timetable for the cycle is as follows:

1st March 2008 - commit fest begins
1st May 2008 - commit fest begins
1st July 2008 - commit fest begins
1st September 2008 - commit fest begins
1st November 2008 - final commit fest begins
1st January 2009 - beta 1
1st March 2009 - 8.4.0 release

Note the lack of any 'feature freeze' date as such. However, any
significant feature patches not submitted by 1st November will clearly
not make it into 8.4.

The hope here is that we will not have enormous, previously unreviewed
patches landing on us at the end of October --- if that happens, we'll
be back in the same position we were in at 8.3 feature freeze.
Although this schedule allows for the final commit fest to take a good
deal of time, we'll reserve the right to reject patches that are too
large to be reviewed in a timely fashion. We want to encourage people
to do development of large features in an incremental fashion, with a
new increment landing during each commit fest.

Regards, Dave (on behalf of the core team)

--
Dave Page
PostgreSQL Core Team

On Wed, 2008-02-06 at 08:56 +0000, Dave Page wrote:

> Hackers,

+1 Very much in favour.

--
Simon Riggs
2ndQuadrant http://www.2ndQuadrant.com

On Wed, Feb 06, 2008 at 08:56:51AM +0000, Dave Page wrote:
> Hackers,

Looks great and well-thought through. Let's hope it works out!

I assume you'll be committing this info to the developer section on the
website?

//Magnus

On Feb 6, 2008 1:49 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Wed, Feb 06, 2008 at 08:56:51AM +0000, Dave Page wrote:
> > Hackers,
>
> Looks great and well-thought through. Let's hope it works out!
>
> I assume you'll be committing this info to the developer section on the
> website?

It's on the developer wiki.

/D

On Wed, Feb 06, 2008 at 02:42:35PM +0000, Dave Page wrote:
> On Feb 6, 2008 1:49 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> > On Wed, Feb 06, 2008 at 08:56:51AM +0000, Dave Page wrote:
> > > Hackers,
> >
> > Looks great and well-thought through. Let's hope it works out!
> >
> > I assume you'll be committing this info to the developer section on the
> > website?
>
> It's on the developer wiki.

Good start. /me thinks it should be on the website. We've usually announced
our feature freeze dates there... (in less details, sure, but something
there)

//Magnus

On Feb 6, 2008 2:44 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>
> Good start. /me thinks it should be on the website. We've usually announced
> our feature freeze dates there... (in less details, sure, but something
> there)

Feel free - you've been hacking that recently!

/D

Dave Page wrote:
> Hackers,
>
> As you know we've finally released PostgreSQL 8.3, after a development
> cycle that lasted well over a year despite our original plans for a 6
> month cycle. The core team are aware that there are a number of
> factors that contributed to this slippage:
>
> - Lack of prompt and early review of patches.
> - A significant rise in the number and complexity of patches submitted.
> - Prioritising completion of incomplete patches over meeting the timetable.
>
> In the 8.4 development cycle we would like to try a new style of
> development, designed to keep the patch queue to a limited size and to
> provide timely feedback to developers on the work they submit. To do
> this we will replace the traditional 'feature freeze' with a series of
> 'commit fests' throughout the development cycle. The idea of commit
> fests was discussed last October in -hackers, and it seemed to meet
> with general approval. Whenever a commit fest is in progress, the
> focus will shift from development to review, feedback and commit of
> patches. Each fest will continue until all patches in the queue have
> either been committed to the CVS repository, returned to the author
> for additional work, or rejected outright, and until that has
> happened, no new patches will be considered. Of course, individual
> developers are free to continue working on their
> patches throughout the fest, but we encourage everyone to do what they
> can to help work through the patch queue. We feel that this idea can
> only be successful if the whole development community is willing to
> focus on patch review during the commit fests, in the same way that
> everyone is expected to focus on testing during beta period.
>
> The proposed timetable for the cycle is as follows:
>
> 1st March 2008 - commit fest begins
> 1st May 2008 - commit fest begins
> 1st July 2008 - commit fest begins
> 1st September 2008 - commit fest begins
> 1st November 2008 - final commit fest begins
> 1st January 2009 - beta 1
> 1st March 2009 - 8.4.0 release
>
> Note the lack of any 'feature freeze' date as such. However, any
> significant feature patches not submitted by 1st November will clearly
> not make it into 8.4.
>
> The hope here is that we will not have enormous, previously unreviewed
> patches landing on us at the end of October --- if that happens, we'll
> be back in the same position we were in at 8.3 feature freeze.
> Although this schedule allows for the final commit fest to take a good
> deal of time, we'll reserve the right to reject patches that are too
> large to be reviewed in a timely fashion. We want to encourage people
> to do development of large features in an incremental fashion, with a
> new increment landing during each commit fest.
>
> Regards, Dave (on behalf of the core team)
>
>

I would like to see this tied down some more. The time for the commit
fests is too open ended. I think we should say something like "All
commit fests will run no more than two weeks, except for the final
commit fest which can run for one month."

If we can't make that work then the whole idea is probably in trouble
anyway.

Another possibility which might help allocating reviewers to projects
(especially large projects) earlier in the process.

cheers

andrew

Am Mittwoch, 6. Februar 2008 schrieb Andrew Dunstan:
> I would like to see this tied down some more. The time for the commit
> fests is too open ended. I think we should say something like "All
> commit fests will run no more than two weeks, except for the final
> commit fest which can run for one month."

Something along those lines was discussed, but we feel that because we have no
experience with how commit fests will run, it is unwise to specify that much
detail already. It is quite possible that as we gain experience with the
process the timeline will be clarified.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Feb 6, 2008 3:57 PM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
> I would like to see this tied down some more. The time for the commit
> fests is too open ended. I think we should say something like "All
> commit fests will run no more than two weeks, except for the final
> commit fest which can run for one month."

I think thats one of the problems - without knowing what patches are
going to come in, or how many there will be, we have no way of knowing
how long each fest will take. What this does mean though is that we
continuously feedback to developers and keep the patch queue down -
kinda like checkpoint smoothing I guess.

/D

This all sounds very promising.

On Feb 6, 2008 7:56 PM, Dave Page <dpage(at)postgresql(dot)org> wrote:
> Each fest will continue until all patches in the queue have
> either been committed to the CVS repository, returned to the author
> for additional work, or rejected outright, and until that has
> happened, no new patches will be considered.

So does this mean we will have a new "patches awaiting the next review
cycle" queue alongside the "patches awaiting review" queue?

Just thinking that we'll need somewhere to park the new patches which
roll in during a commit fest.

Or, you know, start using an actual development tracker =)

Cheers
BJ

Dave Page wrote:
> On Feb 6, 2008 3:57 PM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
>> I would like to see this tied down some more. The time for the commit
>> fests is too open ended. I think we should say something like "All
>> commit fests will run no more than two weeks, except for the final
>> commit fest which can run for one month."
>>
>
> I think thats one of the problems - without knowing what patches are
> going to come in, or how many there will be, we have no way of knowing
> how long each fest will take. What this does mean though is that we
> continuously feedback to developers and keep the patch queue down -
> kinda like checkpoint smoothing I guess.
>
>

I would rather set a target and modify it if necessary based on
experience than have none at all.

The danger of not doing so is that we'll be in almost constant 'commit
fest' mode.

cheers

andrew

On Feb 6, 2008 4:24 PM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
>
>
> I would rather set a target and modify it if necessary based on
> experience than have none at all.
>
> The danger of not doing so is that we'll be in almost constant 'commit
> fest' mode.

Yes, that is something we discussed, and the reason why we used the
wording 'proposed timetable for the cycle'. We will adjust the timing
if need be, but wanted to start out on a confident note :-)

/D

Dave Page wrote:
> On Feb 6, 2008 4:24 PM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
>>
>> I would rather set a target and modify it if necessary based on
>> experience than have none at all.
>>
>> The danger of not doing so is that we'll be in almost constant 'commit
>> fest' mode.
>>
>
> Yes, that is something we discussed, and the reason why we used the
> wording 'proposed timetable for the cycle'. We will adjust the timing
> if need be, but wanted to start out on a confident note :-)
>
>
>

Sometimes I wish we could decide if we want to be wishy or washy ;-)

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> I would rather set a target and modify it if necessary based on
> experience than have none at all.

We felt that we'd like to get a couple of fests under our belts before
trying to nail down very many rules. The process will get more
formalized later, no doubt, but let's see what the actual problems are
before guessing about how to fix them.

The original draft listed the first commit fest as being in May, but
we added a March fest in part to have a "practice run" without too much
stuff being on the plate.

regards, tom lane

"Brendan Jurd" <direvus(at)gmail(dot)com> writes:
> Just thinking that we'll need somewhere to park the new patches which
> roll in during a commit fest.

Bruce has always kept two patch queues, one for the current version and
one for the stuff held for the next version. This won't change anything
except the labels on the queues.

regards, tom lane

Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
>
>> I would rather set a target and modify it if necessary based on
>> experience than have none at all.
>>
>
> We felt that we'd like to get a couple of fests under our belts before
> trying to nail down very many rules. The process will get more
> formalized later, no doubt, but let's see what the actual problems are
> before guessing about how to fix them.
>
> The original draft listed the first commit fest as being in May, but
> we added a March fest in part to have a "practice run" without too much
> stuff being on the plate.
>
>
>

OK, that makes some sense, although I don't know about the "not much
stuff on the plate". We presumably have quite a lot of stuff in the
queue from the last 7 months or so.

cheers

andrew

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Tom Lane wrote:
>> we added a March fest in part to have a "practice run" without too much
>> stuff being on the plate.

> OK, that makes some sense, although I don't know about the "not much
> stuff on the plate". We presumably have quite a lot of stuff in the
> queue from the last 7 months or so.

There is, although I think a large fraction of it will get bounced as
"needs more work", which should reduce the pressure. We'll just be
trying to give feedback to let the patch authors move forward, which
will not take as much time as actually committing would take.

The current queue is
http://momjian.postgresql.org/cgi-bin/pgpatches_hold
Note that a lot of the bulk is discussion of things that aren't
anywhere near committable anyway.

regards, tom lane

On Wednesday 06 February 2008 09:09, Tom Lane wrote:
> "Brendan Jurd" <direvus(at)gmail(dot)com> writes:
> > Just thinking that we'll need somewhere to park the new patches which
> > roll in during a commit fest.
>
> Bruce has always kept two patch queues, one for the current version and
> one for the stuff held for the next version. This won't change anything
> except the labels on the queues.

I think we might want to do something along the lines of what Stefan set up
(at least I think it was he) for the end of 8.4 on developer.postgresql.org.
Bruce's patch list is easy to update, but hard to read. I'll put some effort
into it.

--
Josh Berkus
PostgreSQL @ Sun
San Francisco

The plan looks great. I am +1

> -----Original Message-----
> From: pgsql-hackers-owner(at)postgresql(dot)org
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 4: Have you searched our list archives?
>
> http://archives.postgresql.org
>

Josh Berkus escribió:

> I think we might want to do something along the lines of what Stefan set up
> (at least I think it was he) for the end of 8.4 on developer.postgresql.org.
> Bruce's patch list is easy to update, but hard to read. I'll put some effort
> into it.

Easy to update for Bruce -- for anyone else it is impossible to update
AFAIK.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

Alvaro Herrera wrote:
> Josh Berkus escribió:
> > I think we might want to do something along the lines of what Stefan set
> > up (at least I think it was he) for the end of 8.4 on
> > developer.postgresql.org. Bruce's patch list is easy to update, but hard
> > to read. I'll put some effort into it.
>
> Easy to update for Bruce -- for anyone else it is impossible to update
> AFAIK.

Yes, I feel we could use a group writeable patch queue of some sort. Perhaps
an IMAP server setup could do the job.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Feb 6, 2008 9:56 AM, Dave Page <dpage(at)postgresql(dot)org> wrote:
> Whenever a commit fest is in progress, the
> focus will shift from development to review, feedback and commit of
> patches. Each fest will continue until all patches in the queue have
> either been committed to the CVS repository, returned to the author
> for additional work, or rejected outright, and until that has
> happened, no new patches will be considered.

If we don't have a bench farm anytime soon, I think we should consider
planning a set of benchmarks after each commit fest to prevent
performance regressions on different workloads. I don't expect them to
be comprehensive but it could allow us to prevent the most obvious
regressions.

--
Guillaume

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>> Josh Berkus escribi:
>>> I think we might want to do something along the lines of what Stefan set
>>> up (at least I think it was he) for the end of 8.4 on
>>> developer.postgresql.org. Bruce's patch list is easy to update, but hard
>>> to read. I'll put some effort into it.

> Yes, I feel we could use a group writeable patch queue of some sort. Perhaps
> an IMAP server setup could do the job.

Seems like a wiki page with links to pgsql-patches archive entries would
be easy. But an issue for any of this is who has permissions to edit
the queue? I concur that "Bruce only" is the wrong answer, but I'm not
sure "anyone with a wiki account" is the right answer.

regards, tom lane

On Wed, 06 Feb 2008 15:46:22 -0500
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Seems like a wiki page with links to pgsql-patches archive entries
> would be easy. But an issue for any of this is who has permissions
> to edit the queue? I concur that "Bruce only" is the wrong answer,
> but I'm not sure "anyone with a wiki account" is the right answer.

The Wiki accounts are controlled to a degree. If the page gets abused
we remove the privilege. Remember we can always rollback changes. This
is no different than email moderation imo.

Joshua D. Drake
--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

Joshua D. Drake wrote:
> On Wed, 06 Feb 2008 15:46:22 -0500
> Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
>> Seems like a wiki page with links to pgsql-patches archive entries
>> would be easy. But an issue for any of this is who has permissions
>> to edit the queue? I concur that "Bruce only" is the wrong answer,
>> but I'm not sure "anyone with a wiki account" is the right answer.
>
> The Wiki accounts are controlled to a degree. If the page gets abused
> we remove the privilege. Remember we can always rollback changes. This
> is no different than email moderation imo.

Is it technically possible to set permissions on a per-page basis?

//Magnus

Le Wednesday 06 February 2008 21:35:54 Peter Eisentraut, vous avez écrit :
> Alvaro Herrera wrote:
> > Easy to update for Bruce -- for anyone else it is impossible to update
> > AFAIK.
>
> Yes, I feel we could use a group writeable patch queue of some sort.
> Perhaps an IMAP server setup could do the job.

I've read some developers appreciating the way review board works:
http://review-board.org/
http://code.google.com/p/reviewboard/
http://code.google.com/p/reviewboard/wiki/UserBasics

This last link present the expected workflow when using the tool, and maybe
you'll find this matches the project needs... I don't know if such a tool can
help the project, but mentioning its existence certainly can't arm...

Hope this helps,
--
dim

On Wed, 06 Feb 2008 22:07:06 +0100
Magnus Hagander <magnus(at)hagander(dot)net> wrote:

> Joshua D. Drake wrote:
> > On Wed, 06 Feb 2008 15:46:22 -0500
> > Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >
> >> Seems like a wiki page with links to pgsql-patches archive entries
> >> would be easy. But an issue for any of this is who has permissions
> >> to edit the queue? I concur that "Bruce only" is the wrong answer,
> >> but I'm not sure "anyone with a wiki account" is the right answer.
> >
> > The Wiki accounts are controlled to a degree. If the page gets
> > abused we remove the privilege. Remember we can always rollback
> > changes. This is no different than email moderation imo.
>
> Is it technically possible to set permissions on a per-page basis?

That I can't answer but consider that the people that are putting info
on the wiki are mostly people we trust. We just make it clear that if
you are a jack ass you will have your account removed.

IMO, we are putting entirely too much energy into controlling flow of
text here. You have to log in to change the text, the text is
revertible as is the ability to log in in the first place.

Sincerely,

Joshua D. Drake

>
> //Magnus
>

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>>> Josh Berkus escribió:
>>>> I think we might want to do something along the lines of what Stefan set
>>>> up (at least I think it was he) for the end of 8.4 on
>>>> developer.postgresql.org. Bruce's patch list is easy to update, but hard
>>>> to read. I'll put some effort into it.
>
>> Yes, I feel we could use a group writeable patch queue of some sort. Perhaps
>> an IMAP server setup could do the job.
>
> Seems like a wiki page with links to pgsql-patches archive entries would
> be easy. But an issue for any of this is who has permissions to edit
> the queue? I concur that "Bruce only" is the wrong answer, but I'm not
> sure "anyone with a wiki account" is the right answer.

this is basically what I did during the 8.3 cycle on the wiki - I would
be willing to maintain a similiar thing for 8.4 if people think it is a
good idea.

Stefan

On Wed, 6 Feb 2008, Magnus Hagander wrote:

> Is it technically possible to set permissions on a per-page basis?

Technically possible? Of course. It's sure not easy to do, though; the
Mediawiki team considers having any real ACL structure added onto their
code a non-feature and last time I checked you had to patch the source.

I'd say it's really more trouble than it's worth here. It's not like the
developer site is open to the whole world or something. The number of
people capable of noticing and reverting bad changes in a critical,
popular page vastly outnumbers those likely to do something stupid (with
all the stuff I've done on the developer's wiki I don't think I've had to
revert a change bigger than a grammatical error so far).

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

Dimitri Fontaine <dfontaine(at)hi-media(dot)com> writes:
> Le Wednesday 06 February 2008 21:35:54 Peter Eisentraut, vous avez crit:
>> Yes, I feel we could use a group writeable patch queue of some sort.
>> Perhaps an IMAP server setup could do the job.

> I've read some developers appreciating the way review board works:
> http://review-board.org/
> http://code.google.com/p/reviewboard/
> http://code.google.com/p/reviewboard/wiki/UserBasics

Hmm, the info on that last page might be out of date, but what it says is
that the only SCMS they really support 100% is SVN. The other ones they
claim support for don't work [well/at all] with the post-review tool.

It looks interesting though, and would alleviate a few of the problems
people have mentioned with reviewing stuff that's posted as diffs.
Has anyone here got any direct experience with it?

regards, tom lane

On Wed, 06 Feb 2008 18:50:34 -0500
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> > I've read some developers appreciating the way review board works:
> > http://review-board.org/
> > http://code.google.com/p/reviewboard/
> > http://code.google.com/p/reviewboard/wiki/UserBasics
>
> Hmm, the info on that last page might be out of date, but what it
> says is that the only SCMS they really support 100% is SVN. The

O.k. I am not too interested in starting a whole war here (again) but
for the record, we have what appears to be a perfectly working
capability to move from cvs to svn. So *if* review board is something
we really like, the SCM should not be the barrier.

Sincerely,

Joshua D. Drake

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> O.k. I am not too interested in starting a whole war here (again) but
> for the record, we have what appears to be a perfectly working
> capability to move from cvs to svn. So *if* review board is something
> we really like, the SCM should not be the barrier.

I believe the compromise that's been reached for the moment is that
the core SCM will remain CVS, because everybody's favorite other SCM
can import from CVS but not necessarily from somebody else's favorite
other SCM. So a diff tool that doesn't work with CVS isn't going to be
especially useful for us.

I would imagine that the problem is mostly a lack of round tuits,
and that if we really fell in love with review board we could probably
teach it to handle diffs against CVS (especially seeing that the rest
of it besides post-review already works with CVS, supposedly).

So, again, the question is has anyone really used it? Is it the
best thing since sliced bread, or not so much?

regards, tom lane

Tom Lane wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
>
>> O.k. I am not too interested in starting a whole war here (again) but
>> for the record, we have what appears to be a perfectly working
>> capability to move from cvs to svn. So *if* review board is something
>> we really like, the SCM should not be the barrier.
>>
>
> I believe the compromise that's been reached for the moment is that
> the core SCM will remain CVS, because everybody's favorite other SCM
> can import from CVS but not necessarily from somebody else's favorite
> other SCM. So a diff tool that doesn't work with CVS isn't going to be
> especially useful for us.
>
> I would imagine that the problem is mostly a lack of round tuits,
> and that if we really fell in love with review board we could probably
> teach it to handle diffs against CVS (especially seeing that the rest
> of it besides post-review already works with CVS, supposedly).
>
> So, again, the question is has anyone really used it? Is it the
> best thing since sliced bread, or not so much?
>
> regards, tom lane

My official role at my place of work is "configuration management
software architect". We primarily use ClearCase and I am responsible for
the software side of the tooling around it. We have several thousands
users and terrabytes of data stored from millions of change sets. Not
that roles or anything matter, but where your job is PostgreSQL, my job
is SCM.

Probably because I am spoiled - I don't understand how your teams get
along so well with CVS. From my perspective, nearly everything available
is better than CVS. If it works good for you, and you don't ever have
merging problems, or history tracking problems, then great - any move is
going to be a hassle and will cause pain wasting at least some time in
the next development cycle.

If you do want to see the benefit of change - here is my experience with
Subversion:

I have been playing with Subversion for just almost two years now in a
small group of people with 3 people on a small project. While working on
the main branch ("trunk") submissions were generally smooth. Conflict
resolution is poor without graphical tool support, but with only three
people and co-ordinated work this was rarely an issue. Atomic
submissions were a pleasant relief and performance was adequate. Commits
are not at the level of functionality that I am accustomed to though.
First, commits are not registered until a person is complete their work
and the work is submitted. Second, merging of commits is very weak in
every production version of Subversion available today (1.4 and before)
because Subversion does not perform merge tracking. As soon as one
begins using multiple branches, it becomes very difficult to keep track
of where things are, and the people who support Subversion are satisfied
writing commit numbers in their comments to keep track of completed
merges. Finally, because the concept of directories, branches, and tags
have all been blurred into one muddle, horrible things happen if you try
to do anything clever. In my case, I had a web project that I intended
to break into web, lib, and source. I renamed trunk to trunk/www and
created trunk/lib, and trunk/source. For this point forwards, I was
completely unable to merge changes from other branches to trunk.
Subversion became completely confused. It was at this point that my
frustration acceptance level was passed, and I switched to GIT. This was
last December. Subversion 1.5 was supposed to be out to address many of
these issues, but it was a hollow promise as it was still not released
the last time I checked, and a review of their discussions on the matter
show that many of the promises they made were likely premature.

Since then, I have been consistently impressed with GIT. I have
completed complex merges and extensive parallel development that would
have been painful or impossible with Subversion. I am not a fan of
de-centralization as most GIT supporters are - but I am a fan of full
feature change sets. In GIT I can merge a change set back and forth
between branches and it will track it. I can rebase the change set to a
later baseline and continue manipulating it. I can save my work space
aside, or use the same work space to switch to another branch and have
my uncommitted work automatically three-way merged to the new context.
Our team on this outside work project is now up to 5 people and
everybody likes GIT better than Subversion.

My story is that Subversion is cute - but it does not scale in terms of
flexible parallel development models, nor does it provide sufficient
functionality over CVS to be considered "the best thing since sliced
bread." It is an improvement over CVS, but it is not a great tool. If
you are going to go through the effort of migrating to another system, I
would seriously consider the benefits of other systems out there before
believing that Subversion is the answer to all problems. GIT is one good
choice. However, my experience with these products in insufficient to
make a recommendation for PostgreSQL. I would like to experiment with Hg
and a few others over the next few months and see what I think of these.

I encourage all to keep their minds open.

Cheers,
mark

--
Mark Mielke <mark(at)mielke(dot)cc>

Tom Lane wrote:
> "Brendan Jurd" <direvus(at)gmail(dot)com> writes:
> > Just thinking that we'll need somewhere to park the new patches which
> > roll in during a commit fest.
>
> Bruce has always kept two patch queues, one for the current version and
> one for the stuff held for the next version. This won't change anything
> except the labels on the queues.

Sure, I can do that. One is already called the _hold_ patch queue.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Tom Lane wrote:
> There is, although I think a large fraction of it will get bounced as
> "needs more work", which should reduce the pressure. We'll just be
> trying to give feedback to let the patch authors move forward, which
> will not take as much time as actually committing would take.
>
> The current queue is
> http://momjian.postgresql.org/cgi-bin/pgpatches_hold
> Note that a lot of the bulk is discussion of things that aren't
> anywhere near committable anyway.
>
>
Wouldn't it make sense try to catch up a bit and fix as much of this as
is feasible (including
return and resubmit) for things where the desire is uncontroversial,
even if the implementation
is flawed, and accept other things that are fully acceptable in the time
it takes to do that, and
then call it a wrap?

The curre nt *plan* is for a 14 month cycle. And it will probably
slip. Some of the
queued items are going to be very old by the time you go to 8.4 on this
program,
which seems a shame. That sort of plan looks to me more like a 'major
refactoring
to get to 9.0' sort of plan, than an incremental release.

James

On Wed, Feb 06, 2008 at 06:50:34PM -0500, Tom Lane wrote:
> Dimitri Fontaine <dfontaine(at)hi-media(dot)com> writes:
> > Le Wednesday 06 February 2008 21:35:54 Peter Eisentraut, vous avez crit:
> >> Yes, I feel we could use a group writeable patch queue of some sort.
> >> Perhaps an IMAP server setup could do the job.
>
> > I've read some developers appreciating the way review board works:
> > http://review-board.org/
> > http://code.google.com/p/reviewboard/
> > http://code.google.com/p/reviewboard/wiki/UserBasics
>
> Hmm, the info on that last page might be out of date, but what it says is
> that the only SCMS they really support 100% is SVN. The other ones they
> claim support for don't work [well/at all] with the post-review tool.

Not having looked into exactly how it works and if it's something we want,
but if we want to, any reason we can't just point it at the svn mirror?

Plus, I see something on their blog saying taht they do support cvs as long
as it's pserver. So perhaps part of the docs aren't up-to-date...

> It looks interesting though, and would alleviate a few of the problems
> people have mentioned with reviewing stuff that's posted as diffs.
> Has anyone here got any direct experience with it?

Can't say I do. But even if nobody does, maybe we should just set it up and
test? I'd be particularly interested to see how it actually interacts with
email. From a quick reading, you have to do all the discussion on the web
and it can dump them out to a list. But not read in a discussion from a
list.

//Magnus

Le jeudi 07 février 2008, Tom Lane a écrit :
> Hmm, the info on that last page might be out of date, but what it says is
> that the only SCMS they really support 100% is SVN. The other ones they
> claim support for don't work [well/at all] with the post-review tool.

Maybe this is all to naive to ever work, but I'm thinking a diff extracted
from SVN looks perfectly equivalent to a diff extracted from CVS. And svn
diff output should be usable the same way any -patches diff?

It also seems to me that when a patch is applied, still pending patches may
have to get adapted to new head before they can be reviewed.

So, what if a SVN copy of CVS HEAD was used for review-board, and
automatically updated from CVS HEAD. The diff which won't get cleanly applied
after an automatic SVN update would not apply cleanly against CVS HEAD
neither, so will have to get adapted with or without the SVN mirror step.

If all of this can be made automatic, short of hosting facility & service
administration, what's still blocking?

> It looks interesting though, and would alleviate a few of the problems
> people have mentioned with reviewing stuff that's posted as diffs.

And maybe it would ease Bruce's pressure to organize the queues and allow all
participants to easily follow what's happening with the patches, without
having to digest all -hackers mail and without manual big-table wiki page
editing. Is it only a dream? :)

Regards,
--
dim

Magnus Hagander wrote:
> On Wed, Feb 06, 2008 at 06:50:34PM -0500, Tom Lane wrote:
>> Dimitri Fontaine <dfontaine(at)hi-media(dot)com> writes:
>>> Le Wednesday 06 February 2008 21:35:54 Peter Eisentraut, vous avez �crit�:
>>>> Yes, I feel we could use a group writeable patch queue of some sort.
>>>> Perhaps an IMAP server setup could do the job.
>>> I've read some developers appreciating the way review board works:
>>> http://review-board.org/
>>> http://code.google.com/p/reviewboard/
>>> http://code.google.com/p/reviewboard/wiki/UserBasics
>> Hmm, the info on that last page might be out of date, but what it says is
>> that the only SCMS they really support 100% is SVN. The other ones they
>> claim support for don't work [well/at all] with the post-review tool.
>
> Not having looked into exactly how it works and if it's something we want,
> but if we want to, any reason we can't just point it at the svn mirror?
>
> Plus, I see something on their blog saying taht they do support cvs as long
> as it's pserver. So perhaps part of the docs aren't up-to-date...
>
>
>> It looks interesting though, and would alleviate a few of the problems
>> people have mentioned with reviewing stuff that's posted as diffs.
>> Has anyone here got any direct experience with it?
>
> Can't say I do. But even if nobody does, maybe we should just set it up and
> test? I'd be particularly interested to see how it actually interacts with
> email. From a quick reading, you have to do all the discussion on the web
> and it can dump them out to a list. But not read in a discussion from a
> list.

I could do a demo install on the trackerdemo jail - that one seems to
have most of the prequisits and would not need work to get going. Not
sure I want to install MySQL there though - so we would have to go with
the sqlite backend for the test ;-)

Stefan

Stefan Kaltenbrunner wrote:
>
> I could do a demo install on the trackerdemo jail - that one seems to
> have most of the prequisits and would not need work to get going. Not
> sure I want to install MySQL there though - so we would have to go
> with the sqlite backend for the test ;-)
>
>

Umm, we need to eat our own dog food. If it won't run on postgres then
fuggedaboudit.

The settings_local.py.tmpl contains these two lines:

# Database backend. Any supported django database engine should work.
DATABASE_ENGINE = 'mysql' # 'postgresql', 'mysql', 'sqlite3' or
'ado_mssql'.

So let's just go with postgresql ;-)

cheers

andrew

Andrew Dunstan wrote:
>
>
> Stefan Kaltenbrunner wrote:
>>
>> I could do a demo install on the trackerdemo jail - that one seems to
>> have most of the prequisits and would not need work to get going. Not
>> sure I want to install MySQL there though - so we would have to go
>> with the sqlite backend for the test ;-)
>>
>>
>
> Umm, we need to eat our own dog food. If it won't run on postgres then
> fuggedaboudit.

yep

>
> The settings_local.py.tmpl contains these two lines:
>
> # Database backend. Any supported django database engine should work.
> DATABASE_ENGINE = 'mysql' # 'postgresql', 'mysql', 'sqlite3' or
> 'ado_mssql'.
>
> So let's just go with postgresql ;-)

yeah various parts of their docs state that only sqlite and mysql are
support some others say that only those are tested ...

I will take a stab at it later today anyway.

Stefan

James Mansion <james(at)mansionfamily(dot)plus(dot)com> writes:
> The curre nt *plan* is for a 14 month cycle. And it will probably
> slip. Some of the queued items are going to be very old by the time
> you go to 8.4 on this program, which seems a shame.

What? The plan is to deal with them next month (in the first commit
fest). The point of what I was saying is that a large fraction of what
is in the queue is not committable yet. We are not going to commit it
anyway just to get it out of the queue --- what we're going to do is
review it and send it back to the authors with suggestions for rework.
If they don't get the rework done by November, their patches won't get
into 8.4, but that's hardly the fault of the proposed process.

As for "probably slip", the entire point of this process change is
that we're not going to allow schedule slippage as readily as we
have in the past.

regards, tom lane

Magnus Hagander <magnus(at)hagander(dot)net> writes:
> On Wed, Feb 06, 2008 at 06:50:34PM -0500, Tom Lane wrote:
>> Hmm, the info on that last page might be out of date, but what it says is
>> that the only SCMS they really support 100% is SVN. The other ones they
>> claim support for don't work [well/at all] with the post-review tool.

> Not having looked into exactly how it works and if it's something we want,
> but if we want to, any reason we can't just point it at the svn mirror?

Synchronization problems scare me.

The point I tried to make earlier was that if we actually started to
rely on such a tool, we'd want to fix it to talk to CVS. Hey, it's
an open source project, right?

regards, tom lane

On Thu, 07 Feb 2008 11:19:26 -0500
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Magnus Hagander <magnus(at)hagander(dot)net> writes:
> > On Wed, Feb 06, 2008 at 06:50:34PM -0500, Tom Lane wrote:
> >> Hmm, the info on that last page might be out of date, but what it
> >> says is that the only SCMS they really support 100% is SVN. The
> >> other ones they claim support for don't work [well/at all] with
> >> the post-review tool.
>
> > Not having looked into exactly how it works and if it's something
> > we want, but if we want to, any reason we can't just point it at
> > the svn mirror?
>
> Synchronization problems scare me.
>

That's reasonable. Although I am currently reviewing tailor which can
supposedly incrementally update the repo versus bulk convert each time.

> The point I tried to make earlier was that if we actually started to
> rely on such a tool, we'd want to fix it to talk to CVS. Hey, it's
> an open source project, right?

Yes but we are supposed to work smart not hard. CVS doesn't fit that
criteria when we have highly available, highly used and highly trusted
options available. We even have an option with an extremely low barrier
of entry (svn) and options that are proven amongst one of the (if not
the) most active FOSS projects on the planet (git).

I am not arguing any particular solution but home brewing a solution so
people can stay on what is definitely a dying SCM is dumb. There are
so many tools available to us that we *don't* have to modify, bend,
break or if you like, improve that any argument outside of, "We are
used to CVS" is just hand waving and that argument is sad.

Sincerely,

Joshua D. Drake

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

Joshua D. Drake escribió:

> I am not arguing any particular solution but home brewing a solution so
> people can stay on what is definitely a dying SCM is dumb. There are
> so many tools available to us that we *don't* have to modify, bend,
> break or if you like, improve that any argument outside of, "We are
> used to CVS" is just hand waving and that argument is sad.

Actually, in using Subversion I've found it broken enough that a
migration to it does not offer that much of an advantage over staying
with CVS. It would certainly offer us some advantages, but our usage of
CVS has proven successful. Moreover, several developers have started
using the DSCM of their choice on top of our CVS repo, which gives a lot
of the benefits without the hassle of forcing everyone else to convert.

I say we should try Review Board with CVS. If it doesn't work, make
sure it gets fixed.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Thu, 7 Feb 2008 14:00:33 -0300
Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:

> Joshua D. Drake escribió:
>
> > I am not arguing any particular solution but home brewing a
> > solution so people can stay on what is definitely a dying SCM is
> > dumb. There are so many tools available to us that we *don't* have
> > to modify, bend, break or if you like, improve that any argument
> > outside of, "We are used to CVS" is just hand waving and that
> > argument is sad.
>
> Actually, in using Subversion I've found it broken enough that a
> migration to it does not offer that much of an advantage over staying
> with CVS.

I repeat. I am not arguing a particular solution. I am arguing against
creating more internal infrastructure and the relevant support
requirements when other solutions exist.

Joshua D. Drake

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

Joshua D. Drake wrote:
> On Thu, 7 Feb 2008 14:00:33 -0300
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
>
>> Joshua D. Drake escribió:
>>
>>> I am not arguing any particular solution but home brewing a
>>> solution so people can stay on what is definitely a dying SCM is
>>> dumb. There are so many tools available to us that we *don't* have
>>> to modify, bend, break or if you like, improve that any argument
>>> outside of, "We are used to CVS" is just hand waving and that
>>> argument is sad.
>> Actually, in using Subversion I've found it broken enough that a
>> migration to it does not offer that much of an advantage over staying
>> with CVS.
>
> I repeat. I am not arguing a particular solution. I am arguing against
> creating more internal infrastructure and the relevant support
> requirements when other solutions exist.

Just so we can stop talking about this, it does seem it works with CVS -
it's just not necessarily documented that way...

//Magnus

Josh,

Try it out. Setup a review-board installation, point it at your SVN
mirror. As long as people can "post" diffs (and from the the
screenshots, it looks like it has a "diff file" browse button), it
doesnt' really matter what "it" uses as it's backend, does it?

And if it turns out to be a good means for discussion patches, or at
lease "recording" patches and status, then good.

And the nice thing is that it could even be "managed" by observers at
first, much like the wiki patch status page was until it's been shown
(if it is) to be a good tool for tracking patches, etc.

I'm slightly wary of it, but that's probably just because my normal
development work-flow *doesn't* include a web-browser for anything, and
being forced to use some AJAXy web-interface to do/see anything probably
means I won't do/see anything... Thankfully, my absence in the
development process isn't something that the PostgreSQL community will
greatly miss...

a.

* Joshua D. Drake <jd(at)commandprompt(dot)com> [080207 11:46]:

> > The point I tried to make earlier was that if we actually started to
> > rely on such a tool, we'd want to fix it to talk to CVS. Hey, it's
> > an open source project, right?
>
> Yes but we are supposed to work smart not hard. CVS doesn't fit that
> criteria when we have highly available, highly used and highly trusted
> options available. We even have an option with an extremely low barrier
> of entry (svn) and options that are proven amongst one of the (if not
> the) most active FOSS projects on the planet (git).
>
> I am not arguing any particular solution but home brewing a solution so
> people can stay on what is definitely a dying SCM is dumb. There are
> so many tools available to us that we *don't* have to modify, bend,
> break or if you like, improve that any argument outside of, "We are
> used to CVS" is just hand waving and that argument is sad.
>
> Sincerely,
>
> Joshua D. Drake
>
> --
> The PostgreSQL Company since 1997: http://www.commandprompt.com/
> PostgreSQL Community Conference: http://www.postgresqlconference.org/
> Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
> PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit
>

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Joshua D. Drake wrote:
> On Thu, 7 Feb 2008 14:00:33 -0300
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
>
>> Joshua D. Drake escribió:
>>
>>> I am not arguing any particular solution but home brewing a
>>> solution so people can stay on what is definitely a dying SCM is
>>> dumb. There are so many tools available to us that we *don't* have
>>> to modify, bend, break or if you like, improve that any argument
>>> outside of, "We are used to CVS" is just hand waving and that
>>> argument is sad.
>> Actually, in using Subversion I've found it broken enough that a
>> migration to it does not offer that much of an advantage over staying
>> with CVS.
>
> I repeat. I am not arguing a particular solution. I am arguing against
> creating more internal infrastructure and the relevant support
> requirements when other solutions exist.

yep - I just got a prototype install of review board running against
anoncvs.postgresql.org and it just seems to work fine at a first glance
using CVS (and postgresql for that matter). So no need for any strange
workarounds :-)

Stefan

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> I repeat. I am not arguing a particular solution. I am arguing against
> creating more internal infrastructure and the relevant support
> requirements when other solutions exist.

Who said anything about internal infrastructure? We'd be helping
another open source project flesh out and test a possibly-incomplete
area of their code, not undertaking a fork. (Now, if they rejected
patches on the grounds that they don't care about CVS, then this
doesn't work, but I can't imagine they would; they do have partial
support for it.)

Now, switching to some other SCM might indeed create some new support
requirements. I was a bit surprised to read this on another mailing
list yesterday:

>> From a relative time to install from source standpoint it looks like
>> this:
>>
>> CVS - 10 minutes (no external dependencies)
>> GIT - 8 minutes (no external dependencies)
>> Mercurial - 1 minute (depends on Python)
>> Subversion - 4-6 hours (depends on a multitude of packages and will
>> only work with specific versions which you
>> learn about the hard way at build time).

For those on platforms where SVN comes prepackaged, this might not be
a big problem (except maybe for pulling in packages they don't want).
For other developers this kind of thing could be a showstopper.

regards, tom lane

Tom Lane wrote:
>>> From a relative time to install from source standpoint it looks like
>>> this:
>>>
>>> CVS - 10 minutes (no external dependencies)
>>> GIT - 8 minutes (no external dependencies)
>>> Mercurial - 1 minute (depends on Python)
>>> Subversion - 4-6 hours (depends on a multitude of packages and will
>>> only work with specific versions which you
>>> learn about the hard way at build time).
>>>
>
> For those on platforms where SVN comes prepackaged, this might not be
> a big problem (except maybe for pulling in packages they don't want).
> For other developers this kind of thing could be a showstopper

As with anything you are likely to see on this issue, the above seems
highly suspect as hard numbers. In my own case I believe installing
Subversion is in the 10 minute time frame as well unless you get into
linking it with Apache and such which becomes unfair. Setting up any of
these solutions to be securely accessible from the network takes longer
than 10 minutes, so the numbers listed can only be for local installs,
and not all systems have Python. I think think Solaris 8 does?

In terms of picking an SCM candidate, I don't think "time to install
from source" is a legitimate concern. Installing from source is great,
but if the package needs to be installed from source, it is not well
enough supported by the community to be worth using.

Cheers,
mark

--
Mark Mielke <mark(at)mielke(dot)cc>

Mark Mielke <mark(at)mark(dot)mielke(dot)cc> writes:
> In terms of picking an SCM candidate, I don't think "time to install
> from source" is a legitimate concern. Installing from source is great,
> but if the package needs to be installed from source, it is not well
> enough supported by the community to be worth using.

That is 100.0% wrong. Some people want to install from source, and
some don't have any choice because they are on platforms where there's
not a prebuilt binary available. I am *not* willing to say that we
will blow off developers on any platform that some other project is
choosing not to provide binaries for.

As a fairly well related example, note how CVSup never became the de
facto standard, because it wasn't portable enough, or at least had made
the wrong decisions about what to depend on.

regards, tom lane

Le Thursday 07 February 2008 17:19:26 Tom Lane, vous avez écrit :
> > Not having looked into exactly how it works and if it's something we
> > want, but if we want to, any reason we can't just point it at the svn
> > mirror?
>
> Synchronization problems scare me.

AIUI we're talking about one way synchronization (from CVS to SVN) only, and
that's considering review-board is not able to do CVS directly (which seems
wrong).
The quick doc reading I've made showed a read-only tool at the SCM side ---
the accepted patch won't get commited for you by the tool.

> The point I tried to make earlier was that if we actually started to
> rely on such a tool, we'd want to fix it to talk to CVS.

Others are pointing it does in fact talk to CVS even if the documentation
about this is... to be written.

Regards,
--
dim

Dimitri Fontaine wrote:
> Le Thursday 07 February 2008 17:19:26 Tom Lane, vous avez écrit :
>>> Not having looked into exactly how it works and if it's something we
>>> want, but if we want to, any reason we can't just point it at the svn
>>> mirror?
>> Synchronization problems scare me.
>
> AIUI we're talking about one way synchronization (from CVS to SVN) only, and
> that's considering review-board is not able to do CVS directly (which seems
> wrong).

yes - CVS works just fine

> The quick doc reading I've made showed a read-only tool at the SCM side ---
> the accepted patch won't get commited for you by the tool.

this seems to be true too from what I see on the test-install

>
>> The point I tried to make earlier was that if we actually started to
>> rely on such a tool, we'd want to fix it to talk to CVS.
>
> Others are pointing it does in fact talk to CVS even if the documentation
> about this is... to be written.

well CVS is a simply as selecting "CVS" in the admin interface and seems
to be the only scm that it works without installing additional python
libraries :-)

Stefan

Tom Lane wrote:
> Mark Mielke <mark(at)mark(dot)mielke(dot)cc> writes:
>
>> In terms of picking an SCM candidate, I don't think "time to install
>> from source" is a legitimate concern. Installing from source is great,
>> but if the package needs to be installed from source, it is not well
>> enough supported by the community to be worth using.
>>
>
> That is 100.0% wrong. Some people want to install from source, and
> some don't have any choice because they are on platforms where there's
> not a prebuilt binary available. I am *not* willing to say that we
> will blow off developers on any platform that some other project is
> choosing not to provide binaries for.
>
> As a fairly well related example, note how CVSup never became the de
> facto standard, because it wasn't portable enough, or at least had made
> the wrong decisions about what to depend on

You are not correct. Different SCM systems have different capabilities.
Value and portability are important factors. Time to install from source
is not. Comparing how easy they are to install from source in terms of
minutes is not a fair assessment of the value they provide nor the
portability of the systems. The numbers you quoted are obviously invalid
as SVN is based very significantly on the APR libraries that Apache is
based on in a well established and successful effort to provide
portability. Would you say that Apache is not a good choice? None of
this proves that SVN is a good choice - but I believe it does prove that
using numbers such as you quoted to draw a conclusion about what is best
for PostgreSQL is a backwards way of thinking about the problem.

GIT is currently poor at portability primarily because it is new, and if
you tried to compile it on Windows (a common platform these days)
without CYGWIN you would have a lot of trouble. This does not make GIT a
worse choice. That it lacks in portability is a current concern that
should be weighed along with the rest of the issues such as ease of use,
productivity, integration with other valuable tools, parallel
development support (reliable merge tracking being a major factor here),
and offline capabilities.

I think what you missed was my statement that if package installs do not
exist for the majority of the platforms you intend people to develop
PostgreSQL on, then the SCM system you are considering is not mature or
enough, or supported well enough by the community, to be a good
candidate and there will be trouble down the road if the system that is
chosen goes into disrepair. Being able to compile from source is a
virtue shared by open source developers - but a requirement that it must
be compiled from source is not a virtue. If it must be compiled from
source, it means the product is not valuable enough to people to create
a demand for a binary install package. Nobody should be forced to
compile an SCM system from source in order to contribute to PostgreSQL.
That would be just silly.

Cheers,
mark

--
Mark Mielke <mark(at)mielke(dot)cc>

Am Donnerstag, 7. Februar 2008 schrieb Tom Lane:
> So, again, the question is has anyone really used it?  Is it the
> best thing since sliced bread, or not so much?

I think it is about the equivalent of replacing a mailing list by Yahoo
Groups. It has more special effects, and no doubt some people will like it,
but it cannot replace the original.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Mark Mielke <mark(at)mark(dot)mielke(dot)cc> writes:
>> In terms of picking an SCM candidate, I don't think "time to install
>> from source" is a legitimate concern. Installing from source is great,
>> but if the package needs to be installed from source, it is not well
>> enough supported by the community to be worth using.
>
> That is 100.0% wrong. Some people want to install from source, and
> some don't have any choice because they are on platforms where there's
> not a prebuilt binary available. I am *not* willing to say that we
> will blow off developers on any platform that some other project is
> choosing not to provide binaries for.

I'm not sure we're talking about the same thing. I've never heard any
complaints about building svn from source before for *developers*. I think
that's just as easy as anything else.

What I have heard in the distance past is that it was difficult to set up a
server. That isn't something developers would have to do. And in any case I
understood that to be mostly about how it used to depend on a web server which
is no longer true anyways.

> As a fairly well related example, note how CVSup never became the de
> facto standard, because it wasn't portable enough, or at least had made
> the wrong decisions about what to depend on.

This is all predicated on a bit of ridiculous FUD. Apply the logic in reverse
and it should be obvious. Subversion is a mature package being used by
thousands of open source projects. At this point I would hazard it's more
widely used than CVS amongst open source projects. Therefore it *doesn't* have
any poor choices of dependencies.

For what it's worth I think GIT is a better fit for our needs.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's Slony Replication support!

Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> I'm not sure we're talking about the same thing. I've never heard any
> complaints about building svn from source before for *developers*. I think
> that's just as easy as anything else.

[ shrug... ] The message I quoted was from Bob Friesenhahn, who is
certainly a competent C programmer. If it took him half a day to
install SVN from scratch, I'm prepared to believe it's not trivial.
At the very least, I suggest you replicate the experiment before
asserting you know more about it than someone who's tried.

regards, tom lane

Gregory Stark escribió:

> For what it's worth I think GIT is a better fit for our needs.

Perhaps it would be, if it worked on Windows ... Not that I care, but I
bet Magnus would.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Thu, 7 Feb 2008, Tom Lane wrote:

>>> Subversion - 4-6 hours (depends on a multitude of packages and will
>>> only work with specific versions which you
>>> learn about the hard way at build time).

I have seen one of these nightmare Subversion installs before so I can
attest to the fact that they happen. Was close to two days actually.
Hours spent just trying to sort out which version of apr, neon, etc. all
worked right. The versions that shipped with the OS were broken, trying
to use the whole Subversion dependency bundle introduced "which version am
I linking with now?" issues and even that set wasn't completely
consistant. Complete disaster all around.

That said, I think that's an exceptional case due to using a Linux
distribution that should have been retired already, but dismissing the
wild Subversion build dependency tree problem as FUD would be wrong. It
happens.

> For those on platforms where SVN comes prepackaged, this might not be
> a big problem (except maybe for pulling in packages they don't want).
> For other developers this kind of thing could be a showstopper.

It's only recently that I started getting prepackaged SVN versions that
were new enough to be completely useful included with the Linux
distributions I use. The only thing that's saved me on RHEL4 are the RPM
packages at summersoft.fay.ar.us, and even for those you have to pull down
many packages and install them just right.

As others have pointed out this is really kind of moot. Subversion is
really only a small step forward from CVS and it has merge issues that
make it less suitable the larger the number of developers there are. As
much as I'd like to move off of CVS, if the forward step is Subversion I'd
say why bother; too much work for a small gain.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

On Feb 7, 2008 9:23 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> At the very least, I suggest you replicate the experiment before
> asserting you know more about it than someone who's tried.

Will you accept the testimony of someone who has built an SVN *server*
entirely from source on Slackware Linux? It took about an hour the
first time I did it, with no previous SVN experience.

--
Dave Page
EnterpriseDB UK: http://www.enterprisedb.com
The Oracle-compatible database company

Tom Lane wrote:
> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>> I'm not sure we're talking about the same thing. I've never heard any
>> complaints about building svn from source before for *developers*. I think
>> that's just as easy as anything else.
>
> [ shrug... ] The message I quoted was from Bob Friesenhahn, who is
> certainly a competent C programmer. If it took him half a day to
> install SVN from scratch, I'm prepared to believe it's not trivial.
> At the very least, I suggest you replicate the experiment before
> asserting you know more about it than someone who's tried.

I've built it from source several times, and it's certainly not taken
half a day. It took quite a while to build, but it wasn't hard to do at all.

Goes to replicate the experiment, not to argue for or against any of the
products.

//Magnus

Tom Lane wrote:
> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>
>> I'm not sure we're talking about the same thing. I've never heard any
>> complaints about building svn from source before for *developers*. I think
>> that's just as easy as anything else.
>>
>
> [ shrug... ] The message I quoted was from Bob Friesenhahn, who is
> certainly a competent C programmer. If it took him half a day to
> install SVN from scratch, I'm prepared to believe it's not trivial.
> At the very least, I suggest you replicate the experiment before
> asserting you know more about it than someone who's tried.

Perhaps he didn't read the instructions. See below for a 5 minutes 34
elapsed time. This includes extracting SVN over the network using SVN.

Cheers,
mark

$ time zsh
$ svn checkout http://svn.collab.net/repos/svn/trunk svn-devel
...
Checked out revision 29228.
$ cd svn-devel
$ ./autogen.sh
...
You can run ./configure now.
...
$ su
Password:
[root(at)andromeda]/stage/mark/svn-devel# mkdir /opt/svn-devel
[root(at)andromeda]/stage/mark/svn-devel# chown mark:mark /opt/svn-devel
[root(at)andromeda]/stage/mark/svn-devel# exit
$ ./configure --prefix=/opt/svn-devel

configure: Configuring Subversion 1.6.0
...
$ make -j4
...
$ make install
...
$ /opt/svn-devel/bin/svn --version
svn, version 1.6.0 (dev build)
compiled Feb 7 2008, 16:46:21

Copyright (C) 2000-2008 CollabNet.
Subversion is open source software, see http://subversion.tigris.org/
This product includes software developed by CollabNet
(http://www.Collab.Net/).

The following repository access (RA) modules are available:

* ra_svn : Module for accessing a repository using the svn network protocol.
- handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
- handles 'file' scheme
$ exit
zsh 179.01s user 67.59s system 73% cpu 5:33.51 total

--
Mark Mielke <mark(at)mielke(dot)cc>

Alvaro Herrera wrote:
> Gregory Stark escribió:
>
>> For what it's worth I think GIT is a better fit for our needs.
>
> Perhaps it would be, if it worked on Windows ... Not that I care, but I
> bet Magnus would.
>

To summarize what I care about: I don't really care if I can't *commit*
from Windows - I never do that anyway, for fear of breaking line
endings. I do care, and a lot, if I can't pull down latest-and-greatest
HEAD revision without risk of getting something that's not correct.
Which means that if there is a gateway to something that works well,
that is *stable and capable of being up to date*, I can live with that.
(for example, doing a dump like the current svn mirror does which lags
behind a lot, would be something I'd absolutely object to)

//Magnus

Peter Eisentraut wrote:
> Am Donnerstag, 7. Februar 2008 schrieb Tom Lane:
>> So, again, the question is has anyone really used it? Is it the
>> best thing since sliced bread, or not so much?
>
> I think it is about the equivalent of replacing a mailing list by Yahoo
> Groups. It has more special effects, and no doubt some people will like it,
> but it cannot replace the original.

yeah - the test install is available on http://reviewdemo.postgresql.org
if people want to test judge for themself - contact magnus or me if you
need permissions to do/test stuff there.

Stefan

Mark Mielke wrote:
> Perhaps he didn't read the instructions. See below for a 5 minutes 34
> elapsed time. This includes extracting SVN over the network using SVN.

And just to be complete, here is git at 2 minutes 13 seconds. Not that
these times matter at all, but in case anybody thinks they do...

$ time zsh
$ cd /stage/mark
$ wget http://kernel.org/pub/software/scm/git/git-1.5.4.tar.bz2
...
16:56:41 (450.83 KB/s) - `git-1.5.4.tar.bz2' saved [1583166/1583166]

$ tar xfj git-1.5.4.tar.bz2
$ cd git-1.5.4
$ su
Password:
[root(at)andromeda]/stage/mark/git-1.5.4# mkdir /opt/git-1.5.4
[root(at)andromeda]/stage/mark/git-1.5.4# chown mark:mark /opt/git-1.5.4
[root(at)andromeda]/stage/mark/git-1.5.4# exit
$ ./configure --prefix=/opt/git-1.5.4
...
$ make -j4
...
$ make install
...
$ /opt/git-1.5.4/bin/git version
git version 1.5.4
$ exit
zsh 63.61s user 12.94s system 57% cpu 2:13.77 total

--
Mark Mielke <mark(at)mielke(dot)cc>

Alvaro Herrera wrote:
> Gregory Stark escribió:
>
>> For what it's worth I think GIT is a better fit for our needs.
>
> Perhaps it would be, if it worked on Windows ... Not that I care, but I
> bet Magnus would.

There's fairly good tools to convert from one version control system to
another. Especially: from CVS to others. And it can be done in an
incremental fashion.

Therefore, we can provide mirrors of the CVS repository in multiple
formats. And those mirrors exist already, I remember a GIT and a
Subversion mirror off the top of my head, and I bet there's others.
After we have that, the master version control system used doesn't
matter for developers (except committers), as everyone can choose to use
whichever mirror he wants. The patches submitted to pgsql-patches will
look exactly the same regardless of the version control system the patch
submitter used to check out the source code.

We can agree to disagree. No need for the project to switch.

Personally, I've been playing with GIT recently, and it does feel quite
nice. The mirror seems to be missing all tags, but other than that, I've
been happy with it.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Fabien COELHO wrote:
> ISTM that a decentralized or distributed SCM for PostgreSQL would be a
> bad move, however great it would be at branching and merging. For me
> it is a philosophy question: if PGSQL is a "common work", then
> everything should be open and shared, and a centralized systems make
> sense to embodied this. Even if one can publish one's branch easily
> with GIT, it's not the same, because it is still a personnal branch
> somehow.

> I'm not sure I would be proud to use such a stupidly named tool for a
> "common work". I really do not share Linus humor, and apparent
> contempt for other people. GIT implements "I want to chose whom I work
> with, and don't care about the others, and don't ever want to have to
> look at their ugly patches", or at least it is what I understood from
> his talk at Google last year. Would this be the future spirit of PG
> devel? I hope not.

I don't particularly care what it is called or what Linus' intents were.
Linus has changed his public face on git several times since its
creation, and I think he is playing with people, manipulating them into
launching Linux and GIT into open source history. From a political stand
point, it's about attracting the right sort of people to donate their
time to your project. Different types of honey attract different types
of folk. :-)

Your points on centralization are ones that I mostly agree with and
share. I think it depends on whether you believe that freedom of
software needs to be enforced, or whether you trust that freedom of
software will occur on its own as a natural result. Many of us,
including me, are confused about where we sit on this. It's true that
people should be encouraged to share their patches with others - as a
centralized system would do, but should this be enforced on people as a
centralized system would do?

What happens in PostgreSQL today with CVS?

From a pragmatic standpoint - there is no such thing as a centralized
system, and there is no such thing as a de-centralized system. People
work on their patches offline - whether they do this by downloading a
tar file and patching a local copy, or whether they use CVS to keep up
to date with HEAD, or whether they employ elaborate mirroring techniques
to insulate themselves from CVS - they are not doing their actual work
on a public centralized server. Only their final committed work -
whatever they choose to commit - reaches the public centralized server.
In many cases, patches are not welcome on the public centralized server
because they are either immature, poorly designed, or contain
unacceptable defects. If I want to work on a piece of PostgreSQL, I
would probably work in private first, then share my changes on the list,
and only once I was confident with my change, would I submit it for
review and possible inclusion. Whether I use GIT or SVN or CVS or
whether I use my local file system and diff between two directory
hierarchies, these are merely tools to accomplish my ends. My process is
basically the same no matter which tool I use. I might be more
comfortable with one tool, and perhaps my productivity is artificially
high on one over another because I am unwilling to invest time in
learning the other - but it's all irrelevant from an open source / free
software perspective. Some code is shared with the world and some is
not. One hopes that the valuable code is shared. :-)

Do you have reason to believe that a de-centralized system would hurt
the future of PostgreSQL? Are there any cases of existing open source
projects that you are aware of, that have broken due to a switch to a
de-centralized system? Or is this fear of the unknown? This is an honest
question - and it is a question I have asked myself.

In my case, I see benefits to a de-centralized system, and benefits to a
centralized system, but find neither to be compelling in terms of
choosing a product. My focus has always been on tighter control of the
changes, reliable merge techniques, and proper change set history
storage and retrieval. I find it unfortunate that the open source / free
software community has been unable to produce a "best of all worlds"
solutions. There is no reason why SVN needs to suck at merging - except
that the people who cared about merging didn't like the look of SVN and
moved on to create other tools instead. :-(

From a PostgreSQL perspective - it is probable inevitable that people
will choose their favourite tool to use, create or re-use existing
mirror technology to have their way, and the side with the most
resources will win and have their way in the end. One hopes for an
overall improvement. :-)

Haha - that's my opinion.

Cheers,
mark

--
Mark Mielke <mark(at)mielke(dot)cc>

Fabien COELHO wrote:
> I'm not sure I would be proud to use such a stupidly named tool for a
> "common work". I really do not share Linus humor, and apparent contempt
> for other people. GIT implements "I want to chose whom I work with, and
> don't care about the others, and don't ever want to have to look at their
> ugly patches", or at least it is what I understood from his talk at Google
> last year. Would this be the future spirit of PG devel? I hope not.

Yea, I saw a video of that speech and thought Linus was an
embarrassment. Was he trying to be funny or engaging perhaps? If so,
it didn't work.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

Stefan Kaltenbrunner escribió:

> yeah - the test install is available on http://reviewdemo.postgresql.org
> if people want to test judge for themself - contact magnus or me if you
> need permissions to do/test stuff there.

Thanks. I tried submitting a review request against anoncvs but it
failed with
No valid separator after the filename was found in the diff header

"patch" can apply the patch correctly -- I'm not sure what does this
patch have that RB does not like.

The patch is attached in case you want to play with it.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

"Heikki Linnakangas" <heikki(at)enterprisedb(dot)com> writes:

> Therefore, we can provide mirrors of the CVS repository in multiple formats.
> And those mirrors exist already, I remember a GIT and a Subversion mirror off
> the top of my head, and I bet there's others. After we have that, the master
> version control system used doesn't matter for developers (except committers),
> as everyone can choose to use whichever mirror he wants. The patches submitted
> to pgsql-patches will look exactly the same regardless of the version control
> system the patch submitter used to check out the source code.

I don't think that's right. Developers care about more than just looking at
individual commits of individual files.

If I have a development version to which I've applied a bunch of pending
patches, then fix some of them I want to be able to generate updated versions
of those patches. I also want to be able to take updated versions of the
patches without having to manually roll back the old versions.

And most importantly I need to be able to take the eventually committed
version. If it's coming from a mirror of a CVS repository then there's no
information of which patch the committer is actually committing or even
anything linking the commits to the various files together.

subversion would allow committers to keep going as they are with a number of
CVS problems eliminated (such as "thou shalt not rename files").

git or its ilk would impact the lives of submitters and reviewers most.
Basically it would allow two non-committers to collaborate, something which we
can't really do effectively now.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's PostGIS support!

On Feb 7, 2008 9:42 PM, Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
> Gregory Stark escribió:
>
> > For what it's worth I think GIT is a better fit for our needs.
>
> Perhaps it would be, if it worked on Windows ... Not that I care, but I
> bet Magnus would.

http://code.google.com/p/msysgit/

"Unfortunately, Git on Windows is only officially supported using
Cygwin. However, there is a fork (currently in the process of being
merged with "official" git) which enables you to compile git using
MinGW/MSys.

This project aims to make installing this port easy. "

I just installed MSys/Git; seems to work...

Evidently this fork is bearing fruit...
--
http://linuxfinances.info/info/linuxdistributions.html
"The definition of insanity is doing the same thing over and over and
expecting different results." -- assortedly attributed to Albert
Einstein, Benjamin Franklin, Rita Mae Brown, and Rudyard Kipling

Alvaro Herrera wrote:
> Stefan Kaltenbrunner escribió:
>
>> yeah - the test install is available on http://reviewdemo.postgresql.org
>> if people want to test judge for themself - contact magnus or me if you
>> need permissions to do/test stuff there.
>
> Thanks. I tried submitting a review request against anoncvs but it
> failed with
> No valid separator after the filename was found in the diff header
>
> "patch" can apply the patch correctly -- I'm not sure what does this
> patch have that RB does not like.
>
> The patch is attached in case you want to play with it.

If I'm not mistaken, I read somewhere in the docs that the diff has to
be unified, not context. Could that be it?

//Magnus

"Fabien COELHO" <coelho(at)cri(dot)ensmp(dot)fr> writes:

> ISTM that a decentralized or distributed SCM for PostgreSQL would be a bad
> move, however great it would be at branching and merging. For me it is a
> philosophy question: if PGSQL is a "common work", then everything should be
> open and shared, and a centralized systems make sense to embodied this.

Well not really. Our current model is that only stuff that's ready for
widespread use goes into CVS. That means "everything" isn't open and shared at
all. "everything" is mostly sitting on people's local hard drives where you
can't use do anything with it, let alone contribute.

The patches mailing list is basically our poor-man's distributed SCM today.
It's awful since a) you never know if you're looking at the most recent
version b) updating your tree from an old version to a newer version is
painful c) people only release versions when they think they have something to
say or a question; they don't know you want to try out their stuff until you
complain about last month's silly bugs d) you never know what version of the
tree the patch was against and of course e) if you make any changes they have
all the same problems dealing with your changes to their patch.

And it's hardly any more centralized than a distributed SCM system would be.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's PostGIS support!

Dear Mark,

> I encourage all to keep their minds open.

Good:-)

My 0.02 EUR (or even less) on the recurrent SCM flame war on the list.

ISTM that a decentralized or distributed SCM for PostgreSQL would be a bad
move, however great it would be at branching and merging. For me it is a
philosophy question: if PGSQL is a "common work", then everything should
be open and shared, and a centralized systems make sense to embodied this.
Even if one can publish one's branch easily with GIT, it's not the same,
because it is still a personnal branch somehow.

> From WordNet (r) 3.0 (2006) [wn]:

git
n 1: a person who is deemed to be despicable or contemptible;
"only a rotter would do that"; "kill the rat"; "throw the
bum out"; "you cowardly little pukes!"; "the British call a
contemptible person a `git'" [syn: {rotter}, {dirty dog},
{rat}, {skunk}, {stinker}, {stinkpot}, {bum}, {puke},
{crumb}, {lowlife}, {scum bag}, {so-and-so}, {git}]

I'm not sure I would be proud to use such a stupidly named tool for a
"common work". I really do not share Linus humor, and apparent contempt
for other people. GIT implements "I want to chose whom I work with, and
don't care about the others, and don't ever want to have to look at their
ugly patches", or at least it is what I understood from his talk at Google
last year. Would this be the future spirit of PG devel? I hope not.

--
Fabien.

Christopher Browne wrote:
> On Feb 7, 2008 9:42 PM, Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
>> Gregory Stark escribió:
>>
>>> For what it's worth I think GIT is a better fit for our needs.
>> Perhaps it would be, if it worked on Windows ... Not that I care, but I
>> bet Magnus would.
>
> http://code.google.com/p/msysgit/
>
> "Unfortunately, Git on Windows is only officially supported using
> Cygwin. However, there is a fork (currently in the process of being
> merged with "official" git) which enables you to compile git using
> MinGW/MSys.
>

Mercurial, which is similar to GIT, offer native windows client. See
there
http://www.selenic.com/mercurial/wiki/index.cgi/BinaryPackages#head-adac70dc1664bb9eac334d5c8b57483d300254f3

It support binaries for most of PG supported platforms.

I also recommend to see following page
http://en.wikipedia.org/wiki/Comparison_of_revision_control_software

It compares a lot of SCMs.

Zdenek

Magnus Hagander wrote:
> Alvaro Herrera wrote:
>> Stefan Kaltenbrunner escribió:
>>
>>> yeah - the test install is available on
>>> http://reviewdemo.postgresql.org if people want to test judge for
>>> themself - contact magnus or me if you need permissions to do/test
>>> stuff there.
>>
>> Thanks. I tried submitting a review request against anoncvs but it
>> failed with
>> No valid separator after the filename was found in the diff header
>>
>> "patch" can apply the patch correctly -- I'm not sure what does this
>> patch have that RB does not like.
>>
>> The patch is attached in case you want to play with it.
>
> If I'm not mistaken, I read somewhere in the docs that the diff has to
> be unified, not context. Could that be it?

Yes, and second problem is CVS diff. When you use CVS diff you must
strip root (/home/alvherre/Code/cvs/) from following line:

RCS file: /home/alvherre/Code/cvs/pgsql/src/backend/commands/analyze.c,v

and ",v" as well. After that RB accepted your patch.

Zdenek

Gregory Stark wrote:
> "Heikki Linnakangas" <heikki(at)enterprisedb(dot)com> writes:
>
>> Therefore, we can provide mirrors of the CVS repository in multiple formats.
>> And those mirrors exist already, I remember a GIT and a Subversion mirror off
>> the top of my head, and I bet there's others. After we have that, the master
>> version control system used doesn't matter for developers (except committers),
>> as everyone can choose to use whichever mirror he wants. The patches submitted
>> to pgsql-patches will look exactly the same regardless of the version control
>> system the patch submitter used to check out the source code.
>
> I don't think that's right. Developers care about more than just looking at
> individual commits of individual files.
>
> If I have a development version to which I've applied a bunch of pending
> patches, then fix some of them I want to be able to generate updated versions
> of those patches. I also want to be able to take updated versions of the
> patches without having to manually roll back the old versions.

Doesn't those capabilities depend only on the version control system
you're using, not on the one used in the master repository.

> And most importantly I need to be able to take the eventually committed
> version.

I've never found that to be a problem. When my patch gets committed, I
sometimes do a diff between my patch and the one that was committed to
see what was changed, but after that i just do fresh checkout. Perhaps
your patches are committed more often than mine? ;-)

> If it's coming from a mirror of a CVS repository then there's no
> information of which patch the committer is actually committing or even
> anything linking the commits to the various files together.

That's not true. At least in the git mirror, the conversion programs
group together commits to different files, so that they form a single
commit in the git repository. Since CVS doesn't have that information
explicitly, it's done by matching commit messages and the commit
timestamp. It seems to work just fine.

> subversion would allow committers to keep going as they are with a number of
> CVS problems eliminated (such as "thou shalt not rename files").

Now that is certainly true.

> git or its ilk would impact the lives of submitters and reviewers most.
> Basically it would allow two non-committers to collaborate, something which we
> can't really do effectively now.

Two git-using non-committers can do that already, regardless of the
master repository.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

2008/2/8, Heikki Linnakangas <heikki(at)enterprisedb(dot)com>:
> Gregory Stark wrote:
> > git or its ilk would impact the lives of submitters and reviewers most.
> > Basically it would allow two non-committers to collaborate, something
which we
> > can't really do effectively now.
>
> Two git-using non-committers can do that already, regardless of the
> master repository.

Maybe the existing SVN, git and other mirrors could just become more
official and supported in the sense that users can rely on them to be
updated often enough? At the moment what is there are some links on
http://developer.postgresql.org/index.php/Working_with_CVS#Other_versions_of_the_PostgreSQL_Repositoryand
no indication of how reliable these repositories are. I suppos that a
lot of reason for discussion would disappear if these repositories were made
official and supported.

Markus

On Friday 08 February 2008 00:52:04 Gregory Stark wrote:

> Well not really. Our current model is that only stuff that's ready for
> widespread use goes into CVS. That means "everything" isn't open and shared
> at all. "everything" is mostly sitting on people's local hard drives where
> you can't use do anything with it, let alone contribute.
>
> The patches mailing list is basically our poor-man's distributed SCM today.
> It's awful since a) you never know if you're looking at the most recent
> version b) updating your tree from an old version to a newer version is
> painful c) people only release versions when they think they have something
> to say or a question; they don't know you want to try out their stuff until
> you complain about last month's silly bugs d) you never know what version
> of the tree the patch was against and of course e) if you make any changes
> they have all the same problems dealing with your changes to their patch.
>
> And it's hardly any more centralized than a distributed SCM system would
> be.

One of the things I would like to see in the project is more modularisation
during development . In particular, it may be useful to allow different
maintainers to look after different parts of the backend, much in the way
that different linux kernel developers are in charge of different subsystems.

This strikes me as being advantageous in a couple of ways:

i) It lowers the bar for entry into the project. Knowing the ins and outs of
one subsystem is going to take a developer much less time than it does to
learn about the entire backend.

ii) Some of the larger patches we have seen during 8.3 would be broken into
smaller chunks based upon the part of the backend they touch; so reviewing 3
or 4 smaller incremental patches across 3 or 4 people will take much less
time than having to wait for someone like Alvaro or Tom to review and commit
several hundred KB of code.

This seems to fit more with the idea of a distributed SCM, although it
wouldn't be entirely out of the question to set up permissions using CVS/SVN.

ATB,

Mark.

--
Mark Cave-Ayland
Sirius Corporation - The Open Source Experts
http://www.siriusit.co.uk
T: +44 870 608 0063

Am Freitag, 8. Februar 2008 schrieb Markus Bertheau:
> Maybe the existing SVN, git and other mirrors could just become more
> official and supported in the sense that users can rely on them to be
> updated often enough?

I think you are right. Some of that is already being worked on. It certainly
seems that a lot of people are interested in these services.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

On Feb 8, 2008 10:29 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> Am Freitag, 8. Februar 2008 schrieb Markus Bertheau:
> > Maybe the existing SVN, git and other mirrors could just become more
> > official and supported in the sense that users can rely on them to be
> > updated often enough?
>
> I think you are right. Some of that is already being worked on. It certainly
> seems that a lot of people are interested in these services.
>

+1

In particular, if the git repos were officially supported, and best
practises for use with Postgres documented, I think a lot more hackers
would be comfortable using git to do their work, which is good for
collaboration (as mentioned by Greg Stark and Heikki upthread).

Seems if we could get some of the advantages of using a modern
distributed SCM without the "hard sell" of changing the central repos,
that's worth going for.

Cheers
BJ

Am Freitag, 8. Februar 2008 schrieb Brendan Jurd:
> In particular, if the git repos were officially supported, and best
> practises for use with Postgres documented, I think a lot more hackers
> would be comfortable using git to do their work, which is good for
> collaboration (as mentioned by Greg Stark and Heikki upthread).

Well, I didn't want to announce anything before anything existed, but this is
precisely what is being worked on. Watch for an announcement in this forum.

--
Peter Eisentraut
http://developer.postgresql.org/~petere/

Peter Eisentraut wrote:
> Am Freitag, 8. Februar 2008 schrieb Brendan Jurd:
>> In particular, if the git repos were officially supported, and best
>> practises for use with Postgres documented, I think a lot more hackers
>> would be comfortable using git to do their work, which is good for
>> collaboration (as mentioned by Greg Stark and Heikki upthread).
>
> Well, I didn't want to announce anything before anything existed, but this is
> precisely what is being worked on. Watch for an announcement in this forum.

I've tried with both the SVN and the GIT mirror. Things worked well
initialled, but in *both* cases pulling changes from the mirror stopped
working after a few weeks or so. It seems that both of these mirrors
create the SVN/GIT repo from scratch every time they are updated,
instead of incrementally pulling the changes from CVS. Since the mapping
of CVS updates to changesets is based on heuristics, the mapping can
change for recent commits upon recreation of the mirror. This confuses
both the GIT and the SVN client, and "svn update" (or "git pull") stops
working :-(.

For GIT, I've found a workaround - I've hacked together a script which
uses git-cherry and git-cherry-pick to find changesets on the GIT mirror
which are not in my local tree.

Is there any chance that these mirrors can be updated in a way that
doesn't "change the past"?

regards, Florian Pflug

The Git repo certainly is an "incremental" update.

If you ever see a "rewind" (non-fastforward) of the the repo.or.cz
PostgreSQL repo, please let me know...

a.

* Florian Pflug <fgp(dot)phlo(dot)org(at)gmail(dot)com> [080208 07:50]:

> I've tried with both the SVN and the GIT mirror. Things worked well
> initialled, but in *both* cases pulling changes from the mirror stopped
> working after a few weeks or so. It seems that both of these mirrors
> create the SVN/GIT repo from scratch every time they are updated,
> instead of incrementally pulling the changes from CVS. Since the mapping
> of CVS updates to changesets is based on heuristics, the mapping can
> change for recent commits upon recreation of the mirror. This confuses
> both the GIT and the SVN client, and "svn update" (or "git pull") stops
> working :-(.
>
> For GIT, I've found a workaround - I've hacked together a script which
> uses git-cherry and git-cherry-pick to find changesets on the GIT mirror
> which are not in my local tree.
>
> Is there any chance that these mirrors can be updated in a way that
> doesn't "change the past"?

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Florian Pflug escribió:

> I've tried with both the SVN and the GIT mirror. Things worked well
> initialled, but in *both* cases pulling changes from the mirror stopped
> working after a few weeks or so. It seems that both of these mirrors
> create the SVN/GIT repo from scratch every time they are updated,
> instead of incrementally pulling the changes from CVS.

Yeah, the SVN mirror in commandprompt.com is recreated from scratch
every time. This sucks, we know -- I think it's on Joshua's list to
change it to update incrementally.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

Aidan Van Dyk wrote:
> The Git repo certainly is an "incremental" update.
>
> If you ever see a "rewind" (non-fastforward) of the the repo.or.cz
> PostgreSQL repo, please let me know...

Hm... interesting...

I'm pretty sure that the "past" changed at least once - at least I once
got loud complaints from git about being unable to merge because there
is no common anchestor, or something like that.

I seem the remember that I fixed that manually, and only switched to
using git-cherry when it happened again - but that memory could be wrong...

For reference, here is the script I use for fetching changesets ATM
--------------------------
#Checkout pgsql-head.
git-checkout pgsql-head 2>&1 || exit 1

#Pull the latest changesets
git-fetch pgsql-upstream-git 2>&1 || exit 1

#Now find all unapplied commits from upstream,
#and commit them
set -o pipefail
nice git-cherry \
pgsql-head \
pgsql-upstream-git/master \
pgsql-upstream-git-lastmerged \
| sed -n 's/^\+ \([A-Fa-f0-9][A-Fa-f0-9]*\)$/\1/p' \
| xargs -n1 --no-run-if-empty \
git-cherry-pick \
2>&1 \
|| exit 1

#Now, update pgsql-upstream-git-lastmerged
git tag -f pgsql-upstream-git-lastmerged pgsql-upstream-git/master \
|| exit 1
--------------------------

regards, Florian Pflug

* Florian Pflug <fgp(dot)phlo(dot)org(at)gmail(dot)com> [080208 09:25]:
> Aidan Van Dyk wrote:
> >The Git repo certainly is an "incremental" update.
> >
> >If you ever see a "rewind" (non-fastforward) of the the repo.or.cz
> >PostgreSQL repo, please let me know...
>
> Hm... interesting...
>
> I'm pretty sure that the "past" changed at least once - at least I once
> got loud complaints from git about being unable to merge because there
> is no common anchestor, or something like that.

Very strange - I don't recall it rewinding ever for me. In fact, I'm
pretty sure it *can't* rewind heads, because I *don't* push with -f.

> I seem the remember that I fixed that manually, and only switched to
> using git-cherry when it happened again - but that memory could be wrong...

Wow, the following scheme seems like an awful workaround for what should
be a simple:

# fetch any remote CVS commits
git fetch # defaults to origin, use whatever remote you prefer

# And now let's try and rebase my changes onto CVS HEAD
git rebase origin/master # again - use whatever remote/branch you want.
<edit and fix conflicts/problems>
git commit && git rebase --continue

> For reference, here is the script I use for fetching changesets ATM
> --------------------------
> #Checkout pgsql-head.
> git-checkout pgsql-head 2>&1 || exit 1
>
> #Pull the latest changesets
> git-fetch pgsql-upstream-git 2>&1 || exit 1
>
> #Now find all unapplied commits from upstream,
> #and commit them
> set -o pipefail
> nice git-cherry \
> pgsql-head \
> pgsql-upstream-git/master \
> pgsql-upstream-git-lastmerged \
> | sed -n 's/^\+ \([A-Fa-f0-9][A-Fa-f0-9]*\)$/\1/p' \
> | xargs -n1 --no-run-if-empty \
> git-cherry-pick \
> 2>&1 \
> || exit 1
>
> #Now, update pgsql-upstream-git-lastmerged
> git tag -f pgsql-upstream-git-lastmerged pgsql-upstream-git/master \
> || exit 1
> --------------------------
>
> regards, Florian Pflug

--
Aidan Van Dyk Create like a god,
aidan(at)highrise(dot)ca command like a king,
http://www.highrise.ca/ work like a slave.

Tom Lane wrote:
> Dimitri Fontaine <dfontaine(at)hi-media(dot)com> writes:
>> Le Wednesday 06 February 2008 21:35:54 Peter Eisentraut, vous avez écrit :
>>> Yes, I feel we could use a group writeable patch queue of some sort.
>>> Perhaps an IMAP server setup could do the job.
>
>> I've read some developers appreciating the way review board works:
>> http://review-board.org/
>> http://code.google.com/p/reviewboard/
>> http://code.google.com/p/reviewboard/wiki/UserBasics
>
> Hmm, the info on that last page might be out of date, but what it says is
> that the only SCMS they really support 100% is SVN. The other ones they
> claim support for don't work [well/at all] with the post-review tool.

Btw, wasnt a group already playing with Trac/svn? This one also has
something like above: http://trac-hacks.org/wiki/PeerReviewPlugin

And a lot of more nice features as well as posgres backend support :)

Greets
Tino

Alvaro Herrera wrote:
> Florian Pflug escribió:
>
>
>> I've tried with both the SVN and the GIT mirror. Things worked well
>> initialled, but in *both* cases pulling changes from the mirror stopped
>> working after a few weeks or so. It seems that both of these mirrors
>> create the SVN/GIT repo from scratch every time they are updated,
>> instead of incrementally pulling the changes from CVS.
>>
>
> Yeah, the SVN mirror in commandprompt.com is recreated from scratch
> every time. This sucks, we know -- I think it's on Joshua's list to
> change it to update incrementally.
>
>

Can you document what you actually do on the developers' wiki?

cheers

andrew

On Fri, 08 Feb 2008 10:25:33 -0500
Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:

> > Yeah, the SVN mirror in commandprompt.com is recreated from scratch
> > every time. This sucks, we know -- I think it's on Joshua's list to
> > change it to update incrementally.
> >
> >
>
> Can you document what you actually do on the developers' wiki?

Which? What we are doing now? Sure, it's pretty dumb simple though. I
am looking at tailor which in theory should allow us to have
incremental updates.

Joshua D. Drake

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

Andrew Dunstan escribió:

> Alvaro Herrera wrote:
>> Florian Pflug escribió:

>> Yeah, the SVN mirror in commandprompt.com is recreated from scratch
>> every time. This sucks, we know -- I think it's on Joshua's list to
>> change it to update incrementally.
>
> Can you document what you actually do on the developers' wiki?

I'd prefer we do not advertise it until it's actually usable.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

On Fri, 8 Feb 2008 12:39:36 -0300
Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:

> Andrew Dunstan escribió:
>
> > Alvaro Herrera wrote:
> >> Florian Pflug escribió:
>
> >> Yeah, the SVN mirror in commandprompt.com is recreated from scratch
> >> every time. This sucks, we know -- I think it's on Joshua's list
> >> to change it to update incrementally.
> >
> > Can you document what you actually do on the developers' wiki?
>
> I'd prefer we do not advertise it until it's actually usable.
>

Wait, what are we talking about? The SVN mirror we have right now is
perfectly usable. It just has to be rebuilt a couple of times a day. Is
that what you want documented? Or is it the incremental stuff you want
documented? I have to have the incremental stuff working before we try
that.

Joshua D. Drake

--
The PostgreSQL Company since 1997: http://www.commandprompt.com/
PostgreSQL Community Conference: http://www.postgresqlconference.org/
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL SPI Liaison | SPI Director | PostgreSQL political pundit

Joshua D. Drake wrote:
> On Fri, 8 Feb 2008 12:39:36 -0300
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> wrote:
>
>> Andrew Dunstan escribió:
>>
>>> Alvaro Herrera wrote:
>>>> Florian Pflug escribió:
>>>> Yeah, the SVN mirror in commandprompt.com is recreated from scratch
>>>> every time. This sucks, we know -- I think it's on Joshua's list
>>>> to change it to update incrementally.
>>> Can you document what you actually do on the developers' wiki?
>> I'd prefer we do not advertise it until it's actually usable.
>>
>
> Wait, what are we talking about? The SVN mirror we have right now is
> perfectly usable. It just has to be rebuilt a couple of times a day. Is
> that what you want documented? Or is it the incremental stuff you want
> documented? I have to have the incremental stuff working before we try
> that.

It used to be that you had problems if you tried to hit it at the same
time as it was updating, IIRC. But it could be that it was fixed a long
time ago :)

//Magnus

Joshua D. Drake wrote:
>>>
>>> Can you document what you actually do on the developers' wiki?
>>>
>> I'd prefer we do not advertise it until it's actually usable.
>>
>>
>
> Wait, what are we talking about? The SVN mirror we have right now is
> perfectly usable. It just has to be rebuilt a couple of times a day. Is
> that what you want documented? Or is it the incremental stuff you want
> documented? I have to have the incremental stuff working before we try
> that.
>
>
>

How about starting by telling us exactly what you're doing now.

cheers

andrew

Andrew Dunstan escribió:
>
> Joshua D. Drake wrote:
>>>>
>>>> Can you document what you actually do on the developers' wiki?
>>>>
>>> I'd prefer we do not advertise it until it's actually usable.
>>
>> Wait, what are we talking about? The SVN mirror we have right now is
>> perfectly usable. It just has to be rebuilt a couple of times a day. Is
>> that what you want documented? Or is it the incremental stuff you want
>> documented? I have to have the incremental stuff working before we try
>> that.

It's usable as long as you checkout a copy and then avoid doing anything
much with it. Trying to actually use it and "svn update" is likely to
cause problems at some point. (I know I tried and failed once. I
learned to avoid the fire when I was a child.)

> How about starting by telling us exactly what you're doing now.

Twice a day a "cvs2svn" process runs, which creates a complete SVN
repository from the complete CVS repository.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.