Re: Big 7.4 items

I wanted to outline some of the big items we are looking at for 7.4:

Win32 Port:

Katie Ward and Jan are working on contributing their Win32
port for 7.4. They plan to have a patch available by the end of
December.

Point-In-Time Recovery (PITR)

J. R. Nield did a PITR patch late in 7.3 development, and Patrick
MacDonald from Red Hat is working on merging it into CVS and
adding any missing pieces. Patrick, do you have an ETA on that?

Replication

I have talked to Darren Johnson and I believe 7.4 is the time to
merge the Postgres-R source tree into our main CVS. Most of the
replication code will be in its own directory, with only minor
changes to our existing tree. They have single-master
replication working now, so we may have that feature in some
capacity for 7.4. I know others are working on replication
solutions. This is probably the time to decide for certain if
this is the direction we want to go for replication. Most who
have have studied Postgres-R feel it is the most promising
multi-master replication solution for reliably networked hosts.

Comments?

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On 13 Dec 2002 at 1:22, Bruce Momjian wrote:
> Replication
>
> I have talked to Darren Johnson and I believe 7.4 is the time to
> merge the Postgres-R source tree into our main CVS. Most of the
> replication code will be in its own directory, with only minor
> changes to our existing tree. They have single-master
> replication working now, so we may have that feature in some
> capacity for 7.4. I know others are working on replication
> solutions. This is probably the time to decide for certain if
> this is the direction we want to go for replication. Most who
> have have studied Postgres-R feel it is the most promising
> multi-master replication solution for reliably networked hosts.
>
> Comments?

Some.

1) What kind of replication are we looking at? log file replay/syncnronous etc.
If it is real time, like usogres( I hope I am in line with things here), that
would be real good .Choice is always good..

2 If we are going to have replication, can we have built in load balancing? Is
it a good idea to have it in postgresql or a separate application would be way
to go?

And where are nested transactions?

Bye
Shridhar

--
Booker's Law: An ounce of application is worth a ton of abstraction.

On Fri, 2002-12-13 at 06:22, Bruce Momjian wrote:
> I wanted to outline some of the big items we are looking at for 7.4:
> Point-In-Time Recovery (PITR)
>
> J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> MacDonald from Red Hat is working on merging it into CVS and
> adding any missing pieces. Patrick, do you have an ETA on that?

How hard would it be to extend PITR for master-slave (hot backup)
repliaction, which should then amount to continuously shipping logs to
slave and doing nonstop PITR there :)

It will never be usable for multi-master replication, but somehow it
feels that for master-slave replication simple log replay would be most
simple and robust solution.

--
Hannu Krosing <hannu(at)tm(dot)ee>

Bruce Momjian wrote:
> I wanted to outline some of the big items we are looking at for 7.4:
>
> Win32 Port:
>
> Katie Ward and Jan are working on contributing their Win32
> port for 7.4. They plan to have a patch available by the end of
> December.
>
> Point-In-Time Recovery (PITR)
>
> J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> MacDonald from Red Hat is working on merging it into CVS and
> adding any missing pieces. Patrick, do you have an ETA on that?
>
> Replication
>
> I have talked to Darren Johnson and I believe 7.4 is the time to
> merge the Postgres-R source tree into our main CVS. Most of the
> replication code will be in its own directory, with only minor
> changes to our existing tree. They have single-master
> replication working now, so we may have that feature in some
> capacity for 7.4. I know others are working on replication
> solutions. This is probably the time to decide for certain if
> this is the direction we want to go for replication. Most who
> have have studied Postgres-R feel it is the most promising
> multi-master replication solution for reliably networked hosts.
>
> Comments?

What about distributed TX support:

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=20021106111554.69ae1dcd.pgsql%40snaga.org&rnum=2&prev=/groups%3Fq%3DNAGAYASU%2BSatoshi%26ie%3DUTF-8%26oe%3DUTF-8%26hl%3Den

Mike Mascari
mascarm(at)mascari(dot)com

Bruce Momjian wrote:
> Win32 Port:
>
> Katie Ward and Jan are working on contributing their Win32
> port for 7.4. They plan to have a patch available by the end of
> December.

I have .Net Studio available to me, so if you need help in merging or testing
or whatever, let me know.

> Point-In-Time Recovery (PITR)
>
> J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> MacDonald from Red Hat is working on merging it into CVS and
> adding any missing pieces. Patrick, do you have an ETA on that?

As Hannu asked (and related to your question below), is there any thought of
extending this to allow simple log based replication? In many important
scenarios that would be more than adequate, and simpler to set up.

> Replication
>
> I have talked to Darren Johnson and I believe 7.4 is the time to
> merge the Postgres-R source tree into our main CVS. Most of the
> replication code will be in its own directory, with only minor
> changes to our existing tree. They have single-master
> replication working now, so we may have that feature in some
> capacity for 7.4. I know others are working on replication
> solutions. This is probably the time to decide for certain if
> this is the direction we want to go for replication. Most who
> have have studied Postgres-R feel it is the most promising
> multi-master replication solution for reliably networked hosts.

I'd question if we would want the one-and-only builtin replication method to
be dependent on an external communication library (Spread). I would like to
see Postgres-R merged, but I'd also like to see a simple log-based option.

> Comments?
>

I'd also second Mike Mascari's question -- whatever happened to the person
working on two-phase commit? Is that likely to be done for 7.4? Did he ever
send in a patch?

Joe

Shridhar Daithankar wrote:
> And where are nested transactions?

I didn't mention nested transactions because it didn't seem to be a
_big_ item like the others.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Friday 13 December 2002 17:51, Bruce Momjian wrote:
> Shridhar Daithankar wrote:
> > And where are nested transactions?
>
> I didn't mention nested transactions because it didn't seem to be a
> _big_ item like the others.

This is big item

regards
Haris Peco

Shridhar Daithankar wrote:
> 1) What kind of replication are we looking at? log file
> replay/synchronous etc. If it is real time, like usogres( I
> hope I am in line with things here), that would be real good.
> Choice is always good.

Good. This is the discussion we need. Let me quote the TODO list
replication section first:

* Add replication of distributed databases [replication]
o automatic failover
o load balancing
o master/slave replication
o multi-master replication
o partition data across servers
o sample implementation in contrib/rserv
o queries across databases or servers (two-phase commit)
o allow replication over unreliable or non-persistent links
o http://gborg.postgresql.org/project/pgreplication/projdisplay.php

OK, the first thing is that there isn't any one replication solution
that will behave optimally in all situations.

Now, let me describe Postgres-R and then the other replication
solutions. Postgres-R is multi-master, meaning you can send SELECT and
UPDATE/DELETE queries to any of the servers in the cluster, and get the
same result. It is also synchronous, meaning it doesn't update the
local copy until it is sure the other nodes agree to the change. It
allows failover, because if one node goes down, the others keep going.

Now, let me contrast:

rserv and dbmirror do master/slave. There is no mechanism to allow you
to do updates on the slave, and have them propagate to the master. You
can, however, send SELECT queries to the slave, and in fact that's how
usogres does load balancing.

Two-phase commit is probably the most popular commercial replication
solution. While it works for multi-master, it suffers from poor
performance and doesn't handle cases where one node disappears very
well.

Another replication need is for asynchronous replication, most
traditionally for traveling salesmen who need to update their databases
periodically. The only solution I know for that is PeerDirect's
PostgreSQL commercial offering at http://www.peerdirect.com. It is
possible PITR may help with this, but we need to handle propagating
changes made by the salesmen back up into the server, and to do that, we
will need a mechanism to handle conflicts that occur when two people
update the same records. This is always a problem for asynchronous
replication.

> 2 If we are going to have replication, can we have built in load
> balancing? Is it a good idea to have it in postgresql or a
> separate application would be way to go?

Well, because Postgres-R is multi-master, it has automatic load
balancing. You can have your users point to whatever node you want.
You can implement this "pointing" by using dns IP address cycling, or
have a router that auto-load balances, though you would need to keep a
db session on the same node, of course.

So, in summary, I think we will eventually have two directions for
replication. One is Postgres-R for multi-master, synchronous
replication, and PITR, for asynchronous replication. I don't think
there is any value to use PITR for synchronous replication because by
definition, you don't _store_ the changes for later use because it is
synchronous. In synchronous, you communicate your changes to all the
nodes involved, then commit them.

I will describe the use of 'spread' and the Postgres-R internal issues
in my next email.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Hannu Krosing wrote:
> On Fri, 2002-12-13 at 06:22, Bruce Momjian wrote:
> > I wanted to outline some of the big items we are looking at for 7.4:
> > Point-In-Time Recovery (PITR)
> >
> > J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> > MacDonald from Red Hat is working on merging it into CVS and
> > adding any missing pieces. Patrick, do you have an ETA on that?
>
> How hard would it be to extend PITR for master-slave (hot backup)
> repliaction, which should then amount to continuously shipping logs to
> slave and doing nonstop PITR there :)
>
> It will never be usable for multi-master replication, but somehow it
> feels that for master-slave replication simple log replay would be most
> simple and robust solution.

Exactly. See my previous email. We eventually have two replication
solutions: one, Postgres-R for multi-master, and PITR used for for
asynchonous master/slave.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Mike Mascari wrote:
> What about distributed TX support:
>
> http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=20021106111554.69ae1dcd.pgsql%40snaga.org&rnum=2&prev=/groups%3Fq%3DNAGAYASU%2BSatoshi%26ie%3DUTF-8%26oe%3DUTF-8%26hl%3Den

OK, yes, that is Satoshi's 2-phase commit implementation. I will
address 2-phase commit vs Postgres-R in my next email about spread.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Okay. But please keep in mind that a 2-phase commit implementation is used for more than just replication. Any distributed TX will require a 2PC protocol. As an example, for the DBLINK implementation to ultimately be transaction safe (at least amongst multiple PostgreSQL installations), the players in the distributed transaction must all be participants in a 2PC exchange. And a participant whose communications link is dropped needs to be able to recover by asking the coordinator whether or not to complete or abort the distributed TX. I am 100% ignorant of the distributed TX standard Tom referenced earlier, but I'd guess there might be an assumption of 2PC support in the implementation. In other words, I think we still need 2PC, regardless of the method of replication. And if Satoshi Nagayasu has an implementation ready, why not investigate its possibilities?

Mike Mascari
mascarm(at)mascari(dot)com

----- Original Message -----
From: "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us>

> Mike Mascari wrote:
> > What about distributed TX support:

> OK, yes, that is Satoshi's 2-phase commit implementation. I will
> address 2-phase commit vs Postgres-R in my next email about spread.

Joe Conway wrote:
> Bruce Momjian wrote:
> > Win32 Port:
> >
> > Katie Ward and Jan are working on contributing their Win32
> > port for 7.4. They plan to have a patch available by the end of
> > December.
>
> I have .Net Studio available to me, so if you need help in merging or testing
> or whatever, let me know.

OK, Jan, let him know how he can help.

> > Point-In-Time Recovery (PITR)
> >
> > J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> > MacDonald from Red Hat is working on merging it into CVS and
> > adding any missing pieces. Patrick, do you have an ETA on that?
>
> As Hannu asked (and related to your question below), is there any thought of
> extending this to allow simple log based replication? In many important
> scenarios that would be more than adequate, and simpler to set up.

Yes, see previous email.

> I'd question if we would want the one-and-only builtin replication method to
> be dependent on an external communication library (Spread). I would like to
> see Postgres-R merged, but I'd also like to see a simple log-based option.

OK, let me reiterate I think we will have two replication solutions in
the end --- on Postgres-R for multi-master/synchronous, and PITR for
master/slave asynchronous replication.

Let me address the Spread issue and two-phase commit. (Spread is an
open source piece of software used by Postgres-R.)

In two-phase commit, when one node is about to commit, it gets a lock
from all the other nodes, does its commit, then releases the lock. (Of
course, this is simplified.) It is called two-phase because it says to
all the other nodes "I am about to do something, is that OK?", then when
gets all OK's, it does the commit and says "I did the commit".

Postgres-R uses a different mechanism. This method is shown on page 22
and 24 and following of:

ftp://gborg.postgresql.org/pub/pgreplication/stable/PostgreSQLReplication.pdf.gz

The basic difference is that Spread groups all the write sets into a
queue who's ordering is the same on all the nodes. Instead of asking
for approval for a commit, a node puts its commit in the Spread queue,
and then waits for its own commit to come back in the queue, meaning all
the other nodes saw its commit too.

The only tricky part is that while reading the other node's write sets
before its own arrives, it has to check to see if any of these conflict
with its own write set. If it conflicts, it has to assume the earlier
write set succeeded and its own failed. It also has to check the write
set stream and apply only those changes that don't conflict.

As stated before in Postgres-R discussion, this mechanism hinges on
being able to determine which write sets conflict because there is no
explicit "I aborted", only a stream of write sets, and each node has to
accept the non-conflicting ones and reject the conflicting ones.

> I'd also second Mike Mascari's question -- whatever happened to the person
> working on two-phase commit? Is that likely to be done for 7.4? Did he ever
> send in a patch?

I have not seen a patch from him, but it is very likely he could have
one for 7.4. This is why it is good we discuss this now and figure out
where we want to go for 7.4 so we can get started.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Mike Mascari wrote:
> Okay. But please keep in mind that a 2-phase commit implementation
> is used for more than just replication. Any distributed TX will
> require a 2PC protocol. As an example, for the DBLINK implementation
> to ultimately be transaction safe (at least amongst multiple
> PostgreSQL installations), the players in the distributed
> transaction must all be participants in a 2PC exchange. And a
> participant whose communications link is dropped needs to be
> able to recover by asking the coordinator whether or not to
> complete or abort the distributed TX. I am 100% ignorant of the
> distributed TX standard Tom referenced earlier, but I'd guess
> there might be an assumption of 2PC support in the implementation.
> In other words, I think we still need 2PC, regardless of the
> method of replication. And if Satoshi Nagayasu has an implementation
> ready, why not investigate its possibilities?

This is a good point. I don't want to push Postgres-R as our solution.
Rather, I have looked at both and like Postgres-R, but others need to
look at both and decide so we are all in agreement when we move forward.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:

> OK, the first thing is that there isn't any one replication solution
> that will behave optimally in all situations.

Right

> Now, let me describe Postgres-R and then the other replication
> solutions. Postgres-R is multi-master, meaning you can send SELECT and
> UPDATE/DELETE queries to any of the servers in the cluster, and get the
> same result. It is also synchronous, meaning it doesn't update the
> local copy until it is sure the other nodes agree to the change. It
> allows failover, because if one node goes down, the others keep going.

Wrong

It is asynchronous without the need of 2 phase commit. It is group
communication based and requires the group communication system to
guarantee total order. The tricky part is, that the local transaction
must be on hold until the own commit message comes back without a prior
lock conflict by a replication transaction. If such a lock confict
occurs, the replication transaction wins and the local transaction rolls
back.

>
> Now, let me contrast:
>
> rserv and dbmirror do master/slave. There is no mechanism to allow you
> to do updates on the slave, and have them propagate to the master. You
> can, however, send SELECT queries to the slave, and in fact that's how
> usogres does load balancing.

But you cannot use the result of such a SELECT to update anything. So
you can only phase out complete read only transaction to the slaves.
Requires support from the application since the load balancing system
cannot know automatically what will be a read only transaction and what
not.

>
> Two-phase commit is probably the most popular commercial replication
> solution. While it works for multi-master, it suffers from poor
> performance and doesn't handle cases where one node disappears very
> well.
>
> Another replication need is for asynchronous replication, most
> traditionally for traveling salesmen who need to update their databases
> periodically. The only solution I know for that is PeerDirect's
> PostgreSQL commercial offering at http://www.peerdirect.com. It is
> possible PITR may help with this, but we need to handle propagating
> changes made by the salesmen back up into the server, and to do that, we
> will need a mechanism to handle conflicts that occur when two people
> update the same records. This is always a problem for asynchronous
> replication.

PITR doesn't help here at all, since PeerDirect's replication is trigger
and control table based. What makes our replication system unique is
that it works bidirectional in a heterogenious world.

> I will describe the use of 'spread' and the Postgres-R internal issues
> in my next email.

The last time i was playing with spread (that was at Great Bridge in
Norfolk), it was IMHO useless (for Postgres-R) because it sometimes
dropped messages when the network load got too high. This occured
without any indication, no error, nothing. This is not exactly what I
understand as total order. I hope they have made some substantial
progress on that.

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #

----- Original Message -----
From: "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us>

> Mike Mascari wrote:
> > Okay. But please keep in mind that a 2-phase commit implementation
> > is used for more than just replication.
>
> This is a good point. I don't want to push Postgres-R as our solution.
> Rather, I have looked at both and like Postgres-R, but others need to
> look at both and decide so we are all in agreement when we move forward.
>

After having read your post regarding Spread, I see that it is an alternative to 2PC as a distributed TX protocol. If I understand you correctly, a DBLINK implementation built atop Spread would also be possible. Correct? The question then is, do other RDBMS expose a 2PC implementation which could not then be leveraged at a later time? For example imagine:

1. 7.4 includes a native 2PC protocol with:

CREATE DATABASE LINK accounting
CONNECT TO accounting.acme.com:5432
IDENTIFIED BY mascarm/mascarm;

SELECT *
FROM employees(at)accounting;

INSERT INTO employees(at)accounting
VALUES (1, 'Mike', 'Mascari');

That would be great, allowing PostgreSQL servers running in different departments to participate in a distributed tx.

2. 7.5 includes a DBLINK which supports PostgreSQL participating in a heterogenous distributed transaction (with say, an Oracle database):

CREATE DATABASE LINK finance
CONNECT TO <oracle names entry>
IDENTIFIED BY mascarm/mascarm
USING INTERFACE 'pg2oracle.so';

INSERT INTO employees(at)finance
VALUES (1, 'Mike', 'Mascari');

I guess I'm basically asking:

1) Is it necessary to *choose* between support for 2PC and Spread (Postgres-R) or can't we have both? Spread for Replication, 2PC for non-replicating distributed TX?

2) Do major SQL DBMS vendors which support distributed options expose a callable interface into a 2PC protocol that would allow PostgreSQL to participate? I could check on this...

3) Are there any standards (besides ODBC, which, the last time I looked just had COMMIT/ABORT APIs), that have been defined and adopted by the industry for distributed tx?

Again, I'd guess most people want:

1) High performance Master/Master replication *and* (r.e. Postgres-R)
2) Ability to participate in distrubuted tx's (r.e. 2PC?)

Mike Mascari
mascarm(at)mascari(dot)com

On Fri, 2002-12-13 at 13:20, Bruce Momjian wrote:
> Let me address the Spread issue and two-phase commit. (Spread is an
> open source piece of software used by Postgres-R.)

Note that while Spread is open source in the sense that "the source is
available", it's license is significantly more restrictive than
PostgreSQL's:

http://www.spread.org/license/

Just FYI...

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC

Neil Conway wrote:
> On Fri, 2002-12-13 at 13:20, Bruce Momjian wrote:
> > Let me address the Spread issue and two-phase commit. (Spread is an
> > open source piece of software used by Postgres-R.)
>
> Note that while Spread is open source in the sense that "the source is
> available", it's license is significantly more restrictive than
> PostgreSQL's:
>
> http://www.spread.org/license/
>

Interesting. It looks like a modified version of the old BSD license
where you are required to mention you are using Spread. I believe we
can get that reduced. (I think Darren already addressed this with
them.) We certainly are not going to accept software that requires all
PostgreSQL user sites to mention Spread.

The whole "mention" aspect of the old BSD license was pretty ambiguous,
and I assume this is similar.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

I wrote:
>
> I guess I'm basically asking:
>
> 1) Is it necessary to *choose* between support for 2PC and Spread (Postgres-R) or can't we have both? Spread for Replication, 2PC for non-replicating distributed TX?
>
> 2) Do major SQL DBMS vendors which support distributed options expose a callable interface into a 2PC protocol that would allow PostgreSQL to participate? I could check on this...
>
> 3) Are there any standards (besides ODBC, which, the last time I looked just had COMMIT/ABORT APIs), that have been defined and adopted by the industry for distributed tx?

Answer:

The Open Group's Open/XA C193 specificiation for API for distributed transactions:

http://www.opengroup.org/public/pubs/catalog/c193.htm

I couldn't find any draft copies on the web, but a good description at the Sybase site:

http://manuals.sybase.com/onlinebooks/group-xs/xsg1111e/xatuxedo/@ebt-link;pt=61?target=%25N%13_446_START_RESTART_N%25

The standard is 2PC based.

Mike Mascari
mascarm(at)mascari(dot)com

Bruce Momjian wrote:
>
> Joe Conway wrote:
> > Bruce Momjian wrote:
> > > Win32 Port:
> > >
> > > Katie Ward and Jan are working on contributing their Win32
> > > port for 7.4. They plan to have a patch available by the end of
> > > December.
> >
> > I have .Net Studio available to me, so if you need help in merging or testing
> > or whatever, let me know.
>
> OK, Jan, let him know how he can help.

My current plan is to comb out the Win32 port only from what we've done
all together against 7.2.1. The result should be a clean patch that
applied against 7.2.1 builds a native windows port.

From there, this patch must be lifted up to 7.4.

I have the original context diff now down from 160,000 lines to 80,000
lines. I think I will have the clean diff against 7.2.1 somewhere next
week. That would IMHO be a good time for Tom to start complaining so
that we can work in the required changes during the 7.4 lifting. ;-)

Jan

--
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #

Jan Wieck wrote:
> Bruce Momjian wrote:
>
> > OK, the first thing is that there isn't any one replication solution
> > that will behave optimally in all situations.
>
> Right
>
> > Now, let me describe Postgres-R and then the other replication
> > solutions. Postgres-R is multi-master, meaning you can send SELECT and
> > UPDATE/DELETE queries to any of the servers in the cluster, and get the
> > same result. It is also synchronous, meaning it doesn't update the
> > local copy until it is sure the other nodes agree to the change. It
> > allows failover, because if one node goes down, the others keep going.
>
> Wrong
>
> It is asynchronous without the need of 2 phase commit. It is group

Well, Darren's PDF at:

ftp://gborg.postgresql.org/pub/pgreplication/stable/PostgreSQLReplication.pdf.gz

calls Postgres-R "Type: Embedded, Peer-to-Peer, Sync". I don't know
enough about replication so I will let you fight it out with him. ;-)

> communication based and requires the group communication system to
> guarantee total order. The tricky part is, that the local transaction
> must be on hold until the own commit message comes back without a prior
> lock conflict by a replication transaction. If such a lock confict
> occurs, the replication transaction wins and the local transaction rolls
> back.

Yep, that's the tricky part.

> >
> > Now, let me contrast:
> >
> > rserv and dbmirror do master/slave. There is no mechanism to allow you
> > to do updates on the slave, and have them propagate to the master. You
> > can, however, send SELECT queries to the slave, and in fact that's how
> > usogres does load balancing.
>
> But you cannot use the result of such a SELECT to update anything. So
> you can only phase out complete read only transaction to the slaves.
> Requires support from the application since the load balancing system
> cannot know automatically what will be a read only transaction and what
> not.

Good point. It has to be a read-only session because you can't jump
nodes during a session. That definately limits its usefulness.

> > Two-phase commit is probably the most popular commercial replication
> > solution. While it works for multi-master, it suffers from poor
> > performance and doesn't handle cases where one node disappears very
> > well.
> >
> > Another replication need is for asynchronous replication, most
> > traditionally for traveling salesmen who need to update their databases
> > periodically. The only solution I know for that is PeerDirect's
> > PostgreSQL commercial offering at http://www.peerdirect.com. It is
> > possible PITR may help with this, but we need to handle propagating
> > changes made by the salesmen back up into the server, and to do that, we
> > will need a mechanism to handle conflicts that occur when two people
> > update the same records. This is always a problem for asynchronous
> > replication.
>
> PITR doesn't help here at all, since PeerDirect's replication is trigger
> and control table based. What makes our replication system unique is
> that it works bidirectional in a heterogenious world.

I was only suggesting that PITR _may_ help as an archive method for the
changes. PeerDirect stores those changes in a table?

> > I will describe the use of 'spread' and the Postgres-R internal issues
> > in my next email.
>
> The last time i was playing with spread (that was at Great Bridge in
> Norfolk), it was IMHO useless (for Postgres-R) because it sometimes
> dropped messages when the network load got too high. This occured
> without any indication, no error, nothing. This is not exactly what I
> understand as total order. I hope they have made some substantial
> progress on that.

That's a serious problem, clearly. Hopefully it is either fixed or it
will get fixed. We can't use it without reliability.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Fri, 2002-12-13 at 13:36, Jan Wieck wrote:
> But you cannot use the result of such a SELECT to update anything. So
> you can only phase out complete read only transaction to the slaves.
> Requires support from the application since the load balancing system
> cannot know automatically what will be a read only transaction and what
> not.

Interesting -- SQL contains the concept of "read only" and "read write"
transactions (the default is RW). If we implemented that (which
shouldn't be too difficult[1]), it might make differentiating between
classes of transactions a little easier. Client applications would still
need to be modified, but not nearly as much.

Does this sound like it's worth doing?

[1] -- AFAICS, the only tricky implementation detail is deciding exactly
which database operations are "writes". Does nextval() count, for
example?

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC

Neil Conway wrote:
> On Fri, 2002-12-13 at 13:36, Jan Wieck wrote:
> > But you cannot use the result of such a SELECT to update anything. So
> > you can only phase out complete read only transaction to the slaves.
> > Requires support from the application since the load balancing system
> > cannot know automatically what will be a read only transaction and what
> > not.
>
> Interesting -- SQL contains the concept of "read only" and "read write"
> transactions (the default is RW). If we implemented that (which
> shouldn't be too difficult[1]), it might make differentiating between
> classes of transactions a little easier. Client applications would still
> need to be modified, but not nearly as much.
>
> Does this sound like it's worth doing?
>
> [1] -- AFAICS, the only tricky implementation detail is deciding exactly
> which database operations are "writes". Does nextval() count, for
> example?

You can't migrate a session between nodes, so the entire _session_ has
to be read-only.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

> This is a good point. I don't want to push Postgres-R as our solution.
> Rather, I have looked at both and like Postgres-R, but others need to
> look at both and decide so we are all in agreement when we move forward.

I think in either way, it's clear that they need to be in the main CVS, in
order for it to get up to speed.

Chris

On Fri, 2002-12-13 at 20:20, Christopher Kings-Lynne wrote:
> > This is a good point. I don't want to push Postgres-R as our solution.
> > Rather, I have looked at both and like Postgres-R, but others need to
> > look at both and decide so we are all in agreement when we move forward.
>
> I think in either way, it's clear that they need to be in the main CVS, in
> order for it to get up to speed.

Why's that?

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC

On Friday 13 December 2002 11:01 pm, you wrote:
> Good. This is the discussion we need. Let me quote the TODO list
> replication section first:
>
> * Add replication of distributed databases [replication]
> o automatic failover

Very good. We need that for HA.

> o load balancing

> o master/slave replication
> o multi-master replication
> o partition data across servers

I am interested in this for multitude of reasons. Scalability is obviously one
of them. But I was just wondering about some things(After going thr. all the
messages on this).

Once we have partitioning and replication, that effectively means database
cache can span multiple machines and no longer restricted by shared memory.
So will it work on mosix now? Just a thought.

> OK, the first thing is that there isn't any one replication solution
> that will behave optimally in all situations.
>
> Now, let me describe Postgres-R and then the other replication
> solutions. Postgres-R is multi-master, meaning you can send SELECT and
> UPDATE/DELETE queries to any of the servers in the cluster, and get the
> same result. It is also synchronous, meaning it doesn't update the
> local copy until it is sure the other nodes agree to the change. It
> allows failover, because if one node goes down, the others keep going.
>
> Now, let me contrast:
>
> rserv and dbmirror do master/slave. There is no mechanism to allow you
> to do updates on the slave, and have them propagate to the master. You
> can, however, send SELECT queries to the slave, and in fact that's how
> usogres does load balancing.

Seems like mirroring v/s striping to me. Can we have both combined in either
fashion just like RAID.

Most importantly will it be able to resize the cluster on the fly? Are we
looking at network management of database like Oracle does. (OK the tools are
unwarranted in many situation but it has to offer it).

Most importantly I would like to see this thing easy to setup from a one
point-of-administration view.

Something like declare a cluster of these n1 machines as database partitions
and have these another n2 machine do a slave sync with them for handling
loads. If these kind of command-line options are there, adding easy tools on
top of them should be a pop.

And please, in place database upgrades. Otherwise it will be a huge pain to
maintain such a cluster over long period of times.

> Another replication need is for asynchronous replication, most
> traditionally for traveling salesmen who need to update their databases
> periodically. The only solution I know for that is PeerDirect's
> PostgreSQL commercial offering at http://www.peerdirect.com. It is
> possible PITR may help with this, but we need to handle propagating
> changes made by the salesmen back up into the server, and to do that, we
> will need a mechanism to handle conflicts that occur when two people
> update the same records. This is always a problem for asynchronous
> replication.

We need not offer entire asynchronous replication all at once. We can have
levels of asynchronous replication like read only(Syncing essentially) and
Read-write. Even if we get slave sync only at first, that will be huge plus.

> > 2 If we are going to have replication, can we have built in load
> > balancing? Is it a good idea to have it in postgresql or a
> > separate application would be way to go?
>
> Well, because Postgres-R is multi-master, it has automatic load
> balancing. You can have your users point to whatever node you want.
> You can implement this "pointing" by using dns IP address cycling, or
> have a router that auto-load balances, though you would need to keep a
> db session on the same node, of course.

Umm. W.r.t above point i.e. combining data partitioning and slave-sync, will
it take a partitioned cluster as a single server or that cluster can take
care of itself in such situattions?

>
> So, in summary, I think we will eventually have two directions for
> replication. One is Postgres-R for multi-master, synchronous
> replication, and PITR, for asynchronous replication. I don't think

I would put that as two options rather than directions. We need to be able to
deploy them both if required.

Imagine postgresql running over 500 machine cluster..;-)

Shridhar

Bruce Momjian wrote:
> Joe Conway wrote:
<snip>
>>>Point-In-Time Recovery (PITR)
>>>
>>> J. R. Nield did a PITR patch late in 7.3 development, and Patrick
>>> MacDonald from Red Hat is working on merging it into CVS and
>>> adding any missing pieces. Patrick, do you have an ETA on that?
>>
>>As Hannu asked (and related to your question below), is there any thought of
>>extending this to allow simple log based replication? In many important
>>scenarios that would be more than adequate, and simpler to set up.
<snip>

For PITR-log-based-replication, how much data would be required to be pushed out to each slave system in order to bring
it up to date?

I'm having visions of a 16MB WAL file being pushed out to slave systems in order to update them with a few rows of data...

:-/

Regards and best wishes,

Justin Clift

--
"My grandfather once told me that there are two kinds of people: those
who work and those who take the credit. He told me to try to be in the
first group; there was less competition there."
- Indira Gandhi

On 14 Dec 2002 at 18:02, Justin Clift wrote:
> For PITR-log-based-replication, how much data would be required to be pushed out to each slave system in order to bring
> it up to date?
>
> I'm having visions of a 16MB WAL file being pushed out to slave systems in order to update them with a few rows of data...

I was under impression that data is pushed to slave after a checkpoint is
complete. i.e. 16MB of WAL file has recycled.

Conversely a slave would contain accurate data upto last WAL checkpoint.

I think tunable WAL size should be of some help in such scenario. Otherwise the
system designer has to use async. replication. for granularity upto a
transaction.

Bye
Shridhar

--
Conference, n.: A special meeting in which the boss gathers subordinates to
hear what they have to say, so long as it doesn't conflict with what he's
already decided to do.

> > I think in either way, it's clear that they need to be in the main CVS, in
> > order for it to get up to speed.
>
> Why's that?

Because until replication is in CVS, it won't be used, tested and improved
and developed as fast...

Chris

On Fri, 2002-12-13 at 04:53, Hannu Krosing wrote:
> On Fri, 2002-12-13 at 06:22, Bruce Momjian wrote:
> > I wanted to outline some of the big items we are looking at for 7.4:
> > Point-In-Time Recovery (PITR)
> >
> > J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> > MacDonald from Red Hat is working on merging it into CVS and
> > adding any missing pieces. Patrick, do you have an ETA on that?
>
> How hard would it be to extend PITR for master-slave (hot backup)
> repliaction, which should then amount to continuously shipping logs to
> slave and doing nonstop PITR there :)
>
> It will never be usable for multi-master replication, but somehow it
> feels that for master-slave replication simple log replay would be most
> simple and robust solution.

I'm curious, what would be the recovery strategy for PITR master-slave
replication should the master fail (assuming hot fail over/backup)? A
simple dump/restore? Are there/is there any facilities in PorstgreSQL
for PITR archival which prevents PITR logs from be recycled (or perhaps,
simply archived off)? What about PITR streaming to networked and/or
removable media?

--
Greg Copeland <greg(at)copelandconsulting(dot)net>
Copeland Computer Consulting

On Monday 16 December 2002 07:26 pm, you wrote:
> I'm curious, what would be the recovery strategy for PITR master-slave
> replication should the master fail (assuming hot fail over/backup)? A
> simple dump/restore? Are there/is there any facilities in PorstgreSQL
> for PITR archival which prevents PITR logs from be recycled (or perhaps,
> simply archived off)? What about PITR streaming to networked and/or
> removable media?

In asynchrounous replication, WAL log records are fed to anoter host, which
replays those transactions to sync the data. This way it does not matter if
WAL log is recycled as it is already replicated someplace else..

HTH

Shridhar

I must of miscommunicated here as you're describing PITR replication.
I'm asking about a master failing and the slaving picking up. Now, some
n-time later, how do you recover your master system to be back in sync
with the slave. Obviously, I'm assuming some level of manual recovery.
I'm wondering what the general approach would be.

Consider that on the slave which is now the active server (master dead),
it's possible that the slave's PITR's will be recycled before the master
can come back up. As such, unless there is a, an archival process for
PITR or b, a method of streaming PITR's off for archival, the odds of
using PITR to recover the master (resync if you will) seem greatly
reduced as you will be unable to replay PITR on the master for
synchronization.

Greg

On Mon, 2002-12-16 at 08:02, Shridhar Daithankar wrote:
> On Monday 16 December 2002 07:26 pm, you wrote:
> > I'm curious, what would be the recovery strategy for PITR master-slave
> > replication should the master fail (assuming hot fail over/backup)? A
> > simple dump/restore? Are there/is there any facilities in PorstgreSQL
> > for PITR archival which prevents PITR logs from be recycled (or perhaps,
> > simply archived off)? What about PITR streaming to networked and/or
> > removable media?
>
> In asynchrounous replication, WAL log records are fed to anoter host, which
> replays those transactions to sync the data. This way it does not matter if
> WAL log is recycled as it is already replicated someplace else..
>
> HTH
>
> Shridhar
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
--
Greg Copeland <greg(at)copelandconsulting(dot)net>
Copeland Computer Consulting

On Monday 16 December 2002 07:43 pm, you wrote:
> Consider that on the slave which is now the active server (master dead),
> it's possible that the slave's PITR's will be recycled before the master
> can come back up. As such, unless there is a, an archival process for
> PITR or b, a method of streaming PITR's off for archival, the odds of
> using PITR to recover the master (resync if you will) seem greatly
> reduced as you will be unable to replay PITR on the master for
> synchronization.

I agree. Since we are talking about features in future release, I think it
should be added to TODO if not already there.

I don't know about WAL numbering but AFAIU, it increments and old files are
removed once there are enough WAL files as specified in posgresql.conf. IIRC
there are some perl based replication project exist already which use this
feature.

Shridhar

On Mon, 2002-12-16 at 08:20, Shridhar Daithankar wrote:
> On Monday 16 December 2002 07:43 pm, you wrote:
> > Consider that on the slave which is now the active server (master dead),
> > it's possible that the slave's PITR's will be recycled before the master
> > can come back up. As such, unless there is a, an archival process for
> > PITR or b, a method of streaming PITR's off for archival, the odds of
> > using PITR to recover the master (resync if you will) seem greatly
> > reduced as you will be unable to replay PITR on the master for
> > synchronization.
>
> I agree. Since we are talking about features in future release, I think it
> should be added to TODO if not already there.
>
> I don't know about WAL numbering but AFAIU, it increments and old files are
> removed once there are enough WAL files as specified in posgresql.conf. IIRC
> there are some perl based replication project exist already which use this
> feature.
>

The problem with this is that most people, AFAICT, are going to size WAL
based on their performance/sizing requirements and not based on
theoretical estimates which someone might make to allow for a window of
failure. That is, I don't believe increasing the number of WAL's is
going to satisfactorily address the issue.

--
Greg Copeland <greg(at)copelandconsulting(dot)net>
Copeland Computer Consulting

On Monday 16 December 2002 08:07 pm, you wrote:
> On Mon, 2002-12-16 at 08:20, Shridhar Daithankar wrote:
> > I don't know about WAL numbering but AFAIU, it increments and old files
> > are removed once there are enough WAL files as specified in
> > posgresql.conf. IIRC there are some perl based replication project exist
> > already which use this feature.
>
> The problem with this is that most people, AFAICT, are going to size WAL
> based on their performance/sizing requirements and not based on
> theoretical estimates which someone might make to allow for a window of
> failure. That is, I don't believe increasing the number of WAL's is
> going to satisfactorily address the issue.

Sorry for not being clear. When I said, WAL numbering, I meant WAL naming
conventions where numbers are used to mark WAL files.

It is not number of WAL files. It is entirely upto the installation and IIRC,
even in replication project(Sorry I forgot the exact name), you can set
number of WAL files that it can have.

Shridhar

Shridhar Daithankar wrote:
> On Monday 16 December 2002 08:07 pm, you wrote:
> > On Mon, 2002-12-16 at 08:20, Shridhar Daithankar wrote:
> > > I don't know about WAL numbering but AFAIU, it increments and old files
> > > are removed once there are enough WAL files as specified in
> > > posgresql.conf. IIRC there are some perl based replication project exist
> > > already which use this feature.
> >
> > The problem with this is that most people, AFAICT, are going to size WAL
> > based on their performance/sizing requirements and not based on
> > theoretical estimates which someone might make to allow for a window of
> > failure. That is, I don't believe increasing the number of WAL's is
> > going to satisfactorily address the issue.
>
> Sorry for not being clear. When I said, WAL numbering, I meant WAL naming
> conventions where numbers are used to mark WAL files.
>
> It is not number of WAL files. It is entirely upto the installation and IIRC,
> even in replication project(Sorry I forgot the exact name), you can set
> number of WAL files that it can have.

Basically, PITR is going to have a way to archive off a log of database
changes, either from WAL or from somewhere else. At some point, there
is going to have to be administrative action which says, "I have a
master down for three days. I am going to have to save my PITR logs for
that period." So, PITR will probably be used for recovery of a failed
master, and such recover is going to have to require some administrative
action _if_ the automatic expiration of PITR logs is shorter than the
duration the master is down.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Bruce Momjian wrote:
>
> I wanted to outline some of the big items we are looking at for 7.4:
>
> [snip]
>
> Point-In-Time Recovery (PITR)
>
> J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> MacDonald from Red Hat is working on merging it into CVS and
> adding any missing pieces. Patrick, do you have an ETA on that?

Neil Conway and I will be working on this starting the beginning
of January. By the middle of January, we hope to have a handle on
an ETA.

Cheers,
Patrick
--
Patrick Macdonald
Red Hat Database Development

Patrick Macdonald wrote:
> Bruce Momjian wrote:
> >
> > I wanted to outline some of the big items we are looking at for 7.4:
> >
> > [snip]
> >
> > Point-In-Time Recovery (PITR)
> >
> > J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> > MacDonald from Red Hat is working on merging it into CVS and
> > adding any missing pieces. Patrick, do you have an ETA on that?
>
> Neil Conway and I will be working on this starting the beginning
> of January. By the middle of January, we hope to have a handle on
> an ETA.

Ewe, that is later than I was hoping. I have put J.R's PITR patch up
at:

ftp://candle.pha.pa.us/pub/postgresql/PITR_20020822_02.gz

(I have tried to contact J.R. several times over the past few months,
with no reply.)

J.R felt it was ready to go. I would like to have an evaluation of the
patch to know what it does and what is missing. I would like that
sooner rather than later because:

o I want to avoid too much code drift
o I don't want to find there are major pieces missing and to
not have enough time to implement them in 7.4
o It is a big feature so I would like sufficient testing before beta

OK, I just talked to Patrick on the phone, and he says Neil Conway is
working on merging the code into 7.3, and adding missing pieces like
logging table creation. So, it seems PITR is moving forward. Neil, can
you comment on where you are with this, and what still needs to be done?
Do we need to start looking at log archiving options? How is the PITR
log contents different from the WAL log contents, except of course no
pre-write page images?

If we need to discuss things, perhaps we can do it now and get folks
working on other pieces, or at least thinking about them.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

On Mon, 2002-12-16 at 13:38, Bruce Momjian wrote:
> OK, I just talked to Patrick on the phone, and he says Neil Conway is
> working on merging the code into 7.3, and adding missing pieces like
> logging table creation. So, it seems PITR is moving forward. Neil, can
> you comment on where you are with this, and what still needs to be done?

Well, I should have a preliminary merge of the old PITR patch with CVS
HEAD done by Wednesday or Thursday. It took me a while to merge because
(a) I've got final exams at university at the moment (b) I had to merge
most of it by hand, as much of the diff is a single hunk (!), for some
reason.

As for the status of the code, I haven't really had a chance to evaluate
it; as Patrick noted, I think we should be able to give you an ETA by
the middle of January or so (I'll be offline starting Thursday until the
first week of January).

Cheers,

Neil
--
Neil Conway <neilc(at)samurai(dot)com> || PGP Key ID: DB3C29FC

Bruce Momjian wrote:
>
> Patrick Macdonald wrote:
> > Bruce Momjian wrote:
> > >
> > > I wanted to outline some of the big items we are looking at for 7.4:
> > >
> > > [snip]
> > >
> > > Point-In-Time Recovery (PITR)
> > >
> > > J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> > > MacDonald from Red Hat is working on merging it into CVS and
> > > adding any missing pieces. Patrick, do you have an ETA on that?
> >
> > Neil Conway and I will be working on this starting the beginning
> > of January. By the middle of January, we hope to have a handle on
> > an ETA.
>
> Ewe, that is later than I was hoping. I have put J.R's PITR patch up
> at:
>
> ftp://candle.pha.pa.us/pub/postgresql/PITR_20020822_02.gz
>
> (I have tried to contact J.R. several times over the past few months,
> with no reply.)
>
> J.R felt it was ready to go. I would like to have an evaluation of the
> patch to know what it does and what is missing. I would like that
> sooner rather than later because:
>
> o I want to avoid too much code drift
> o I don't want to find there are major pieces missing and to
> not have enough time to implement them in 7.4
> o It is a big feature so I would like sufficient testing before beta
>
> OK, I just talked to Patrick on the phone, and he says Neil Conway is
> working on merging the code into 7.3, and adding missing pieces like
> logging table creation. So, it seems PITR is moving forward.

Well, sort of. I stated that Neil was already working on merging the
patch into the CVS tip. I also mentioned that there are missing
pieces but have no idea if Neil is currently working on them.

Cheers,
Patrick
--
Patrick Macdonald
Red Hat Database Development

Patrick Macdonald wrote:
> > OK, I just talked to Patrick on the phone, and he says Neil Conway is
> > working on merging the code into 7.3, and adding missing pieces like
> > logging table creation. So, it seems PITR is moving forward.
>
> Well, sort of. I stated that Neil was already working on merging the
> patch into the CVS tip. I also mentioned that there are missing
> pieces but have no idea if Neil is currently working on them.

Oh, OK. What I would like to do is find out what actually needs to be
done so we can get folks started on it. If we can get a 7.3 merge,
maybe we should get it into CVS and then list the items needing
attention and folks can submit patches to implement those.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

The file "ftp://candle.pha.pa.us/pub/postgresql/PITR_20020822_02.gz"
do not
have read permissions to copy. please provide the read permissions to copy.

Regards
jana

>Patrick Macdonald wrote:
>
>
>>Bruce Momjian wrote:
>>
>>
>>>I wanted to outline some of the big items we are looking at for 7.4:
>>>
>>>[snip]
>>>
>>>Point-In-Time Recovery (PITR)
>>>
>>> J. R. Nield did a PITR patch late in 7.3 development, and Patrick
>>> MacDonald from Red Hat is working on merging it into CVS and
>>> adding any missing pieces. Patrick, do you have an ETA on that?
>>>
>>>
>>Neil Conway and I will be working on this starting the beginning
>>of January. By the middle of January, we hope to have a handle on
>>an ETA.
>>
>>
>
>Ewe, that is later than I was hoping. I have put J.R's PITR patch up
>at:
>
> ftp://candle.pha.pa.us/pub/postgresql/PITR_20020822_02.gz
>
>(I have tried to contact J.R. several times over the past few months,
>with no reply.)
>
>J.R felt it was ready to go. I would like to have an evaluation of the
>patch to know what it does and what is missing. I would like that
>sooner rather than later because:
>
> o I want to avoid too much code drift
> o I don't want to find there are major pieces missing and to
> not have enough time to implement them in 7.4
> o It is a big feature so I would like sufficient testing before beta
>
>OK, I just talked to Patrick on the phone, and he says Neil Conway is
>working on merging the code into 7.3, and adding missing pieces like
>logging table creation. So, it seems PITR is moving forward. Neil, can
>you comment on where you are with this, and what still needs to be done?
>Do we need to start looking at log archiving options? How is the PITR
>log contents different from the WAL log contents, except of course no
>pre-write page images?
>
>If we need to discuss things, perhaps we can do it now and get folks
>working on other pieces, or at least thinking about them.
>
>
>

Oops, sorry. Permissions fixed.

---------------------------------------------------------------------------

Janardhan wrote:
> The file "ftp://candle.pha.pa.us/pub/postgresql/PITR_20020822_02.gz"
> do not
> have read permissions to copy. please provide the read permissions to copy.
>
> Regards
> jana
>
> >Patrick Macdonald wrote:
> >
> >
> >>Bruce Momjian wrote:
> >>
> >>
> >>>I wanted to outline some of the big items we are looking at for 7.4:
> >>>
> >>>[snip]
> >>>
> >>>Point-In-Time Recovery (PITR)
> >>>
> >>> J. R. Nield did a PITR patch late in 7.3 development, and Patrick
> >>> MacDonald from Red Hat is working on merging it into CVS and
> >>> adding any missing pieces. Patrick, do you have an ETA on that?
> >>>
> >>>
> >>Neil Conway and I will be working on this starting the beginning
> >>of January. By the middle of January, we hope to have a handle on
> >>an ETA.
> >>
> >>
> >
> >Ewe, that is later than I was hoping. I have put J.R's PITR patch up
> >at:
> >
> > ftp://candle.pha.pa.us/pub/postgresql/PITR_20020822_02.gz
> >
> >(I have tried to contact J.R. several times over the past few months,
> >with no reply.)
> >
> >J.R felt it was ready to go. I would like to have an evaluation of the
> >patch to know what it does and what is missing. I would like that
> >sooner rather than later because:
> >
> > o I want to avoid too much code drift
> > o I don't want to find there are major pieces missing and to
> > not have enough time to implement them in 7.4
> > o It is a big feature so I would like sufficient testing before beta
> >
> >OK, I just talked to Patrick on the phone, and he says Neil Conway is
> >working on merging the code into 7.3, and adding missing pieces like
> >logging table creation. So, it seems PITR is moving forward. Neil, can
> >you comment on where you are with this, and what still needs to be done?
> >Do we need to start looking at log archiving options? How is the PITR
> >log contents different from the WAL log contents, except of course no
> >pre-write page images?
> >
> >If we need to discuss things, perhaps we can do it now and get folks
> >working on other pieces, or at least thinking about them.
> >
> >
> >
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

So if this gets added to the 7.3 branch, will there be documentation
accompanying it?

-tfo

In article <200212161838(dot)gBGIcb717436(at)candle(dot)pha(dot)pa(dot)us>,
pgman(at)candle(dot)pha(dot)pa(dot)us (Bruce Momjian) wrote:

> OK, I just talked to Patrick on the phone, and he says Neil Conway is
> working on merging the code into 7.3, and adding missing pieces like
> logging table creation. So, it seems PITR is moving forward.

I meant he is merging it into HEAD, not the 7.3 CVS. Sorry for the
confusion.

---------------------------------------------------------------------------

Thomas O'Connell wrote:
> So if this gets added to the 7.3 branch, will there be documentation
> accompanying it?
>
> -tfo
>
> In article <200212161838(dot)gBGIcb717436(at)candle(dot)pha(dot)pa(dot)us>,
> pgman(at)candle(dot)pha(dot)pa(dot)us (Bruce Momjian) wrote:
>
> > OK, I just talked to Patrick on the phone, and he says Neil Conway is
> > working on merging the code into 7.3, and adding missing pieces like
> > logging table creation. So, it seems PITR is moving forward.
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073