Re: pg_export_snapshot on standby side

Hi,

We cannot run parallel pg_dump on the standby server because
pg_export_snapshot() always fails on the standby. Is this the oversight
of parallel pg_dump or pg_export_snapshot()?

pg_export_snapshot() fails in the standby because it always assigns
new XID and which is not allowed in the standby. Do we really need
to assign new XID even in the standby for the exportable snapshot?

Regards,

--
Fujii Masao

On 21 May 2013 19:16, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:

> We cannot run parallel pg_dump on the standby server because
> pg_export_snapshot() always fails on the standby. Is this the oversight
> of parallel pg_dump or pg_export_snapshot()?
>
> pg_export_snapshot() fails in the standby because it always assigns
> new XID and which is not allowed in the standby. Do we really need
> to assign new XID even in the standby for the exportable snapshot?

Having looked at the code, I say No, we don't *need* to.

There are various parts of the code that deal with
takenDuringRecovery, so much of this was clearly intended to work in
recovery.

We use the topXid for the name of the snapshot file. That is clearly
unnecessary and we should be using the virtualxid instead like we do
elsewhere. We also use the topXid to test whether it is still running,
though again, we could equally use the virtualxid instead. There is no
problem with virtualxids possibly not being active anymore, since if
we didn't have an xid before and don't have one now, and the xmin is
the same, the snapshot is still valid.

I think we should treat this as a bug and fix it in 9.3 while we're
still in beta. Why? Because once we begin using the topXid as the
filename we can't then change later to using the vxid.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Sat, May 25, 2013 at 6:18 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On 21 May 2013 19:16, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>
>> We cannot run parallel pg_dump on the standby server because
>> pg_export_snapshot() always fails on the standby. Is this the oversight
>> of parallel pg_dump or pg_export_snapshot()?
>>
>> pg_export_snapshot() fails in the standby because it always assigns
>> new XID and which is not allowed in the standby. Do we really need
>> to assign new XID even in the standby for the exportable snapshot?
>
> Having looked at the code, I say No, we don't *need* to.

Good to hear.

> There are various parts of the code that deal with
> takenDuringRecovery, so much of this was clearly intended to work in
> recovery.
>
> We use the topXid for the name of the snapshot file. That is clearly
> unnecessary and we should be using the virtualxid instead like we do
> elsewhere. We also use the topXid to test whether it is still running,
> though again, we could equally use the virtualxid instead. There is no
> problem with virtualxids possibly not being active anymore, since if
> we didn't have an xid before and don't have one now, and the xmin is
> the same, the snapshot is still valid.
>
> I think we should treat this as a bug and fix it in 9.3 while we're
> still in beta.

+1

> Why? Because once we begin using the topXid as the
> filename we can't then change later to using the vxid.

I'm afraid that pg_export_snapshot() in 9.2 has already been using
topXid as the filename.

Regards,

--
Fujii Masao

Fujii Masao escribió:
> On Sat, May 25, 2013 at 6:18 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:

> > I think we should treat this as a bug and fix it in 9.3 while we're
> > still in beta.
>
> +1
>
> > Why? Because once we begin using the topXid as the
> > filename we can't then change later to using the vxid.
>
> I'm afraid that pg_export_snapshot() in 9.2 has already been using
> topXid as the filename.

I don't think this matters at all. The filename is an implementation
detail which we don't even need to keep compatible across pg_upgrade.
If we think this is a bug, we should backpatch the fix to 9.2.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Uh, was this ever addressed? I don't think so.

---------------------------------------------------------------------------

On Sat, May 25, 2013 at 10:18:57AM +0100, Simon Riggs wrote:
> On 21 May 2013 19:16, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>
> > We cannot run parallel pg_dump on the standby server because
> > pg_export_snapshot() always fails on the standby. Is this the oversight
> > of parallel pg_dump or pg_export_snapshot()?
> >
> > pg_export_snapshot() fails in the standby because it always assigns
> > new XID and which is not allowed in the standby. Do we really need
> > to assign new XID even in the standby for the exportable snapshot?
>
> Having looked at the code, I say No, we don't *need* to.
>
> There are various parts of the code that deal with
> takenDuringRecovery, so much of this was clearly intended to work in
> recovery.
>
> We use the topXid for the name of the snapshot file. That is clearly
> unnecessary and we should be using the virtualxid instead like we do
> elsewhere. We also use the topXid to test whether it is still running,
> though again, we could equally use the virtualxid instead. There is no
> problem with virtualxids possibly not being active anymore, since if
> we didn't have an xid before and don't have one now, and the xmin is
> the same, the snapshot is still valid.
>
> I think we should treat this as a bug and fix it in 9.3 while we're
> still in beta. Why? Because once we begin using the topXid as the
> filename we can't then change later to using the vxid.
>
> --
> Simon Riggs http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

On 11 January 2014 18:34, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> Uh, was this ever addressed? I don't think so.

It appears not.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

Seems we still have not addressed this.

---------------------------------------------------------------------------

On Sat, May 25, 2013 at 10:18:57AM +0100, Simon Riggs wrote:
> On 21 May 2013 19:16, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
>
> > We cannot run parallel pg_dump on the standby server because
> > pg_export_snapshot() always fails on the standby. Is this the oversight
> > of parallel pg_dump or pg_export_snapshot()?
> >
> > pg_export_snapshot() fails in the standby because it always assigns
> > new XID and which is not allowed in the standby. Do we really need
> > to assign new XID even in the standby for the exportable snapshot?
>
> Having looked at the code, I say No, we don't *need* to.
>
> There are various parts of the code that deal with
> takenDuringRecovery, so much of this was clearly intended to work in
> recovery.
>
> We use the topXid for the name of the snapshot file. That is clearly
> unnecessary and we should be using the virtualxid instead like we do
> elsewhere. We also use the topXid to test whether it is still running,
> though again, we could equally use the virtualxid instead. There is no
> problem with virtualxids possibly not being active anymore, since if
> we didn't have an xid before and don't have one now, and the xmin is
> the same, the snapshot is still valid.
>
> I think we should treat this as a bug and fix it in 9.3 while we're
> still in beta. Why? Because once we begin using the topXid as the
> filename we can't then change later to using the vxid.
>
> --
> Simon Riggs http://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Training & Services
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ Everyone has their own god. +

On Thu, Aug 7, 2014 at 2:17 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
>
> Seems we still have not addressed this.

Thanks for the reminder :) I'm not sure if I have time to work on this
for a while. If anyone is interested in this, please feel free to try it.

Regards,

--
Fujii Masao

On Thu, Aug 7, 2014 at 01:26:24PM +0900, Fujii Masao wrote:
> On Thu, Aug 7, 2014 at 2:17 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> >
> > Seems we still have not addressed this.
>
> Thanks for the reminder :) I'm not sure if I have time to work on this
> for a while. If anyone is interested in this, please feel free to try it.

Added to TODO:

Allow pg_export_snapshot() to run on hot standby servers

This will allow parallel pg_dump on such servers.

pg_export_snapshot on standby side

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription +