Re: Autonomous Transaction (WIP)

Lists: pgsql-hackers
From: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
To: Simon Riggs <simon(at)2ndQuadrant(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-04-15 06:30:31
Message-ID: BF2827DCCE55594C8D7A8F7FFD3AB7713DDE136A@SZXEML508-MBX.china.huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 14 April 2014 20:10, Simon Riggs wrote:

>>Autonomous Transaction Storage:
>>As for main transaction, structure PGXACT is used to store main transactions, which are created in shared memory of size:
>> (Number of process)*sizeof(struct PGXACT)
>>Similarly a new structure will be defined to store autonomous transaction:
>> Struct PGAutonomousXACT

Oh...I had already added this patch for 2014-June CommitFest, thinking that everyone is busy with work to wrap up 9.4.

> I already proposed exactly this design two years ago and it was rejected at the PgCon hackers meeting.
> I have a better design worked out now and will likely be working on it for 9.5

Can we work together to take this feature to final goal.
May be you can go through my complete patch and see whatever part of the patch and related design can be re-used along with your new design.
Also if possible you can share your design (even rough is OK), I will see if I can contribute to that in some-way.

Thanks and Regards,
Kumar Rajeev Rastogi


From: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-16 20:30:45
Message-ID: 20140616203045.GD18688@eldon.alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

What's the status of this patch?

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


From: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
To: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-18 09:19:53
Message-ID: BF2827DCCE55594C8D7A8F7FFD3AB7713DE01695@SZXEML508-MBX.china.huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 17 June 2014 02:01, Alvaro Herrera Wrote:

> What's the status of this patch?

I have completed work on this and some more changes are done on top of earlier patch shared:
1. Fixed all of the issues observed.
2. Addressed some of the feedback from community like
a. Change the syntax to
START AUTONOMOUS TRANSACTION [READ ONLY | READ WRITE]
b. As Pavan had pointed, I have made transaction behavior (only read-only properties) of main and autonomous transaction independent.
3. Added documentation for this feature.
4. Rebased to latest git code.

Please find the attached latest patch and provide opinion.

Thanks and Regards,
Kumar Rajeev Rastogi

Attachment Content-Type Size
autonomous_tx_v2.patch application/octet-stream 113.5 KB

From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-24 16:39:13
Message-ID: CAFj8pRD5zoA=6PqSncPJ2+sckFOvH5yJAWw-r4ow893V1070Ew@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hello

regress tests fails:

plancache ... ok
limit ... ok
plpgsql ... ok
copy2 ... ok
temp ... FAILED
domain ... ok
rangefuncs ... ok
prepare ... ok
without_oid ... ok
conversion ... ok
truncate ... ok
alter_table ... ok
sequence ... ok

I did some small tests and it works well. When I looked to code, I was
surprised by hardcoded max nesting level of autonomous transactions

#define MAX_AUTOX_NESTING_LEVEL 3

why? Is not it too restrictive?

I am missing a regress tests.

Regards

Pavel

2014-06-18 11:19 GMT+02:00 Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>:

> On 17 June 2014 02:01, Alvaro Herrera Wrote:
>
> > What's the status of this patch?
>
> I have completed work on this and some more changes are done on top of
> earlier patch shared:
> 1. Fixed all of the issues observed.
> 2. Addressed some of the feedback from community like
> a. Change the syntax to
> START AUTONOMOUS TRANSACTION [READ ONLY | READ
> WRITE]
> b. As Pavan had pointed, I have made transaction behavior (only
> read-only properties) of main and autonomous transaction independent.
> 3. Added documentation for this feature.
> 4. Rebased to latest git code.
>
> Please find the attached latest patch and provide opinion.
>
> Thanks and Regards,
> Kumar Rajeev Rastogi
>
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>
>

Attachment Content-Type Size
regression.diffs application/octet-stream 1.4 KB

From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-24 16:40:53
Message-ID: CAFj8pRBOZiZgNUePGJ8Fj_AJ2WhYJEOYu76QzF775np=OaVwOQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

postgres=# select version();

version
-----------------------------------------------------------------------------------------------------------------
PostgreSQL 9.5devel on x86_64-unknown-linux-gnu, compiled by gcc (GCC)
4.8.2 20131212 (Red Hat 4.8.2-7), 64-bit
(1 row)

2014-06-24 18:39 GMT+02:00 Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>:

> Hello
>
> regress tests fails:
>
> plancache ... ok
> limit ... ok
> plpgsql ... ok
> copy2 ... ok
> temp ... FAILED
> domain ... ok
> rangefuncs ... ok
> prepare ... ok
> without_oid ... ok
> conversion ... ok
> truncate ... ok
> alter_table ... ok
> sequence ... ok
>
> I did some small tests and it works well. When I looked to code, I was
> surprised by hardcoded max nesting level of autonomous transactions
>
> #define MAX_AUTOX_NESTING_LEVEL 3
>
> why? Is not it too restrictive?
>
> I am missing a regress tests.
>
> Regards
>
> Pavel
>
>
>
>
>
>
> 2014-06-18 11:19 GMT+02:00 Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>:
>
>> On 17 June 2014 02:01, Alvaro Herrera Wrote:
>>
>> > What's the status of this patch?
>>
>> I have completed work on this and some more changes are done on top of
>> earlier patch shared:
>> 1. Fixed all of the issues observed.
>> 2. Addressed some of the feedback from community like
>> a. Change the syntax to
>> START AUTONOMOUS TRANSACTION [READ ONLY | READ
>> WRITE]
>> b. As Pavan had pointed, I have made transaction behavior (only
>> read-only properties) of main and autonomous transaction independent.
>> 3. Added documentation for this feature.
>> 4. Rebased to latest git code.
>>
>> Please find the attached latest patch and provide opinion.
>>
>> Thanks and Regards,
>> Kumar Rajeev Rastogi
>>
>>
>>
>> --
>> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-hackers
>>
>>
>

Attachment Content-Type Size
temp.out application/octet-stream 3.9 KB

From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-24 16:47:59
Message-ID: CAFj8pRArDxCGFCkd5DK+ZkTtbQwh0XNjWh_taaoTBZ-9FV=h_A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hello

There are lot of unnecessary block over one statement in code

+ if ((inAutoX) && (chunk == events->head) && ((char *)event
< afterTriggers->events_stack[my_level].tailfree))
+ {
+ continue;
+ }
+

and there a few too long lines

Regards

Pavel

2014-06-24 18:40 GMT+02:00 Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>:

> postgres=# select version();
>
> version
>
> -----------------------------------------------------------------------------------------------------------------
> PostgreSQL 9.5devel on x86_64-unknown-linux-gnu, compiled by gcc (GCC)
> 4.8.2 20131212 (Red Hat 4.8.2-7), 64-bit
> (1 row)
>
>
>
> 2014-06-24 18:39 GMT+02:00 Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>:
>
> Hello
>>
>> regress tests fails:
>>
>> plancache ... ok
>> limit ... ok
>> plpgsql ... ok
>> copy2 ... ok
>> temp ... FAILED
>> domain ... ok
>> rangefuncs ... ok
>> prepare ... ok
>> without_oid ... ok
>> conversion ... ok
>> truncate ... ok
>> alter_table ... ok
>> sequence ... ok
>>
>> I did some small tests and it works well. When I looked to code, I was
>> surprised by hardcoded max nesting level of autonomous transactions
>>
>> #define MAX_AUTOX_NESTING_LEVEL 3
>>
>> why? Is not it too restrictive?
>>
>> I am missing a regress tests.
>>
>> Regards
>>
>> Pavel
>>
>>
>>
>>
>>
>>
>> 2014-06-18 11:19 GMT+02:00 Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>:
>>
>>> On 17 June 2014 02:01, Alvaro Herrera Wrote:
>>>
>>> > What's the status of this patch?
>>>
>>> I have completed work on this and some more changes are done on top of
>>> earlier patch shared:
>>> 1. Fixed all of the issues observed.
>>> 2. Addressed some of the feedback from community like
>>> a. Change the syntax to
>>> START AUTONOMOUS TRANSACTION [READ ONLY | READ
>>> WRITE]
>>> b. As Pavan had pointed, I have made transaction behavior (only
>>> read-only properties) of main and autonomous transaction independent.
>>> 3. Added documentation for this feature.
>>> 4. Rebased to latest git code.
>>>
>>> Please find the attached latest patch and provide opinion.
>>>
>>> Thanks and Regards,
>>> Kumar Rajeev Rastogi
>>>
>>>
>>>
>>> --
>>> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
>>> To make changes to your subscription:
>>> http://www.postgresql.org/mailpref/pgsql-hackers
>>>
>>>
>>
>


From: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
To: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-27 11:22:53
Message-ID: BF2827DCCE55594C8D7A8F7FFD3AB7713DE0F537@SZXEML508-MBX.china.huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 24 June 2014 22:18, Pavel Stehule Wrote:

Thanks for looking into this patch.

>There are lot of unnecessary block over one statement in code

>+ if ((inAutoX) && (chunk == events->head) && ((char *)event < afterTriggers->events_stack[my_level].tailfree))
>+ {
>+ continue;
>+ }
>+
>and there a few too long lines
I have removed unnecessary blocks and long lines are broken in shorted lines.

> plancache ... ok
> limit ... ok
> plpgsql ... ok
> copy2 ... ok
> temp ... FAILED
> domain ... ok
> rangefuncs ... ok
> prepare ... ok
> without_oid ... ok
> conversion ... ok
> truncate ... ok
> alter_table ... ok
> sequence ... ok
Fixed the regression test failure.
>I did some small tests and it works well. When I looked to code, I was surprised by hardcoded max nesting level of autonomous transactions

>#define MAX_AUTOX_NESTING_LEVEL 3
>why? Is not it too restrictive?
Yes you are right. I had plan to make it configurable parameters but wanted to take feedback of community. Please let me know if configurable
parameter with minimum value as zero (which is also default value to disable this feature) and maximum as 100 will be OK.? In current patch this
Change is not available.
Apart from the issue reported by you, in the latest patch I have also added support for local transaction ID for autonomous transaction also, as it has dependency on
CONCURRENT INDEX and CHECKPOINT.

Updated patch is attached.

Thanks and Regards,
Kumar Rajeev Rastogi

Attachment Content-Type Size
autonomous_tx_v3.patch application/octet-stream 130.1 KB

From: Abhijit Menon-Sen <ams(at)2ndQuadrant(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-30 10:38:56
Message-ID: 20140630103856.GA32185@toroid.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

If I understand correctly, the design of this patch has already been
considered earlier and rejected. So I guess the patch should also be
marked rejected?

-- Abhijit


From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-06-30 17:19:57
Message-ID: CAFj8pRBwgjhv__LcDG615hnARLu4-3TRjp9DdjoCO5_yH_CAog@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

2014-06-30 12:38 GMT+02:00 Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>:

> If I understand correctly, the design of this patch has already been
> considered earlier and rejected. So I guess the patch should also be
> marked rejected?
>

I didn't find a related message.

?

Regards

Pavel

>
> -- Abhijit
>


From: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
To: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-01 06:16:38
Message-ID: BF2827DCCE55594C8D7A8F7FFD3AB7713DE134B6@SZXEML508-MBX.china.huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 30 June 2014 22:50, Pavel Stehule Wrote:
2014-06-30 12:38 GMT+02:00 Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com<mailto:ams(at)2ndquadrant(dot)com>>:
>>If I understand correctly, the design of this patch has already been
>>considered earlier and rejected. So I guess the patch should also be
>>marked rejected?

>I didn't find a related message.
>?
I think there have been some confusion, the design idea were never rejected but yes there were few feedback/ concern, which I had clarified. Also some of the other concerns are already fixed in latest patch.
So I wanted to have this patch in commitfest application, so that we can have a healthy discussion and rectify all the issues.
But now I see that this patch has already been moved to rejected category, which will put break on further review.
So is there any way to bring back and continue reviewing this patch.
Please let me know if any issue or I am missing something.

Thanks and Regards,
Kumar Rajeev Rastogi


From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-01 06:22:50
Message-ID: CAFj8pRCLzdVR5qdGHuiHSnqoW07jYBu_BvU9qcVtBE-5b=m75Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

2014-07-01 8:16 GMT+02:00 Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>:

> On 30 June 2014 22:50, Pavel Stehule Wrote:
>
> 2014-06-30 12:38 GMT+02:00 Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>:
>
> >>If I understand correctly, the design of this patch has already been
> >>considered earlier and rejected. So I guess the patch should also be
> >>marked rejected?
>
>
>
> >I didn't find a related message.
> >?
>
> I think there have been some confusion, the design idea were never
> rejected but yes there were few feedback/ concern, which I had clarified.
> Also some of the other concerns are already fixed in latest patch.
>
> So I wanted to have this patch in commitfest application, so that we can
> have a healthy discussion and rectify all the issues.
>
> But now I see that this patch has already been moved to rejected category,
> which will put break on further review.
>
> So is there any way to bring back and continue reviewing this patch.
>
> Please let me know if any issue or I am missing something.
>

I didn't watch a discuss about internal implementation, but now, when I am
testing this feature - it works well.

Surely - this feature has important but with relatively large impact and
should be extremely well tested. Now there are no any special test.
Probably we can reuse a tests for nested transactions.

I prefer this feature will be part of first commitfest due high complexity.

Regards

Pavel

>
>
> *Thanks and Regards,*
>
> *Kumar Rajeev Rastogi*
>


From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-01 06:29:40
Message-ID: CAA4eK1LrUed_=L+ZP7zgLNMAXd3uhRgKiiJp2EWm+SJGqORkVA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, Jul 1, 2014 at 11:46 AM, Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
wrote:
> On 30 June 2014 22:50, Pavel Stehule Wrote:
>
> >I didn't find a related message.
> >?
>
> I think there have been some confusion, the design idea were never
rejected but yes there were few feedback/ concern, which I had clarified.
Also some of the other concerns are already fixed in latest patch.

Simon has mentioned that exactly this idea has been rejected at
PGCon 2 years back. Please refer that in below mail:
http://www.postgresql.org/message-id/BF2827DCCE55594C8D7A8F7FFD3AB7713DDE136A@SZXEML508-MBX.china.huawei.com

As far as I can see, you never came back with the different solution.

Have you checked the discussion in Developer meeting notes. Please
check the same at below link:
http://wiki.postgresql.org/wiki/PgCon_2012_Developer_Meeting#Autonomous_Transactions

> So I wanted to have this patch in commitfest application, so that we can
have a healthy discussion and rectify all the issues.
> But now I see that this patch has already been moved to rejected
category, which will put break on further review.

I believe ideally this patch should have been marked as
"Returned with feedback" as you already got a feedback long
back and never come up with solution for same.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com


From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>, Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-01 06:56:22
Message-ID: CAFj8pRAfLteQJA6ercASJhUPzH5F2WZYUQmWofzs_6F5=mJuJQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

2014-07-01 8:29 GMT+02:00 Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>:

> On Tue, Jul 1, 2014 at 11:46 AM, Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
> wrote:
> > On 30 June 2014 22:50, Pavel Stehule Wrote:
> >
> > >I didn't find a related message.
> > >?
> >
> > I think there have been some confusion, the design idea were never
> rejected but yes there were few feedback/ concern, which I had clarified.
> Also some of the other concerns are already fixed in latest patch.
>
> Simon has mentioned that exactly this idea has been rejected at
> PGCon 2 years back. Please refer that in below mail:
>
> http://www.postgresql.org/message-id/BF2827DCCE55594C8D7A8F7FFD3AB7713DDE136A@SZXEML508-MBX.china.huawei.com
>
> As far as I can see, you never came back with the different solution.
>
> Have you checked the discussion in Developer meeting notes. Please
> check the same at below link:
>
> http://wiki.postgresql.org/wiki/PgCon_2012_Developer_Meeting#Autonomous_Transactions
>

Are these notes still valid?

* Why autonomous transaction should be close to functions? We can
implement AT as first step and next step can be implementation of
integration AT to stored procedures.

* When autonomous transaction is independent on parent transaction, then
locks parent and autonomous transaction should be in conflict

I though about integration to PL/pgSQL and I don't think so close
integration between autonomous transaction and procedure is optimal. More
practical is design so autonomous transaction is similar to subtransaction.

Then we can simply wrote some code like

BEGIN
.. some code
WHEN OTHERS THEN
.. I would to write permanently to log
BEGIN AUTONOMOUS
INSERT INTO log VALUES(..);
WHEN OTHERS
RAISE WARNING 'Cannot to write to log ..';
RAISE EXCEPTION ' ...' forward up exception from autonomous
transaction to parent transaction
END
END;

Now I am thinking so PL/SQL design of autonomous transactions is relatively
limited and is not best to follow it.

Regards

Pavel

>
> > So I wanted to have this patch in commitfest application, so that we can
> have a healthy discussion and rectify all the issues.
> > But now I see that this patch has already been moved to rejected
> category, which will put break on further review.
>
> I believe ideally this patch should have been marked as
> "Returned with feedback" as you already got a feedback long
> back and never come up with solution for same.
>
> With Regards,
> Amit Kapila.
> EnterpriseDB: http://www.enterprisedb.com
>


From: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
To: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-01 08:38:11
Message-ID: BF2827DCCE55594C8D7A8F7FFD3AB7713DE1355E@SZXEML508-MBX.china.huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 01 July 2014 12:26, Pavel Stehule Wrote:

>>Have you checked the discussion in Developer meeting notes. Please
>>check the same at below link:
>>http://wiki.postgresql.org/wiki/PgCon_2012_Developer_Meeting#Autonomous_Transactions

>Are these notes still valid?
>* Why autonomous transaction should be close to functions? We can implement AT as first step and next step can be implementation of integration AT to stored procedures.
We have implemented AT on the line of sub-transaction. Also we have integrated AT with stored procedure i.e. we can create an autonomous transaction inside the store procedure, which can be also committed.
>* When autonomous transaction is independent on parent transaction, then locks parent and autonomous transaction should be in conflict
Yes our implementation makes the autonomous transaction independent of main transaction and hence as per our design parent (main) transaction and autonomous may get conflicted. For which we have implemented deadlock detection mechanism between autonomous transaction and its parent transaction.
> I though about integration to PL/pgSQL and I don't think so close integration between autonomous transaction and procedure is optimal. More practical is design so autonomous transaction is similar to subtransaction.
Yes as mentioned above, our implementation of autonomous transaction is on track of subtransaction.
>Then we can simply wrote some code like
> BEGIN
> .. some code
> WHEN OTHERS THEN
> .. I would to write permanently to log
> BEGIN AUTONOMOUS
> INSERT INTO log VALUES(..);
> WHEN OTHERS
> RAISE WARNING 'Cannot to write to log ..';
> RAISE EXCEPTION ' ...' forward up exception from autonomous transaction to parent transaction
> END
> END;
>Now I am thinking so PL/SQL design of autonomous transactions is relatively limited and is not best to follow it.
With our approach, we can use autonomous transaction in procedure as given below:
BEGIN
.. some code
WHEN OTHERS THEN
.. I would to write permanently to log
START AUTONOMOUS TRANSACTION
INSERT INTO log VALUES(..);
COMMIT:
WHEN OTHERS
RAISE WARNING 'Cannot to write to log ..';
RAISE EXCEPTION ' ...' forward up exception from autonomous transaction to parent transaction
END
END;
Please let me know if I have missed to answer any of your queries.
Thanks and Regards,
Kumar Rajeev Rastogi


From: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Simon Riggs <simon(at)2ndquadrant(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-01 08:52:27
Message-ID: CAFj8pRB4saDV66efvB80=+VOsXQrqMe6JK_FnV7DEqFNLScJGg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

2014-07-01 10:38 GMT+02:00 Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>:

> On 01 July 2014 12:26, Pavel Stehule Wrote:
>
>
>
> >>Have you checked the discussion in Developer meeting notes. Please
>
> >>check the same at below link:
>
> >>
> http://wiki.postgresql.org/wiki/PgCon_2012_Developer_Meeting#Autonomous_Transactions
>
>
>
> >Are these notes still valid?
>
> >* Why autonomous transaction should be close to functions? We can
> implement AT as first step and next step can be implementation of
> integration AT to stored procedures.
>
> We have implemented AT on the line of sub-transaction. Also we have
> integrated AT with stored procedure i.e. we can create an autonomous
> transaction inside the store procedure, which can be also committed.
>
> >* When autonomous transaction is independent on parent transaction, then
> locks parent and autonomous transaction should be in conflict
>
> Yes our implementation makes the autonomous transaction independent of
> main transaction and hence as per our design parent (main) transaction and
> autonomous may get conflicted. For which we have implemented deadlock
> detection mechanism between autonomous transaction and its parent
> transaction.
>
> > I though about integration to PL/pgSQL and I don't think so close
> integration between autonomous transaction and procedure is optimal. More
> practical is design so autonomous transaction is similar to subtransaction.
>
> Yes as mentioned above, our implementation of autonomous transaction is on
> track of subtransaction.
>

ok

> >Then we can simply wrote some code like
>
> > BEGIN
>
> > .. some code
>
> > WHEN OTHERS THEN
>
> > .. I would to write permanently to log
>
> > BEGIN AUTONOMOUS
>
> > INSERT INTO log VALUES(..);
>
> > WHEN OTHERS
>
> > RAISE WARNING 'Cannot to write to log ..';
>
> > RAISE EXCEPTION ' ...' forward up exception from autonomous
> transaction to parent transaction
>
> > END
>
> > END;
>
> >Now I am thinking so PL/SQL design of autonomous transactions is
> relatively limited and is not best to follow it.
>
> With our approach, we can use autonomous transaction in procedure as given
> below:
>
> BEGIN
>
> .. some code
>
> WHEN OTHERS THEN
>
> .. I would to write permanently to log
>
> *START AUTONOMOUS TRANSACTION*
>
> INSERT INTO log VALUES(..);
>
> *COMMIT: *
>
> WHEN OTHERS
>
> RAISE WARNING 'Cannot to write to log ..';
>
> RAISE EXCEPTION ' ...' forward up exception from autonomous
> transaction to parent transaction
>
> END
>
> END;
>
I don't like this design (really) - it can be used in later implementation
of procedures - but I don't like a explicit transaction manipulation in
functions. It is Oracleism (and this part I don't want to follow, because
Oracle design is not lucky) - and it is one reason, why Oracle SP are
significantly complex than PostgreSQL SP. After all I am thinking so
PostgreSQL relation between transactions and procedures are better, simply
for usage, simply for learning. But it is little bit different topic.

Regards

Pavel

> Please let me know if I have missed to answer any of your queries.
>
> Thanks and Regards,
>
> Kumar Rajeev Rastogi
>
>
>
>
>


From: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-03 06:33:25
Message-ID: BF2827DCCE55594C8D7A8F7FFD3AB7713DE1458B@SZXEML508-MBX.china.huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 01 July 2014 12:00, Amit Kapila Wrote:

>On Tue, Jul 1, 2014 at 11:46 AM, Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com<mailto:rajeev(dot)rastogi(at)huawei(dot)com>> wrote:
>> On 30 June 2014 22:50, Pavel Stehule Wrote:
>> >I didn't find a related message.
>> >?
>>
>> I think there have been some confusion, the design idea were never rejected but yes there were few feedback/ concern, which I had clarified. Also some of the other concerns are already fixed in latest patch.

>Simon has mentioned that exactly this idea has been rejected at
>PGCon 2 years back. Please refer that in below mail:
>http://www.postgresql.org/message-id/BF2827DCCE55594C8D7A8F7FFD3AB7713DDE136A@SZXEML508-MBX.china.huawei.com

>As far as I can see, you never came back with the different solution.

Yeah right. So for this I tried to search archived mails to get the details about the discussion but I could not find anything regarding design.
So I am not sure how shall I make my solution different from earlier as earlier solution is not accessible to me. Any help regarding this will be really great help to me.

Also from the current Autonomous transaction discussion thread (including CA+U5nMKEUm4abRQBndLYt5LEdekTAe8rbiYW3977YHMeOWQ1kA(at)mail(dot)gmail(dot)com<http://www.postgresql.org/message-id/CA+U5nMKEUm4abRQBndLYt5LEdekTAe8rbiYW3977YHMeOWQ1kA@mail.gmail.com>),
I have summarized all important feedbacks as mentioned below along with the resolution suggested:

1. Pavel Stehule (07-04-2014): -1 for Oracle syntax - it is hardly inconsistent with Postgres

Changed the syntax to “START AUTONOMOUS TRANSACTION”

2. Pavan (10-04-2014): Making autonomous transaction properties independent of main transaction.
Made all properties of autonomous transaction (including read-only) independent from main transaction except isolation level, which I did not find very useful to keep different. But others opinion is different then we can make this property also independent.

3. Alvaro Herrarta (09-04-2014): Autonomous transaction to have their own separate proc entry.
This was concluded to not have separate proc entry for autonomous transaction and same concluded.

4. Tom Lane (09-04-2014): The point being that you need to change both pg_subtrans and pg_clog to make that state transition.
This is handled for autonomous transaction.

5. Robert Haas (09-04-2014): Not in favour of current design related to "maintaining lockmask for autonomous transaction".

I had replied for this mail regarding why this design is kept but still if design for this part is not acceptable, then I can rework to make it better. In order to do so I would be very happy to have more discussion to get concrete feedback and direction to improve this.

6. Tom Lane (09-04-2014): no justification for distinguishing normal and autonomous transactions at this level (locking level).
I had replied this also earlier. Reason for distinguishing at this level is to handle any kind of deadlock possibility between main and autonomous transaction. Deadlock handling between main and autonomous transaction was one of the requirement discussed at PGCon 2012 as part of autonomous transaction discussion. Please let me know if I am missing something in this.

All of the above mentioned changes are included in latest patch shared.
Please let me know if I have missed any other important points from the earlier discussion, I would like to address that also.
>Have you checked the discussion in Developer meeting notes. Please
>check the same at below link:
>http://wiki.postgresql.org/wiki/PgCon_2012_Developer_Meeting#Autonomous_Transactions

From the discussion, I am able to make out two important points:

1. Main transaction and autonomous transaction should be independent and can conflict.

This is already included in our latest patch.

2. Utility commands like VACUUM and CREATE INDEX CONCURRENTLY should be able to work from autonomous transaction.

Both of the above mentioned utility commands are not supported even inside the main transaction. So it is not working within autonomous transaction.

Any opinion about this?
Please let me know if I have missed any points from the link given.

>> So I wanted to have this patch in commitfest application, so that we can have a healthy discussion and rectify all the issues.
>> But now I see that this patch has already been moved to rejected category, which will put break on further review.
>I believe ideally this patch should have been marked as
>"Returned with feedback" as you already got a feedback long
>back and never come up with solution for same.

Since this patch is very big and complex, it is better we continue discussing from the first CommitFest itself so that we can get ample time to share everyone’s opinion and then address all possible issue.

Any Opinions/Suggestions are welcome. Also let me know if I have missed something.

Thanks and Regards,
Kumar Rajeev Rastogi


From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
Cc: Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Abhijit Menon-Sen <ams(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: Autonomous Transaction (WIP)
Date: 2014-07-03 07:20:23
Message-ID: CAA4eK1JR=ORtGuKX1W+-PffNs13TYQj2BXpR+2cLsf65H-cb1w@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Thu, Jul 3, 2014 at 12:03 PM, Rajeev rastogi <rajeev(dot)rastogi(at)huawei(dot)com>
wrote:
> On 01 July 2014 12:00, Amit Kapila Wrote:
> >Simon has mentioned that exactly this idea has been rejected at
>
> >PGCon 2 years back. Please refer that in below mail:
>
> >
http://www.postgresql.org/message-id/BF2827DCCE55594C8D7A8F7FFD3AB7713DDE136A@SZXEML508-MBX.china.huawei.com
>
>
>
> >As far as I can see, you never came back with the different solution.
>
>
>
> Yeah right. So for this I tried to search archived mails to get the
details about the discussion but I could not find anything regarding design.
> So I am not sure how shall I make my solution different from earlier as
earlier solution is not accessible to me.

I haven't read your idea/patch in any detail, so can't comment
on whether it is good or bad. However I think if one of the
Committers has already mentioned that the same idea has been
rejected previously, then it makes little sense to further review
or update the patch unless you know the reason of rejection and
handle it in an acceptable way.

Now as far as I can understand, the problem seems to be in a way
you have defined Autonomous Transaction Storage which can lead
to consumption of additional client slots, this is just what I could make
sense from above mail but I am not completely sure on this matter.

With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com