Somebody has broken autovacuum's abort path

http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jaguar&dt=2010-01-05%2004:00:03
(and the same a couple times before this...)

Core was generated by `postgres: autovacuum worker process regression '.
Program terminated with signal 6, Aborted.
[New process 9209]
#0 0x0091c402 in __kernel_vsyscall ()
#0 0x0091c402 in __kernel_vsyscall ()
#1 0x007a9d80 in raise () from /lib/libc.so.6
#2 0x007ab691 in abort () from /lib/libc.so.6
#3 0x083393be in ExceptionalCondition (
conditionName=0x8498e40 "!(rel->rd_refcnt > 0)",
errorType=0x836d218 "FailedAssertion", fileName=0x8498d55 "relcache.c",
lineNumber=1612) at assert.c:57
#4 0x083324c2 in RelationDecrementReferenceCount (rel=0xb5c641e0)
at relcache.c:1612
#5 0x0835b562 in ResourceOwnerReleaseInternal (owner=0x92d1058,
phase=RESOURCE_RELEASE_BEFORE_LOCKS, isCommit=0 '\0', isTopLevel=1 '\001')
at resowner.c:251
#6 0x0835b86f in ResourceOwnerRelease (owner=0x92d1058,
phase=RESOURCE_RELEASE_BEFORE_LOCKS, isCommit=0 '\0', isTopLevel=1 '\001')
at resowner.c:185
#7 0x080cc5d9 in AbortTransaction () at xact.c:2179
#8 0x080cc7c7 in AbortOutOfAnyTransaction () at xact.c:3676
#9 0x0824063e in do_autovacuum () at autovacuum.c:2259
#10 0x08240b25 in AutoVacWorkerMain (argc=<value optimized out>,
argv=<value optimized out>) at autovacuum.c:1602
#11 0x08240c51 in StartAutoVacWorker () at autovacuum.c:1406
#12 0x0824c0f5 in sigusr1_handler (postgres_signal_arg=10)
at postmaster.c:4307
#13 <signal handler called>
#14 0x0091c402 in __kernel_vsyscall ()
#15 0x0084b1dd in ___newselect_nocancel () from /lib/libc.so.6
#16 0x082486e0 in ServerLoop () at postmaster.c:1364
#17 0x08249d96 in PostmasterMain (argc=6, argv=0x924d918) at postmaster.c:1069
#18 0x081eb080 in main (argc=6, argv=0x0) at main.c:188

I think this can likely be blamed on the HS changes in transaction
abort, since I'm not aware of any other recent changes near here.

regards, tom lane

On Tue, 2010-01-05 at 03:09 -0500, Tom Lane wrote:

> I think this can likely be blamed on the HS changes in transaction
> abort, since I'm not aware of any other recent changes near here.

I'll take a look.

--
Simon Riggs www.2ndQuadrant.com

On Tue, 2010-01-05 at 03:09 -0500, Tom Lane wrote:
> http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jaguar&dt=2010-01-05%2004:00:03
> (and the same a couple times before this...)
>
> Core was generated by `postgres: autovacuum worker process regression '.
> Program terminated with signal 6, Aborted.
> [New process 9209]
> #0 0x0091c402 in __kernel_vsyscall ()
> #0 0x0091c402 in __kernel_vsyscall ()
> #1 0x007a9d80 in raise () from /lib/libc.so.6
> #2 0x007ab691 in abort () from /lib/libc.so.6
> #3 0x083393be in ExceptionalCondition (
> conditionName=0x8498e40 "!(rel->rd_refcnt > 0)",
> errorType=0x836d218 "FailedAssertion", fileName=0x8498d55 "relcache.c",
> lineNumber=1612) at assert.c:57
> #4 0x083324c2 in RelationDecrementReferenceCount (rel=0xb5c641e0)
> at relcache.c:1612
> #5 0x0835b562 in ResourceOwnerReleaseInternal (owner=0x92d1058,
> phase=RESOURCE_RELEASE_BEFORE_LOCKS, isCommit=0 '\0', isTopLevel=1 '\001')
> at resowner.c:251
> #6 0x0835b86f in ResourceOwnerRelease (owner=0x92d1058,
> phase=RESOURCE_RELEASE_BEFORE_LOCKS, isCommit=0 '\0', isTopLevel=1 '\001')
> at resowner.c:185
> #7 0x080cc5d9 in AbortTransaction () at xact.c:2179
> #8 0x080cc7c7 in AbortOutOfAnyTransaction () at xact.c:3676
> #9 0x0824063e in do_autovacuum () at autovacuum.c:2259
> #10 0x08240b25 in AutoVacWorkerMain (argc=<value optimized out>,
> argv=<value optimized out>) at autovacuum.c:1602
> #11 0x08240c51 in StartAutoVacWorker () at autovacuum.c:1406
> #12 0x0824c0f5 in sigusr1_handler (postgres_signal_arg=10)

> I think this can likely be blamed on the HS changes in transaction
> abort, since I'm not aware of any other recent changes near here.

I haven't knowingly made changes to this code path, nor were there
changes to transaction abort in HS. xact_redo_abort() was changed, but
that's not what's failing.

http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/access/transam/xact.c?r1=1.277&r2=1.278

--
Simon Riggs www.2ndQuadrant.com

http://www.postgresql.org/docs/8.4/static/ddl-partitioning.html

Is there a way to automatically extend table partitions? I'm curious if
/ when a table is getting full if there is a way for postgres to create
additional tables. Or is it all manual?

Thanks :D

On Tue, Jan 5, 2010 at 8:00 PM, u235sentinel <u235sentinel(at)gmail(dot)com> wrote:
> http://www.postgresql.org/docs/8.4/static/ddl-partitioning.html
>
> Is there a way to automatically extend table partitions?  I'm curious if /
> when a table is getting full if there is a way for postgres to create
> additional tables.

Getting full?

...Robert

Robert Haas wrote:
>
> Getting full?
>
> ...Robert
>
>
Ok. Bad analogy. We have the tables setup to write data according to
the month it was loaded. We have a December table, a January table and
so on. Basically following the examples given on the 8.4 web site.

I'm thinking it would be nice if there was a way to automatically add
the next month without having to script it all out.

Just a thought :D

On Tue, Jan 05, 2010 at 08:50:25PM -0700, u235sentinel wrote:
> Robert Haas wrote:
> >
> >Getting full?
> >
> >...Robert
> >
> Ok. Bad analogy. We have the tables setup to write data according
> to the month it was loaded. We have a December table, a January
> table and so on. Basically following the examples given on the 8.4
> web site.
>
> I'm thinking it would be nice if there was a way to automatically
> add the next month without having to script it all out.

There's no good reason you can't add 5 years' worth of tables right up
front.

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On Wed, Jan 6, 2010 at 12:06 PM, David Fetter <david(at)fetter(dot)org> wrote:
> On Tue, Jan 05, 2010 at 08:50:25PM -0700, u235sentinel wrote:
>> Robert Haas wrote:
>> >
>> >Getting full?
>> >
>> >...Robert
>> >
>> Ok.  Bad analogy.  We have the tables setup to write data according
>> to the month it was loaded.  We have a December table, a January
>> table and so on.  Basically following the examples given on the 8.4
>> web site.
>>
>> I'm thinking it would be nice if there was a way to automatically
>> add the next month without having to script it all out.
>
> There's no good reason you can't add 5 years' worth of tables right up
> front.

Different people might want different naming conventions for those
tables, too. We've heard this request before so maybe we should
consider it, but it seems like a lot of work for something that's not
too hard to code up for yourself, and can be handled much more
flexibly that way.

...Robert

On 1/6/10 9:13 AM, Robert Haas wrote:
> On Wed, Jan 6, 2010 at 12:06 PM, David Fetter <david(at)fetter(dot)org> wrote:
>> On Tue, Jan 05, 2010 at 08:50:25PM -0700, u235sentinel wrote:
>>> Robert Haas wrote:
>>>> Getting full?
>>>>
>>>> ...Robert
>>>>
>>> Ok. Bad analogy. We have the tables setup to write data according
>>> to the month it was loaded. We have a December table, a January
>>> table and so on. Basically following the examples given on the 8.4
>>> web site.

FWIW, our roadmap is to add a 2nd type or partitioning which would be on
the sub-table level and much more automated for that reason.

--Josh Berkus

Adding on to this use case:
what do we do when we reach end of year?
Probably auto-archive as per weekly, monthly , quarterly or yearly tables?

On Thu, Jan 7, 2010 at 1:14 AM, Josh Berkus <josh(at)agliodbs(dot)com> wrote:

> On 1/6/10 9:13 AM, Robert Haas wrote:
> > On Wed, Jan 6, 2010 at 12:06 PM, David Fetter <david(at)fetter(dot)org> wrote:
> >> On Tue, Jan 05, 2010 at 08:50:25PM -0700, u235sentinel wrote:
> >>> Robert Haas wrote:
> >>>> Getting full?
> >>>>
> >>>> ...Robert
> >>>>
> >>> Ok. Bad analogy. We have the tables setup to write data according
> >>> to the month it was loaded. We have a December table, a January
> >>> table and so on. Basically following the examples given on the 8.4
> >>> web site.
>
> FWIW, our roadmap is to add a 2nd type or partitioning which would be on
> the sub-table level and much more automated for that reason.
>
> --Josh Berkus
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

--
Chetan Sutrave
http://www.enterprisedb.com

On Thu, Jan 07, 2010 at 08:01:16PM +0530, Chetan Suttraway wrote:
> Adding on to this use case:
> what do we do when we reach end of year?
> Probably auto-archive as per weekly, monthly , quarterly or yearly tables?

Because such requirements are so specific to each place, it's easier
to do this in your own code. While managing partitions may get
simpler than our current table inheritance, it's unlikely that
automated tools will ever be able to handle all the cases for it, so
that coding is likely to be part of your partitioning strategy for the
foreseeable future.

Cheers,
David.

P.S. Please do trim, as I just did, and don't top post.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate