Re: Moving pg_autovacuum from contrib to src/bin

Having no response from my email to hackers yesterday, I will assume
(hope) that no news is good news and proceed onwards.

As outlined in my email, step 1 is to move pg_autovacuum from contrib to
src/bin. Attached is a patch that modifies src/bin/Makefile and the
pg_autovacuum Makefile which will be in src/bin/pg_autovacuum/Makefile
once the pg_autovacuum directory is moved.

If there is anything else I can to to facilitate this move, let me know
and I'll do it.

Matthew O'Connor

On Sat, 2004-05-29 at 01:31, Matthew T. O'Connor wrote:
> Having no response from my email to hackers yesterday, I will assume
> (hope) that no news is good news and proceed onwards.
>
> As outlined in my email, step 1 is to move pg_autovacuum from contrib to
> src/bin. Attached is a patch that modifies src/bin/Makefile and the
> pg_autovacuum Makefile which will be in src/bin/pg_autovacuum/Makefile
> once the pg_autovacuum directory is moved.
>
> If there is anything else I can to to facilitate this move, let me know
> and I'll do it.

Actually attaching the patch might be something else I could do :-)

Anyway, here is the patch for real this time....

Matthew

"Matthew T. O'Connor" <matthew(at)zeut(dot)net> writes:
> As outlined in my email, step 1 is to move pg_autovacuum from contrib to
> src/bin. Attached is a patch that modifies src/bin/Makefile and the
> pg_autovacuum Makefile which will be in src/bin/pg_autovacuum/Makefile
> once the pg_autovacuum directory is moved.

Um, you didn't actually attach the patch ... but in any case this is not
the direction I would have expected. An integrated autovacuum daemon
would certainly not live in src/bin. It would be a module within the
postgres executable, most likely living within src/backend/postmaster
since that seems to be where we're putting integrated postmaster children.

(To clarify: that's where the pgstat daemon is now, and that's where I'm
about to put the bgwriter daemon, so if anyone thinks this is clearly
the wrong place then now would be a good time to mention it...)

regards, tom lane

On Sat, 2004-05-29 at 02:15, Tom Lane wrote:
> "Matthew T. O'Connor" <matthew(at)zeut(dot)net> writes:
> > As outlined in my email, step 1 is to move pg_autovacuum from contrib to
> > src/bin. Attached is a patch that modifies src/bin/Makefile and the
> > pg_autovacuum Makefile which will be in src/bin/pg_autovacuum/Makefile
> > once the pg_autovacuum directory is moved.
>
> Um, you didn't actually attach the patch ... but in any case this is not
> the direction I would have expected. An integrated autovacuum daemon
> would certainly not live in src/bin. It would be a module within the
> postgres executable, most likely living within src/backend/postmaster
> since that seems to be where we're putting integrated postmaster children.
>
> (To clarify: that's where the pgstat daemon is now, and that's where I'm
> about to put the bgwriter daemon, so if anyone thinks this is clearly
> the wrong place then now would be a good time to mention it...)

The only reason I put it in src/bin is because I thought people wanted
pg_autovacuum to still be a runable stand alone app. But, I guess there
is nothing preventing it from being a runable stand alone app located in
src/backend/postmaster.

Ok, I will move it (again) and send another patch.

Thanks for the feedback.

Matthew

Matthew T. O'Connor wrote:
> The only reason I put it in src/bin is because I thought people
> wanted pg_autovacuum to still be a runable stand alone app.

I see no reason for that. If it's integrated as a postmaster child,
then it is run in a daemon-like mode automatically.

> But, I
> guess there is nothing preventing it from being a runable stand alone
> app located in src/backend/postmaster.

Yes, there are a lot of things. You didn't understand what Tom was
saying. We were expecting the autovacuum daemon to be integrated as a
postmaster child, not a separate program.

On Sat, 2004-05-29 at 02:45, Peter Eisentraut wrote:
> Matthew T. O'Connor wrote:
> > The only reason I put it in src/bin is because I thought people
> > wanted pg_autovacuum to still be a runable stand alone app.
>
> I see no reason for that. If it's integrated as a postmaster child,
> then it is run in a daemon-like mode automatically.

I don't really see a strong reason to keep it runable as a standalone
except that I thought that was what was decided in previous discussion
on hackers.

Unfortunately, I can't find the thread anywhere on
archives.postgresql.org, when I search for "pg_autovacuum" it doesn't
return any hits more recent than Jan 2004, and I know there have been a
few pg_autovacuum related discussions since then so it seems like
something is broken.

> > But, I
> > guess there is nothing preventing it from being a runable stand alone
> > app located in src/backend/postmaster.
>
> Yes, there are a lot of things. You didn't understand what Tom was
> saying. We were expecting the autovacuum daemon to be integrated as a
> postmaster child, not a separate program.

I did understand Tom, but based on the hacker discussion I think the
"postmaster integration" will consist of the postmaster launching and
killing the pg_autovacuum standalone executable as required. In that
sense, I don't think it matters if pg_autovacuum is located in src/bin
or src/backend/postmaster.

Is this form of integration something people would object to? If so, I
can fold it into the backend such that pg_autovacuum can no longer be
run from the command line.

Comments anyone?

Matthew T. O'Connor wrote:
> I did understand Tom, but based on the hacker discussion I think the
> "postmaster integration" will consist of the postmaster launching and
> killing the pg_autovacuum standalone executable as required. In that
> sense, I don't think it matters if pg_autovacuum is located in
> src/bin or src/backend/postmaster.

It is supposed to be linked into the postmaster and forked from there.

Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> Matthew T. O'Connor wrote:
>> I did understand Tom, but based on the hacker discussion I think the
>> "postmaster integration" will consist of the postmaster launching and
>> killing the pg_autovacuum standalone executable as required. In that
>> sense, I don't think it matters if pg_autovacuum is located in
>> src/bin or src/backend/postmaster.

> It is supposed to be linked into the postmaster and forked from there.

In the current state of pg_autovacuum it wouldn't matter a lot, but
I am assuming that we will soon migrate it to depend on being part of
the postmaster environment. For instance, it ought to be configured
from GUC, which will mean it has to receive SIGHUP from the postmaster.
In an only slightly longer timeframe, it will probably want access to
shared memory so it can look at stats maintained in the FSM. These
attributes would make it quite inappropriate for autovacuum to live in
src/bin.

BTW, Matthew, I am currently working on promoting the bgwriter into a
more full-fledged postmaster child. If you can wait a day or so you
should have a decent model to work from. I'll try to commit as soon
as a working skeleton is in place.

regards, tom lane

On Sat, 2004-05-29 at 11:04, Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > It is supposed to be linked into the postmaster and forked from there.
>
> In the current state of pg_autovacuum it wouldn't matter a lot, but
> I am assuming that we will soon migrate it to depend on being part of
> the postmaster environment. For instance, it ought to be configured
> from GUC, which will mean it has to receive SIGHUP from the postmaster.
> In an only slightly longer timeframe, it will probably want access to
> shared memory so it can look at stats maintained in the FSM. These
> attributes would make it quite inappropriate for autovacuum to live in
> src/bin.

Ok.

> BTW, Matthew, I am currently working on promoting the bgwriter into a
> more full-fledged postmaster child. If you can wait a day or so you
> should have a decent model to work from. I'll try to commit as soon
> as a working skeleton is in place.

I can wait, but I am really trying not to miss the feature freeze which
AFAIK, is still happening in a few days. Is that changing? Will I have
time if I wait a few days? Especially given that my backend hacking
skill leave much to be desired.

My main priority is that I want pg_autovacuum to move ahead and I don't
want to miss 7.5.

Thanks,

Matthew

On Sat, 29 May 2004, Matthew T. O'Connor wrote:

> On Sat, 2004-05-29 at 11:04, Tom Lane wrote:
>> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
>>> It is supposed to be linked into the postmaster and forked from there.
>>
>> In the current state of pg_autovacuum it wouldn't matter a lot, but
>> I am assuming that we will soon migrate it to depend on being part of
>> the postmaster environment. For instance, it ought to be configured
>> from GUC, which will mean it has to receive SIGHUP from the postmaster.
>> In an only slightly longer timeframe, it will probably want access to
>> shared memory so it can look at stats maintained in the FSM. These
>> attributes would make it quite inappropriate for autovacuum to live in
>> src/bin.
>
> Ok.
>
>> BTW, Matthew, I am currently working on promoting the bgwriter into a
>> more full-fledged postmaster child. If you can wait a day or so you
>> should have a decent model to work from. I'll try to commit as soon
>> as a working skeleton is in place.
>
> I can wait, but I am really trying not to miss the feature freeze which
> AFAIK, is still happening in a few days. Is that changing? Will I have
> time if I wait a few days? Especially given that my backend hacking
> skill leave much to be desired.

We're discussing it in core right now, but Tom's feel is that we can have
PITR if we wait the extra month, which will also give a bit more time to
hammer out any bugs in Win32 ... so figure on having the extra 4 weeks to
work with ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email: scrappy(at)hub(dot)org Yahoo!: yscrappy ICQ: 7615664

On Sat, 2004-05-29 at 15:02, Marc G. Fournier wrote:
> On Sat, 29 May 2004, Matthew T. O'Connor wrote:
> > I can wait, but I am really trying not to miss the feature freeze which
> > AFAIK, is still happening in a few days. Is that changing? Will I have
> > time if I wait a few days? Especially given that my backend hacking
> > skill leave much to be desired.
>
> We're discussing it in core right now, but Tom's feel is that we can have
> PITR if we wait the extra month, which will also give a bit more time to
> hammer out any bugs in Win32 ... so figure on having the extra 4 weeks to
> work with ...

That will help me a lot with pg_autovacuum integration also. So, I will
wait until Tom finishes work on the bgwriter and use that as a template
for autovacuum integration.

Please let me know when the official decision is made so that I can plan
appropriately.

Thanks much,

Matthew