Re: Reduce the time to know trigger_fi​le's existence

Hi,

We are trying to introduce a thread that monitors the creation of the
trigger_file. As and when the file is created, the process that monitors
postgres server needs to be notified through the inotify API.

This is to reduce the 3-4 seconds delay that exists with the current
implementation in postgres. As per the current implementation, the thread
checks for the existence of the file every 5 seconds. If the file got
created just 1 second after the sleep, there is a wait time of 4 seconds
before we know whether the file is present or not. We intend to avoid this
delay by using inotify().

PostgreSQL version number you are running: postgres 9.1.5

How you installed PostgreSQL: Downloaded and compiled the sources

Does anyone have suggestions on the appropriate place to
add inotify_add_watch to achieve our objective?
Thanks in advance.

Hi,

We are trying to introduce a thread that monitors the creation of the
trigger_file. As and when the file is created, the process that monitors
postgres server needs to be notified through the inotify API.

This is to reduce the 3-4 seconds delay that exists with the current
implementation in postgres. As per the current implementation, the thread
checks for the existence of the file every 5 seconds. If the file got
created just 1 second after the sleep, there is a wait time of 4 seconds
before we know whether the file is present or not. We intend to avoid this
delay by using inotify().

PostgreSQL version number you are running: postgres 9.1.5

How you installed PostgreSQL: Downloaded and compiled the sources

Does anyone have suggestions on the appropriate place to
add inotify_add_watch to achieve our objective?
Thanks in advance.

On 09/03/2012 04:30 PM, Harshitha S wrote:
> Hi,
> We are trying to introduce a thread that monitors the creation of the
> trigger_file. As and when the file is created, the process that monitors
> postgres server needs to be notified through the inotify API.

File system notification APIs aren't portable. If you want to implement
this you'll need to support at least the Windows FSNotify stuff,
notification features compatible with several different Linux versions
and with the BSDs, with a fallback to polling for other platforms.

For Windows:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa365261(v=vs.85).aspx

This will mean adding a new internal API and then adding backend/port/
implementations for various platforms, as well as taking the current
code and splitting it out into a port file that gets used if no other is
chosen.

This page is likely to be useful:

http://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer%3F

--
Craig Ringer

On 09/03/2012 04:30 PM, Harshitha S wrote:
> Hi,
> We are trying to introduce a thread that monitors the creation of the
> trigger_file. As and when the file is created, the process that monitors
> postgres server needs to be notified through the inotify API.

File system notification APIs aren't portable. If you want to implement
this you'll need to support at least the Windows FSNotify stuff,
notification features compatible with several different Linux versions
and with the BSDs, with a fallback to polling for other platforms.

For Windows:

http://msdn.microsoft.com/en-us/library/windows/desktop/aa365261(v=vs.85).aspx

For Linux it appears that inotify has been present since 2005:

http://en.wikipedia.org/wiki/Inotify

but it's probably best to add a configure test and fall back to polling
if it isn't found.

This will mean adding a new internal API and then adding backend/port/
implementations for various platforms, as well as taking the current
code and splitting it out into a port file that gets used if no other is
chosen.

This page is likely to be useful:

http://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer%3F

--
Craig Ringer

Hi,

Can Latch - that postgres already uses, be used to achieve the same?

On Tue, Sep 4, 2012 at 6:31 AM, Craig Ringer <ringerc(at)ringerc(dot)id(dot)au> wrote:

> On 09/03/2012 04:30 PM, Harshitha S wrote:
>
>> Hi,
>> We are trying to introduce a thread that monitors the creation of the
>> trigger_file. As and when the file is created, the process that monitors
>> postgres server needs to be notified through the inotify API.
>>
>
> File system notification APIs aren't portable. If you want to implement
> this you'll need to support at least the Windows FSNotify stuff,
> notification features compatible with several different Linux versions and
> with the BSDs, with a fallback to polling for other platforms.
>
> For Windows:
>
> http://msdn.microsoft.com/en-**us/library/windows/desktop/**
> aa365261(v=vs.85).aspx<http://msdn.microsoft.com/en-us/library/windows/desktop/aa365261%28v=vs.85%29.aspx>
>
> For Linux it appears that inotify has been present since 2005:
>
> http://en.wikipedia.org/wiki/**Inotify<http://en.wikipedia.org/wiki/Inotify>
>
> but it's probably best to add a configure test and fall back to polling if
> it isn't found.
>
>
>
>
> This will mean adding a new internal API and then adding backend/port/
> implementations for various platforms, as well as taking the current code
> and splitting it out into a port file that gets used if no other is chosen.
>
> This page is likely to be useful:
>
> http://wiki.postgresql.org/**wiki/So,_you_want_to_be_a_**developer%3F<http://wiki.postgresql.org/wiki/So,_you_want_to_be_a_developer%3F>
>
> --
> Craig Ringer
>

On Mon, Sep 3, 2012 at 8:30 AM, Harshitha S <hershetha(at)gmail(dot)com> wrote:
> Hi,
>
> We are trying to introduce a thread that monitors the creation of the
> trigger_file. As and when the file is created, the process that monitors
> postgres server needs to be notified through the inotify API.
>
> This is to reduce the 3-4 seconds delay that exists with the current
> implementation in postgres. As per the current implementation, the thread
> checks for the existence of the file every 5 seconds. If the file got
> created just 1 second after the sleep, there is a wait time of 4 seconds
> before we know whether the file is present or not. We intend to avoid this
> delay by using inotify().
>
> PostgreSQL version number you are running: postgres 9.1.5
>
> How you installed PostgreSQL: Downloaded and compiled the sources
>
>
> Does anyone have suggestions on the appropriate place to add
> inotify_add_watch to achieve our objective?
> Thanks in advance.

"pg_ctl promote" already uses a signal to remove this time delay. Is
there a reason you can't just use this?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 09/04/2012 02:15 PM, Harshitha S wrote:
> Hi,
>
> Can Latch - that postgres already uses, be used to achieve the same?

With a quick look at latches I'm not sure what that'd have to do with
file change notification.

Magnus's suggestion to use a signal - especially as one is already
understood for the purpose - makes a lot more sense. He should know,
given how much work he's done on the server.

--
Craig Ringer