Re: How are ppl monitoring PostgreSQL ... ? What is being

The subject kinda says it all ... I know there are SNMP patches available
out there now, but without those ... ?

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
Yahoo . yscrappy Skype: hub.org ICQ . 7615664

I'm using a great little Linux program called "monit" to check that
there's something listening on the 5432 port. It also monitors
individual process memory and CPU usage etc. Quite good.

http://www.tildeslash.com/monit/

Andy

Marc G. Fournier wrote:
>
> The subject kinda says it all ... I know there are SNMP patches
> available out there now, but without those ... ?
>
> ----
> Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
> Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
> Yahoo . yscrappy Skype: hub.org ICQ . 7615664
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>
> !DSPAM:14,448b0f39256841201919775!
>
>

On Sat, Jun 10, 2006 at 07:29:52PM +0100, Andy Shellam wrote:
> I'm using a great little Linux program called "monit" to check that
> there's something listening on the 5432 port. It also monitors
> individual process memory and CPU usage etc. Quite good.

A server can be quite broken yet still respond to transport- and
network-layer probes like TCP connections and pings. Some NMSs
support custom monitors, so whenever possible I like to do
application-layer tests to ensure that the server not only accepts
connections but is indeed behaving as expected.

> Marc G. Fournier wrote:
> >
> > The subject kinda says it all ... I know there are SNMP patches
> > available out there now, but without those ... ?

MRTG can generate graphs of anything you can write a script to
measure, as long as the script returns output in a certain format.
Other packages of that ilk probably have similar capabilities.

--
Michael Fuhr

mike(at)fuhr(dot)org (Michael Fuhr) wrote:
> On Sat, Jun 10, 2006 at 07:29:52PM +0100, Andy Shellam wrote:
>> I'm using a great little Linux program called "monit" to check that
>> there's something listening on the 5432 port. It also monitors
>> individual process memory and CPU usage etc. Quite good.
>
> A server can be quite broken yet still respond to transport- and
> network-layer probes like TCP connections and pings. Some NMSs
> support custom monitors, so whenever possible I like to do
> application-layer tests to ensure that the server not only accepts
> connections but is indeed behaving as expected.

I'd generally agree with this; it points to having tests that aren't
so much about PostgreSQL as they are about the applications using
PostgreSQL...

>> Marc G. Fournier wrote:
>> >
>> > The subject kinda says it all ... I know there are SNMP patches
>> > available out there now, but without those ... ?
>
> MRTG can generate graphs of anything you can write a script to
> measure, as long as the script returns output in a certain format.
> Other packages of that ilk probably have similar capabilities.

Our NOC group runs "replication tests" against various servers that
feed MRTG; they point at a view that is normally frequently updated,
and check to see how elderly the latest value is.

On replicated nodes, this checks the health of replication.

On "master" nodes, this checks the health of the application itself.
--
select 'cbbrowne' || '@' || 'gmail.com';
http://linuxdatabases.info/info/slony.html
Rules of the Evil Overlord #66. "My security keypad will actually be a
fingerprint scanner. Anyone who watches someone press a sequence of
buttons or dusts the pad for fingerprints then subsequently tries to
enter by repeating that sequence will trigger the alarm system."
<http://www.eviloverlord.com/>

Marc G. Fournier wrote:
>
> The subject kinda says it all ... I know there are SNMP patches
> available out there now, but without those ... ?

We use Nagios to call shell scripts that perform specific requests to a
PostgreSQL server. Usually over SSH via psql, but you could use anything.

Joshua D. Drake

>
> ----
> Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
> Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
> Yahoo . yscrappy Skype: hub.org ICQ . 7615664
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

On Sat, 10 Jun 2006, Joshua D. Drake wrote:

> Marc G. Fournier wrote:
>>
>> The subject kinda says it all ... I know there are SNMP patches available
>> out there now, but without those ... ?
>
> We use Nagios to call shell scripts that perform specific requests to a
> PostgreSQL server. Usually over SSH via psql, but you could use anything.

'k, so what are you testing for? Just that a query runs? time it takes
to run? specific results?

SELECT count(1) FROM pg_connections;

would be a good one to graph ...

----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
Yahoo . yscrappy Skype: hub.org ICQ . 7615664

>
> SELECT count(1) FROM pg_connections;
>
> would be a good one to graph ...

Well you should use pg_stat_databases for that :)

We have complete graphing and monitoring service coming out at the end
of this month.

Joshua D. Drake

>
> ----
> Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
> Email . scrappy(at)hub(dot)org MSN . scrappy(at)hub(dot)org
> Yahoo . yscrappy Skype: hub.org ICQ . 7615664
>

--

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
Providing the most comprehensive PostgreSQL solutions since 1997
http://www.commandprompt.com/

On 6/10/06, Marc G. Fournier <scrappy(at)postgresql(dot)org> wrote:
>
> The subject kinda says it all ... I know there are SNMP patches available
> out there now, but without those ... ?
>

i can tell you what we do monitor:
1. general server things (free disk space, cpu-idle, load, memory used, swap
used, context switches, io operations, sockets in state close_wait).
2. db stuff:
- numer of transactions per second
- database size
- size of specific tables/indices that we know have nasty tendency to grow
- number of objects in database
- duration of longest query
- replication lag (in events, and in time)

to make mrtg graphs of db stuff we made some perl scripts, which are bound
to snmp using snmp's "exec" feature. then we just grab the values using mrtg
and draw. quite simple actually.

depesz

--
http://www.depesz.com/ - nowy, lepszy depesz

Interesting, Hubert.
Where/how do you get the data for:
- number of transactions per second
- duration of longest query
?

Thanks,
Otis

----- Original Message ----
From: hubert depesz lubaczewski
To: Marc G. Fournier
Cc: pgsql-admin(at)postgresql(dot)org
Sent: Sunday, June 11, 2006 11:10:20 AM
Subject: Re: [ADMIN] How are ppl monitoring PostgreSQL ... ? What is being monitored ... ?

On 6/10/06, Marc G. Fournier wrote: The subject kinda says it all ... I know there are SNMP patches available
out there now, but without those ... ?


i can tell you what we do monitor:
1. general server things (free disk space, cpu-idle, load, memory used, swap used, context switches, io operations, sockets in state close_wait).
2. db stuff:
- numer of transactions per second
- database size
- size of specific tables/indices that we know have nasty tendency to grow
- number of objects in database
- duration of longest query
- replication lag (in events, and in time)

to make mrtg graphs of db stuff we made some perl scripts, which are bound to snmp using snmp's "exec" feature. then we just grab the values using mrtg and draw. quite simple actually.

depesz

--
http://www.depesz.com/ - nowy, lepszy depesz

On 7/7/06, ogjunk-pgjedan(at)yahoo(dot)com <ogjunk-pgjedan(at)yahoo(dot)com> wrote:
>
> Interesting, Hubert.
> Where/how do you get the data for:
> - number of transactions per second

periodically (every 5 minutes) i do:
select sum(xact_commit) + sum(xact_rollback) from pg_stat_database
and then just check against previous value, and calculate "per/second". this
is actually handled nicely by mrtg itself.

- duration of longest query
>

select * from pg_stat_activity where current_query !~ '<IDLE>' order by
query_start asc limit 1;
this has some problems though, but are more or less easily avoidable.

depesz

--
http://www.depesz.com/ - nowy, lepszy depesz

On Fri, 7 Jul 2006, hubert depesz lubaczewski wrote:

> periodically (every 5 minutes) i do:
> select sum(xact_commit) + sum(xact_rollback) from pg_stat_database
> and then just check against previous value, and calculate "per/second". this
> is actually handled nicely by mrtg itself.

Hubert,

Do you already have a MRTG plugin for postgres that you've written and if so,
care to share it with the rest of us? Or better yet, share it with the
mrtg.org folks? I was thinking of writing one, but if you have one already, I
hate to reinvent the wheel.

--
Jeff Frost, Owner <jeff(at)frostconsultingllc(dot)com>
Frost Consulting, LLC http://www.frostconsultingllc.com/
Phone: 650-780-7908 FAX: 650-649-1954

On 7/9/06, Jeff Frost <jeff(at)frostconsultingllc(dot)com> wrote:
>
> Do you already have a MRTG plugin for postgres that you've written and if
> so,
> care to share it with the rest of us? Or better yet, share it with the
> mrtg.org folks? I was thinking of writing one, but if you have one
> already, I
> hate to reinvent the wheel.
>

this is not mrtg plugin. i use snmpd (net-snmpd to be exact), and connect
custom scripts to it using "exec" directive in config.
so basically i have series of scripts which all look more or less like this:
#!/bin/bash
psql -U pgdba -d some_database -qAt 'select .....';

and a series of:
Exec some-name /etc/snmpd/addons/somescript.sh

that's all.

depesz

--
http://www.depesz.com/ - nowy, lepszy depesz