Re: Using Postgres to store high volume streams of sensor readings

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Ciprian Dorin Craciun <ciprian(dot)craciun(at)gmail(dot)com>
Cc: Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>, m_lists(at)yahoo(dot)it, pgsql-general(at)postgresql(dot)org
Subject: Re: Using Postgres to store high volume streams of sensor readings
Date: 2008-11-23 13:28:49
Message-ID: 20081123132849.GC4452@tamriel.snowman.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

* Ciprian Dorin Craciun (ciprian(dot)craciun(at)gmail(dot)com) wrote:
> > Even better might be partitioning on the timestamp. IF all access is
> > in a certain timestamp range it's usually a big win, especially
> > because he can move to a new table every hour / day / week or whatever
> > and merge the old one into a big "old data" table.
>
> Yes, If i would speed the inserts tremendously... I've tested it
> and the insert speed is somewhere at 200k->100k.
>
> But unfortunately the query speed is not good at all because most
> queries are for a specific client (and sensor) in a given time
> range...

Have you set up your partitions correctly (eg, with appropriate CHECK
constraints and with constraint_exclusion turned on)? Also, you'd want
to keep your indexes on the individual partitions, of course.. That
should improve query time quite a bit since it should only be hitting
the partitions where the data might be.

Stephen

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Scara Maccai 2008-11-23 15:07:51 Re: Using Postgres to store high volume streams of sensor readings
Previous Message Gregory Stark 2008-11-23 12:50:30 Re: Postgres mail list traffic over time