Re: horizontal clustering?

[Decibel! <decibel(at)decibel(dot)org>]

On Aug 3, 2007, at 3:35 PM, Chirag Patel wrote:
> My application is designed in such a way that the amount of data is  
> expected to be enormous (in the terabytes). I would like to cluster  
> the data and have the luxury of being able to slice up  
> (denormalize) the database tables by user. In other words, users  
> 1-50 and user 51-100 can be divided so that queries across users is  
> not needed.
>
> Does PostgresSQL or any third party add-ons allow this scaling to  
> automatically occur? When I say automatically, I mean:
>
> 1.       New users are routed to a different database when the  
> current database reaches a pre-defined capacity
>
> 2.       Denormalization logic is provided so that the querying  
> across databases is transparent to the user
>
>
>
> I’ve heard this feature described as “federation” or “horizontal  
> scaling” in the O'Reilly book “Building Scalable Web Sites”
>
>
>
> I’ve never done clustering  before so I’m clueless. Any ideas or  
> better suggestions?
There's no built-in support for any of that in any database I'm  
familiar with. Some databases do have support for clustering, such as  
GridSQL (formerly ExtenDB) or MPP. But generally folks roll their own  
when it comes to this.

One thing to be careful of... for many sites it's common for people  
to sign up but not generate much data for some period of time. If all  
your new accounts are going to a single server, that server can  
suddenly be very overloaded.
--
Decibel!, aka Jim Nasby                        decibel(at)decibel(dot)org
EnterpriseDB      http://enterprisedb.com      512.569.9461 (cell)