Re: Detailed documentation for external calls (threading, shared resources etc)

Greetings,
This is actually a request for documentation guidance. I intend to
develop an extension to postgresql. Basically I'd like to place calls
to network using ZeroMQ, and I need to have detailed information about
a lot of things, especially threading issues. I need to have some
global resources which will be presumably used by multiple threads.
I can see that there is a lot of documentation, but I'd really
appreciate pointers towards the books, or key documents that'd help me
move forward faster (docs/books about inner workings of key
functionality) I'll be using C (most likely the best option) to
develop code, so which books/documents would you recommend?

Cheers
Seref

On Jun12, 2011, at 19:26 , Seref Arikan wrote:
> This is actually a request for documentation guidance. I intend to
> develop an extension to postgresql. Basically I'd like to place calls
> to network using ZeroMQ, and I need to have detailed information about
> a lot of things, especially threading issues. I need to have some
> global resources which will be presumably used by multiple threads.
> I can see that there is a lot of documentation, but I'd really
> appreciate pointers towards the books, or key documents that'd help me
> move forward faster (docs/books about inner workings of key
> functionality) I'll be using C (most likely the best option) to
> develop code, so which books/documents would you recommend?

There are no threading issues in postgres, because postgres doesn't
use threads. Each client connection is serviced by one backend process,
launched by the postmaster when a new client connects. Communication
between backend processes takes places via a shared memory segment
and at times also via signals.

The documentation contains extensive information about how to interface
3rd-party code with postgres. To see how to interface C functions with
the SQL layer, read http://www.postgresql.org/docs/9.0/interactive/xfunc-c.html.
If you need to also access the database from your C-language functions,
also read http://www.postgresql.org/docs/9.0/interactive/spi.html.

More exhaustive documentation is spread around the source tree in
the form of README files. I suggest you read the ones concerned with
the postgres subsystems you're dealing with. At the very least, you
should read .//src/backend/utils/mmgr/README which explains how
postgres manages memory.

The various contrib modules, found in contrib/ in the source tree,
are also a good reference.

best regards,
Florian Pflug

Well process or thread, the issues are the same.

Check out the existing modules and how they use spinlocks and lwlocks to
protect shared memory data structures.

One thing to beware of is that there's no shared memory manager so all
shared data structures need to be fixed size allocated on startup. Anything
where that isn't good enough usually ends up as an on disk data structure.
On Jun 12, 2011 8:37 PM, "Florian Pflug" <fgp(at)phlo(dot)org> wrote:
> On Jun12, 2011, at 19:26 , Seref Arikan wrote:
>> This is actually a request for documentation guidance. I intend to
>> develop an extension to postgresql. Basically I'd like to place calls
>> to network using ZeroMQ, and I need to have detailed information about
>> a lot of things, especially threading issues. I need to have some
>> global resources which will be presumably used by multiple threads.
>> I can see that there is a lot of documentation, but I'd really
>> appreciate pointers towards the books, or key documents that'd help me
>> move forward faster (docs/books about inner workings of key
>> functionality) I'll be using C (most likely the best option) to
>> develop code, so which books/documents would you recommend?
>
>
> There are no threading issues in postgres, because postgres doesn't
> use threads. Each client connection is serviced by one backend process,
> launched by the postmaster when a new client connects. Communication
> between backend processes takes places via a shared memory segment
> and at times also via signals.
>
> The documentation contains extensive information about how to interface
> 3rd-party code with postgres. To see how to interface C functions with
> the SQL layer, read
http://www.postgresql.org/docs/9.0/interactive/xfunc-c.html.
> If you need to also access the database from your C-language functions,
> also read http://www.postgresql.org/docs/9.0/interactive/spi.html.
>
> More exhaustive documentation is spread around the source tree in
> the form of README files. I suggest you read the ones concerned with
> the postgres subsystems you're dealing with. At the very least, you
> should read .//src/backend/utils/mmgr/README which explains how
> postgres manages memory.
>
> The various contrib modules, found in contrib/ in the source tree,
> are also a good reference.
>
> best regards,
> Florian Pflug
>
>
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

Seref Arikan <serefarikan(at)kurumsalteknoloji(dot)com> writes:
> This is actually a request for documentation guidance. I intend to
> develop an extension to postgresql. Basically I'd like to place calls
> to network using ZeroMQ, and I need to have detailed information about

You didn't tell us about what you want to achieve, only how. It could
be that your problem could be solved with a custom PGQ consumer, that
leaves well outside the database server. PGQ is the generic queue layer
on which the replication system Londiste is built.

http://wiki.postgresql.org/wiki/Skytools
http://wiki.postgresql.org/wiki/PGQ_Tutorial
http://wiki.postgresql.org/wiki/Londiste_Tutorial

Regards,
--
Dimitri Fontaine
http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support