Re: [HACKERS] - WIP Patch Updatable Cursor

Hi,

This is the first posting to the community of the WIP patch for the
Updatable Cursor implementation.

I want to confirm that the community is satisfied that the effort to date is
in a suitable direction and to get comments on the development to date.

The patch is in the following state:

The grammar definition is complete and 'yacc'ed to produce gram.y.c.

The functions transformUpdateStmt and transformDeleteStmt have been updated
to process the cursor name and obtain the related portal.

The change to save the current tuple id (ctid) into the portal, related to
the Fetch command has been done.

The ctids relating to the Update/Delete statements' TidScan are being
extracted to be saved in the executor.

The parts in progress are to complete the saving of the ctids from the
TidScan into a list stored in a file, plus related searching the list for an
individual ctid obtained from the Update/Delete statements.

Unstarted as yet:

1) Correctly process, in the database, the Delete / Update of the
tuple from the cursor.

2) To enable the cursor name to be defined as a parameter in a
PREPARE statement and provided as part if an EXECUTE statement.

The community may wish to comment on the following issue:

1) At present the file that will contain the list of ctids is going into
a new directory called pg_ctids, analogous to pg_twophase, and also stored
in the pg_data directory.

Regards,
John Bartlett

This is an email from Fujitsu Australia Software Technology Pty Ltd, ABN 27 003 693 481. It is confidential to the ordinary user of the email address to which it was addressed and may contain copyright and/or legally privileged information. No one else may read, print, store, copy or forward all or any of it or its attachments. If you receive this email in error, please return to sender. Thank you.

If you do not wish to receive commercial email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(at)fast(dot)fujitsu(dot)com(dot)au

John Bartlett wrote:
> The community may wish to comment on the following issue:
>
> 1) At present the file that will contain the list of ctids is going into
> a new directory called pg_ctids, analogous to pg_twophase, and also stored
> in the pg_data directory.

I don't understand this. What's stored in the file and why? If they're
only needed within the transaction, surely a temp file would be more
appropriate?

The new ctidListStore.c file in the patch is not in a valid diff-format.
I also noticed that you've moved the line beginning with "CREATE_ROLE"
in gram.y so that it's not in alphabetical order anymore.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

Hi,

A list of ctids is stored in the file.

The file is used to store the ctids during an updatable cursor transaction.

It is set up as a permanent file as it has a potential lifetime of
preserving data between crashes of the backend. Temporary files tend to be
used for data that is defined within a single command. In this case the file
needs to exist within a transaction and across backend processes.

The file gram.y has been corrected in my version.

The files ctidListStore.c and ctidListStore.h were pasted into the patch
file, as the diff -N command produced a file of several hundred thousand
lines.

Regards,
John Bartlett

-----Original Message-----
From: Heikki Linnakangas [mailto:hlinnaka(at)gmail(dot)com] On Behalf Of Heikki
Linnakangas
Sent: Tuesday, 27 February 2007 10:03 PM
To: John Bartlett
Cc: pgsql-patches(at)postgresql(dot)org; pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCHES]

John Bartlett wrote:
> The community may wish to comment on the following issue:
>
> 1) At present the file that will contain the list of ctids is going
into
> a new directory called pg_ctids, analogous to pg_twophase, and also stored
> in the pg_data directory.

I don't understand this. What's stored in the file and why? If they're
only needed within the transaction, surely a temp file would be more
appropriate?

The new ctidListStore.c file in the patch is not in a valid diff-format.
I also noticed that you've moved the line beginning with "CREATE_ROLE"
in gram.y so that it's not in alphabetical order anymore.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com

This is an email from Fujitsu Australia Software Technology Pty Ltd, ABN 27 003 693 481. It is confidential to the ordinary user of the email address to which it was addressed and may contain copyright and/or legally privileged information. No one else may read, print, store, copy or forward all or any of it or its attachments. If you receive this email in error, please return to sender. Thank you.

If you do not wish to receive commercial email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(at)fast(dot)fujitsu(dot)com(dot)au

On Wed, 28 Feb 2007, John Bartlett wrote:

> Hi,
>
> A list of ctids is stored in the file.

I would have thought these would be stored in memory. If the set got
large, you'd use a temporary file the way other systems which overflow to
disk do?

>
> The file is used to store the ctids during an updatable cursor transaction.
>
> It is set up as a permanent file as it has a potential lifetime of
> preserving data between crashes of the backend. Temporary files tend to be
> used for data that is defined within a single command. In this case the file
> needs to exist within a transaction and across backend processes.

It does not. Cursors are implicitly closed when a session is closed. A
backend crash or system restart closes all open sessions.

>
> The file gram.y has been corrected in my version.
>
> The files ctidListStore.c and ctidListStore.h were pasted into the patch
> file, as the diff -N command produced a file of several hundred thousand
> lines.

Edit the file with a text editor. If you know which files should be
excluded (like tags files), use diff --exclude=<pattern>.

Thanks,

Gavin

Hi Gavin,

Thanks for your comments.

I shall set up the ctids list to be stored in memory initially and dump to a
temp file if needed.

Regards,
John Bartlett

-----Original Message-----
From: pgsql-patches-owner(at)postgresql(dot)org
[mailto:pgsql-patches-owner(at)postgresql(dot)org] On Behalf Of Gavin Sherry
Sent: Wednesday, 28 February 2007 3:23 PM
To: John Bartlett
Cc: pgsql-hackers(at)postgresql(dot)org; pgsql-patches(at)postgresql(dot)org; 'Heikki
Linnakangas'
Subject: Re: [HACKERS] [PATCHES] - WIP Patch Updatable Cursor

On Wed, 28 Feb 2007, John Bartlett wrote:

> Hi,
>
> A list of ctids is stored in the file.

I would have thought these would be stored in memory. If the set got
large, you'd use a temporary file the way other systems which overflow to
disk do?

>
> The file is used to store the ctids during an updatable cursor
transaction.
>
> It is set up as a permanent file as it has a potential lifetime of
> preserving data between crashes of the backend. Temporary files tend to be
> used for data that is defined within a single command. In this case the
file
> needs to exist within a transaction and across backend processes.

It does not. Cursors are implicitly closed when a session is closed. A
backend crash or system restart closes all open sessions.

>
> The file gram.y has been corrected in my version.
>
> The files ctidListStore.c and ctidListStore.h were pasted into the patch
> file, as the diff -N command produced a file of several hundred thousand
> lines.

Edit the file with a text editor. If you know which files should be
excluded (like tags files), use diff --exclude=<pattern>.

Thanks,

Gavin

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
message can get through to the mailing list cleanly

This is an email from Fujitsu Australia Software Technology Pty Ltd, ABN 27 003 693 481. It is confidential to the ordinary user of the email address to which it was addressed and may contain copyright and/or legally privileged information. No one else may read, print, store, copy or forward all or any of it or its attachments. If you receive this email in error, please return to sender. Thank you.

If you do not wish to receive commercial email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(at)fast(dot)fujitsu(dot)com(dot)au