Review : Add hooks for pre- and post-processor executables for COPY and \copy

From: Amit Kapila <amit(dot)kapila(at)huawei(dot)com>
To: "'pgsql-hackers'" <pgsql-hackers(at)postgresql(dot)org>
Subject: Review : Add hooks for pre- and post-processor executables for COPY and \copy
Date: 2012-12-18 12:42:25
Message-ID: 006801cddd1d$22773440$67659cc0$@kapila@huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Basic stuff:
------------
- Rebase of Patch is required.
- Compiles cleanly without any errors/warnings
- Regression tests pass.

What it does:
---------------------
This patch is useful when COPY command input/output are stored in
compression format or in any command/script uses these output/input in any
means; without generating intermediate temporary files.
This feature can be used in server side using "COPY statement" by
administrator. Or can be used in psql internal "\copy" command by any user.

Code Review comments:
---------------------

1. Modify the comment in function header of: parse_slash_copy (needs to
modify for new syntax)
2. Comments for functions OpenPipeStream & ClosePipeStream are missing.

3. Any Script errors are not directly visible to user; If there problems in
script no way to cleanup.

Shouldn't this be mentioned in User Manual.

Test case issues:
------------------
1. "Broken pipe" is not handled in case of psql "\copy" command;
Issue are as follows:
Following are verified on SuSE-Linux 10.2.
1) psql is exiting when "\COPY xxx TO" command is issued and
command/script is not found

When popen is called in write mode it is creating valid
file descriptor and when it tries to write to file "Broken pipe" error is
coming which is not handled.
psql# \copy pgbench_accounts TO PROGRAM
'../compress.sh pgbench_accounts4.txt'
2) When "\copy" command is in progress then program/command is
killed/"crashed due to any problem"
psql is exiting.

Script used in testcases:
------------------
1. compress.sh
echo 'cat > $1' > compress.sh
echo 'bzip2 -z $1' >> compress.sh
chmod +x compress.sh

2. decompress.sh
echo 'bzip2 -d -c -k $*' > decompress.sh
chmod +x decompress.sh

Testcases executed are attached with this mail.

With Regards,

Amit Kapila.

Attachment Content-Type Size
testcases.txt text/plain 3.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Christoph Berg 2012-12-18 12:47:33 Re: configure.in and setproctitle/optreset problem
Previous Message Andres Freund 2012-12-18 12:25:04 Re: XLByte* usage