Fwd: psql include file using relative path

psql has the ability to execute commands from a file, but if one wishes
to develop and provide a modularized set of sql files, then psql is not very
helpful because the \i command can open file paths either if they are
absolute paths or if they are palced correctly relative to psql's current
working directory.

Attached patch adds a new meta-command to psql, '\ir' that allows the user
to process files relative to currently processing file.

Also attached is a sample use case ir_sample. One can extract it _anywhere_
on the filesystem and invoke psql with the path to main.sql and the rest of
the files will be automatically included from that location.

Sample session:

[/tmp]$ psql -f ~/dev/ir_sample/main.sql
processing main.sql
BEGIN
processing subdir1/1.sql
processing subdir1/2.sql
processing subdir2/1.sql
processing subdir2/subdir2.1/1.sql
processing subdir2/2.sql
processing subdir2/3.sql
COMMIT

And here's what the sample's directory structure and files look like:

[ir_sample]$ find ./ -name "*.sql" | while read f; do echo === $f ====; cat
$f; done
=== ./main.sql ====
\echo processing main.sql
BEGIN;
\ir subdir1/1.sql
\ir subdir1/2.sql
\ir subdir2/1.sql
\ir subdir2/2.sql
\ir subdir2/3.sql
COMMIT;
=== ./subdir1/1.sql ====
\echo processing subdir1/1.sql
=== ./subdir1/2.sql ====
\echo processing subdir1/2.sql
=== ./subdir2/subdir2.1/1.sql ====
\echo processing subdir2/subdir2.1/1.sql
=== ./subdir2/1.sql ====
\echo processing subdir2/1.sql
\ir subdir2.1/1.sql
=== ./subdir2/2.sql ====
\echo processing subdir2/2.sql
=== ./subdir2/3.sql ====
\echo processing subdir2/3.sql

Regards,
--
Gurjeet Singh
EnterpriseDB Corporation <http://www.enterprisedb.com/>
The Enterprise PostgreSQL <http://www.postgresql.org/> Company

On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh <singh(dot)gurjeet(at)gmail(dot)com> wrote:
>     psql has the ability to execute commands from a file, but if one wishes
> to develop and provide a modularized set of sql files, then psql is not very
> helpful because the \i command can open file paths either if they are
> absolute paths or if they are palced correctly relative to psql's current
> working directory.
>
> Attached patch adds a new meta-command to psql, '\ir' that allows the user
> to process files relative to currently processing file.

Please add this patch to the currently open CommitFest at:

https://commitfest.postgresql.org/action/commitfest_view/open

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Gurjeet!

What about tab completion, like in \i command?

On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh <singh(dot)gurjeet(at)gmail(dot)com> wrote:
>>     psql has the ability to execute commands from a file, but if one wishes
>> to develop and provide a modularized set of sql files, then psql is not very
>> helpful because the \i command can open file paths either if they are
>> absolute paths or if they are palced correctly relative to psql's current
>> working directory.
>>
>> Attached patch adds a new meta-command to psql, '\ir' that allows the user
>> to process files relative to currently processing file.
>
> Please add this patch to the currently open CommitFest at:
>
> https://commitfest.postgresql.org/action/commitfest_view/open
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers
>

--
   Ibrar Ahmed

Attached patch implements tab completion. It also introduces the long-form
alternative \include_relative for \ir

Regards,

On Tue, Mar 8, 2011 at 2:18 PM, Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com> wrote:

> Gurjeet!
>
> What about tab completion, like in \i command?
>
> On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas <robertmhaas(at)gmail(dot)com>
> wrote:
> > On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh <singh(dot)gurjeet(at)gmail(dot)com>
> wrote:
> >> psql has the ability to execute commands from a file, but if one
> wishes
> >> to develop and provide a modularized set of sql files, then psql is not
> very
> >> helpful because the \i command can open file paths either if they are
> >> absolute paths or if they are palced correctly relative to psql's
> current
> >> working directory.
> >>
> >> Attached patch adds a new meta-command to psql, '\ir' that allows the
> user
> >> to process files relative to currently processing file.
> >
> > Please add this patch to the currently open CommitFest at:
> >
> > https://commitfest.postgresql.org/action/commitfest_view/open
> >
> > --
> > Robert Haas
> > EnterpriseDB: http://www.enterprisedb.com
> > The Enterprise PostgreSQL Company
> >
> > --
> > Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-hackers
> >
>
>
>
> --
> Ibrar Ahmed
>

--
Gurjeet Singh
EnterpriseDB <http://www.enterprisedb.com/> Corporation
The Enterprise PostgreSQL <http://www.postgresql.org/> Company

Being able to include relative paths is a really great feature, but
should it have a UI (well, API) distinct from fixed-path includes? My
first instinct is that it shouldn't, but I haven't really thought it
through thoroughly.

Cheers,
David (the tough coughs as he ploughs the dough)
On Tue, Mar 08, 2011 at 03:19:40PM -0500, Gurjeet Singh wrote:
> Attached patch implements tab completion. It also introduces the long-form
> alternative \include_relative for \ir
>
> Regards,
>
> On Tue, Mar 8, 2011 at 2:18 PM, Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com> wrote:
>
> > Gurjeet!
> >
> > What about tab completion, like in \i command?
> >
> > On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas <robertmhaas(at)gmail(dot)com>
> > wrote:
> > > On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh <singh(dot)gurjeet(at)gmail(dot)com>
> > wrote:
> > >> psql has the ability to execute commands from a file, but if one
> > wishes
> > >> to develop and provide a modularized set of sql files, then psql is not
> > very
> > >> helpful because the \i command can open file paths either if they are
> > >> absolute paths or if they are palced correctly relative to psql's
> > current
> > >> working directory.
> > >>
> > >> Attached patch adds a new meta-command to psql, '\ir' that allows the
> > user
> > >> to process files relative to currently processing file.
> > >
> > > Please add this patch to the currently open CommitFest at:
> > >
> > > https://commitfest.postgresql.org/action/commitfest_view/open
> > >
> > > --
> > > Robert Haas
> > > EnterpriseDB: http://www.enterprisedb.com
> > > The Enterprise PostgreSQL Company
> > >
> > > --
> > > Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> > > To make changes to your subscription:
> > > http://www.postgresql.org/mailpref/pgsql-hackers
> > >
> >
> >
> >
> > --
> > Ibrar Ahmed
> >
>
>
>
> --
> Gurjeet Singh
> EnterpriseDB <http://www.enterprisedb.com/> Corporation
> The Enterprise PostgreSQL <http://www.postgresql.org/> Company

>
> --
> Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-hackers

--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

Good question, I hadn't thought of that either, and thinking about it a bit
I think we'd want to keep the current behaviour of \i and provide new
behaviour using a new command.

Say when we are processing a pretty nested file after multiple \ir commands,
a \i <relative path file> in any of those files should look for that file in
psql's CWD/PWD. That is what the user expects from \i command currently and
I don't think it'd be desirable to break that assumption.

Regards,

On Wed, Mar 9, 2011 at 4:41 PM, David Fetter <david(at)fetter(dot)org> wrote:

> Being able to include relative paths is a really great feature, but
> should it have a UI (well, API) distinct from fixed-path includes? My
> first instinct is that it shouldn't, but I haven't really thought it
> through thoroughly.
>
> Cheers,
> David (the tough coughs as he ploughs the dough)
> On Tue, Mar 08, 2011 at 03:19:40PM -0500, Gurjeet Singh wrote:
> > Attached patch implements tab completion. It also introduces the
> long-form
> > alternative \include_relative for \ir
> >
> > Regards,
> >
> > On Tue, Mar 8, 2011 at 2:18 PM, Ibrar Ahmed <ibrar(dot)ahmad(at)gmail(dot)com>
> wrote:
> >
> > > Gurjeet!
> > >
> > > What about tab completion, like in \i command?
> > >
> > > On Fri, Feb 25, 2011 at 5:07 AM, Robert Haas <robertmhaas(at)gmail(dot)com>
> > > wrote:
> > > > On Thu, Feb 24, 2011 at 6:21 PM, Gurjeet Singh <
> singh(dot)gurjeet(at)gmail(dot)com>
> > > wrote:
> > > >> psql has the ability to execute commands from a file, but if one
> > > wishes
> > > >> to develop and provide a modularized set of sql files, then psql is
> not
> > > very
> > > >> helpful because the \i command can open file paths either if they
> are
> > > >> absolute paths or if they are palced correctly relative to psql's
> > > current
> > > >> working directory.
> > > >>
> > > >> Attached patch adds a new meta-command to psql, '\ir' that allows
> the
> > > user
> > > >> to process files relative to currently processing file.
> > > >
> > > > Please add this patch to the currently open CommitFest at:
> > > >
> > > > https://commitfest.postgresql.org/action/commitfest_view/open
> > > >
> > > > --
> > > > Robert Haas
> > > > EnterpriseDB: http://www.enterprisedb.com
> > > > The Enterprise PostgreSQL Company
> > > >
> > > > --
> > > > Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> > > > To make changes to your subscription:
> > > > http://www.postgresql.org/mailpref/pgsql-hackers
> > > >
> > >
> > >
> > >
> > > --
> > > Ibrar Ahmed
> > >
> >
> >
> >
> > --
> > Gurjeet Singh
> > EnterpriseDB <http://www.enterprisedb.com/> Corporation
> > The Enterprise PostgreSQL <http://www.postgresql.org/> Company
>
>
> >
> > --
> > Sent via pgsql-hackers mailing list (pgsql-hackers(at)postgresql(dot)org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-hackers
>
>
> --
> David Fetter <david(at)fetter(dot)org> http://fetter.org/
> Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
> Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
> iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics
>
> Remember to vote!
> Consider donating to Postgres: http://www.postgresql.org/about/donate
>

--
Gurjeet Singh
EnterpriseDB <http://www.enterprisedb.com/> Corporation
The Enterprise PostgreSQL <http://www.postgresql.org/> Company

On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
> Good question, I hadn't thought of that either, and thinking about
> it a bit I think we'd want to keep the current behaviour of \i and
> provide new behaviour using a new command.
>
> Say when we are processing a pretty nested file after multiple \ir
> commands, a \i <relative path file> in any of those files should
> look for that file in psql's CWD/PWD. That is what the user expects
> from \i command currently and I don't think it'd be desirable to
> break that assumption.

I'm not sure I understand. Stuff that worked before would still work.

Should stuff break when it has a legitimately accessible path in it
just because that path is relative?

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

On Wed, Mar 9, 2011 at 7:32 PM, David Fetter <david(at)fetter(dot)org> wrote:

> On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
> > Good question, I hadn't thought of that either, and thinking about
> > it a bit I think we'd want to keep the current behaviour of \i and
> > provide new behaviour using a new command.
> >
> > Say when we are processing a pretty nested file after multiple \ir
> > commands, a \i <relative path file> in any of those files should
> > look for that file in psql's CWD/PWD. That is what the user expects
> > from \i command currently and I don't think it'd be desirable to
> > break that assumption.
>
> I'm not sure I understand. Stuff that worked before would still work.
>

files present:
~/1.sql
~/package/main.sql
~/package/1.sql

psql's CWD: $HOME
psql executed as: psql -f ~/package/main.sql

Now if we kept \i and \ir separate then if main.sql does

\i 1.sql

it'd read ~/1.sql, which is the current behaviour,

and

\ir 1.sql

would read ~/package/1.sql .

If we folded \ir into \i then what would you want `\i 1.sql` to do? Read
1.sql from $HOME or the one that is main.sql's sibling.

>
> Should stuff break when it has a legitimately accessible path in it
> just because that path is relative?
>

Given the above test case, I think it'd be best if we introduced a new
command for this feature.

Regards,
--
Gurjeet Singh
EnterpriseDB <http://www.enterprisedb.com/> Corporation
The Enterprise PostgreSQL <http://www.postgresql.org/> Company

On 03/09/2011 09:36 PM, Gurjeet Singh wrote:
> If we folded \ir into \i then what would you want `\i 1.sql` to do?
> Read 1.sql from $HOME or the one that is main.sql's sibling.
>
>
> Should stuff break when it has a legitimately accessible path in it
> just because that path is relative?
>
>
> Given the above test case, I think it'd be best if we introduced a new
> command for this feature.
>
>

I agree there's a good case for the new feature. I think someone
mentioned tab completion upthread, and that doesn't make so much sense
to me. This only makes sense nested in a script - in fact if it's not
called from inside an included script (via -f or \i) it should possibly
error out (if it already does this I apologise - I haven't looked at the
patch).

cheers

andrew

On Wed, Mar 9, 2011 at 10:07 PM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:

>
> I agree there's a good case for the new feature. I think someone mentioned
> tab completion upthread, and that doesn't make so much sense to me. This
> only makes sense nested in a script - in fact if it's not called from inside
> an included script (via -f or \i) it should possibly error out (if it
> already does this I apologise - I haven't looked at the patch).
>
>
I think \ir can stand on its own. In the patch, the \ir command falls back
to \i behaviour if there's no file being processed currently. So, I think
tab-completion makes sense for this command. And if someone wishes they can
stop using \i altogether and \ir will give them old and new bheaviour
seamlessly.

Regards,
--
Gurjeet Singh
EnterpriseDB <http://www.enterprisedb.com/> Corporation
The Enterprise PostgreSQL <http://www.postgresql.org/> Company

On Wed, Mar 9, 2011 at 7:32 PM, David Fetter <david(at)fetter(dot)org> wrote:
> On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
>> Good question, I hadn't thought of that either, and thinking about
>> it a bit I think we'd want to keep the current behaviour of \i and
>> provide new behaviour using a new command.
>>
>> Say when we are processing a pretty nested file after multiple \ir
>> commands, a \i <relative path file> in any of those files should
>> look for that file in psql's CWD/PWD. That is what the user expects
>> from \i command currently and I don't think it'd be desirable to
>> break that assumption.
>
> I'm not sure I understand.  Stuff that worked before would still work.
>
> Should stuff break when it has a legitimately accessible path in it
> just because that path is relative?

You're confused. The point is whether the path is relative to PWD or
to the directory in which the currently executing script is located.
If you want to allow people to get either interpretation, you need two
commands.

In interactive use, I believe there's no difference between the two.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

On Wed, Mar 09, 2011 at 10:57:53PM -0500, Robert Haas wrote:
> On Wed, Mar 9, 2011 at 7:32 PM, David Fetter <david(at)fetter(dot)org> wrote:
> > On Wed, Mar 09, 2011 at 07:05:19PM -0500, Gurjeet Singh wrote:
> >> Good question, I hadn't thought of that either, and thinking
> >> about it a bit I think we'd want to keep the current behaviour of
> >> \i and provide new behaviour using a new command.
> >>
> >> Say when we are processing a pretty nested file after multiple
> >> \ir commands, a \i <relative path file> in any of those files
> >> should look for that file in psql's CWD/PWD. That is what the
> >> user expects from \i command currently and I don't think it'd be
> >> desirable to break that assumption.
> >
> > I'm not sure I understand.  Stuff that worked before would still
> > work.
> >
> > Should stuff break when it has a legitimately accessible path in
> > it just because that path is relative?
>
> You're confused. The point is whether the path is relative to PWD
> or to the directory in which the currently executing script is
> located. If you want to allow people to get either interpretation,
> you need two commands.
>
> In interactive use, I believe there's no difference between the two.

Thanks for clearing that up :)

OK, nothing to see here. Move along ;)

Cheers,
David.
--
David Fetter <david(at)fetter(dot)org> http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david(dot)fetter(at)gmail(dot)com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate