Policy on pulling in code from other projects?

Lists: pgsql-hackers
From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Policy on pulling in code from other projects?
Date: 2011-07-21 17:43:06
Message-ID: 4E28652A.8050208@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hey,

So I am looking intently on what it is going to take to get the URI
patch done for psql [1] and was digging around the web and have a URI
parser library. It is under the New BSD license and is strictly RFC RFC
3986 [2] compliant .

Now I have not dug into the code but the parser is used by other
projects. So question is:

Assuming the code actually makes this patch easier, do we:

A. Pull in the code into the main tree
B. Instead have it as a requirement via configure?

1.
http://archives.postgresql.org/message-id/1302114698.23164.17.camel@jd-desktop

2. http://tools.ietf.org/html/rfc3986

Sincerely,

Joshua D. Drake

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: Peter Geoghegan <peter(at)2ndquadrant(dot)com>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-21 18:11:13
Message-ID: CAEYLb_WdFoJPEcxxD5kX0XdFQ7beQP=URQRf=32b1CF8MsBamA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 21 July 2011 18:43, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> So I am looking intently on what it is going to take to get the URI patch
> done for psql [1] and was digging around the web and have a URI parser
> library. It is under the New BSD license and is strictly RFC  RFC 3986 [2]
> compliant .
>
> Now I have not dug into the code but the parser is used by other projects.
> So question is:
>
> Assuming the code actually makes this patch easier, do we:
>
> A. Pull in the code into the main tree
> B. Instead have it as a requirement via configure?
>
> 1.
> http://archives.postgresql.org/message-id/1302114698.23164.17.camel@jd-desktop
>
> 2. http://tools.ietf.org/html/rfc3986

Without commenting on the practicalities of what you'd like to do,
including code from other projects in the tree is well precedented.
Off the top of my head, I can tell you that pgcrypto uses code from
various sources, while preserving the original copyright notices.

--
Peter Geoghegan       http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training and Services


From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-21 18:13:17
Message-ID: 17411.1311271997@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"Joshua D. Drake" <jd(at)commandprompt(dot)com> writes:
> So I am looking intently on what it is going to take to get the URI
> patch done for psql [1] and was digging around the web and have a URI
> parser library. It is under the New BSD license and is strictly RFC RFC
> 3986 [2] compliant .

Surely we do not need a whole library to parse URIs.

regards, tom lane


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-21 18:18:14
Message-ID: 4E286D66.5060301@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/21/2011 11:13 AM, Tom Lane wrote:
> "Joshua D. Drake"<jd(at)commandprompt(dot)com> writes:
>> So I am looking intently on what it is going to take to get the URI
>> patch done for psql [1] and was digging around the web and have a URI
>> parser library. It is under the New BSD license and is strictly RFC RFC
>> 3986 [2] compliant .
>
> Surely we do not need a whole library to parse URIs.

Shrug, standards compliant, already runs on windows....

Seems like a good idea to me?

http://uriparser.sourceforge.net/

Sincerely,

Joshua D. Drake

>
> regards, tom lane
>

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-21 18:19:27
Message-ID: 4E286DAF.80005@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/21/2011 11:13 AM, Tom Lane wrote:
> "Joshua D. Drake"<jd(at)commandprompt(dot)com> writes:
>> So I am looking intently on what it is going to take to get the URI
>> patch done for psql [1] and was digging around the web and have a URI
>> parser library. It is under the New BSD license and is strictly RFC RFC
>> 3986 [2] compliant .
>
> Surely we do not need a whole library to parse URIs.

Also:

http://uriparser.git.sourceforge.net/git/gitweb-index.cgi

Sincerely,

Joshua D. Drake

>
> regards, tom lane
>

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 14:51:35
Message-ID: 4E298E77.3010203@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/21/2011 11:13 AM, Tom Lane wrote:
> "Joshua D. Drake"<jd(at)commandprompt(dot)com> writes:
>> So I am looking intently on what it is going to take to get the URI
>> patch done for psql [1] and was digging around the web and have a URI
>> parser library. It is under the New BSD license and is strictly RFC RFC
>> 3986 [2] compliant .
>
> Surely we do not need a whole library to parse URIs.
>
> regards, tom lane
>

Any other comments? I don't want to do a bunch of work just to have it
tossesd on a technicality.

JD

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 15:36:30
Message-ID: CA+TgmobS+-SsML5Ym6oN0zBcfL+T_=RciARf=cfgRKvuC+1dNQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 22, 2011 at 10:51 AM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> On 07/21/2011 11:13 AM, Tom Lane wrote:
>>
>> "Joshua D. Drake"<jd(at)commandprompt(dot)com>  writes:
>>>
>>> So I am looking intently on what it is going to take to get the URI
>>> patch done for psql [1] and was digging around the web and have a URI
>>> parser library. It is under the New BSD license and is strictly RFC  RFC
>>> 3986 [2] compliant .
>>
>> Surely we do not need a whole library to parse URIs.
>
> Any other comments? I don't want to do a bunch of work just to have it
> tossesd on a technicality.

Well, you haven't explained what feature you are trying to develop, so
it's a bit premature to speculate on whether that feature is valuable
enough to justify the carrying cost of another library.

Like Tom, I'm not real sure why we would need a library for this. It
seems like a hundred lines of C would be sufficient for most things
you'd likely want to do, and less work than integrating someone else's
code.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 15:43:47
Message-ID: 4E299AB3.3050700@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/22/2011 08:36 AM, Robert Haas wrote:
> On Fri, Jul 22, 2011 at 10:51 AM, Joshua D. Drake<jd(at)commandprompt(dot)com> wrote:
>> On 07/21/2011 11:13 AM, Tom Lane wrote:
>>>
>>> "Joshua D. Drake"<jd(at)commandprompt(dot)com> writes:
>>>>
>>>> So I am looking intently on what it is going to take to get the URI
>>>> patch done for psql [1] and was digging around the web and have a URI
>>>> parser library. It is under the New BSD license and is strictly RFC RFC
>>>> 3986 [2] compliant .
>>>
>>> Surely we do not need a whole library to parse URIs.
>>
>> Any other comments? I don't want to do a bunch of work just to have it
>> tossesd on a technicality.
>
> Well, you haven't explained what feature you are trying to develop, so
> it's a bit premature to speculate on whether that feature is valuable
> enough to justify the carrying cost of another library.

Yes I did:

OP of this thread:

http://archives.postgresql.org/pgsql-hackers/2011-07/msg01144.php

It is letting pgsql use URI syntax.

Sincerely,

Joshua D. Drake

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 16:03:59
Message-ID: CA+TgmoYRyEcHUhLQw44_nUoHt+Nu1k2Hx_96wR4vnqZckkjMhA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 22, 2011 at 11:43 AM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> OP of this thread:
>
> http://archives.postgresql.org/pgsql-hackers/2011-07/msg01144.php
>
> It is letting pgsql use URI syntax.

Sorry, I missed that the first time.

IMHO, it seems like it would be simpler to do that by rolling our own
code rather than importing someone else's.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 17:26:44
Message-ID: 4E29B2D4.5000902@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/22/2011 09:03 AM, Robert Haas wrote:
> On Fri, Jul 22, 2011 at 11:43 AM, Joshua D. Drake<jd(at)commandprompt(dot)com> wrote:
>> OP of this thread:
>>
>> http://archives.postgresql.org/pgsql-hackers/2011-07/msg01144.php
>>
>> It is letting pgsql use URI syntax.
>
> Sorry, I missed that the first time.
>
> IMHO, it seems like it would be simpler to do that by rolling our own
> code rather than importing someone else's.

I guess it would depend on how full featured we want the code. The
library provides validation of various things that rolling our own code
likely wouldn't. I mean, it is a library just like libssl or any other
such thing.

I would imagine that rolling our own code would likely be simpler but
the code itself would be dumber (not bad, just not as capable) and we
would be duplicating the effort of an already established project. Do we
really want to do that?

Sincerely,

Joshua D. Drake

>

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 17:57:46
Message-ID: CA+TgmobkYkOdT0sZYR45S0NCGwUWXgKYdoczyveHvyheddx8xw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 22, 2011 at 1:26 PM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
>> IMHO, it seems like it would be simpler to do that by rolling our own
>> code rather than importing someone else's.
>
> I guess it would depend on how full featured we want the code. The library
> provides validation of various things that rolling our own code likely
> wouldn't. I mean, it is a library just like libssl or any other such thing.

Uhm, that's a complete apples-and-oranges comparison. libssl is at
least three orders of magnitude more complicated than URL parsing, and
provides a critical feature we would otherwise lack. Parsing URLs is
trivial and something we can easily code ourselves, and supports a
marginal feature that would be nice to have but is not make-or-break.

If we decide to pull their source code into our repo, then will you
also expect us to update it every time they release a new feature?
What about when they fix a bug? What about when they fix a security
bug? And what happens when the next 15 people who want similarly
minor features want some library included to support their feature,
too? Shall we now be responsible for syncing up with all of those
projects?

> I would imagine that rolling our own code would likely be simpler but the
> code itself would be dumber (not bad, just not as capable) and we would be
> duplicating the effort of an already established project. Do we really want
> to do that?

Odds are good that most of the extra features they've added are things
we don't need or can't benefit from. AFAIK, we only care about pgsql
URLs, not http or wais or whatever other crazy things they support.
So we'll be increasing either the amount of code we have to maintain -
and the size of our binaries on disk - for basically no benefit. Do
we really want to do that?

I mean, look, there is precedent for doing this. We sucked in chunks
of snowball for tsearch and, separately, somebody's regex
implementation. We depend on libedit or libreadline for command-line
editing, libssl for encryption, and various other libraries for LDAP
and Kerberos. So if you're asking: would we ever consider doing this?
Sure, because we already have. There is no rule against it. But if
you're arguing that the benefits of this library for this feature are
sufficient to justify sucking it in, I'm so far unconvinced. What
does it do that is so much better than what we could code up on our
own in a couple of hours?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 18:09:26
Message-ID: 4E29BCD6.3010304@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/22/2011 10:57 AM, Robert Haas wrote:
> On Fri, Jul 22, 2011 at 1:26 PM, Joshua D. Drake<jd(at)commandprompt(dot)com> wrote:
>>> IMHO, it seems like it would be simpler to do that by rolling our own
>>> code rather than importing someone else's.

> I mean, look, there is precedent for doing this. We sucked in chunks
> of snowball for tsearch and, separately, somebody's regex
> implementation. We depend on libedit or libreadline for command-line
> editing, libssl for encryption, and various other libraries for LDAP
> and Kerberos. So if you're asking: would we ever consider doing this?
> Sure, because we already have. There is no rule against it. But if
> you're arguing that the benefits of this library for this feature are
> sufficient to justify sucking it in, I'm so far unconvinced. What
> does it do that is so much better than what we could code up on our
> own in a couple of hours?
>

I am not neccessarily looking to include it in our tarball. What I asked
was:

Assuming the code actually makes this patch easier, do we:

A. Pull in the code into the main tree
B. Instead have it as a requirement via configure?

I am not arguing one way or the other. I am trying to find the best
solution. I personally think you are handwaving if you think we can
build out a robust parser in a couple of hours.

Remember this library follows the RFC for URIs which is why I even
brought it up. If it was just some random parser, I wouldn't even have
bothered. Do we care about the RFC for URIs? (I am not being sarcastic),
if we don't let's just throw some simple code together, if we do, I
think this library makes sense, whether in our tree or not.

Sincerely,

Joshua D. Drake

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 18:21:53
Message-ID: CA+TgmoZCc9tfypMNvpVz0kuGuz__nDo0OvfgHQkzFFyE-NjpWw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 22, 2011 at 2:09 PM, Joshua D. Drake <jd(at)commandprompt(dot)com> wrote:
> I am not neccessarily looking to include it in our tarball. What I asked
> was:
>
> Assuming the code actually makes this patch easier, do we:
>
> A. Pull in the code into the main tree
> B. Instead have it as a requirement via configure?

Well, both ways are going to cause a certain number of problems.
Pulling it into the tree means we've got to maintain it forever;
requiring it via configure means that everyone who packages PostgreSQL
now has to include that library if they want to include PostgreSQL.

> I am not arguing one way or the other. I am trying to find the best
> solution. I personally think you are handwaving if you think we can build
> out a robust parser in a couple of hours.

I respect your opinion, but, in this case, I disagree.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, "Robert Haas" <robertmhaas(at)gmail(dot)com>
Cc: <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 18:22:09
Message-ID: 4E297981020000250003F70E@gw.wicourts.gov
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

"Joshua D. Drake" <jd(at)commandprompt(dot)com> wrote:

> I personally think you are handwaving if you think we can
> build out a robust parser in a couple of hours.

I don't think so. If you have a regular expression engine
available, you should be able to get there by picking one of these
and modifying:

http://regexlib.com/DisplayPatterns.aspx?cattabindex=1&categoryId=2

-Kevin


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Kevin Grittner <Kevin(dot)Grittner(at)wicourts(dot)gov>
Cc: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 18:25:09
Message-ID: CA+TgmoaAvx8yzgWph51jzjxFP8-UEBdzSLCvBY_-roxJZeaUQA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Jul 22, 2011 at 2:22 PM, Kevin Grittner
<Kevin(dot)Grittner(at)wicourts(dot)gov> wrote:
> "Joshua D. Drake" <jd(at)commandprompt(dot)com> wrote:
>> I personally think you are handwaving if you think we can
>> build out a robust parser in a couple of hours.
>
> I don't think so.  If you have a regular expression engine
> available, you should be able to get there by picking one of these
> and modifying:
>
> http://regexlib.com/DisplayPatterns.aspx?cattabindex=1&categoryId=2

Well, JD is talking about doing this client-side, so I don't think our
regex stuff would be available. Still, the fact that you can even
write a regex for it means it can't be that hard. It's usually pretty
darn straightforward to translate a regex into C code.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Peter Eisentraut <peter_e(at)gmx(dot)net>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 19:08:39
Message-ID: 1311361719.7322.0.camel@vanquo.pezone.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On fre, 2011-07-22 at 11:09 -0700, Joshua D. Drake wrote:
> Assuming the code actually makes this patch easier, do we:
>
> A. Pull in the code into the main tree
> B. Instead have it as a requirement via configure?

B


From: Greg Smith <greg(at)2ndQuadrant(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-22 20:25:34
Message-ID: 4E29DCBE.3050406@2ndQuadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/22/2011 02:09 PM, Joshua D. Drake wrote:
> Remember this library follows the RFC for URIs which is why I even
> brought it up. If it was just some random parser, I wouldn't even have
> bothered. Do we care about the RFC for URIs?

The main components of the RFC involve:

-Decoding escaped characters entered by percent-encoding
-Parsing the permissible IPv4 and IPv6 addresses
-Handling absolute vs. relative addresses. This is a lot of the spec,
and it's not really relevant for PostgreSQL URIs
-Splitting the URI into its five main components

I know I've seen a URL-oriented %-encoding decoder as a PostgreSQL
function already (I think Gabriele here wrote one). Surely useful IP
address decoding functions are already around. And the splitting part
seems like a fairly straightforward bit of regular expression work.

I think one crossover point where it's absolutely worth using the
external library for this purpose is if you have an app that has to
follow all of the rules around path names. If this project didn't
already have libraries around for things like IP address parsing, using
the library instead would also make more sense. The remaining chores if
you don't worry about all the path name trivia, and know how to
interpret an IP address, seem feasible to do directly.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us


From: Josh Berkus <josh(at)agliodbs(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-23 00:00:07
Message-ID: 4E2A0F07.3070001@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Arguments in favor of coding from scratch:

1) Does not introduce new dependencies into postgresql-client packages.
(note how much of a problem Readline has been)

2) keeps psql as lightweight as possible

3) We don't need to be able to parse any potential URI, just the ones we
accept.

Arguments against:

a) waste of time coding a parser

b) eventual feature creep as people ask us for more and more syntax support

Overall, though, I'd say that argument (1) against is pretty powerful.
Count the number of complaints and build bug reports about readline &
libedit on the lists.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com


From: Christopher Browne <cbbrowne(at)gmail(dot)com>
To: Josh Berkus <josh(at)agliodbs(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-23 02:10:51
Message-ID: CAFNqd5Vdb5Gsi8NoY6fEpYyuhrBwgmmFjTAm0Pttx8b9KFocCw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

I generally agree, Josh, but I think readline is getting pointed at a bit
too much. Yeah, it's a bad one, but we also include other stuff like zlib
that doesn't commonly come up as an issue.

I'd argue something just a wee bit different...

By the time we would add in:

- autoconf rules to detect it,
- makefile rules to link it in
- include file changes
- wrappers to ensure use of pmalloc
- Debian guys add build dependancies
- rpm dependencies get added
- BSD ports dependencies

That is likely rather more code than 1 not terribly large file of C needed
to do it ourselves. And this code is rather worse, as it is in a bunch of
languages, spread all over.

If we were gaining a lot of extra functionality "for free" it would be one
thing. That is true for libssl, and likely zlib, but not here.


From: Dave Page <dpage(at)pgadmin(dot)org>
To: Christopher Browne <cbbrowne(at)gmail(dot)com>
Cc: Josh Berkus <josh(at)agliodbs(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-23 06:25:30
Message-ID: CA+OCxowTt-qaiRVijqWM0i-_oEN7fUXDcny6k1Mknc16WWZhmQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Saturday, July 23, 2011, Christopher Browne <cbbrowne(at)gmail(dot)com> wrote:
> I generally agree, Josh, but I think readline is getting pointed at a bit too much.  Yeah, it's a bad one, but we also include other stuff like zlib that doesn't commonly come up as an issue.
> I'd argue something just a wee bit different...
> By the time we would add in:
> - autoconf rules to detect it,
> - makefile rules to link it in
> - include file changes
> - wrappers to ensure use of pmalloc
> - Debian guys add build dependancies
> -  rpm dependencies get added
> - BSD ports dependencies
> That is likely rather more code than 1 not terribly large file of C needed to do it ourselves.  And this code is rather worse, as it is in a bunch of languages, spread all over.
> If we were gaining a lot of extra functionality "for free" it would be one thing.  That is true for libssl, and likely zlib, but not here.
>

Also consider if the library is widely available on common distros or
not. If not, packagers are going to have to start packaging that
first, in order to build the PostgreSQL packages. This is a *huge*
issue for use if we want to use wxWidgets addon libraries with
pgAdmin.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-23 10:39:10
Message-ID: 4E2AA4CE.8000005@dunslane.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/22/2011 10:51 AM, Joshua D. Drake wrote:
> On 07/21/2011 11:13 AM, Tom Lane wrote:
>> "Joshua D. Drake"<jd(at)commandprompt(dot)com> writes:
>>> So I am looking intently on what it is going to take to get the URI
>>> patch done for psql [1] and was digging around the web and have a URI
>>> parser library. It is under the New BSD license and is strictly RFC
>>> RFC
>>> 3986 [2] compliant .
>>
>> Surely we do not need a whole library to parse URIs.
>>
>> regards, tom lane
>>
>
> Any other comments? I don't want to do a bunch of work just to have it
> tossesd on a technicality.
>
>

1. I think the proposed use is of very marginal value at best, and
certainly not worth importing an external library for.

2. Even if we have the feature, we do not need to parse URIs generally.
A small amount of hand written C code should suffice. If it doesn't that
would itself be an argument against it.

cheers

andrew


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-23 17:35:41
Message-ID: 4E2B066D.7080006@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/22/2011 05:00 PM, Josh Berkus wrote:
> Arguments in favor of coding from scratch:
>
> 1) Does not introduce new dependencies into postgresql-client packages.
> (note how much of a problem Readline has been)

Readline has license issues, this doesn't.

>
> 2) keeps psql as lightweight as possible
>

Agreed on that one.

> 3) We don't need to be able to parse any potential URI, just the ones we
> accept.

True.

>
> Arguments against:
>
> a) waste of time coding a parser
>
> b) eventual feature creep as people ask us for more and more syntax support

Yep.

>
> Overall, though, I'd say that argument (1) against is pretty powerful.
> Count the number of complaints and build bug reports about readline&
> libedit on the lists.
>

That is a good argument.

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-23 17:36:57
Message-ID: 4E2B06B9.9060307@commandprompt.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On 07/23/2011 03:39 AM, Andrew Dunstan wrote:
>
> 1. I think the proposed use is of very marginal value at best, and
> certainly not worth importing an external library for.
>
> 2. Even if we have the feature, we do not need to parse URIs generally.
> A small amount of hand written C code should suffice. If it doesn't that
> would itself be an argument against it.

Well the one area that I think this might be useful in the future is the
pg_hba.conf but that is for a whole other thread.

JD

>
> cheers
>
> andrew
>
>

--
Command Prompt, Inc. - http://www.commandprompt.com/
PostgreSQL Support, Training, Professional Services and Development
The PostgreSQL Conference - http://www.postgresqlconference.org/
@cmdpromptinc - @postgresconf - 509-416-6579


From: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
To: Dave Page <dpage(at)pgadmin(dot)org>
Cc: Christopher Browne <cbbrowne(at)gmail(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-25 02:12:36
Message-ID: 1311559785-sup-9065@alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011:

> Also consider if the library is widely available on common distros or
> not. If not, packagers are going to have to start packaging that
> first, in order to build the PostgreSQL packages. This is a *huge*
> issue for use if we want to use wxWidgets addon libraries with
> pgAdmin.

More likely, they are going to ignore it and pass the --disable-liburi
(whatever) configure parameter and the functionality is going to be
absent most of the time anyway.

--
Álvaro Herrera <alvherre(at)commandprompt(dot)com>
The PostgreSQL Company - Command Prompt, Inc.
PostgreSQL Replication, Consulting, Custom Development, 24x7 support


From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Cc: Dave Page <dpage(at)pgadmin(dot)org>, Christopher Browne <cbbrowne(at)gmail(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-25 10:56:12
Message-ID: CA+TgmoYQ6HEgsdmAo-HWBkk=m4_cZbM-8puPNjSZxSt3Tuqg=A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sun, Jul 24, 2011 at 10:12 PM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011:
>
>> Also consider if the library is widely available on common distros or
>> not. If not, packagers are going to have to start packaging that
>> first, in order to build the PostgreSQL packages. This is a *huge*
>> issue for use if we want to use wxWidgets addon libraries with
>> pgAdmin.
>
> More likely, they are going to ignore it and pass the --disable-liburi
> (whatever) configure parameter and the functionality is going to be
> absent most of the time anyway.

That wouldn't be too good either...

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Dave Page <dpage(at)pgadmin(dot)org>
To: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Cc: Christopher Browne <cbbrowne(at)gmail(dot)com>, Josh Berkus <josh(at)agliodbs(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-07-25 11:03:17
Message-ID: CA+OCxowVKeSnyb6RHSansEjiAsZ__m7i5Kz6G0NdChDDWrFYqA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Jul 25, 2011 at 3:12 AM, Alvaro Herrera
<alvherre(at)commandprompt(dot)com> wrote:
> Excerpts from Dave Page's message of sáb jul 23 02:25:30 -0400 2011:
>
>> Also consider if the library is widely available on common distros or
>> not. If not, packagers are going to have to start packaging that
>> first, in order to build the PostgreSQL packages. This is a *huge*
>> issue for use if we want to use wxWidgets addon libraries with
>> pgAdmin.
>
> More likely, they are going to ignore it and pass the --disable-liburi
> (whatever) configure parameter and the functionality is going to be
> absent most of the time anyway.

Yup.

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitter: @pgsnake

EnterpriseDB UK: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


From: Peter van Hardenberg <pvh(at)pvh(dot)ca>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-08-10 01:00:42
Message-ID: CAAcg=kWRkA+Cv8XBawMhQ2VhSwVbK9VR+OR2Emh8rY0Xh4XY4Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Sat, Jul 23, 2011 at 3:39 AM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:

>
> 1. I think the proposed use is of very marginal value at best, and
> certainly not worth importing an external library for.
>
>
Now that I've seen two people who seem to think that this is not an
important feature I'll wade in and respond to this idea.

I think it's very easy to doubt the value of a definitively recognizable
string that represents a postgres database when you don't have a
heterogenous environment with more than a hundred thousand applications of
all types in it. To make matters worse, as language support on that platform
continues to widen beyond its humble beginnings, there isn't a standard
across those languages for what constitutes a postgres URL. This is the
current situation at Heroku, where we currently run ~150,000 individual
databases on our infrastructure as well as a variety of other databases such
as MySQL, Redis, Mongo, Couch, Riak, Cassandra, &c.

To head off the most obvious criticism, we aren't using connection strings
in our system because there isn't any reasonable way to recognize them. A PG
conninfo string is just a set of key value pairs with no dependably present
signifier. This is why almost every database library from Ruby to Python to
Java takes some form of a URL with a protocol called "postgres" in it in
order to help select which driver to use.

Further, support (and syntax!) for the more esoteric connection parameters
varies from library to library as well as between languages. A good spec by
the project would go a long way in resolving this, and I can at least be
confident that we could get it adopted very quickly by all three of the
Ruby-community Postgres libraries.

In conclusion, this is a serious operational concern for me and my team and
I will be personally dealing with fires caused by this for years to come
regardless of the outcome of this thread.

Best,
-pvh

--
Peter van Hardenberg
Department of Data
Heroku
"Everything was beautiful, and nothing hurt." -- Kurt Vonnegut


From: "David E(dot) Wheeler" <david(at)kineticode(dot)com>
To: Peter van Hardenberg <pvh(at)pvh(dot)ca>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>, "Joshua D(dot) Drake" <jd(at)commandprompt(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Policy on pulling in code from other projects?
Date: 2011-08-10 17:01:43
Message-ID: 4BFA5243-BBFD-4884-8D77-77B40DE95C18@kineticode.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Aug 9, 2011, at 6:00 PM, Peter van Hardenberg wrote:

> In conclusion, this is a serious operational concern for me and my team and I will be personally dealing with fires caused by this for years to come regardless of the outcome of this thread.

Do you have an interest in funding development of the necessary URI-parsing code to get the feature done for 9.2?

Best,

David