Re: Deal with <>s in message IDs

Thunderbird (and presumably other things) will wrap message IDs in <>s, which pgarchives doesn't handle correctly. I'd like to fix this; presumably the correct way to do so is the same way we're handling /'s in message IDs?
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com

On Wed, Oct 29, 2014 at 12:17 AM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
> Thunderbird (and presumably other things) will wrap message IDs in <>s,
> which pgarchives doesn't handle correctly. I'd like to fix this; presumably
> the correct way to do so is the same way we're handling /'s in message IDs?

Um. So what does it actually do :) Do you have an example?

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 10/29/14, 5:01 AM, Magnus Hagander wrote:
> On Wed, Oct 29, 2014 at 12:17 AM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
>> Thunderbird (and presumably other things) will wrap message IDs in <>s,
>> which pgarchives doesn't handle correctly. I'd like to fix this; presumably
>> the correct way to do so is the same way we're handling /'s in message IDs?
>
> Um. So what does it actually do :) Do you have an example?

The backslash stuff? I don't know, I was hoping you did...

If you're referring to the <> issue, my problem is this: If you paste '<message-ID>' into commitfest instead of just 'message-ID' you get no results, because it creates a URL that contains the <>s. Obviously commitfest could be taught to strip <>, but ISTM it's more useful to have pgarchives do it.
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com

On Wed, Oct 29, 2014 at 5:28 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
> On 10/29/14, 5:01 AM, Magnus Hagander wrote:
>>
>> On Wed, Oct 29, 2014 at 12:17 AM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>
>> wrote:
>>>
>>> Thunderbird (and presumably other things) will wrap message IDs in <>s,
>>> which pgarchives doesn't handle correctly. I'd like to fix this;
>>> presumably
>>> the correct way to do so is the same way we're handling /'s in message
>>> IDs?
>>
>>
>> Um. So what does it actually do :) Do you have an example?
>
>
> The backslash stuff? I don't know, I was hoping you did...

There's a backslash thing as well?

> If you're referring to the <> issue, my problem is this: If you paste
> '<message-ID>' into commitfest instead of just 'message-ID' you get no
> results, because it creates a URL that contains the <>s. Obviously
> commitfest could be taught to strip <>, but ISTM it's more useful to have
> pgarchives do it.

Yeah, I still don't understand what you mean. And I'd still need an
example link that shows the wrong thing, both to understand it and to
verify a fix..

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 10/30/2014 12:46 PM, Magnus Hagander wrote:
> On Wed, Oct 29, 2014 at 5:28 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
>> If you're referring to the <> issue, my problem is this: If you paste
>> '<message-ID>' into commitfest instead of just 'message-ID' you get no
>> results, because it creates a URL that contains the <>s. Obviously
>> commitfest could be taught to strip <>, but ISTM it's more useful to have
>> pgarchives do it.
>
> Yeah, I still don't understand what you mean. And I'd still need an
> example link that shows the wrong thing, both to understand it and to
> verify a fix..

http://www.postgresql.org/message-id/%3CCABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com%3E

versus

http://www.postgresql.org/message-id/CABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com
--
Vik

On Thu, Oct 30, 2014 at 12:56 PM, Vik Fearing <vik(at)xocolatl(dot)community> wrote:
> On 10/30/2014 12:46 PM, Magnus Hagander wrote:
>> On Wed, Oct 29, 2014 at 5:28 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
>>> If you're referring to the <> issue, my problem is this: If you paste
>>> '<message-ID>' into commitfest instead of just 'message-ID' you get no
>>> results, because it creates a URL that contains the <>s. Obviously
>>> commitfest could be taught to strip <>, but ISTM it's more useful to have
>>> pgarchives do it.
>>
>> Yeah, I still don't understand what you mean. And I'd still need an
>> example link that shows the wrong thing, both to understand it and to
>> verify a fix..
>
>
> http://www.postgresql.org/message-id/%3CCABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com%3E
>
> versus
>
> http://www.postgresql.org/message-id/CABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com

Ok, I'm sorry, but I'm still confused here. Where do we actually
generate the first link, the one that's wrong? Do we put the wrong
thing in our thread browsing somewhere? If so, I'm still failing to
find it :)

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Magnus Hagander wrote:
> On Thu, Oct 30, 2014 at 12:56 PM, Vik Fearing <vik(at)xocolatl(dot)community> wrote:
> > On 10/30/2014 12:46 PM, Magnus Hagander wrote:
> >> On Wed, Oct 29, 2014 at 5:28 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
> >>> If you're referring to the <> issue, my problem is this: If you paste
> >>> '<message-ID>' into commitfest instead of just 'message-ID' you get no
> >>> results, because it creates a URL that contains the <>s. Obviously
> >>> commitfest could be taught to strip <>, but ISTM it's more useful to have
> >>> pgarchives do it.
> >>
> >> Yeah, I still don't understand what you mean. And I'd still need an
> >> example link that shows the wrong thing, both to understand it and to
> >> verify a fix..
> >
> >
> > http://www.postgresql.org/message-id/%3CCABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com%3E
> >
> > versus
> >
> > http://www.postgresql.org/message-id/CABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com
>
> Ok, I'm sorry, but I'm still confused here. Where do we actually
> generate the first link, the one that's wrong? Do we put the wrong
> thing in our thread browsing somewhere? If so, I'm still failing to
> find it :)

Some people enter message-ids with the < > delimiters in the commitfest app.

I'm not really sure this really belongs in the archives app. We could
just have the commitfest app remove them on input.

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services

On Oct 30, 2014 1:13 PM, "Alvaro Herrera" <alvherre(at)2ndquadrant(dot)com> wrote:
>
> Magnus Hagander wrote:
> > On Thu, Oct 30, 2014 at 12:56 PM, Vik Fearing <vik(at)xocolatl(dot)community>
wrote:
> > > On 10/30/2014 12:46 PM, Magnus Hagander wrote:
> > >> On Wed, Oct 29, 2014 at 5:28 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>
wrote:
> > >>> If you're referring to the <> issue, my problem is this: If you
paste
> > >>> '<message-ID>' into commitfest instead of just 'message-ID' you get
no
> > >>> results, because it creates a URL that contains the <>s. Obviously
> > >>> commitfest could be taught to strip <>, but ISTM it's more useful
to have
> > >>> pgarchives do it.
> > >>
> > >> Yeah, I still don't understand what you mean. And I'd still need an
> > >> example link that shows the wrong thing, both to understand it and to
> > >> verify a fix..
> > >
> > >
> > >
http://www.postgresql.org/message-id/%3CCABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com%3E
> > >
> > > versus
> > >
> > >
http://www.postgresql.org/message-id/CABUevEyy05BHq21BA0CgYcsmvd06ZKXpXPAwr1khZ+RGk+4PUA@mail.gmail.com
> >
> > Ok, I'm sorry, but I'm still confused here. Where do we actually
> > generate the first link, the one that's wrong? Do we put the wrong
> > thing in our thread browsing somewhere? If so, I'm still failing to
> > find it :)
>
> Some people enter message-ids with the < > delimiters in the commitfest
app.
>
> I'm not really sure this really belongs in the archives app. We could
> just have the commitfest app remove them on input.
>

Oh, now I get it. Yeah, based on that it does sound like the wrong place to
fix it. Though if it's easy enough it might be worth doing for convenience.
I'll see if there's some place to stick a regexp that can do it safely.

/Magnus

On 10/30/14, 7:19 AM, Magnus Hagander wrote:
> > I'm not really sure this really belongs in the archives app. We could
> > just have the commitfest app remove them on input.
> >
>
> Oh, now I get it. Yeah, based on that it does sound like the wrong place to fix it. Though if it's easy enough it might be worth doing for convenience. I'll see if there's some place to stick a regexp that can do it safely.

Yes, my original question was if we could basically do the same thing we're doing now to handle /'s. See http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/urls.py;h=d524999dcca8a123b05c294c218d7d93977f4060;hb=HEAD#l28 (line 28) and http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/mailarchives/views.py;h=f6c401a6458ee16050fcd5837a33e848d3cb7644;hb=HEAD#l557 (line 557).
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com

On Thu, Oct 30, 2014 at 7:16 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
> On 10/30/14, 7:19 AM, Magnus Hagander wrote:
>>
>> > I'm not really sure this really belongs in the archives app. We could
>> > just have the commitfest app remove them on input.
>> >
>>
>> Oh, now I get it. Yeah, based on that it does sound like the wrong place
>> to fix it. Though if it's easy enough it might be worth doing for
>> convenience. I'll see if there's some place to stick a regexp that can do it
>> safely.
>
>
> Yes, my original question was if we could basically do the same thing we're
> doing now to handle /'s. See
> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/urls.py;h=d524999dcca8a123b05c294c218d7d93977f4060;hb=HEAD#l28
> (line 28) and
> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/mailarchives/views.py;h=f6c401a6458ee16050fcd5837a33e848d3cb7644;hb=HEAD#l557
> (line 557).

Thats actually something different - that deals with the training
slash sometimes added by django, and is done for cache limitation.
Slashes *in* messageids is independent of that. That's why I
misunderstood you :)

That doesn't mean it's not possible to work around it of course. We
have a grand total of 6 messages in our archives that have < or > in
the messageid, but as long as we only eat one of them, and only at
beginning or end, it should be safe.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 10/31/14, 6:11 AM, Magnus Hagander wrote:
> On Thu, Oct 30, 2014 at 7:16 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
>> On 10/30/14, 7:19 AM, Magnus Hagander wrote:
>>>
>>> > I'm not really sure this really belongs in the archives app. We could
>>> > just have the commitfest app remove them on input.
>>> >
>>>
>>> Oh, now I get it. Yeah, based on that it does sound like the wrong place
>>> to fix it. Though if it's easy enough it might be worth doing for
>>> convenience. I'll see if there's some place to stick a regexp that can do it
>>> safely.
>>
>>
>> Yes, my original question was if we could basically do the same thing we're
>> doing now to handle /'s. See
>> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/urls.py;h=d524999dcca8a123b05c294c218d7d93977f4060;hb=HEAD#l28
>> (line 28) and
>> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/mailarchives/views.py;h=f6c401a6458ee16050fcd5837a33e848d3cb7644;hb=HEAD#l557
>> (line 557).
>
> Thats actually something different - that deals with the training
> slash sometimes added by django, and is done for cache limitation.
> Slashes *in* messageids is independent of that. That's why I
> misunderstood you :)
>
> That doesn't mean it's not possible to work around it of course. We
> have a grand total of 6 messages in our archives that have < or > in
> the messageid, but as long as we only eat one of them, and only at
> beginning or end, it should be safe.

Were you going to take a stab at it or do you want me to? I don't know that I'd be able to test whatever I come up with...
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com

On Fri, Oct 31, 2014 at 11:26 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
> On 10/31/14, 6:11 AM, Magnus Hagander wrote:
>>
>> On Thu, Oct 30, 2014 at 7:16 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>
>> wrote:
>>>
>>> On 10/30/14, 7:19 AM, Magnus Hagander wrote:
>>>>
>>>>
>>>> > I'm not really sure this really belongs in the archives app. We
>>>> could
>>>> > just have the commitfest app remove them on input.
>>>> >
>>>>
>>>> Oh, now I get it. Yeah, based on that it does sound like the wrong place
>>>> to fix it. Though if it's easy enough it might be worth doing for
>>>> convenience. I'll see if there's some place to stick a regexp that can
>>>> do it
>>>> safely.
>>>
>>>
>>>
>>> Yes, my original question was if we could basically do the same thing
>>> we're
>>> doing now to handle /'s. See
>>>
>>> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/urls.py;h=d524999dcca8a123b05c294c218d7d93977f4060;hb=HEAD#l28
>>> (line 28) and
>>>
>>> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/mailarchives/views.py;h=f6c401a6458ee16050fcd5837a33e848d3cb7644;hb=HEAD#l557
>>> (line 557).
>>
>>
>> Thats actually something different - that deals with the training
>> slash sometimes added by django, and is done for cache limitation.
>> Slashes *in* messageids is independent of that. That's why I
>> misunderstood you :)
>>
>> That doesn't mean it's not possible to work around it of course. We
>> have a grand total of 6 messages in our archives that have < or > in
>> the messageid, but as long as we only eat one of them, and only at
>> beginning or end, it should be safe.
>
>
> Were you going to take a stab at it or do you want me to? I don't know that
> I'd be able to test whatever I come up with...

I've put it on my TODO list yeah. But if you want to take a stab at it
go ahead - though I'm not sure it's worth too much without setting up
an env to test it, since that's likely the part that takes the most
time :) But if you want to do that.. :)

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On Sun, Nov 2, 2014 at 11:26 AM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Fri, Oct 31, 2014 at 11:26 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
>> On 10/31/14, 6:11 AM, Magnus Hagander wrote:
>>>
>>> On Thu, Oct 30, 2014 at 7:16 PM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>
>>> wrote:
>>>>
>>>> On 10/30/14, 7:19 AM, Magnus Hagander wrote:
>>>>>
>>>>>
>>>>> > I'm not really sure this really belongs in the archives app. We
>>>>> could
>>>>> > just have the commitfest app remove them on input.
>>>>> >
>>>>>
>>>>> Oh, now I get it. Yeah, based on that it does sound like the wrong place
>>>>> to fix it. Though if it's easy enough it might be worth doing for
>>>>> convenience. I'll see if there's some place to stick a regexp that can
>>>>> do it
>>>>> safely.
>>>>
>>>>
>>>>
>>>> Yes, my original question was if we could basically do the same thing
>>>> we're
>>>> doing now to handle /'s. See
>>>>
>>>> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/urls.py;h=d524999dcca8a123b05c294c218d7d93977f4060;hb=HEAD#l28
>>>> (line 28) and
>>>>
>>>> http://git.postgresql.org/gitweb/?p=pgarchives.git;a=blob;f=django/archives/mailarchives/views.py;h=f6c401a6458ee16050fcd5837a33e848d3cb7644;hb=HEAD#l557
>>>> (line 557).
>>>
>>>
>>> Thats actually something different - that deals with the training
>>> slash sometimes added by django, and is done for cache limitation.
>>> Slashes *in* messageids is independent of that. That's why I
>>> misunderstood you :)
>>>
>>> That doesn't mean it's not possible to work around it of course. We
>>> have a grand total of 6 messages in our archives that have < or > in
>>> the messageid, but as long as we only eat one of them, and only at
>>> beginning or end, it should be safe.
>>
>>
>> Were you going to take a stab at it or do you want me to? I don't know that
>> I'd be able to test whatever I come up with...
>
> I've put it on my TODO list yeah. But if you want to take a stab at it
> go ahead - though I'm not sure it's worth too much without setting up
> an env to test it, since that's likely the part that takes the most
> time :) But if you want to do that.. :)

Turns out that was pretty easy - I think. I've applied a fix that does
this redirect - let's hope it works.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

On 11/4/14, 8:00 AM, Magnus Hagander wrote:
> Turns out that was pretty easy - I think. I've applied a fix that does
> this redirect - let's hope it works.

It works: http://archives.postgresql.org/message-id/%3C544AF3FB.5000604@BlueTreble.com%3E

Thanks for fixing !
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com