Re: New functions in sslinfo module

On Mon, Apr 21, 2014 at 1:48 PM, Воронин Дмитрий <carriingfate92(at)yandex(dot)ru>
wrote:
> Hello,
>
> I make an a patch, which adds 4 functions to sslinfo extension module:
> 1) ssl_get_count_of_extensions() --- get count of X509v3 extensions from
> client certificate;
> 2) ssl_get_extension_names() --- get short names of X509v3 extensions from
> client certificate;
> 3) ssl_get_extension_value(text) --- get value of extension from
certificate
> (argument --- short name of extension);
> 4) ssl_is_critical_extension(text) --- returns true, if extension is
> critical and false, if is not (argument --- short name of extension).
>
> You can view some information of certificate's extensions via those
> functions.
> I want, that my functions will be included in PostgreSQL release.
>
> What do you think about it?
Please avoid creating a new thread each time you send a new version of the
same patch. Previous message was here:
http://www.postgresql.org/message-id/1135491397673046@web9m.yandex.ru
With my previous answer here:
http://www.postgresql.org/message-id/CAB7nPqRVFhnPnQL9ND+K=WA-YF_N1fAirx=s6fawk9F6ANLwBQ@mail.gmail.com

As I already mentioned last time, please register this patch to the
upcoming commit fest beginning in June:
https://commitfest.postgresql.org/action/commitfest_view?id=22
This way, you will be sure that your patch will get at least one fair
review and that progress will be made on the feature you are proposing.

The development cycle of 9.4 is over, but your patch could get into 9.5.
You seem as well to have developed this patch using a tarball of 9.3.4 code
by generating diffs from it, you will need a development environment with
git. Here are some guidelines you can refer to (those are the same URLs as
in my previous email btw...):
https://wiki.postgresql.org/wiki/Submitting_a_Patch
https://wiki.postgresql.org/wiki/Working_with_Git
https://wiki.postgresql.org/wiki/Creating_Clean_Patches
Regards,
--
Michael

On Mon, Apr 21, 2014 at 2:51 PM, Воронин Дмитрий
<carriingfate92(at)yandex(dot)ru>wrote:

> I put patch generated on git diffs to this letter. I make an a thread in
> postgresql commit fest:
> https://commitfest.postgresql.org/action/patch_view?id=1438
>
Thanks!
--
Michael

On 04/21/2014 07:51 AM, Воронин Дмитрий wrote:
> I put patch generated on git diffs to this letter. I make an a thread in
> postgresql commit fest:
> https://commitfest.postgresql.org/action/patch_view?id=1438

Thanks for the patch, it seems like a useful feature.

=== General ===

- Applies cleanly to HEAD and compiles without warnings.

- The test suite passes.

- x509v3 support was added in OpenSSL 0.9.2 and sslinfo already depends
heavily on OpenSSL so no new dependencies.

=== User functionality ===

- If you are a user of the sslinfo extension the new functions should be
useful additions.

- I tested the code without SSL, with certificate but without client
certificate, with client certificates first without extensions and the
with. All of this worked fine except for some usability which could be
improved, see below.

- I cannot see the use for ssl_get_count_of_extensions(). When would
anyone need to know the number of extensions? I think this is an
implementation detail of OpenSSL which we do not need to expose. If any
user wants this feature he can count the extension names.

- Documentation is missing for the new functions.

- I think the names of the new functions should be change. Below are my
suggestions. Other suggestions are welcome.

* ssl_extension_value(text)
* ssl_extension_is_critical()
* ssl_extension_names()
* ssl_extension_count() (If we should keep it.)

- Would it be interesting to have a set returning function which returns
all extensions with both the names and the values? Like the below.

$ SELECT * FROM ssl_extensions();
name | value
------------------+------------------------------------------------------
basicConstraints | CA:FALSE
keyUsage | Digital Signature, Non Repudiation, Key Encipherment
(2 rows)

- I do not think that ssl_get_extension_names() should return a single
row with NULL when the certificate has no extensions or when there is no
certificate at all. Returning zero rows in this case should make it
easier to use.

- Currently ssl_is_critical_extension() and ssl_get_extension_value()
throw an error when the requested extension is not in the certificate.

I am not sure if I like this behavior. I think I would prefer if the
code always throws an error when name lookup fails, and never when it is
successful. For a valid extension name I think NULL should be returned
when it does not exist.

=== Code review: main ===

- Why are X509_NAME_field_to_text(), X509_NAME_to_text(),
ASN1_STRING_to_text() and get_extension() not static? None of these are
a symbol which should be exported.

- I have not worked with extension myself, but since your change adds
functions to the extension I think you need to create a version 1.1
instead of modifying 1.0 in place. If so you also need to write an
upgrade script from 1.0 to 1.1. See dblink--1.0--1.1.sql for an example.

- Please break out the comment fix for ssl_cipher() into a separate patch.

- Why do you use pg_do_encoding_conversion() over pg_any_to_server()?
pg_any_to_server() is implemented using pg_do_encoding_conversion().

- I think you should use OBJ_txt2nid() rather than OBJ_sn2nid() +
OBJ_ln2nid(). You should probably also use OBJ_txt2obj() since
X509_get_ext_by_NID() will call OBJ_nid2obj() anyway.

- You should free the extension_name string. I do not think it is ok to
leak it to the end of the query.

- I think you need to convert the extension values and names to the
server encoding. I just wonder if we need to support data which is
incorrectly encoded.

=== Code review: style issues ===

- Trailing whitespace in sslinfo--1.0.sql and sslinfo.c.q

- sslinfo--1.0.sql does not end in a newline.

- I do not think the PostgreSQL project adds authors in the top comment
of files in cases like this. Authors get credit in the commit messages.

- I think you can remove the prototypes of all the ssl_* functions.

- Adding the have in "Indicates whether current client have provided a
certificate" is not necessary. The previous wording looks correct to my
non-native speaker eyes.

- Too much white space in variable declarations in get_extension().

- Extra space before -1 in "X509_get_ext_by_NID(certificate,
extension_nid, -1);"

- Please do not initialize variables unless necessary. Compilers are
pretty good at warning about uninitialized usage. For example both
locate and extension_nid do not need to be initialized.

- Remove empty line directly before ssl_get_extension_value().

- Try to follow variable naming conventions from other functions (e.g.
use nid rather than extension_nid, membuf rather than bio, sp rather
than value).

- I am pretty sure the variable you call locate should be called
location (or loc for short).

- There should not be any spaces around "->".

- The declaration of *extension in ssl_get_extension_value is not
aligned properly.

- Remove white space in variable declaration in
ssl_get_count_of_extensions().

- Incorrectly alignment of variable declarations in
ssl_get_extension_names().

- All the "Returns X datum" comments look redundant to me, but this is a
matter of preference.

- The star when declaring result in ssl_get_extension_names() should be
put on the other side of the white space.

--
Andreas Karlsson

<p>24.06.2014, 00:07, "Andreas Karlsson" &lt;<a href="mailto:andreas(at)proxel(dot)se">andreas(at)proxel(dot)se</a>&gt;:</p><blockquote> On 04/21/2014 07:51 AM, Воронин Дмитрий wrote:<br /><blockquote>  I put patch generated on git diffs to this letter. I make an a thread in<br />  postgresql commit fest:<br />  <a href="https://commitfest.postgresql.org/action/patch_view?id=1438">https://commitfest.postgresql.org/action/patch_view?id=1438</a></blockquote> Thanks for the patch, it seems like a useful feature.<br /><br /> === General ===<br /><br /> - Applies cleanly to HEAD and compiles without warnings.<br /><br /> - The test suite passes.<br /><br /> - x509v3 support was added in OpenSSL 0.9.2 and sslinfo already depends<br /> heavily on OpenSSL so no new dependencies.<br /><br /> === User functionality ===<br /><br /> - If you are a user of the sslinfo extension the new functions should be<br /> useful additions.<br /><br /> - I tested the code without SSL, with certificate but without client<br /> certificate, with client certificates first without extensions and the<br /> with. All of this worked fine except for some usability which could be<br /> improved, see below.<br /><br /> - I cannot see the use for ssl_get_count_of_extensions(). When would<br /> anyone need to know the number of extensions? I think this is an<br /> implementation detail of OpenSSL which we do not need to expose. If any<br /> user wants this feature he can count the extension names.<br /><br /> - Documentation is missing for the new functions.<br /><br /> - I think the names of the new functions should be change. Below are my<br /> suggestions. Other suggestions are welcome.<br /><br /> * ssl_extension_value(text)<br /> * ssl_extension_is_critical()<br /> * ssl_extension_names()<br /> * ssl_extension_count() (If we should keep it.)<br /><br /> - Would it be interesting to have a set returning function which returns<br /> all extensions with both the names and the values? Like the below.<br /><br /> $ SELECT * FROM ssl_extensions();<br />         name       |                        value<br /> ------------------+------------------------------------------------------<br />   basicConstraints | CA:FALSE<br />   keyUsage         | Digital Signature, Non Repudiation, Key Encipherment<br /> (2 rows)<br /><br /> - I do not think that ssl_get_extension_names() should return a single<br /> row with NULL when the certificate has no extensions or when there is no<br /> certificate at all. Returning zero rows in this case should make it<br /> easier to use.<br /><br /> - Currently ssl_is_critical_extension() and ssl_get_extension_value()<br /> throw an error when the requested extension is not in the certificate.<br /><br /> I am not sure if I like this behavior. I think I would prefer if the<br /> code always throws an error when name lookup fails, and never when it is<br /> successful. For a valid extension name I think NULL should be returned<br /> when it does not exist.<br /><br /> === Code review: main ===<br /><br /> - Why are X509_NAME_field_to_text(), X509_NAME_to_text(),<br /> ASN1_STRING_to_text() and get_extension() not static? None of these are<br /> a symbol which should be exported.<br /><br /> - I have not worked with extension myself, but since your change adds<br /> functions to the extension I think you need to create a version 1.1<br /> instead of modifying 1.0 in place. If so you also need to write an<br /> upgrade script from 1.0 to 1.1. See dblink--1.0--1.1.sql for an example.<br /><br /> - Please break out the comment fix for ssl_cipher() into a separate patch.<br /><br /> - Why do you use pg_do_encoding_conversion() over pg_any_to_server()?<br /> pg_any_to_server() is implemented using pg_do_encoding_conversion().<br /><br /> - I think you should use OBJ_txt2nid() rather than OBJ_sn2nid() +<br /> OBJ_ln2nid(). You should probably also use OBJ_txt2obj() since<br /> X509_get_ext_by_NID() will call OBJ_nid2obj() anyway.<br /><br /> - You should free the extension_name string. I do not think it is ok to<br /> leak it to the end of the query.<br /><br /> - I think you need to convert the extension values and names to the<br /> server encoding. I just wonder if we need to support data which is<br /> incorrectly encoded.<br /><br /> === Code review: style issues ===<br /><br /> - Trailing whitespace in sslinfo--1.0.sql and sslinfo.c.q<br /><br /> - sslinfo--1.0.sql does not end in a newline.<br /><br /> - I do not think the PostgreSQL project adds authors in the top comment<br /> of files in cases like this. Authors get credit in the commit messages.<br /><br /> - I think you can remove the prototypes of all the ssl_* functions.<br /><br /> - Adding the have in "Indicates whether current client have provided a<br /> certificate" is not necessary. The previous wording looks correct to my<br /> non-native speaker eyes.<br /><br /> - Too much white space in variable declarations in get_extension().<br /><br /> - Extra space before -1 in "X509_get_ext_by_NID(certificate,<br /> extension_nid,  -1);"<br /><br /> - Please do not initialize variables unless necessary. Compilers are<br /> pretty good at warning about uninitialized usage. For example both<br /> locate and extension_nid do not need to be initialized.<br /><br /> - Remove empty line directly before ssl_get_extension_value().<br /><br /> - Try to follow variable naming conventions from other functions (e.g.<br /> use nid rather than extension_nid, membuf rather than bio, sp rather<br /> than value).<br /><br /> - I am pretty sure the variable you call locate should be called<br /> location (or loc for short).<br /><br /> - There should not be any spaces around "-&gt;".<br /><br /> - The declaration of *extension in ssl_get_extension_value is not<br /> aligned properly.<br /><br /> - Remove white space in variable declaration in<br /> ssl_get_count_of_extensions().<br /><br /> - Incorrectly alignment of variable declarations in<br /> ssl_get_extension_names().<br /><br /> - All the "Returns X datum" comments look redundant to me, but this is a<br /> matter of preference.<br /><br /> - The star when declaring result in ssl_get_extension_names() should be<br /> put on the other side of the white space.<br /><br /> --<br /> Andreas Karlsson</blockquote><p>Hello, Andreas!<br /><br /><span lang="en"><span>I apologize, that I am writing this message today. Thank you for testing my patch!<br /><br />About user functionality</span></span>. <br />I rename my functions, your suggestions are seemed great. When I wrote those functions, I created names like others in sslinfo extension. OK, I will rename it.<br /><br />About ssl_get_extension_count(). I will delete this function. <br />I will modify functions ssl_extensions(), that it returns a set (key, value). Could you get me an example of code those function? <br /><br /> &gt;&gt;&gt; - Why are X509_NAME_field_to_text(), X509_NAME_to_text(), ASN1_STRING_to_text() and get_extension() not static? None of these are a symbol which should be exported.<br />&gt;&gt;&gt; Why do you use pg_do_encoding_conversion() over pg_any_to_server()? pg_any_to_server() is implemented using pg_do_encoding_conversion().<br /><br />I don't write a code of those functions and I can't answer on your question.</p><p>I will fix your notes and create a new patch version. Thank you.<br />--</p><p>Best regards, Dmitry Voronin</p>

Oh, how can I write a documentation for my functions?

02.07.2014, 16:17, "Воронин Дмитрий" <carriingfate92(at)yandex(dot)ru>:
> 24.06.2014, 00:07, "Andreas Karlsson" <andreas(at)proxel(dot)se>:
>>  On 04/21/2014 07:51 AM, Воронин Дмитрий wrote:
>>>   I put patch generated on git diffs to this letter. I make an a thread in
>>>   postgresql commit fest:
>>>   https://commitfest.postgresql.org/action/patch_view?id=1438
>>  Thanks for the patch, it seems like a useful feature.
>>
>>  === General ===
>>
>>  - Applies cleanly to HEAD and compiles without warnings.
>>
>>  - The test suite passes.
>>
>>  - x509v3 support was added in OpenSSL 0.9.2 and sslinfo already depends
>>  heavily on OpenSSL so no new dependencies.
>>
>>  === User functionality ===
>>
>>  - If you are a user of the sslinfo extension the new functions should be
>>  useful additions.
>>
>>  - I tested the code without SSL, with certificate but without client
>>  certificate, with client certificates first without extensions and the
>>  with. All of this worked fine except for some usability which could be
>>  improved, see below.
>>
>>  - I cannot see the use for ssl_get_count_of_extensions(). When would
>>  anyone need to know the number of extensions? I think this is an
>>  implementation detail of OpenSSL which we do not need to expose. If any
>>  user wants this feature he can count the extension names.
>>
>>  - Documentation is missing for the new functions.
>>
>>  - I think the names of the new functions should be change. Below are my
>>  suggestions. Other suggestions are welcome.
>>
>>  * ssl_extension_value(text)
>>  * ssl_extension_is_critical()
>>  * ssl_extension_names()
>>  * ssl_extension_count() (If we should keep it.)
>>
>>  - Would it be interesting to have a set returning function which returns
>>  all extensions with both the names and the values? Like the below.
>>
>>  $ SELECT * FROM ssl_extensions();
>>          name       |                        value
>>  ------------------+------------------------------------------------------
>>    basicConstraints | CA:FALSE
>>    keyUsage         | Digital Signature, Non Repudiation, Key Encipherment
>>  (2 rows)
>>
>>  - I do not think that ssl_get_extension_names() should return a single
>>  row with NULL when the certificate has no extensions or when there is no
>>  certificate at all. Returning zero rows in this case should make it
>>  easier to use.
>>
>>  - Currently ssl_is_critical_extension() and ssl_get_extension_value()
>>  throw an error when the requested extension is not in the certificate.
>>
>>  I am not sure if I like this behavior. I think I would prefer if the
>>  code always throws an error when name lookup fails, and never when it is
>>  successful. For a valid extension name I think NULL should be returned
>>  when it does not exist.
>>
>>  === Code review: main ===
>>
>>  - Why are X509_NAME_field_to_text(), X509_NAME_to_text(),
>>  ASN1_STRING_to_text() and get_extension() not static? None of these are
>>  a symbol which should be exported.
>>
>>  - I have not worked with extension myself, but since your change adds
>>  functions to the extension I think you need to create a version 1.1
>>  instead of modifying 1.0 in place. If so you also need to write an
>>  upgrade script from 1.0 to 1.1. See dblink--1.0--1.1.sql for an example.
>>
>>  - Please break out the comment fix for ssl_cipher() into a separate patch.
>>
>>  - Why do you use pg_do_encoding_conversion() over pg_any_to_server()?
>>  pg_any_to_server() is implemented using pg_do_encoding_conversion().
>>
>>  - I think you should use OBJ_txt2nid() rather than OBJ_sn2nid() +
>>  OBJ_ln2nid(). You should probably also use OBJ_txt2obj() since
>>  X509_get_ext_by_NID() will call OBJ_nid2obj() anyway.
>>
>>  - You should free the extension_name string. I do not think it is ok to
>>  leak it to the end of the query.
>>
>>  - I think you need to convert the extension values and names to the
>>  server encoding. I just wonder if we need to support data which is
>>  incorrectly encoded.
>>
>>  === Code review: style issues ===
>>
>>  - Trailing whitespace in sslinfo--1.0.sql and sslinfo.c.q
>>
>>  - sslinfo--1.0.sql does not end in a newline.
>>
>>  - I do not think the PostgreSQL project adds authors in the top comment
>>  of files in cases like this. Authors get credit in the commit messages.
>>
>>  - I think you can remove the prototypes of all the ssl_* functions.
>>
>>  - Adding the have in "Indicates whether current client have provided a
>>  certificate" is not necessary. The previous wording looks correct to my
>>  non-native speaker eyes.
>>
>>  - Too much white space in variable declarations in get_extension().
>>
>>  - Extra space before -1 in "X509_get_ext_by_NID(certificate,
>>  extension_nid,  -1);"
>>
>>  - Please do not initialize variables unless necessary. Compilers are
>>  pretty good at warning about uninitialized usage. For example both
>>  locate and extension_nid do not need to be initialized.
>>
>>  - Remove empty line directly before ssl_get_extension_value().
>>
>>  - Try to follow variable naming conventions from other functions (e.g.
>>  use nid rather than extension_nid, membuf rather than bio, sp rather
>>  than value).
>>
>>  - I am pretty sure the variable you call locate should be called
>>  location (or loc for short).
>>
>>  - There should not be any spaces around "->".
>>
>>  - The declaration of *extension in ssl_get_extension_value is not
>>  aligned properly.
>>
>>  - Remove white space in variable declaration in
>>  ssl_get_count_of_extensions().
>>
>>  - Incorrectly alignment of variable declarations in
>>  ssl_get_extension_names().
>>
>>  - All the "Returns X datum" comments look redundant to me, but this is a
>>  matter of preference.
>>
>>  - The star when declaring result in ssl_get_extension_names() should be
>>  put on the other side of the white space.
>>
>>  --
>>  Andreas Karlsson
>
> Hello, Andreas!
>
> I apologize, that I am writing this message today. Thank you for testing my patch!
>
> About user functionality.
> I rename my functions, your suggestions are seemed great. When I wrote those functions, I created names like others in sslinfo extension. OK, I will rename it.
>
> About ssl_get_extension_count(). I will delete this function.
> I will modify functions ssl_extensions(), that it returns a set (key, value). Could you get me an example of code those function?
>
>  >>> - Why are X509_NAME_field_to_text(), X509_NAME_to_text(), ASN1_STRING_to_text() and get_extension() not static? None of these are a symbol which should be exported.
>>>> Why do you use pg_do_encoding_conversion() over pg_any_to_server()? pg_any_to_server() is implemented using pg_do_encoding_conversion().
>
> I don't write a code of those functions and I can't answer on your question.
>
> I will fix your notes and create a new patch version. Thank you.
> --
>
> Best regards, Dmitry Voronin

--
С уважением, Дмитрий Воронин

On Wed, Jul 2, 2014 at 9:19 PM, Воронин Дмитрий
<carriingfate92(at)yandex(dot)ru> wrote:
>
> Oh, how can I write a documentation for my functions?

You will need to edit the sgml documentation and to include the diffs
in your patch. Hence in your case simply list the new functions and a
description of what they do in doc/src/sgml/sslinfo.sgml.
--
Michael

On 07/02/2014 02:17 PM, Воронин Дмитрий wrote:
> I apologize, that I am writing this message today. Thank you for testing
> my patch!

You are welcome!

> I will modify functions ssl_extensions(), that it returns a set (key,
> value). Could you get me an example of code those function?

You can look at hstore_each() in hstore_op.c.

> >>> - Why are X509_NAME_field_to_text(), X509_NAME_to_text(),
> ASN1_STRING_to_text() and get_extension() not static? None of these are
> a symbol which should be exported.
> >>> Why do you use pg_do_encoding_conversion() over pg_any_to_server()?
> pg_any_to_server() is implemented using pg_do_encoding_conversion().
>
> I don't write a code of those functions and I can't answer on your question.

Hm, I thought I saw them changed from static to not in the diff after
applying your patch. Maybe I just misread the patch.

--
Andreas Karlsson

On Fri, Jul 18, 2014 at 3:12 PM, Воронин Дмитрий
<carriingfate92(at)yandex(dot)ru> wrote:
> I make a new version of patch. I corrected your notes for my previous
> version of patch. Could you test it? Thank you.

I just had a look at the new version of this patch, and this is
lacking a couple of things.

First, this patch has been developed using a tarball of 9.3.4. sslinfo
is not a module changing frequently so you are lucky that this is not
generating diffs with HEAD but you should try to get yourself familiar
with git and submit patches that are based on HEAD to facilitate their
review and integration. You can have a look here to begin with:
https://wiki.postgresql.org/wiki/Working_with_GIT

Then, here are more comments about the patch:
- Update sslinfo to 1.1. Here are all the details about how to do it:
http://www.postgresql.org/docs/devel/static/extend-extensions.html#AEN57147
Well, basically this is only creating sslinfo--1.0--1.1.sql to be able
to do an upgrade, copy sslinfo--1.0.sql to sslinfo--1.1.sql with the
new objects defined and dumping sslinfo.control.
- In your previous patches I saw that you defined the new functions in
sslinfo--1.0.sql but in the new version of your patch it is not the
case.
- Please remove all the diff noise in sslinfo.sgml, like this one:
*** 157,167 ****
<varlistentry>
<term>
<function>ssl_client_dn_field(fieldname text) returns text</function>
- <indexterm>
- <primary>ssl_client_dn_field</primary>
- </indexterm>
</term>
<listitem>
<para>
--- 157,167 ----
</varlistentry>

<varlistentry>
+ <indexterm>
+ <primary>ssl_client_dn_field</primary>
+ </indexterm>
Your patch should only include documentation for the new functions.
- Please remove whitespaces, there are quite a lot of them.
- in 9.5, PG_FUNCTION_INFO_V1 does the function declaration for you,
making following block useless:
[...]
+Datum ssl_is_used(PG_FUNCTION_ARGS);
+Datum ssl_version(PG_FUNCTION_ARGS);
+Datum ssl_cipher(PG_FUNCTION_ARGS);
[...]
- Please avoid if blocks of this type, this is not consistent with the
project style:
if (SRF_IS_FIRSTCALL()) {
[...]
}
One newline for each bracket. Here is the manual page referencing code
conventions:
http://www.postgresql.org/docs/current/static/source.html

Most of those comments have been already mentioned by Andreas in one
of his emails upthread but you have obviously not solved the issues he
has reported.

This looks like a useful feature, but at this point of the commit fest
and considering the number of structural changes still needed I will
mark this patch as "Returned with feedback". Feel free to resubmit to
the next commit fest in August with an updated patch of course!

Regards,
--
Michael