Re: Documentation and explanatory diagrams

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello

I am wondering if we have ever evaluated the use of explanatory diagrams
in the PostgreSQL documentation.

I have always thought that a good diagram is worth a thousand words. I
think the PostgreSQL manual would improve its quality if we get some
good diagrams on the right places.

They will help novice and advanced users who are not involved in the
development of PostgreSQL, to get a quick overview of some central
components in a PostgreSQL system when they read the documentation.

Some of the possible diagrams that come to my mind in none particular
order are:

- PITR
- Streaming Replication
- Hot Standby
- Write-Ahead Log
- System Catalogs
- Architectural Fundamentals overview
- PostgreSQL server overview
- Disk layout
- Database Roles and Privileges
- Database Physical Storage
- Security and authentication

What do you think about the idea? I could start working on an initial
version of some of these diagrams if you are interested in the idea.

regards,
- --
Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAku55ocACgkQBhuKQurGihRbJQCfVA9SYVWkyfudFg7XPDlLXYeV
tKAAmgMjg1tfU63sf0RTMRoFTtfsn/Pj
=2+9o
-----END PGP SIGNATURE-----

That would be very nice to have !

Oleg
On Mon, 5 Apr 2010, Rafael Martinez wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello
>
> I am wondering if we have ever evaluated the use of explanatory diagrams
> in the PostgreSQL documentation.
>
> I have always thought that a good diagram is worth a thousand words. I
> think the PostgreSQL manual would improve its quality if we get some
> good diagrams on the right places.
>
> They will help novice and advanced users who are not involved in the
> development of PostgreSQL, to get a quick overview of some central
> components in a PostgreSQL system when they read the documentation.
>
> Some of the possible diagrams that come to my mind in none particular
> order are:
>
> - PITR
> - Streaming Replication
> - Hot Standby
> - Write-Ahead Log
> - System Catalogs
> - Architectural Fundamentals overview
> - PostgreSQL server overview
> - Disk layout
> - Database Roles and Privileges
> - Database Physical Storage
> - Security and authentication
>
> What do you think about the idea? I could start working on an initial
> version of some of these diagrams if you are interested in the idea.
>
> regards,
> - --
> Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
> Center for Information Technology Services
> University of Oslo, Norway
>
> PGP Public Key: http://folk.uio.no/rafael/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAku55ocACgkQBhuKQurGihRbJQCfVA9SYVWkyfudFg7XPDlLXYeV
> tKAAmgMjg1tfU63sf0RTMRoFTtfsn/Pj
> =2+9o
> -----END PGP SIGNATURE-----
>
>

Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: oleg(at)sai(dot)msu(dot)su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

Rafael Martinez <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no> writes:
> I am wondering if we have ever evaluated the use of explanatory diagrams
> in the PostgreSQL documentation.

There actually used to be some. We've moved away from them, I believe,
because of the difficulty of maintenance; which stems mainly from the
lack of any universally-available tools. Unless you can tell me what
you're going to draw the diagram with and how people on Unix, Windows,
or Mac could modify it later, I'm going to not be very interested ...

regards, tom lane

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Tom Lane wrote:
> Rafael Martinez <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no> writes:
>> I am wondering if we have ever evaluated the use of explanatory diagrams
>> in the PostgreSQL documentation.
>
> There actually used to be some. We've moved away from them, I believe,
> because of the difficulty of maintenance; which stems mainly from the
> lack of any universally-available tools. Unless you can tell me what
> you're going to draw the diagram with and how people on Unix, Windows,
> or Mac could modify it later, I'm going to not be very interested ...
>
> regards, tom lane

Well, I was thinking about DIA [1]. It runs on Unix, Windows and Mac.
It loads and saves diagrams to a custom XML format and it can export
diagrams to a number of formats, including EPS, SVG, XFIG, WMF and PNG.

I have to admit that I have only use this program on Linux.

[1] http://live.gnome.org/Dia/

regards
- --
Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAku5+QcACgkQBhuKQurGihQhigCeIL+aEVrgL9yMtLsttxiYzHAU
uQ0An1N6VufuYzPBiAZc16uKzsU4HxV0
=NySa
-----END PGP SIGNATURE-----

On mån, 2010-04-05 at 16:52 +0200, Rafael Martinez wrote:
> Tom Lane wrote:
> > Rafael Martinez <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no> writes:
> >> I am wondering if we have ever evaluated the use of explanatory diagrams
> >> in the PostgreSQL documentation.
> >
> > There actually used to be some. We've moved away from them, I believe,
> > because of the difficulty of maintenance; which stems mainly from the
> > lack of any universally-available tools. Unless you can tell me what
> > you're going to draw the diagram with and how people on Unix, Windows,
> > or Mac could modify it later, I'm going to not be very interested ...

> Well, I was thinking about DIA [1]. It runs on Unix, Windows and Mac.
> It loads and saves diagrams to a custom XML format and it can export
> diagrams to a number of formats, including EPS, SVG, XFIG, WMF and PNG.

Preferably, any tool that we would use would save a reasonably plain
text source file that we could check into VCS and would provide a tool
for automatically converting to a variety of target formats. For
example, graphviz could work well. (Not saying that graphviz is the
right tool for producing the kinds of diagrams that you want, but it
provides the right toolchain interfaces.) Dia might, but it would be
interesting to see how human-readable that XML format really is. (This
is necessary for change tracking. I would like to use a diff tool to
see what happened to a diagram over various revisions. If opening the
file in the editing tool, changing one bit, and saving it produces a
completely different machine-readable-only XML mush, then it's no good.)

Rafael Martinez wrote:
> Well, I was thinking about DIA [1]. It runs on Unix, Windows and Mac.
> It loads and saves diagrams to a custom XML format and it can export
> diagrams to a number of formats, including EPS, SVG, XFIG, WMF and PNG.
>

I keep a copy of Windows around to run exactly four programs: Word,
Excel, Quicken, and Visio, the program Dia was partially inspired by.
Dia has such a terrible UI that I will boot an entire VM just to use
Visio instead.

In any case, the sort of diagrams you're asking about are created
regularly, they just show up in presentations created for conferences
and the like rather than appearing in the official documentation.
http://wiki.postgresql.org/wiki/PostgreSQL_Related_Slides_and_Presentations
is a good starting point. The short list at
http://www.postgresql.org/developer/coding has two of the better ones
around, Tom's "A Tour of PostgreSQL Internals" and Bruce's "Database
Internals Presentation", which has now expanded to be "PostgreSQL
Internals Through Pictures" at
http://momjian.us/main/writings/pgsql/internalpics.pdf

Given that's how the work is being done by community members right now,
I think you'd make better progress asking how to index the material in
those slides better and provide a hotlist of good slides on the Wiki. I
consider fighting the battle to get the diagrams directly into the docs
the way it would need to be done to be both impractical and a waste of
resources, given many of these diagrams do already exist (and new ones
are constantly generated) if you follow the presentation scene.

--
Greg Smith 2ndQuadrant US Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com www.2ndQuadrant.us

Peter Eisentraut wrote:

> Preferably, any tool that we would use would save a reasonably plain
> text source file that we could check into VCS and would provide a tool
> for automatically converting to a variety of target formats. For
> example, graphviz could work well. (Not saying that graphviz is the
> right tool for producing the kinds of diagrams that you want, but it
> provides the right toolchain interfaces.) Dia might, but it would be
> interesting to see how human-readable that XML format really is. (This
> is necessary for change tracking. I would like to use a diff tool to
> see what happened to a diagram over various revisions. If opening the
> file in the editing tool, changing one bit, and saving it produces a
> completely different machine-readable-only XML mush, then it's no good.)

It's ugly, but it seems to work reasonably. You have to explicitely
tell it not to compress the .dia files.

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Peter Eisentraut wrote:
> On mån, 2010-04-05 at 16:52 +0200, Rafael Martinez wrote:

>> Well, I was thinking about DIA [1]. It runs on Unix, Windows and Mac.
>> It loads and saves diagrams to a custom XML format and it can export
>> diagrams to a number of formats, including EPS, SVG, XFIG, WMF and PNG.
>
> Preferably, any tool that we would use would save a reasonably plain
> text source file that we could check into VCS and would provide a tool
> for automatically converting to a variety of target formats. For
> example, graphviz could work well. (Not saying that graphviz is the
> right tool for producing the kinds of diagrams that you want, but it
> provides the right toolchain interfaces.) Dia might, but it would be
> interesting to see how human-readable that XML format really is. (This
> is necessary for change tracking. I would like to use a diff tool to
> see what happened to a diagram over various revisions. If opening the
> file in the editing tool, changing one bit, and saving it produces a
> completely different machine-readable-only XML mush, then it's no good.)
>

Hello

I mentioned Dia because is the one I have experience with and I know it
works with different operative systems. It is not perfect but in my
opinion it is good enough and it does a decent job.

With Dia:

* We have a program that works on Unix, Windows and Mac.
* We can create/update a diagram in an easy way without having to learn
a new complex/powerful system.
* We can save the diagrams in a plain text format that can be check into
CVS.
* We can convert to a variety of target formats with commandline tools.
I have had good experience converting to png and eps.

I have made a quick check of some of your concerns:

* Changing one bit a diagram and saving it, does not produce a
completely different machine-readable-only XML mush. It changes only
the portion of the code affected by your changes. A diff will show
only the lines affected by your changes.

* It looks like the objects in a diagram are saved in the same order
they were created.

* Modified objects keep their position in the saved file.

* The XML format used is human-readable and not so difficult to
understand but it has so much information that it would not be a good
idea to edit it manually. Straightforward changes will not be
difficult to do manually, complex changes have to be done via the
program.

regards,
- --
Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAku6axAACgkQBhuKQurGihR2YQCeJCE8ZNR3TgP5rkcBYyeZ/9nl
ZfEAn1/HQlw/1IrhY1mGNLRGD9bHzdns
=fXkD
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greg Smith wrote:
[.......]
>
> Given that's how the work is being done by community members right now,
> I think you'd make better progress asking how to index the material in
> those slides better and provide a hotlist of good slides on the Wiki. I
> consider fighting the battle to get the diagrams directly into the docs
> the way it would need to be done to be both impractical and a waste of
> resources, given many of these diagrams do already exist (and new ones
> are constantly generated) if you follow the presentation scene.
>

I still think that the right place to have explanatory diagrams is in
the official documentation and not a wiki or external document with
pointers to other documents/presentations.

They would make some important parts of the documentation much more
easier to understand. This is especially important for new users of
PostgreSQL and users who are trying to get an overview of PostgreSQL in
a evaluation process.

I know that most contributors have more than enough to do with the new
release, and I don't want to create more work at this time. So maybe it
will be better to fight this battle after 9.0 is released. :-)

regards
- --
Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAku6a1gACgkQBhuKQurGihTJjgCgjaCCkghoP8vr1auJJKwczM/4
EIMAnAsRhFfgj9pIG3hR6fsG0Bw8LjWa
=lkA6
-----END PGP SIGNATURE-----

Rafael Martinez wrote:
> > Given that's how the work is being done by community members right now,
> > I think you'd make better progress asking how to index the material in
> > those slides better and provide a hotlist of good slides on the Wiki. I
> > consider fighting the battle to get the diagrams directly into the docs
> > the way it would need to be done to be both impractical and a waste of
> > resources, given many of these diagrams do already exist (and new ones
> > are constantly generated) if you follow the presentation scene.
> >
>
> I still think that the right place to have explanatory diagrams is in
> the official documentation and not a wiki or external document with
> pointers to other documents/presentations.
>
> They would make some important parts of the documentation much more
> easier to understand. This is especially important for new users of
> PostgreSQL and users who are trying to get an overview of PostgreSQL in
> a evaluation process.
>
> I know that most contributors have more than enough to do with the new
> release, and I don't want to create more work at this time. So maybe it
> will be better to fight this battle after 9.0 is released. :-)

OK, DIA sounds like it meets the requirements, and it would be good to
get some images into our documentation. I wonder if it could import
some of my xfig diagrams; if so, I could probably keep the images you
took from me updated in CVS.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bruce Momjian wrote:
[...]
>
> OK, DIA sounds like it meets the requirements, and it would be good to
> get some images into our documentation. I wonder if it could import
> some of my xfig diagrams; if so, I could probably keep the images you
> took from me updated in CVS.
>

Where can I get these xfig diagrams? I can play with them and see how it
works to import/convert them.

- --
Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkvRb8gACgkQBhuKQurGihRxmwCgo6fpECJOSCS93DTdoMJJAC8e
0JQAoIseSJKVzW+ktui3Ma++1suLeTwR
=tNyp
-----END PGP SIGNATURE-----

Rafael Martinez wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Bruce Momjian wrote:
> [...]
> >
> > OK, DIA sounds like it meets the requirements, and it would be good to
> > get some images into our documentation. I wonder if it could import
> > some of my xfig diagrams; if so, I could probably keep the images you
> > took from me updated in CVS.
> >
>
> Where can I get these xfig diagrams? I can play with them and see how it
> works to import/convert them.

I would have to email them to you; here is one for testing:

http://momjian.us/expire/mvcc.fig

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Bruce Momjian wrote:
> Rafael Martinez wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Bruce Momjian wrote:
>> [...]
>>> OK, DIA sounds like it meets the requirements, and it would be good to
>>> get some images into our documentation. I wonder if it could import
>>> some of my xfig diagrams; if so, I could probably keep the images you
>>> took from me updated in CVS.
>>>
>> Where can I get these xfig diagrams? I can play with them and see how it
>> works to import/convert them.
>
> I would have to email them to you; here is one for testing:
>
> http://momjian.us/expire/mvcc.fig
>

I have been playing with this file and DIA-0.96.1 on Linux and this is
the result:

* Importing/converting from xfig to dia format worked with almost no
problems.

* DIA complained that it could not find 'Latex font 16' and that it
would use 'sans' instead. An explanation about this could be what they
explain in this url: http://projects.gnome.org/dia/fonts.html

* The diagram was imported automatically with the right layout but
without the red color in the boxes. This was fixed without problems
manually.

After converting this file to DIA I have generated several other formats
from it. Here are they: http://folk.uio.no/rafael/pgsql_diagrams/

* png, jpeg and eps are generated with the right layout, color and font
size.

* xfig is generated with the right layout and color but the font size is
too big

* svg has big problems with the font size under google chrome and is
'almost' right under firefox.

- --
Rafael Martinez, <r(dot)m(dot)guerrero(at)usit(dot)uio(dot)no>
Center for Information Technology Services
University of Oslo, Norway

PGP Public Key: http://folk.uio.no/rafael/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkvVRmUACgkQBhuKQurGihREXwCdHzYcLsSw4CKnk/TT/Qh4ruAH
5MkAn1DU0UKs04YaFJaYznTxAursM64i
=0ed1
-----END PGP SIGNATURE-----

Rafael Martinez wrote:
> I have been playing with this file and DIA-0.96.1 on Linux and this is
> the result:
>
> * Importing/converting from xfig to dia format worked with almost no
> problems.
>
> * DIA complained that it could not find 'Latex font 16' and that it
> would use 'sans' instead. An explanation about this could be what they
> explain in this url: http://projects.gnome.org/dia/fonts.html
>
> * The diagram was imported automatically with the right layout but
> without the red color in the boxes. This was fixed without problems
> manually.
>
> After converting this file to DIA I have generated several other formats
> from it. Here are they: http://folk.uio.no/rafael/pgsql_diagrams/
>
> * png, jpeg and eps are generated with the right layout, color and font
> size.
>
> * xfig is generated with the right layout and color but the font size is
> too big
>
> * svg has big problems with the font size under google chrome and is
> 'almost' right under firefox.

Looks good. I have created a tarball of all my xfig figures:

http://momjian.us/expire/xfig.tgz

While they are licensed as Creative Commons Attribution, you can
consider them BSD-licensed for inclusion into our Postgres docs.

Let me know what else I can do to help.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com