Re: patch for new feature: Buffer Cache Hibernation

Hi,

I revised the patch against HEAD, it's available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110506.patch

Implemented hibernation file validations:
- comparison with pg_control
At shutdown:
pg_control state should be DB_SHUTDOWNED.
At startup:
pg_control state should be DB_SHUTDOWNED.
hibernation files should be newer than pg_control.

- CRC check
At shutdown:
compute CRC values for hibernation files and store them into a file.
At startup:
CRC values for hibernation files should be the same with read from the
file created at shutdown.

- file size
At startup:
The size of hibernation file should be the same with calculated file
size based on shared_buffers.

- buffer descriptors validation
At startup:
The descriptor flags should not include BM_DIRTY, BM_IO_IN_PROGRESS,
BM_IO_ERROR, BM_JUST_DIRTIED and BM_PIN_COUNT_WAITER.
Sanity checks for usage_count and usage_count should be done.
(wait_backend_pid is zero-cleared because the process was terminated already)

- system call error checking
At shutdown and startup:
Evaluation for return value system call (eg. open(), read(), write()
and etc) should be done.

> > How do you protect against the cached buffers getting out-of-sync with
> > the actual disk files (especially during recovery scenarios)? What
>
> Saving DB buffer cahce is called at shutdown after finishing
> bgwriter's final checkpoint process, so dirty-buffers should not exist
> I believe.
> For recovery scenarios, I need to research it though...
> Could you describe what is need to be consider?

I think hibernation should be allowed only when the system is shutdown
normaly by checking pg_control state.
And once the abnormal shutdown was detected, the hibernation files
should be ignored.
The latest patch includes this.
# modifications for xlog.c:ReadControlFile() was required though...

> > about crash-induced corruption in the cache file itself (consider the
> > not-unlikely possibility that init will kill the database before it's
> > had time to dump all the buffers during a system shutdown)? Do you have
>
> I think this is important point. I'll implement validation function for
> hibernation file.

Added validations seem enough for me.
# because my understanding on postgres is not enough ;)
If any other considerations are required, please point them out.

Thanks