Doc Patch and test for Patch to compute Max LSN of Data Pages

Lists: pgsql-hackers
From: Amit kapila <amit(dot)kapila(at)huawei(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: Doc Patch and test for Patch to compute Max LSN of Data Pages
Date: 2012-08-29 11:39:55
Message-ID: 6C0B27F7206C9E4CA54AE035729E9C382852F497@szxeml509-mbs
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

>>> Based on the discussion and suggestions in this mail chain, following features can be implemented:
>>>
>>> 1. To compute the value of max LSN in data pages based on user input whether he wants it for an individual

>>> file, a particular directory or whole database.
>>
>>> 2a. To search the available WAL files for the latest checkpoint record and prints the value.
>>> 2b. To search the available WAL files for the latest checkpoint record and recreates a pg_control file pointing at

>>> that checkpoint.

>>> I have kept both options to address different kind of corruption scenarios.

>> I think I can see all of those things being potentially useful. There
>> are a couple of pending patches that will revise the WAL format
>> slightly; not sure how much those are likely to interfere with any
>> development you might do on (2) in the meantime.

> Based on above conclusion, I have prepared a patch which implements Option-1

This mail contains doc patch Option-1 and test cases.

Below are test scenarios corresponding to which testcases are in Test_find_max_lsn_from_datafiles

Scenario-1:
Validation of the maximum LSN number & in data base directory
Steps:
1. Start the server, create table, insert some records into the table.
2. Shutdown the server in normal mode.
3. ./pg_resetxlog -P data to find the maximum LSN number and validate with the current pg_xlog directory and in pg_control file
Expected behavior:
Displayed maximum LSN number should to same as in pg_control file &
WAL segment number displayed (fileid, segnum) should be same current file in pg_xlog directory.

Scenario-2:
Validation of the maximum LSN number & in specific directory
Steps:
1. Start the server, create table, insert some records into the table.
2. Shutdown the server in normal mode.
3. ./pg_resetxlog -p base/1/12557/ data
Expected behavior:
Displayed maximum LSN number should to same as in pg_control file &
WAL segment number displayed (fileid, segnum) should be same current file in pg_xlog directory.

Scenario-3:
Validation of the maximum LSN number & in specific file
Steps:
1. Start the server, create table, insert some records into the table.
2. Shutdown the server in normal mode.
3. ./pg_resetxlog -p base/1/12557/16384 data
Expected behavior:
Displayed maximum LSN number should to same as in pg_control file &
WAL segment number displayed (fileid, segnum) should be same current file in pg_xlog directory.

With Regards,

Amit Kapila.

Attachment Content-Type Size
documentation_for_pg_resetxlog_find_max_lsn_from_datafiles.patch text/plain 2.5 KB
Test_find_max_lsn_from_datafiles.txt text/plain 1.8 KB

From: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
To: Amit Kapila <amit(dot)kapila(at)huawei(dot)com>
Cc: 'Robert Haas' <robertmhaas(at)gmail(dot)com>, 'Tom Lane' <tgl(at)sss(dot)pgh(dot)pa(dot)us>, 'Cédric Villemain' <cedric(at)2ndquadrant(dot)com>, 'Pg Hackers' <pgsql-hackers(at)postgresql(dot)org>, 'Simon Riggs' <simon(at)2ndQuadrant(dot)com>, 'Heikki Linnakangas' <hlinnakangas(at)vmware(dot)com>, 'Andres Freund' <andres(at)2ndquadrant(dot)com>
Subject: Re: Allow WAL information to recover corrupted pg_controldata
Date: 2012-11-05 14:02:53
Message-ID: 20121105140253.GB12444@alvh.no-ip.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

I'm not sure what to do with this patch. There was some resistance to
the idea originally; then after some discussion, there was some
apparent agreement that it might be useful on occasion. Later, a patch
was posted, but there was almost no review of it; except to say that it
should probably be reworked on top of an hypothetical, future XLogReader
feature.

Since it doesn't look like we're going anywhere with it soon, I'm going
to close it as returned with feedback. Hopefully, if we get XLogReader
in 9.3, we will have time to rebase this patch on top of that. (I
invite Amit to give Heikki's version of XLogReader patch a look.)

(It is very hard to track down vague references to old threads that
aren't properly linked in new threads. Please make sure to reply to old
emails, or at least to give Message-Ids or URLs when starting new
threads. I am replying to one message of each old thread here.)

--
Álvaro Herrera http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


From: Amit Kapila <amit(dot)kapila(at)huawei(dot)com>
To: "'Alvaro Herrera'" <alvherre(at)2ndquadrant(dot)com>
Cc: "'Robert Haas'" <robertmhaas(at)gmail(dot)com>, "'Tom Lane'" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, 'Cédric Villemain' <cedric(at)2ndquadrant(dot)com>, "'Pg Hackers'" <pgsql-hackers(at)postgresql(dot)org>, "'Simon Riggs'" <simon(at)2ndQuadrant(dot)com>, "'Heikki Linnakangas'" <hlinnakangas(at)vmware(dot)com>, "'Andres Freund'" <andres(at)2ndquadrant(dot)com>
Subject: Re: Allow WAL information to recover corrupted pg_controldata
Date: 2012-11-05 14:38:37
Message-ID: 001801cdbb63$3e980900$bbc81b00$@kapila@huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Monday, November 05, 2012 7:33 PM Alvaro Herrera wrote:
> I'm not sure what to do with this patch. There was some resistance to
> the idea originally; then after some discussion, there was some
> apparent agreement that it might be useful on occasion. Later, a patch
> was posted, but there was almost no review of it; except to say that it
> should probably be reworked on top of an hypothetical, future XLogReader
> feature.
>
> Since it doesn't look like we're going anywhere with it soon, I'm going
> to close it as returned with feedback. Hopefully, if we get XLogReader
> in 9.3, we will have time to rebase this patch on top of that. (I
> invite Amit to give Heikki's version of XLogReader patch a look.)

The patch for which Heikki has given comment
(https://commitfest.postgresql.org/action/patch_view?id=897) is already
moved to next CF.
This was not related to XLogReader. However as there is not much interest in
this feature, so it is okay.

> (It is very hard to track down vague references to old threads that
> aren't properly linked in new threads. Please make sure to reply to old
> emails, or at least to give Message-Ids or URLs when starting new
> threads. I am replying to one message of each old thread here.)

My mistake, I am sorry for that and I shall try to take care for future
work.

With Regards,
Amit Kapila.