Problem(Abstract)
Tivoli Storage Manager crash
Symptom
Tivoli Storage Manager does not start after crash
Cause
Disk failure on DB2 database files
Diagnosing the problem
During Tivoli Storage Manager start up, the following errors are reported:
ANR0172I rdbdb.c(1465): Error encountered performing action
ActivateDatabase.
ANR0162W Supplemental database diagnostic information: -1224:**No State
Information Available (128)**:-1224 (**No Other Information Available
(206)**).
ANR0172I rdbdb.c(1009): Error encountered performing action
InstanceStop.
ANR0162W Supplemental database diagnostic information: -1032:**No State
Information Available (89)**:-1032 (**No Other Information Available
(72)**).
The db2diag.log reports:
.
2010-02-19-15.29.08.326780+060 I289570185E438 LEVEL: Warning
...
MESSAGE : Crash Recovery is needed.
2010-02-19-15.30.13.948634+060 I289572079E2547 LEVEL: Severe
PID : 8492 TID : 47197773097280PROC : db2sysc 0
INSTANCE: tsmb1 NODE : 000 DB : TSMDB1
APPHDL : 0-7 APPID: *LOCAL.tsmb1.100219142908
AUTHID : TSMB1
EDUID : 65 EDUNAME: db2redow (TSMDB1) 0
FUNCTION: DB2 UDB, buffer pool services, sqlbVerifyCBITS, probe:1110
MESSAGE : ZRC=0x86020019=-2046689255=SQLB_CSUM "Bad Page, Checksum
Error"
DIA8426C A invalid page checksum was found for page "".
DATA #1 : String, 64 bytes
Error encountered trying to read a page - information follows :
DATA #2 : String, 97 bytes
CBIT verification error
bitExpected is 0, userByte is 181, sector 63 (from head of page, 0
based)
DATA #3 : Page ID, PD_TYPE_SQLB_PAGE_ID, 4 bytes
1196993
DATA #4 : Object descriptor, PD_TYPE_SQLB_OBJECT_DESC, 72 bytes
Obj: {pool:6;obj:5;type:1} Parent={5;5}
...
Resolving the problem
The db2diag messages indicate that the table ID 5 has corruption (see last message line: " {pool:6;obj:5;type:1} ".
You can do two different recoveries.
1) Restore the database to a point in time before the disk corruption occurred,
Or
2) Follow DB2 steps to determine if the corruption is only in the index tables and do a DB2 reorg.
Follow these steps to start the DB2 data collection:
$ db2stop
SQL1064N DB2STOP processing was successful.
Now run db2dart command for tablespaceid 6 or run for whole database.
$ db2dart TSMDB1 /DB
The requested DB2DART processing has completed successfully!
Complete DB2DART report found in:
/home/tsminst1/sqllib/db2dump/DART0000/TSMDB1.RPT
Messages in RPT file:
Warning: The database state is not consistent.
Warning: Errors reported about reorg rows may be due to the inconsistent state of the database.
Error: Duplicate found at same slot in unique index. Lvl(1) Page,slot,rid current:2311036p,1008,x6000050319000000 prev:2311036p,1008,x5100AC5E180000
Error: Duplicate found at same slot in unique index. Lvl(1) Page,slot,rid current:2249254p,344,x0D00071D19000000 prev:2249254p,344,x5700AF5E18000000
Error: Duplicate found at same slot in unique index. Lvl(1) Page,slot,rid current:2248938p,951,x4C00FF7419000000 prev:2248938p,951,x4100BC5E18000000
Error: Duplicate found at same slot in unique index. Lvl(1) Page,slot,rid current:2223926p,957,x55002C6A19000000 prev:2223926p,957,x2A00B95E18000000
Error: Duplicate found at same slot in unique index. Lvl(1) Page,slot,rid current:2223926p,958,x81002C6A19000000 prev:2223926p,958,x2900B95E18000000
Error: Duplicate found at same slot in unique index. Lvl(1) Page,slot,rid current:2223926p,959,x17002C6A19000000 prev:2223926p,959,x2800B95E18000000
Error: Duplicate found at same slot in unique index. Lvl(1) Page,slot,rid current:2223926p,960,x18002C6A19000000 prev:2223926p,960,x2700B95E18000000
Index page corruption can be corrected by running the following command below :
Get the table name (This example is using the message output provided in the message: "{pool:6;obj:5;type:1} "):
db2 "select tabname from syscat.tables where tableid=5 and tbspaceid=6"
Get the schema name:
db2 list tables
Run db2 reorg command ( substituting value of schema name. tablename )
db2 " REORG INDEXES ALL FOR TABLE schemaname. table_ame ALLOW NO ACCESS"
If there is still an issue, DB2 support will need the full db2dart output for further analysis.
http://www-01.ibm.com/support/docview.wss?uid=swg21421841
http://www.ibm.com/developerworks/data/library/techarticle/dm-1208corruptiondb2/