Thursday, January 22, 2009

ORA-00600 internal error code

Error Description
In our production server while doing data pump import operation at the end of import it fails with following errors, then the instance crashed and database went down.
Errors in file /var/opt/dumpfile/bdump/shiak_mman_23373.trc:
ORA-00600: internal error code, arguments: [kmgs_pre_process_request_6], [4], [2
], [0], [3], [0xC992C04D0], [], []
Sun Oct 19 18:16:20 2008
MMAN: terminating instance due to error 822
Instance terminated by MMAN, pid = 23373

In the trace file we got the call stack as,
ksedst, ksedmp, ksfdmp, kgerinv, kgeasnmierr, kmgs_pre_process_request, kmgsdrv, ksbabs, ksbrdp, opirip, opidrv, sou2o, opimai_real, main.

Cause of the problem
As we know in Oracle ORA-600 error series are oracle bug. This is oracle Bug 4433838. From alert log we see our MMAN (Memory Manager) process has been terminated. So something wrong happen with oracle MMAN.

This bug actually fires when the initialization parameter SGA_TARGET is set to an exact multiple of 4Gb. So, while setting big SGA be conscious.

A little bit more about this bug is if the SGA_TARGET value in HEXADECIMAL has 00000000 as the last bytes then you can hit this bug.

Detail Explanation of this bug
In out production server we set our SGA_TARGET=48G and this bug fires. We see 48G is multiple of 12 times of 4G. So this bug can error. A little bit elaborate of this issue,
Of 48G we get decimal value as,
SQL> col dec for 9999999999999
SQL> select 48*1024*1024*1024 dec from dual;

DEC
--------------
51539607552

Now let's get hexadecimal value of this number,
SQL> select to_char(51539607552,'XXXXXXXXXXXXX') hex from dual;
HEX
---------------
C00000000

As we see in the hexadecimal representation we get last 8 letters are all 0s; 00000000. The bug happens whenever in hexadecimal representation the 4 bytes have 0. Here in setting of 48G we get last 4 bytes all 0. As for last 0 in binary it represent 0000. So 2 letters of 0 in hexadecimal just form 1 byte(8 bits). So last 8 letters in hexadecimal means we got last 4 bytes 0. And forming this type causes to fire hug.

Solution of the problem

Solution 1)
As bug fires whenever in HEXADECIMAL representation we have 00000000 as the last 4 bytes(8 letters in hex value). So our target is to set SGA_TARGET so that it does not form like this way(4 bytes 0 at last). We can do this if we set our SGA_TARGET not exact multiple of 4G.
Doing set our SGA_TARGET to 47G would happen not the last 4 bytes as all 0s of binary representation.

ALTER SYSTEM SET SGA_TARGET=47G SCOPE=BOTH;
SQL> select to_char(47*1024*1024*1024,'XXXXXXXXXXXXX') hex from dual;
HEX
---------------
BC0000000
We see last 7 letters as 0, and hence the bug will not fire.

Solution 2)
Out database version was 10.2.0.1; base version. And this bug fires. This bug remains in oracle patchset 10.2.0.2 and 10.2.0.3. And is fixed in 10.2.0.4 patchset. So applying patchset 10.2.0.4 would fix the problem.

Solution 3)
Upgrade your database to 11.1.0.6 also will fix the problem.

Solution 4)
Apply the one-off Patch 4433838.

No comments:

Post a Comment