In a RAC database there may contain several instances and the initialization parameters of all of the instances should not have identical setting. In fact the initialization parameters that are critical for database creation and affect certain database operations must have the same value for every instance in an Oracle RAC database.
Below is the list of initialization parameters that must have identical settings for all instances in a RAC database.
1)ACTIVE_INSTANCE_COUNT
2)ARCHIVE_LAG_TARGET
3)CLUSTER_DATABASE
4)CLUSTER_DATABASE_INSTANCES
5)COMPATIBLE
6)CONTROL_FILES
7)DB_BLOCK_SIZE
8)DB_DOMAIN
9)DB_FILES
10)DB_NAME
11)DB_RECOVERY_FILE_DEST
12)DB_RECOVERY_FILE_DEST_SIZE
13)DB_UNIQUE_NAME
14)DML_LOCKS (Only if set to zero).
15)INSTANCE_TYPE (RDBMS or ASM)
16)PARALLEL_MAX_SERVERS
17)REMOTE_LOGIN_PASSWORD_FILE
18)UNDO_MANAGEMENT
Showing posts with label RAC. Show all posts
Showing posts with label RAC. Show all posts
Monday, January 12, 2009
List of Parameters that must have identical in RAC database
In a RAC database there may contain several instances and the initialization parameters of all of the instances should not have identical setting. In fact the initialization parameters that are critical for database creation and affect certain database operations must have the same value for every instance in an Oracle RAC database.
Below is the list of initialization parameters that must have identical settings for all instances in a RAC database.
1)ACTIVE_INSTANCE_COUNT
2)ARCHIVE_LAG_TARGET
3)CLUSTER_DATABASE
4)CLUSTER_DATABASE_INSTANCES
5)COMPATIBLE
6)CONTROL_FILES
7)DB_BLOCK_SIZE
8)DB_DOMAIN
9)DB_FILES
10)DB_NAME
11)DB_RECOVERY_FILE_DEST
12)DB_RECOVERY_FILE_DEST_SIZE
13)DB_UNIQUE_NAME
14)DML_LOCKS (Only if set to zero).
15)INSTANCE_TYPE (RDBMS or ASM)
16)PARALLEL_MAX_SERVERS
17)REMOTE_LOGIN_PASSWORD_FILE
18)UNDO_MANAGEMENT
Below is the list of initialization parameters that must have identical settings for all instances in a RAC database.
1)ACTIVE_INSTANCE_COUNT
2)ARCHIVE_LAG_TARGET
3)CLUSTER_DATABASE
4)CLUSTER_DATABASE_INSTANCES
5)COMPATIBLE
6)CONTROL_FILES
7)DB_BLOCK_SIZE
8)DB_DOMAIN
9)DB_FILES
10)DB_NAME
11)DB_RECOVERY_FILE_DEST
12)DB_RECOVERY_FILE_DEST_SIZE
13)DB_UNIQUE_NAME
14)DML_LOCKS (Only if set to zero).
15)INSTANCE_TYPE (RDBMS or ASM)
16)PARALLEL_MAX_SERVERS
17)REMOTE_LOGIN_PASSWORD_FILE
18)UNDO_MANAGEMENT
Friday, January 9, 2009
Configure Raw Devices for ASM in RAC
Configuring RAW decides in RAC is just similiar as you do configure raw devices for voting disk and OCR. In this example, I have faced a different scenario where in
In racnode-1 disk sdb is mapped to sdf in ranode-2
In racnode-1 disk sdc is mapped to sdd in ranode-2
In racnode-1 disk sdd is mapped to sde in ranode-2
First I have created three raw partitions from my raw devices sdb, sdc and sdd of racnode-1 each of 80GB for ASM.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33): +80000M
Last cylinder or +size or +sizeM or +sizeK (9726-19581, default 19581):
Using default value 19581
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdc
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdd
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
For RACNODE-1
[root@racnode-1 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdc2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-1 ~]# reboot
For RACNODE-2
[root@racnode-2 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdf2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sde2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-2 ~]# reboot
In racnode-1 disk sdb is mapped to sdf in ranode-2
In racnode-1 disk sdc is mapped to sdd in ranode-2
In racnode-1 disk sdd is mapped to sde in ranode-2
First I have created three raw partitions from my raw devices sdb, sdc and sdd of racnode-1 each of 80GB for ASM.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33): +80000M
Last cylinder or +size or +sizeM or +sizeK (9726-19581, default 19581):
Using default value 19581
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdc
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdd
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
For RACNODE-1
[root@racnode-1 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdc2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-1 ~]# reboot
For immediate affect if you don't see any reboot message,
#/bin/raw /dev/raw/raw6 /dev/sdb2
/bin/raw /dev/raw/raw7 /dev/sdc2
/bin/raw /dev/raw/raw8 /dev/sdd2
Adjust the permission settings by,
#chown oracle:oinstall /dev/raw/raw6
chown oracle:oinstall /dev/raw/raw7
chown oracle:oinstall /dev/raw/raw8
chmod 640 /dev/raw/raw6
chmod 640 /dev/raw/raw7
chmod 640 /dev/raw/raw8
For RACNODE-2
[root@racnode-2 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdf2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sde2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-2 ~]# reboot
For immediate affect, if you don't see any reboot message
#/bin/raw /dev/raw/raw6 /dev/sdf2
/bin/raw /dev/raw/raw7 /dev/sdd2
/bin/raw /dev/raw/raw8 /dev/sde2
and adjust the permission settings by,
#chown oracle:oinstall /dev/raw/raw6
chown oracle:oinstall /dev/raw/raw7
chown oracle:oinstall /dev/raw/raw8
chmod 640 /dev/raw/raw6
chmod 640 /dev/raw/raw7
chmod 640 /dev/raw/raw8
Major Oracle Clusterware components
The Oracle Clusterware comprises several background processes that facilitate cluster operations. These processes or components are the main communication links between the Oracle Clusterware high availability components and the Oracle Database as well as they monitor and manage database operations.
Here is the list of major oracle clusterware components or processes.
1)Cluster Synchronization Services (CSS): It manages and controls which nodes are members of the cluster and notify members when a node joins or leaves the cluster.
2)Cluster Ready Services (CRS): It manages high availability operations within a cluster. The CRS process start, stop, monitor and failover operations. It generates events when a resource status changes. When you have installed Oracle RAC, crs monitors the Oracle instance, Listener, and so on, and automatically restarts these components when a failure occurs.
3)Event Management (EVM): It is a background process that publishes events that crs creates.
4)Oracle Notification Service (ONS): It publishes and subscribes service for communicating Fast Application Notification (FAN) events.
5)RACG: It runs server callout scripts when FAN events occur.
6)Process Monitor Daemon (OPROCD): This process is locked in memory to monitor the cluster and provide I/O fencing. OPROCD performs its check, stops running, and if the wake up is beyond the expected time, then OPROCD resets the processor and reboots the node. An OPROCD failure results in Oracle Clusterware restarting the node. OPROCD uses the hangcheck timer on Linux platforms.
Here is the list of major oracle clusterware components or processes.
1)Cluster Synchronization Services (CSS): It manages and controls which nodes are members of the cluster and notify members when a node joins or leaves the cluster.
2)Cluster Ready Services (CRS): It manages high availability operations within a cluster. The CRS process start, stop, monitor and failover operations. It generates events when a resource status changes. When you have installed Oracle RAC, crs monitors the Oracle instance, Listener, and so on, and automatically restarts these components when a failure occurs.
3)Event Management (EVM): It is a background process that publishes events that crs creates.
4)Oracle Notification Service (ONS): It publishes and subscribes service for communicating Fast Application Notification (FAN) events.
5)RACG: It runs server callout scripts when FAN events occur.
6)Process Monitor Daemon (OPROCD): This process is locked in memory to monitor the cluster and provide I/O fencing. OPROCD performs its check, stops running, and if the wake up is beyond the expected time, then OPROCD resets the processor and reboots the node. An OPROCD failure results in Oracle Clusterware restarting the node. OPROCD uses the hangcheck timer on Linux platforms.
Configure Raw Devices for ASM in RAC
Configuring RAW decides in RAC is just similiar as you do configure raw devices for voting disk and OCR. In this example, I have faced a different scenario where in
In racnode-1 disk sdb is mapped to sdf in ranode-2
In racnode-1 disk sdc is mapped to sdd in ranode-2
In racnode-1 disk sdd is mapped to sde in ranode-2
First I have created three raw partitions from my raw devices sdb, sdc and sdd of racnode-1 each of 80GB for ASM.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33): +80000M
Last cylinder or +size or +sizeM or +sizeK (9726-19581, default 19581):
Using default value 19581
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdc
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdd
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
For RACNODE-1
[root@racnode-1 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdc2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-1 ~]# reboot
For RACNODE-2
[root@racnode-2 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdf2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sde2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-2 ~]# reboot
In racnode-1 disk sdb is mapped to sdf in ranode-2
In racnode-1 disk sdc is mapped to sdd in ranode-2
In racnode-1 disk sdd is mapped to sde in ranode-2
First I have created three raw partitions from my raw devices sdb, sdc and sdd of racnode-1 each of 80GB for ASM.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33): +80000M
Last cylinder or +size or +sizeM or +sizeK (9726-19581, default 19581):
Using default value 19581
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdc
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdd
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (33-19581, default 33):
Using default value 33
Last cylinder or +size or +sizeM or +sizeK (33-19581, default 19581): +80000M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
For RACNODE-1
[root@racnode-1 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdb2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdc2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-1 ~]# reboot
For immediate affect if you don't see any reboot message,
#/bin/raw /dev/raw/raw6 /dev/sdb2
/bin/raw /dev/raw/raw7 /dev/sdc2
/bin/raw /dev/raw/raw8 /dev/sdd2
Adjust the permission settings by,
#chown oracle:oinstall /dev/raw/raw6
chown oracle:oinstall /dev/raw/raw7
chown oracle:oinstall /dev/raw/raw8
chmod 640 /dev/raw/raw6
chmod 640 /dev/raw/raw7
chmod 640 /dev/raw/raw8
For RACNODE-2
[root@racnode-2 ~]# vi /etc/udev/rules.d/63-oracle-raw.rules
ACTION=="add", KERNEL=="sdf2", RUN+="/bin/raw /dev/raw/raw6 %N"
ACTION=="add", KERNEL=="sdd2", RUN+="/bin/raw /dev/raw/raw7 %N"
ACTION=="add", KERNEL=="sde2", RUN+="/bin/raw /dev/raw/raw8 %N"
KERNEL=="raw[6-8]*", OWNER="oracle", GROUP="oinstall", MODE="644"
Then reboot the node if it suggest as in this example. The new table will be used at the next reboot.
[root@racnode-2 ~]# reboot
For immediate affect, if you don't see any reboot message
#/bin/raw /dev/raw/raw6 /dev/sdf2
/bin/raw /dev/raw/raw7 /dev/sdd2
/bin/raw /dev/raw/raw8 /dev/sde2
and adjust the permission settings by,
#chown oracle:oinstall /dev/raw/raw6
chown oracle:oinstall /dev/raw/raw7
chown oracle:oinstall /dev/raw/raw8
chmod 640 /dev/raw/raw6
chmod 640 /dev/raw/raw7
chmod 640 /dev/raw/raw8
Major Oracle Clusterware components
The Oracle Clusterware comprises several background processes that facilitate cluster operations. These processes or components are the main communication links between the Oracle Clusterware high availability components and the Oracle Database as well as they monitor and manage database operations.
Here is the list of major oracle clusterware components or processes.
1)Cluster Synchronization Services (CSS): It manages and controls which nodes are members of the cluster and notify members when a node joins or leaves the cluster.
2)Cluster Ready Services (CRS): It manages high availability operations within a cluster. The CRS process start, stop, monitor and failover operations. It generates events when a resource status changes. When you have installed Oracle RAC, crs monitors the Oracle instance, Listener, and so on, and automatically restarts these components when a failure occurs.
3)Event Management (EVM): It is a background process that publishes events that crs creates.
4)Oracle Notification Service (ONS): It publishes and subscribes service for communicating Fast Application Notification (FAN) events.
5)RACG: It runs server callout scripts when FAN events occur.
6)Process Monitor Daemon (OPROCD): This process is locked in memory to monitor the cluster and provide I/O fencing. OPROCD performs its check, stops running, and if the wake up is beyond the expected time, then OPROCD resets the processor and reboots the node. An OPROCD failure results in Oracle Clusterware restarting the node. OPROCD uses the hangcheck timer on Linux platforms.
Here is the list of major oracle clusterware components or processes.
1)Cluster Synchronization Services (CSS): It manages and controls which nodes are members of the cluster and notify members when a node joins or leaves the cluster.
2)Cluster Ready Services (CRS): It manages high availability operations within a cluster. The CRS process start, stop, monitor and failover operations. It generates events when a resource status changes. When you have installed Oracle RAC, crs monitors the Oracle instance, Listener, and so on, and automatically restarts these components when a failure occurs.
3)Event Management (EVM): It is a background process that publishes events that crs creates.
4)Oracle Notification Service (ONS): It publishes and subscribes service for communicating Fast Application Notification (FAN) events.
5)RACG: It runs server callout scripts when FAN events occur.
6)Process Monitor Daemon (OPROCD): This process is locked in memory to monitor the cluster and provide I/O fencing. OPROCD performs its check, stops running, and if the wake up is beyond the expected time, then OPROCD resets the processor and reboots the node. An OPROCD failure results in Oracle Clusterware restarting the node. OPROCD uses the hangcheck timer on Linux platforms.
Thursday, January 8, 2009
Overview of Oracle Clusterware and Oracle RAC
In a database whenever you query from V$database to know the name of the database and want to know the name of the instance from v$thread they return similar result. Suppose,
SQL> select name from v$database;
NAME
---------
DBASE
SQL> select instance from v$thread;
INSTANCE
--------------------------------------------------------------------------------
dbase
which indicates in the database there is only one instance that is database have a one-to-one relationship between datafiles and the instance. Oracle RAC environments, however, have a one-to-many relationship between datafiles and instances. In RAC environments multiple instances form a single database. The instances can reside on different servers which is referred as nodes in RAC environment.
• In RAC environment the set of interconnected instances that allows the servers to work together is called cluster. The physical structures that make up the database must reside on shared storage that is accessible from all servers that are part of the cluster. Each server in the cluster runs the Oracle RAC software.
• In RAC database each instance has its own memory and background processes but every instances share the same data files. So while any data need to write or retrieve they use the same datafiles but each instance use it's own memory.
Now question is if one instance read data and modify it in the buffer cache and if another instance need that data then how oracle manages it?
• In RAC Oracle uses Cache Fusion Technology to synchronize the data stored in the buffer cache of each cluster database instance. Cache Fusion moves current data blocks (which reside in memory) between database instances, rather than having one database instance write the data blocks to disk and requiring another database instance to reread the data blocks from disk.
• When a data block is needed for one instance which resides in the buffer cache of another instance, Cache Fusion transfers the data block directly between the instances using the interconnect, enabling the Oracle RAC database to access and modify data as if the data resided in a single buffer cache.
Oracle RAC provides high availability and scalability.
If one instance fails in RAC environment still another instance can do the task and thus prevents the server from being a single point of failure.
In a RAC environment transactions are distributed over multiple instances. Thus minimize load from one instance.
• Never mix with oracle clusterware with Oracle RAC. The Oracle Clusterware is software that when installed on servers, enables the servers to be bound together to operate as if they were one server. Before you install oracle RAC you need to install clusterware. Without clusterware you can't install oracle RAC.
SQL> select name from v$database;
NAME
---------
DBASE
SQL> select instance from v$thread;
INSTANCE
--------------------------------------------------------------------------------
dbase
which indicates in the database there is only one instance that is database have a one-to-one relationship between datafiles and the instance. Oracle RAC environments, however, have a one-to-many relationship between datafiles and instances. In RAC environments multiple instances form a single database. The instances can reside on different servers which is referred as nodes in RAC environment.
• In RAC environment the set of interconnected instances that allows the servers to work together is called cluster. The physical structures that make up the database must reside on shared storage that is accessible from all servers that are part of the cluster. Each server in the cluster runs the Oracle RAC software.
• In RAC database each instance has its own memory and background processes but every instances share the same data files. So while any data need to write or retrieve they use the same datafiles but each instance use it's own memory.
Now question is if one instance read data and modify it in the buffer cache and if another instance need that data then how oracle manages it?
• In RAC Oracle uses Cache Fusion Technology to synchronize the data stored in the buffer cache of each cluster database instance. Cache Fusion moves current data blocks (which reside in memory) between database instances, rather than having one database instance write the data blocks to disk and requiring another database instance to reread the data blocks from disk.
• When a data block is needed for one instance which resides in the buffer cache of another instance, Cache Fusion transfers the data block directly between the instances using the interconnect, enabling the Oracle RAC database to access and modify data as if the data resided in a single buffer cache.
Oracle RAC provides high availability and scalability.
If one instance fails in RAC environment still another instance can do the task and thus prevents the server from being a single point of failure.
In a RAC environment transactions are distributed over multiple instances. Thus minimize load from one instance.
• Never mix with oracle clusterware with Oracle RAC. The Oracle Clusterware is software that when installed on servers, enables the servers to be bound together to operate as if they were one server. Before you install oracle RAC you need to install clusterware. Without clusterware you can't install oracle RAC.
Overview of Oracle Clusterware and Oracle RAC
In a database whenever you query from V$database to know the name of the database and want to know the name of the instance from v$thread they return similar result. Suppose,
SQL> select name from v$database;
NAME
---------
DBASE
SQL> select instance from v$thread;
INSTANCE
--------------------------------------------------------------------------------
dbase
which indicates in the database there is only one instance that is database have a one-to-one relationship between datafiles and the instance. Oracle RAC environments, however, have a one-to-many relationship between datafiles and instances. In RAC environments multiple instances form a single database. The instances can reside on different servers which is referred as nodes in RAC environment.
• In RAC environment the set of interconnected instances that allows the servers to work together is called cluster. The physical structures that make up the database must reside on shared storage that is accessible from all servers that are part of the cluster. Each server in the cluster runs the Oracle RAC software.
• In RAC database each instance has its own memory and background processes but every instances share the same data files. So while any data need to write or retrieve they use the same datafiles but each instance use it's own memory.
Now question is if one instance read data and modify it in the buffer cache and if another instance need that data then how oracle manages it?
• In RAC Oracle uses Cache Fusion Technology to synchronize the data stored in the buffer cache of each cluster database instance. Cache Fusion moves current data blocks (which reside in memory) between database instances, rather than having one database instance write the data blocks to disk and requiring another database instance to reread the data blocks from disk.
• When a data block is needed for one instance which resides in the buffer cache of another instance, Cache Fusion transfers the data block directly between the instances using the interconnect, enabling the Oracle RAC database to access and modify data as if the data resided in a single buffer cache.
Oracle RAC provides high availability and scalability.
If one instance fails in RAC environment still another instance can do the task and thus prevents the server from being a single point of failure.
In a RAC environment transactions are distributed over multiple instances. Thus minimize load from one instance.
• Never mix with oracle clusterware with Oracle RAC. The Oracle Clusterware is software that when installed on servers, enables the servers to be bound together to operate as if they were one server. Before you install oracle RAC you need to install clusterware. Without clusterware you can't install oracle RAC.
SQL> select name from v$database;
NAME
---------
DBASE
SQL> select instance from v$thread;
INSTANCE
--------------------------------------------------------------------------------
dbase
which indicates in the database there is only one instance that is database have a one-to-one relationship between datafiles and the instance. Oracle RAC environments, however, have a one-to-many relationship between datafiles and instances. In RAC environments multiple instances form a single database. The instances can reside on different servers which is referred as nodes in RAC environment.
• In RAC environment the set of interconnected instances that allows the servers to work together is called cluster. The physical structures that make up the database must reside on shared storage that is accessible from all servers that are part of the cluster. Each server in the cluster runs the Oracle RAC software.
• In RAC database each instance has its own memory and background processes but every instances share the same data files. So while any data need to write or retrieve they use the same datafiles but each instance use it's own memory.
Now question is if one instance read data and modify it in the buffer cache and if another instance need that data then how oracle manages it?
• In RAC Oracle uses Cache Fusion Technology to synchronize the data stored in the buffer cache of each cluster database instance. Cache Fusion moves current data blocks (which reside in memory) between database instances, rather than having one database instance write the data blocks to disk and requiring another database instance to reread the data blocks from disk.
• When a data block is needed for one instance which resides in the buffer cache of another instance, Cache Fusion transfers the data block directly between the instances using the interconnect, enabling the Oracle RAC database to access and modify data as if the data resided in a single buffer cache.
Oracle RAC provides high availability and scalability.
If one instance fails in RAC environment still another instance can do the task and thus prevents the server from being a single point of failure.
In a RAC environment transactions are distributed over multiple instances. Thus minimize load from one instance.
• Never mix with oracle clusterware with Oracle RAC. The Oracle Clusterware is software that when installed on servers, enables the servers to be bound together to operate as if they were one server. Before you install oracle RAC you need to install clusterware. Without clusterware you can't install oracle RAC.
Requirements to check before install Oracle RAC
1)Checking the Hardware Requirements
•Physical memory: At least 1GB RAM.
•Swap space: If RAM is between 1 GB and 2 GB then make swap space to 1.5 times of the size of RAM.
If RMAN is more than 2GB then make swap space to the equal of the size of the RAM.
•Temporary space: At least 400 MB. Typically in /tmp directory.
•Processor type (CPU): Need to be certified with the version of the Oracle software being installed.
•Hard Disk Space: 1.5 GB for oracle database home directory+1.5GB for the ASM home directory+120 oracle clusterware software installation+Two Oracle Clusterware components OCR 256 MB each, or 512 MB total disk space+Three Oracle Clusterware components Voting Disk 256 MB each, or 768 MB total disk space.
•All the nodes in the cluster must have same hardware architecture. However can have machines of different speeds and size in the same cluster.
On *nix system you can check hardware components as follows.
•To determine physical RAM size, # grep MemTotal /proc/meminfo
•To determine the configured swap space, # grep SwapTotal /proc/meminfo
•To determine the amount of disk space available in the /tmp directory, # df -k /tmp
•To determine free disk space on the system, #df -h or #df -k
•To determine whether the system architecture, # grep "model name" /proc/cpuinfo
Checking the Network Requirements
Network Hardware Requirements
•One private interconnect is needed for Oracle Custerware for the use to synchronize each instance's use of the shared resources and for Oracle RAC to interconnect to transmit data blocks that are shared between the instances. Thus each node need at least two network interface cards, or network adapters. One adapter is for the public network and the other adapter is for the private network
•The public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes. For example if in server1/node1, eth0 is the public interface then on server2/node2 eth0 must be the public interface.
•For the public network, each network adapter must support TCP/IP.
•For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP.
•Note UDP is the default interconnect protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware.
•For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. All nodes must be accessible between through private network. You can check the reachability by ping command.
Network Parameter Requirements
If NFS is used for the shared storage, then you must set the values for the NFS buffer size parameters rsize and wsize to at least 16384. Oracle recommends that you use the value 32768.
You can set the value by updating the /etc/fstab file on each node with an entry similar to the following,
clusternode:/vol/DATA/oradata /home/oradata/app nfs
rw,bg,vers=3,tcp,hard,nointr,timeo=600,rsize=32768,wsize=32768,actimeo=0 1 2
IP Address Requirements
•You must have at least three IP addresses available for each node
1.An IP address for the public interface. Interface name should be the name of the node name.
2.An IP address for the private interface. Interface name should be hostname-priv.
3.One virtual IP address with an associated network name. Interface name should be hostname-vip.
•The VIP is on the same subnet as your public interface and it address will be not used currently in the network.
•For public and virtual addresses register with an associated network name in DNS. If you do not have an available DNS, then record the all network names and interface names in the system hosts file, /etc/hosts.
•Identify the interface names and associated IP addresses for all network adapters by running the following command on each node:
# /sbin/ifconfig
Node Time Requirements
Ensure that each member node of the cluster is set as closely as possible to the same date and time. Oracle strongly recommends using the Network Time Protocol (NTP) feature of most operating systems for this purpose.
Verifying the Installed Operating System and Software Requirements
•To determine which distribution and version of Linux is installed, run the following command as the root user:
#cat /etc/issue
•The Linux kernel is updated to fix bugs. These kernel updates are referred to as erratum kernels or errata levels. To determine if the required errata level is installed, use the following procedure as the root user:
#uname -r
•Physical memory: At least 1GB RAM.
•Swap space: If RAM is between 1 GB and 2 GB then make swap space to 1.5 times of the size of RAM.
If RMAN is more than 2GB then make swap space to the equal of the size of the RAM.
•Temporary space: At least 400 MB. Typically in /tmp directory.
•Processor type (CPU): Need to be certified with the version of the Oracle software being installed.
•Hard Disk Space: 1.5 GB for oracle database home directory+1.5GB for the ASM home directory+120 oracle clusterware software installation+Two Oracle Clusterware components OCR 256 MB each, or 512 MB total disk space+Three Oracle Clusterware components Voting Disk 256 MB each, or 768 MB total disk space.
•All the nodes in the cluster must have same hardware architecture. However can have machines of different speeds and size in the same cluster.
On *nix system you can check hardware components as follows.
•To determine physical RAM size, # grep MemTotal /proc/meminfo
•To determine the configured swap space, # grep SwapTotal /proc/meminfo
•To determine the amount of disk space available in the /tmp directory, # df -k /tmp
•To determine free disk space on the system, #df -h or #df -k
•To determine whether the system architecture, # grep "model name" /proc/cpuinfo
Checking the Network Requirements
Network Hardware Requirements
•One private interconnect is needed for Oracle Custerware for the use to synchronize each instance's use of the shared resources and for Oracle RAC to interconnect to transmit data blocks that are shared between the instances. Thus each node need at least two network interface cards, or network adapters. One adapter is for the public network and the other adapter is for the private network
•The public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes. For example if in server1/node1, eth0 is the public interface then on server2/node2 eth0 must be the public interface.
•For the public network, each network adapter must support TCP/IP.
•For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP.
•Note UDP is the default interconnect protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware.
•For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. All nodes must be accessible between through private network. You can check the reachability by ping command.
Network Parameter Requirements
If NFS is used for the shared storage, then you must set the values for the NFS buffer size parameters rsize and wsize to at least 16384. Oracle recommends that you use the value 32768.
You can set the value by updating the /etc/fstab file on each node with an entry similar to the following,
clusternode:/vol/DATA/oradata /home/oradata/app nfs
rw,bg,vers=3,tcp,hard,nointr,timeo=600,rsize=32768,wsize=32768,actimeo=0 1 2
IP Address Requirements
•You must have at least three IP addresses available for each node
1.An IP address for the public interface. Interface name should be the name of the node name.
2.An IP address for the private interface. Interface name should be hostname-priv.
3.One virtual IP address with an associated network name. Interface name should be hostname-vip.
•The VIP is on the same subnet as your public interface and it address will be not used currently in the network.
•For public and virtual addresses register with an associated network name in DNS. If you do not have an available DNS, then record the all network names and interface names in the system hosts file, /etc/hosts.
•Identify the interface names and associated IP addresses for all network adapters by running the following command on each node:
# /sbin/ifconfig
Node Time Requirements
Ensure that each member node of the cluster is set as closely as possible to the same date and time. Oracle strongly recommends using the Network Time Protocol (NTP) feature of most operating systems for this purpose.
Verifying the Installed Operating System and Software Requirements
•To determine which distribution and version of Linux is installed, run the following command as the root user:
#cat /etc/issue
•The Linux kernel is updated to fix bugs. These kernel updates are referred to as erratum kernels or errata levels. To determine if the required errata level is installed, use the following procedure as the root user:
#uname -r
Various Tools for Installing, Configuring Oracle RAC
1)Oracle Universal Installer (OUI)- After configuring the pre-installation tasks of the nodes OUI installs the Oracle Clusterware and the Oracle Database software with Oracle RAC. It also can install oracle software on the other nodes using a network connection.
2)Cluster Verification Utility (CVU)- The CVU is a command-line tool. It is very useful to check the nodes for preinstallation as well as postinstallation requirements of the cluster environment. In fact OUI runs the CVU after the Oracle Clusterware installation to verify the environment.
3)Oracle Enterprise Manager- With EM it is easy to configure RAC environments. It has both the Database Control and Grid Control graphical user interfaces (GUIs).
4)Server Control (SRVCTL)- SRVCTL is a command-line interface that you can use to manage an Oracle RAC database from a single point. Using SRVCTL you can start or stop or manage any instance on the cluster.
5)Cluster Ready Services Control (CRSCTL)- CRSCTL is a command-line tool that you can use to manage Oracle Clusterware. You can use CRSCTL to start and stop Oracle Clusterware and to determine the current status of your Oracle Clusterware installation.
6)Oracle Interface Configuration Tool (OIFCFG)- OIFCFG is a command-line tool which you can use to allocate and de-allocate network interfaces to components. You can use OIFCFG to direct components to use specific network interfaces and to retrieve component configuration information.
7)OCR Configuration Tool (OCRCONFIG)- OCRCONFIG is a command-line tool for OCR administration. You can also use the OCRCHECK and OCRDUMP utilities to troubleshoot configuration problems that affect the OCR.
2)Cluster Verification Utility (CVU)- The CVU is a command-line tool. It is very useful to check the nodes for preinstallation as well as postinstallation requirements of the cluster environment. In fact OUI runs the CVU after the Oracle Clusterware installation to verify the environment.
3)Oracle Enterprise Manager- With EM it is easy to configure RAC environments. It has both the Database Control and Grid Control graphical user interfaces (GUIs).
4)Server Control (SRVCTL)- SRVCTL is a command-line interface that you can use to manage an Oracle RAC database from a single point. Using SRVCTL you can start or stop or manage any instance on the cluster.
5)Cluster Ready Services Control (CRSCTL)- CRSCTL is a command-line tool that you can use to manage Oracle Clusterware. You can use CRSCTL to start and stop Oracle Clusterware and to determine the current status of your Oracle Clusterware installation.
6)Oracle Interface Configuration Tool (OIFCFG)- OIFCFG is a command-line tool which you can use to allocate and de-allocate network interfaces to components. You can use OIFCFG to direct components to use specific network interfaces and to retrieve component configuration information.
7)OCR Configuration Tool (OCRCONFIG)- OCRCONFIG is a command-line tool for OCR administration. You can also use the OCRCHECK and OCRDUMP utilities to troubleshoot configuration problems that affect the OCR.
Requirements to check before install Oracle RAC
1)Checking the Hardware Requirements
•Physical memory: At least 1GB RAM.
•Swap space: If RAM is between 1 GB and 2 GB then make swap space to 1.5 times of the size of RAM.
If RMAN is more than 2GB then make swap space to the equal of the size of the RAM.
•Temporary space: At least 400 MB. Typically in /tmp directory.
•Processor type (CPU): Need to be certified with the version of the Oracle software being installed.
•Hard Disk Space: 1.5 GB for oracle database home directory+1.5GB for the ASM home directory+120 oracle clusterware software installation+Two Oracle Clusterware components OCR 256 MB each, or 512 MB total disk space+Three Oracle Clusterware components Voting Disk 256 MB each, or 768 MB total disk space.
•All the nodes in the cluster must have same hardware architecture. However can have machines of different speeds and size in the same cluster.
On *nix system you can check hardware components as follows.
•To determine physical RAM size, # grep MemTotal /proc/meminfo
•To determine the configured swap space, # grep SwapTotal /proc/meminfo
•To determine the amount of disk space available in the /tmp directory, # df -k /tmp
•To determine free disk space on the system, #df -h or #df -k
•To determine whether the system architecture, # grep "model name" /proc/cpuinfo
Checking the Network Requirements
Network Hardware Requirements
•One private interconnect is needed for Oracle Custerware for the use to synchronize each instance's use of the shared resources and for Oracle RAC to interconnect to transmit data blocks that are shared between the instances. Thus each node need at least two network interface cards, or network adapters. One adapter is for the public network and the other adapter is for the private network
•The public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes. For example if in server1/node1, eth0 is the public interface then on server2/node2 eth0 must be the public interface.
•For the public network, each network adapter must support TCP/IP.
•For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP.
•Note UDP is the default interconnect protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware.
•For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. All nodes must be accessible between through private network. You can check the reachability by ping command.
Network Parameter Requirements
If NFS is used for the shared storage, then you must set the values for the NFS buffer size parameters rsize and wsize to at least 16384. Oracle recommends that you use the value 32768.
You can set the value by updating the /etc/fstab file on each node with an entry similar to the following,
clusternode:/vol/DATA/oradata /home/oradata/app nfs
rw,bg,vers=3,tcp,hard,nointr,timeo=600,rsize=32768,wsize=32768,actimeo=0 1 2
IP Address Requirements
•You must have at least three IP addresses available for each node
1.An IP address for the public interface. Interface name should be the name of the node name.
2.An IP address for the private interface. Interface name should be hostname-priv.
3.One virtual IP address with an associated network name. Interface name should be hostname-vip.
•The VIP is on the same subnet as your public interface and it address will be not used currently in the network.
•For public and virtual addresses register with an associated network name in DNS. If you do not have an available DNS, then record the all network names and interface names in the system hosts file, /etc/hosts.
•Identify the interface names and associated IP addresses for all network adapters by running the following command on each node:
# /sbin/ifconfig
Node Time Requirements
Ensure that each member node of the cluster is set as closely as possible to the same date and time. Oracle strongly recommends using the Network Time Protocol (NTP) feature of most operating systems for this purpose.
Verifying the Installed Operating System and Software Requirements
•To determine which distribution and version of Linux is installed, run the following command as the root user:
#cat /etc/issue
•The Linux kernel is updated to fix bugs. These kernel updates are referred to as erratum kernels or errata levels. To determine if the required errata level is installed, use the following procedure as the root user:
#uname -r
•Physical memory: At least 1GB RAM.
•Swap space: If RAM is between 1 GB and 2 GB then make swap space to 1.5 times of the size of RAM.
If RMAN is more than 2GB then make swap space to the equal of the size of the RAM.
•Temporary space: At least 400 MB. Typically in /tmp directory.
•Processor type (CPU): Need to be certified with the version of the Oracle software being installed.
•Hard Disk Space: 1.5 GB for oracle database home directory+1.5GB for the ASM home directory+120 oracle clusterware software installation+Two Oracle Clusterware components OCR 256 MB each, or 512 MB total disk space+Three Oracle Clusterware components Voting Disk 256 MB each, or 768 MB total disk space.
•All the nodes in the cluster must have same hardware architecture. However can have machines of different speeds and size in the same cluster.
On *nix system you can check hardware components as follows.
•To determine physical RAM size, # grep MemTotal /proc/meminfo
•To determine the configured swap space, # grep SwapTotal /proc/meminfo
•To determine the amount of disk space available in the /tmp directory, # df -k /tmp
•To determine free disk space on the system, #df -h or #df -k
•To determine whether the system architecture, # grep "model name" /proc/cpuinfo
Checking the Network Requirements
Network Hardware Requirements
•One private interconnect is needed for Oracle Custerware for the use to synchronize each instance's use of the shared resources and for Oracle RAC to interconnect to transmit data blocks that are shared between the instances. Thus each node need at least two network interface cards, or network adapters. One adapter is for the public network and the other adapter is for the private network
•The public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes. For example if in server1/node1, eth0 is the public interface then on server2/node2 eth0 must be the public interface.
•For the public network, each network adapter must support TCP/IP.
•For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP.
•Note UDP is the default interconnect protocol for Oracle RAC, and TCP is the interconnect protocol for Oracle Clusterware.
•For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. All nodes must be accessible between through private network. You can check the reachability by ping command.
Network Parameter Requirements
If NFS is used for the shared storage, then you must set the values for the NFS buffer size parameters rsize and wsize to at least 16384. Oracle recommends that you use the value 32768.
You can set the value by updating the /etc/fstab file on each node with an entry similar to the following,
clusternode:/vol/DATA/oradata /home/oradata/app nfs
rw,bg,vers=3,tcp,hard,nointr,timeo=600,rsize=32768,wsize=32768,actimeo=0 1 2
IP Address Requirements
•You must have at least three IP addresses available for each node
1.An IP address for the public interface. Interface name should be the name of the node name.
2.An IP address for the private interface. Interface name should be hostname-priv.
3.One virtual IP address with an associated network name. Interface name should be hostname-vip.
•The VIP is on the same subnet as your public interface and it address will be not used currently in the network.
•For public and virtual addresses register with an associated network name in DNS. If you do not have an available DNS, then record the all network names and interface names in the system hosts file, /etc/hosts.
•Identify the interface names and associated IP addresses for all network adapters by running the following command on each node:
# /sbin/ifconfig
Node Time Requirements
Ensure that each member node of the cluster is set as closely as possible to the same date and time. Oracle strongly recommends using the Network Time Protocol (NTP) feature of most operating systems for this purpose.
Verifying the Installed Operating System and Software Requirements
•To determine which distribution and version of Linux is installed, run the following command as the root user:
#cat /etc/issue
•The Linux kernel is updated to fix bugs. These kernel updates are referred to as erratum kernels or errata levels. To determine if the required errata level is installed, use the following procedure as the root user:
#uname -r
Various Tools for Installing, Configuring Oracle RAC
1)Oracle Universal Installer (OUI)- After configuring the pre-installation tasks of the nodes OUI installs the Oracle Clusterware and the Oracle Database software with Oracle RAC. It also can install oracle software on the other nodes using a network connection.
2)Cluster Verification Utility (CVU)- The CVU is a command-line tool. It is very useful to check the nodes for preinstallation as well as postinstallation requirements of the cluster environment. In fact OUI runs the CVU after the Oracle Clusterware installation to verify the environment.
3)Oracle Enterprise Manager- With EM it is easy to configure RAC environments. It has both the Database Control and Grid Control graphical user interfaces (GUIs).
4)Server Control (SRVCTL)- SRVCTL is a command-line interface that you can use to manage an Oracle RAC database from a single point. Using SRVCTL you can start or stop or manage any instance on the cluster.
5)Cluster Ready Services Control (CRSCTL)- CRSCTL is a command-line tool that you can use to manage Oracle Clusterware. You can use CRSCTL to start and stop Oracle Clusterware and to determine the current status of your Oracle Clusterware installation.
6)Oracle Interface Configuration Tool (OIFCFG)- OIFCFG is a command-line tool which you can use to allocate and de-allocate network interfaces to components. You can use OIFCFG to direct components to use specific network interfaces and to retrieve component configuration information.
7)OCR Configuration Tool (OCRCONFIG)- OCRCONFIG is a command-line tool for OCR administration. You can also use the OCRCHECK and OCRDUMP utilities to troubleshoot configuration problems that affect the OCR.
2)Cluster Verification Utility (CVU)- The CVU is a command-line tool. It is very useful to check the nodes for preinstallation as well as postinstallation requirements of the cluster environment. In fact OUI runs the CVU after the Oracle Clusterware installation to verify the environment.
3)Oracle Enterprise Manager- With EM it is easy to configure RAC environments. It has both the Database Control and Grid Control graphical user interfaces (GUIs).
4)Server Control (SRVCTL)- SRVCTL is a command-line interface that you can use to manage an Oracle RAC database from a single point. Using SRVCTL you can start or stop or manage any instance on the cluster.
5)Cluster Ready Services Control (CRSCTL)- CRSCTL is a command-line tool that you can use to manage Oracle Clusterware. You can use CRSCTL to start and stop Oracle Clusterware and to determine the current status of your Oracle Clusterware installation.
6)Oracle Interface Configuration Tool (OIFCFG)- OIFCFG is a command-line tool which you can use to allocate and de-allocate network interfaces to components. You can use OIFCFG to direct components to use specific network interfaces and to retrieve component configuration information.
7)OCR Configuration Tool (OCRCONFIG)- OCRCONFIG is a command-line tool for OCR administration. You can also use the OCRCHECK and OCRDUMP utilities to troubleshoot configuration problems that affect the OCR.
Configure server to install Oracle RAC
Configuring Operating System Users and Groups in All Nodes
On windows you don't need to create separate user to install oracle or don't need to create groups. OUI automatically do it. However on unix you must create the following operating system groups are required if you are installing Oracle RAC for the first time.
•The OSDBA group (typically, dba) -This is the OS user who has SYSDBA privilege.
•The Oracle Inventory group (typically, oinstall) -This group owns all Oracle software installed on the system.
The following operating system users are required
•A user that owns the Oracle software (typically, oracle).
•An unprivileged user (for example, the nobody user on Linux systems).
Create the required groups and users
Do this steps on all nodes of your cluster.
As a root user,
/usr/sbin/groupadd oinstall
/usr/sbin/groupadd dba
useradd -u 200 -g oinstall -G dba -d /home/oracle -r oracle
Change the password by,
passwd oracle
Verify the user by,
id oracle
Note that in this example in stead of using node1 use your original node name and instead of using node2 use your original node name.
Configuring SSH on All Cluster Nodes
When installing Oracle RAC on UNIX and Linux platforms, the software is installed on one node, and OUI uses secure communication to copy the software binary files to the other cluster nodes. OUI uses the Secure Shell (SSH) for the communication.
So you must have configured SSH on all nodes.
•To configure SSH, you must first create Rivest-Shamir-Adleman (RSA) keys and Digital Signature Algorithm (DSA) keys on each cluster node. After you have created the private and public keys, you copy the keys from all cluster node members into an authorized keys file that is identical on each node.
Generating RSA and DSA Keys
1)Log on as oracle user.
2)See whether .ssh directory exist or not. If does not exist then create one.
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
3)Create the RSA-type public and private encryption keys by,
/usr/bin/ssh-keygen -t rsa
This command creates the public key in the /home/oracle/.ssh/id_rsa.pub file and the private key in the /home/oracle/.ssh/id_rsa file.
4)Create the DSA type public and private keys.
/usr/bin/ssh-keygen -t dsa
This command creates the public key in the /home/oracle/.ssh/id_dsa.pub file and the private key in the /home/oracle/.ssh/id_dsa file.
5)Repeat step 1 through 4 in all the nodes.
Adding the Keys to an Authorized Key File
1)Go to .ssh directory
$ cd ~/.ssh
2)Add the RSA and DSA keys to the authorized_keys files.
$ cat id_rsa.pub >>authorized_keys
$ cat id_dsa.pub >>authorized_keys
3)Using SCP copy the authorized_keys file to the oracle user .ssh directory on a remote node.
scp authorized_keys node2:/home/oracle/.ssh/
4)Using SSH, log in to the node where you copied the authorized_keys file, using the passphrase you created. Then change to the .ssh directory, and using the cat command, add the RSA and DSA keys for the second node to authorized_keys file.
ssh node2
Enter passphrase for key '/home/oracle/.ssh/id_rsa':
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
cat id_dsa.pub >> authorized_keys
5)If you have more than 2 nodes in your cluster, repeat step 3 and step 4 for each node you intend to add to your cluster. Copy the most recently updated authorized_keys file to the next node, then add the public keys for that node to the authorized_keys file.
6)After updating the authorized_keys file on all nodes, use SCP to copy the complete authorized_keys file from the last node to be updated to all the other cluster nodes, overwriting the existing version on the other nodes. For example,
scp authorized_keys node1:/home/oracle/.ssh/
Configure SSH User Equivalency on Cluster Member Nodes
1)Log on as a oracle user.
2)Start the SSH agent and load the SSH keys into memory.
$ exec /usr/bin/ssh-agent $SHELL
$ /usr/bin/ssh-add
3)Complete the SSH configuration by using the ssh command to retrieve the date on each node in the cluster.
$ ssh node1 date
$ ssh node2 date
On windows you don't need to create separate user to install oracle or don't need to create groups. OUI automatically do it. However on unix you must create the following operating system groups are required if you are installing Oracle RAC for the first time.
•The OSDBA group (typically, dba) -This is the OS user who has SYSDBA privilege.
•The Oracle Inventory group (typically, oinstall) -This group owns all Oracle software installed on the system.
The following operating system users are required
•A user that owns the Oracle software (typically, oracle).
•An unprivileged user (for example, the nobody user on Linux systems).
Create the required groups and users
Do this steps on all nodes of your cluster.
As a root user,
/usr/sbin/groupadd oinstall
/usr/sbin/groupadd dba
useradd -u 200 -g oinstall -G dba -d /home/oracle -r oracle
Change the password by,
passwd oracle
Verify the user by,
id oracle
Note that in this example in stead of using node1 use your original node name and instead of using node2 use your original node name.
Configuring SSH on All Cluster Nodes
When installing Oracle RAC on UNIX and Linux platforms, the software is installed on one node, and OUI uses secure communication to copy the software binary files to the other cluster nodes. OUI uses the Secure Shell (SSH) for the communication.
So you must have configured SSH on all nodes.
•To configure SSH, you must first create Rivest-Shamir-Adleman (RSA) keys and Digital Signature Algorithm (DSA) keys on each cluster node. After you have created the private and public keys, you copy the keys from all cluster node members into an authorized keys file that is identical on each node.
Generating RSA and DSA Keys
1)Log on as oracle user.
2)See whether .ssh directory exist or not. If does not exist then create one.
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
3)Create the RSA-type public and private encryption keys by,
/usr/bin/ssh-keygen -t rsa
This command creates the public key in the /home/oracle/.ssh/id_rsa.pub file and the private key in the /home/oracle/.ssh/id_rsa file.
4)Create the DSA type public and private keys.
/usr/bin/ssh-keygen -t dsa
This command creates the public key in the /home/oracle/.ssh/id_dsa.pub file and the private key in the /home/oracle/.ssh/id_dsa file.
5)Repeat step 1 through 4 in all the nodes.
Adding the Keys to an Authorized Key File
1)Go to .ssh directory
$ cd ~/.ssh
2)Add the RSA and DSA keys to the authorized_keys files.
$ cat id_rsa.pub >>authorized_keys
$ cat id_dsa.pub >>authorized_keys
3)Using SCP copy the authorized_keys file to the oracle user .ssh directory on a remote node.
scp authorized_keys node2:/home/oracle/.ssh/
4)Using SSH, log in to the node where you copied the authorized_keys file, using the passphrase you created. Then change to the .ssh directory, and using the cat command, add the RSA and DSA keys for the second node to authorized_keys file.
ssh node2
Enter passphrase for key '/home/oracle/.ssh/id_rsa':
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
cat id_dsa.pub >> authorized_keys
5)If you have more than 2 nodes in your cluster, repeat step 3 and step 4 for each node you intend to add to your cluster. Copy the most recently updated authorized_keys file to the next node, then add the public keys for that node to the authorized_keys file.
6)After updating the authorized_keys file on all nodes, use SCP to copy the complete authorized_keys file from the last node to be updated to all the other cluster nodes, overwriting the existing version on the other nodes. For example,
scp authorized_keys node1:/home/oracle/.ssh/
Configure SSH User Equivalency on Cluster Member Nodes
1)Log on as a oracle user.
2)Start the SSH agent and load the SSH keys into memory.
$ exec /usr/bin/ssh-agent $SHELL
$ /usr/bin/ssh-add
3)Complete the SSH configuration by using the ssh command to retrieve the date on each node in the cluster.
$ ssh node1 date
$ ssh node2 date
Configure server to install Oracle RAC
Configuring Operating System Users and Groups in All Nodes
On windows you don't need to create separate user to install oracle or don't need to create groups. OUI automatically do it. However on unix you must create the following operating system groups are required if you are installing Oracle RAC for the first time.
•The OSDBA group (typically, dba) -This is the OS user who has SYSDBA privilege.
•The Oracle Inventory group (typically, oinstall) -This group owns all Oracle software installed on the system.
The following operating system users are required
•A user that owns the Oracle software (typically, oracle).
•An unprivileged user (for example, the nobody user on Linux systems).
Create the required groups and users
Do this steps on all nodes of your cluster.
As a root user,
/usr/sbin/groupadd oinstall
/usr/sbin/groupadd dba
useradd -u 200 -g oinstall -G dba -d /home/oracle -r oracle
Change the password by,
passwd oracle
Verify the user by,
id oracle
Note that in this example in stead of using node1 use your original node name and instead of using node2 use your original node name.
Configuring SSH on All Cluster Nodes
When installing Oracle RAC on UNIX and Linux platforms, the software is installed on one node, and OUI uses secure communication to copy the software binary files to the other cluster nodes. OUI uses the Secure Shell (SSH) for the communication.
So you must have configured SSH on all nodes.
•To configure SSH, you must first create Rivest-Shamir-Adleman (RSA) keys and Digital Signature Algorithm (DSA) keys on each cluster node. After you have created the private and public keys, you copy the keys from all cluster node members into an authorized keys file that is identical on each node.
Generating RSA and DSA Keys
1)Log on as oracle user.
2)See whether .ssh directory exist or not. If does not exist then create one.
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
3)Create the RSA-type public and private encryption keys by,
/usr/bin/ssh-keygen -t rsa
This command creates the public key in the /home/oracle/.ssh/id_rsa.pub file and the private key in the /home/oracle/.ssh/id_rsa file.
4)Create the DSA type public and private keys.
/usr/bin/ssh-keygen -t dsa
This command creates the public key in the /home/oracle/.ssh/id_dsa.pub file and the private key in the /home/oracle/.ssh/id_dsa file.
5)Repeat step 1 through 4 in all the nodes.
Adding the Keys to an Authorized Key File
1)Go to .ssh directory
$ cd ~/.ssh
2)Add the RSA and DSA keys to the authorized_keys files.
$ cat id_rsa.pub >>authorized_keys
$ cat id_dsa.pub >>authorized_keys
3)Using SCP copy the authorized_keys file to the oracle user .ssh directory on a remote node.
scp authorized_keys node2:/home/oracle/.ssh/
4)Using SSH, log in to the node where you copied the authorized_keys file, using the passphrase you created. Then change to the .ssh directory, and using the cat command, add the RSA and DSA keys for the second node to authorized_keys file.
ssh node2
Enter passphrase for key '/home/oracle/.ssh/id_rsa':
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
cat id_dsa.pub >> authorized_keys
5)If you have more than 2 nodes in your cluster, repeat step 3 and step 4 for each node you intend to add to your cluster. Copy the most recently updated authorized_keys file to the next node, then add the public keys for that node to the authorized_keys file.
6)After updating the authorized_keys file on all nodes, use SCP to copy the complete authorized_keys file from the last node to be updated to all the other cluster nodes, overwriting the existing version on the other nodes. For example,
scp authorized_keys node1:/home/oracle/.ssh/
Configure SSH User Equivalency on Cluster Member Nodes
1)Log on as a oracle user.
2)Start the SSH agent and load the SSH keys into memory.
$ exec /usr/bin/ssh-agent $SHELL
$ /usr/bin/ssh-add
3)Complete the SSH configuration by using the ssh command to retrieve the date on each node in the cluster.
$ ssh node1 date
$ ssh node2 date
On windows you don't need to create separate user to install oracle or don't need to create groups. OUI automatically do it. However on unix you must create the following operating system groups are required if you are installing Oracle RAC for the first time.
•The OSDBA group (typically, dba) -This is the OS user who has SYSDBA privilege.
•The Oracle Inventory group (typically, oinstall) -This group owns all Oracle software installed on the system.
The following operating system users are required
•A user that owns the Oracle software (typically, oracle).
•An unprivileged user (for example, the nobody user on Linux systems).
Create the required groups and users
Do this steps on all nodes of your cluster.
As a root user,
/usr/sbin/groupadd oinstall
/usr/sbin/groupadd dba
useradd -u 200 -g oinstall -G dba -d /home/oracle -r oracle
Change the password by,
passwd oracle
Verify the user by,
id oracle
Note that in this example in stead of using node1 use your original node name and instead of using node2 use your original node name.
Configuring SSH on All Cluster Nodes
When installing Oracle RAC on UNIX and Linux platforms, the software is installed on one node, and OUI uses secure communication to copy the software binary files to the other cluster nodes. OUI uses the Secure Shell (SSH) for the communication.
So you must have configured SSH on all nodes.
•To configure SSH, you must first create Rivest-Shamir-Adleman (RSA) keys and Digital Signature Algorithm (DSA) keys on each cluster node. After you have created the private and public keys, you copy the keys from all cluster node members into an authorized keys file that is identical on each node.
Generating RSA and DSA Keys
1)Log on as oracle user.
2)See whether .ssh directory exist or not. If does not exist then create one.
$ mkdir ~/.ssh
$ chmod 700 ~/.ssh
3)Create the RSA-type public and private encryption keys by,
/usr/bin/ssh-keygen -t rsa
This command creates the public key in the /home/oracle/.ssh/id_rsa.pub file and the private key in the /home/oracle/.ssh/id_rsa file.
4)Create the DSA type public and private keys.
/usr/bin/ssh-keygen -t dsa
This command creates the public key in the /home/oracle/.ssh/id_dsa.pub file and the private key in the /home/oracle/.ssh/id_dsa file.
5)Repeat step 1 through 4 in all the nodes.
Adding the Keys to an Authorized Key File
1)Go to .ssh directory
$ cd ~/.ssh
2)Add the RSA and DSA keys to the authorized_keys files.
$ cat id_rsa.pub >>authorized_keys
$ cat id_dsa.pub >>authorized_keys
3)Using SCP copy the authorized_keys file to the oracle user .ssh directory on a remote node.
scp authorized_keys node2:/home/oracle/.ssh/
4)Using SSH, log in to the node where you copied the authorized_keys file, using the passphrase you created. Then change to the .ssh directory, and using the cat command, add the RSA and DSA keys for the second node to authorized_keys file.
ssh node2
Enter passphrase for key '/home/oracle/.ssh/id_rsa':
cd ~/.ssh
cat id_rsa.pub >> authorized_keys
cat id_dsa.pub >> authorized_keys
5)If you have more than 2 nodes in your cluster, repeat step 3 and step 4 for each node you intend to add to your cluster. Copy the most recently updated authorized_keys file to the next node, then add the public keys for that node to the authorized_keys file.
6)After updating the authorized_keys file on all nodes, use SCP to copy the complete authorized_keys file from the last node to be updated to all the other cluster nodes, overwriting the existing version on the other nodes. For example,
scp authorized_keys node1:/home/oracle/.ssh/
Configure SSH User Equivalency on Cluster Member Nodes
1)Log on as a oracle user.
2)Start the SSH agent and load the SSH keys into memory.
$ exec /usr/bin/ssh-agent $SHELL
$ /usr/bin/ssh-add
3)Complete the SSH configuration by using the ssh command to retrieve the date on each node in the cluster.
$ ssh node1 date
$ ssh node2 date
Pre-Installation RAC environement setup check with CVU
Checking the Network Setup with CVU
To verify node connectivity among all nodes in your cluster use following syntax as an oracle user,
/mount_point/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command.
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node1,node2 -verbose
Checking the Hardware and Operating System Setup with CV
As an oracle user use the following command syntax to start Cluster Verification Utility (CVU) stage verification to check hardware and OS setup:
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node1,node2
Checking the Operating System Requirements Setup with CVU
To check if your system meets the operating system requirement pre-installation tasks use the following syntax,
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node_list -p {crs|database}
-osdba osdba_group -orainv orainv_group -verbose
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom with the OSDBA dba and Oracle inventory group oinstall then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node1,node2 -p crs -osdba crs -orainv oinstall
To verify node connectivity among all nodes in your cluster use following syntax as an oracle user,
/mount_point/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command.
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node1,node2 -verbose
Checking the Hardware and Operating System Setup with CV
As an oracle user use the following command syntax to start Cluster Verification Utility (CVU) stage verification to check hardware and OS setup:
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node1,node2
Checking the Operating System Requirements Setup with CVU
To check if your system meets the operating system requirement pre-installation tasks use the following syntax,
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node_list -p {crs|database}
-osdba osdba_group -orainv orainv_group -verbose
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom with the OSDBA dba and Oracle inventory group oinstall then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node1,node2 -p crs -osdba crs -orainv oinstall
Pre-Installation RAC environement setup check with CVU
Checking the Network Setup with CVU
To verify node connectivity among all nodes in your cluster use following syntax as an oracle user,
/mount_point/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command.
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node1,node2 -verbose
Checking the Hardware and Operating System Setup with CV
As an oracle user use the following command syntax to start Cluster Verification Utility (CVU) stage verification to check hardware and OS setup:
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node1,node2
Checking the Operating System Requirements Setup with CVU
To check if your system meets the operating system requirement pre-installation tasks use the following syntax,
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node_list -p {crs|database}
-osdba osdba_group -orainv orainv_group -verbose
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom with the OSDBA dba and Oracle inventory group oinstall then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node1,node2 -p crs -osdba crs -orainv oinstall
To verify node connectivity among all nodes in your cluster use following syntax as an oracle user,
/mount_point/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command.
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node1,node2 -verbose
Checking the Hardware and Operating System Setup with CV
As an oracle user use the following command syntax to start Cluster Verification Utility (CVU) stage verification to check hardware and OS setup:
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node_list [-verbose]
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node1,node2
Checking the Operating System Requirements Setup with CVU
To check if your system meets the operating system requirement pre-installation tasks use the following syntax,
/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node_list -p {crs|database}
-osdba osdba_group -orainv orainv_group -verbose
If you have two nodes node1 and node2 and your mountpoint is /dev/cdrom with the OSDBA dba and Oracle inventory group oinstall then enter following command,
/dev/cdrom/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node1,node2 -p crs -osdba crs -orainv oinstall
Configure Network for oracle RAC installation
1)Determine public node names, private node names, and virtual node names for each node in the cluster.
For the public node name, use the primary host name of each node.
In my environment my one node name is racnode-1 and another node name is racnode-2. So I determine the public, private and virtual node name for node racnode-1 as racnode-1, racnode-1-priv and racnode-1-vip accordingly.
Similarly on racnode-2 as racnode-2, racnode-2-priv and racnode-2-vip accordingly.
2)Identify the interface names and associated IP addresses.
On racnode-1,
#/sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B0:46:7D
inet addr:192.168.1.91 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb0:467d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:96114 errors:0 dropped:0 overruns:0 frame:0
TX packets:1769 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7704446 (7.3 MiB) TX bytes:215309 (210.2 KiB)
eth1 Link encap:Ethernet HWaddr 00:11:3B:0A:42:DC
inet addr:192.168.150.30 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:3bff:fe0a:42dc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27116 errors:0 dropped:0 overruns:0 frame:0
TX packets:48666 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7171014 (6.8 MiB) TX bytes:4353054 (4.1 MiB)
Interrupt:58 Base address:0xa000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1201 errors:0 dropped:0 overruns:0 frame:0
TX packets:1201 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3069384 (2.9 MiB) TX bytes:3069384 (2.9 MiB)
On racnode-2,
[root@racnode-2 ~]# /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B2:EB:27
inet addr:192.168.1.92 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb2:eb27/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:104567 errors:0 dropped:0 overruns:0 frame:0
TX packets:1424 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8546472 (8.1 MiB) TX bytes:197747 (193.1 KiB)
Interrupt:169 Base address:0x6200
eth1 Link encap:Ethernet HWaddr 00:11:95:1F:02:09
inet addr:192.168.150.20 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:95ff:fe1f:209/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27495 errors:0 dropped:0 overruns:0 frame:0
TX packets:50130 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6697016 (6.3 MiB) TX bytes:4488642 (4.2 MiB)
Interrupt:177 Base address:0x8100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1203 errors:0 dropped:0 overruns:0 frame:0
TX packets:1203 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2857788 (2.7 MiB) TX bytes:2857788 (2.7 MiB)
Here I will use network interface eth0 for the public network and eth1 is for the private network. So, on racnode-1 IP address on public network will be 192.168.1.91 and IP address on private network will be 192.168.150.30. Similarly on racnode-2 IP address on public network will be 192.168.1.92 and IP address on private network is 192.168.150.20
3)On each node in the cluster, determine a third IP address that will serve as a virtual IP address which must not use currently in the network and its subnet must be same as on public network. I determine virtual IP address for racnode-1 as 192.168.1.95 and on racnode-2 as 192.168.1.96
4)After completing the network configuration, the IP address and network interface configuration should be as follows,
5)In all nodes modify /etc/hosts file so that they contain the host IP addresses, virtual IP addresses, and private network IP addresses from both nodes in the cluster, as follows,
6)Verify the network configuration by using the ping command to test the connection.
ping -c 3 racnode-2
ping -c 3 racnode-2-vip
ping -c 3 racnode-2-priv
ping -c 3 racnode-1
ping -c 3 racnode-1-vip
ping -c 3 racnode-1-priv
Note that in this stage ping to the virtual ip address will fail until after Oracle Clusterware is installed and running. But ping to your private and public network address should work.
For the public node name, use the primary host name of each node.
In my environment my one node name is racnode-1 and another node name is racnode-2. So I determine the public, private and virtual node name for node racnode-1 as racnode-1, racnode-1-priv and racnode-1-vip accordingly.
Similarly on racnode-2 as racnode-2, racnode-2-priv and racnode-2-vip accordingly.
2)Identify the interface names and associated IP addresses.
On racnode-1,
#/sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B0:46:7D
inet addr:192.168.1.91 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb0:467d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:96114 errors:0 dropped:0 overruns:0 frame:0
TX packets:1769 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7704446 (7.3 MiB) TX bytes:215309 (210.2 KiB)
eth1 Link encap:Ethernet HWaddr 00:11:3B:0A:42:DC
inet addr:192.168.150.30 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:3bff:fe0a:42dc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27116 errors:0 dropped:0 overruns:0 frame:0
TX packets:48666 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7171014 (6.8 MiB) TX bytes:4353054 (4.1 MiB)
Interrupt:58 Base address:0xa000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1201 errors:0 dropped:0 overruns:0 frame:0
TX packets:1201 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3069384 (2.9 MiB) TX bytes:3069384 (2.9 MiB)
On racnode-2,
[root@racnode-2 ~]# /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B2:EB:27
inet addr:192.168.1.92 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb2:eb27/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:104567 errors:0 dropped:0 overruns:0 frame:0
TX packets:1424 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8546472 (8.1 MiB) TX bytes:197747 (193.1 KiB)
Interrupt:169 Base address:0x6200
eth1 Link encap:Ethernet HWaddr 00:11:95:1F:02:09
inet addr:192.168.150.20 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:95ff:fe1f:209/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27495 errors:0 dropped:0 overruns:0 frame:0
TX packets:50130 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6697016 (6.3 MiB) TX bytes:4488642 (4.2 MiB)
Interrupt:177 Base address:0x8100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1203 errors:0 dropped:0 overruns:0 frame:0
TX packets:1203 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2857788 (2.7 MiB) TX bytes:2857788 (2.7 MiB)
Here I will use network interface eth0 for the public network and eth1 is for the private network. So, on racnode-1 IP address on public network will be 192.168.1.91 and IP address on private network will be 192.168.150.30. Similarly on racnode-2 IP address on public network will be 192.168.1.92 and IP address on private network is 192.168.150.20
3)On each node in the cluster, determine a third IP address that will serve as a virtual IP address which must not use currently in the network and its subnet must be same as on public network. I determine virtual IP address for racnode-1 as 192.168.1.95 and on racnode-2 as 192.168.1.96
4)After completing the network configuration, the IP address and network interface configuration should be as follows,
Node(Hostname) Node Name Type IP Address
racnode-1 racnode-1 public 192.168.1.91
racnode-1 racnode-1-vip virtual 192.168.1.95
racnode-1 racnode-1-priv private 192.168.150.30
racnode-2 racnode-2 public 192.168.1.92
racnode-2 racnode-2-vip virtual 192.168.1.96
racnode-2 racnode-2-priv private 192.168.150.20
5)In all nodes modify /etc/hosts file so that they contain the host IP addresses, virtual IP addresses, and private network IP addresses from both nodes in the cluster, as follows,
[root@racnode-2 ~]# vi /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.1.91 racnode-1
192.168.1.95 racnode-1-vip
192.168.150.30 racnode-1-priv
192.168.1.92 racnode-2
192.168.1.96 racnode-2-vip
192.168.150.20 racnode-2-priv
6)Verify the network configuration by using the ping command to test the connection.
ping -c 3 racnode-2
ping -c 3 racnode-2-vip
ping -c 3 racnode-2-priv
ping -c 3 racnode-1
ping -c 3 racnode-1-vip
ping -c 3 racnode-1-priv
Note that in this stage ping to the virtual ip address will fail until after Oracle Clusterware is installed and running. But ping to your private and public network address should work.
Configure Network for oracle RAC installation
1)Determine public node names, private node names, and virtual node names for each node in the cluster.
For the public node name, use the primary host name of each node.
In my environment my one node name is racnode-1 and another node name is racnode-2. So I determine the public, private and virtual node name for node racnode-1 as racnode-1, racnode-1-priv and racnode-1-vip accordingly.
Similarly on racnode-2 as racnode-2, racnode-2-priv and racnode-2-vip accordingly.
2)Identify the interface names and associated IP addresses.
On racnode-1,
#/sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B0:46:7D
inet addr:192.168.1.91 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb0:467d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:96114 errors:0 dropped:0 overruns:0 frame:0
TX packets:1769 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7704446 (7.3 MiB) TX bytes:215309 (210.2 KiB)
eth1 Link encap:Ethernet HWaddr 00:11:3B:0A:42:DC
inet addr:192.168.150.30 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:3bff:fe0a:42dc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27116 errors:0 dropped:0 overruns:0 frame:0
TX packets:48666 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7171014 (6.8 MiB) TX bytes:4353054 (4.1 MiB)
Interrupt:58 Base address:0xa000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1201 errors:0 dropped:0 overruns:0 frame:0
TX packets:1201 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3069384 (2.9 MiB) TX bytes:3069384 (2.9 MiB)
On racnode-2,
[root@racnode-2 ~]# /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B2:EB:27
inet addr:192.168.1.92 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb2:eb27/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:104567 errors:0 dropped:0 overruns:0 frame:0
TX packets:1424 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8546472 (8.1 MiB) TX bytes:197747 (193.1 KiB)
Interrupt:169 Base address:0x6200
eth1 Link encap:Ethernet HWaddr 00:11:95:1F:02:09
inet addr:192.168.150.20 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:95ff:fe1f:209/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27495 errors:0 dropped:0 overruns:0 frame:0
TX packets:50130 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6697016 (6.3 MiB) TX bytes:4488642 (4.2 MiB)
Interrupt:177 Base address:0x8100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1203 errors:0 dropped:0 overruns:0 frame:0
TX packets:1203 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2857788 (2.7 MiB) TX bytes:2857788 (2.7 MiB)
Here I will use network interface eth0 for the public network and eth1 is for the private network. So, on racnode-1 IP address on public network will be 192.168.1.91 and IP address on private network will be 192.168.150.30. Similarly on racnode-2 IP address on public network will be 192.168.1.92 and IP address on private network is 192.168.150.20
3)On each node in the cluster, determine a third IP address that will serve as a virtual IP address which must not use currently in the network and its subnet must be same as on public network. I determine virtual IP address for racnode-1 as 192.168.1.95 and on racnode-2 as 192.168.1.96
4)After completing the network configuration, the IP address and network interface configuration should be as follows,
5)In all nodes modify /etc/hosts file so that they contain the host IP addresses, virtual IP addresses, and private network IP addresses from both nodes in the cluster, as follows,
6)Verify the network configuration by using the ping command to test the connection.
ping -c 3 racnode-2
ping -c 3 racnode-2-vip
ping -c 3 racnode-2-priv
ping -c 3 racnode-1
ping -c 3 racnode-1-vip
ping -c 3 racnode-1-priv
Note that in this stage ping to the virtual ip address will fail until after Oracle Clusterware is installed and running. But ping to your private and public network address should work.
For the public node name, use the primary host name of each node.
In my environment my one node name is racnode-1 and another node name is racnode-2. So I determine the public, private and virtual node name for node racnode-1 as racnode-1, racnode-1-priv and racnode-1-vip accordingly.
Similarly on racnode-2 as racnode-2, racnode-2-priv and racnode-2-vip accordingly.
2)Identify the interface names and associated IP addresses.
On racnode-1,
#/sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B0:46:7D
inet addr:192.168.1.91 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb0:467d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:96114 errors:0 dropped:0 overruns:0 frame:0
TX packets:1769 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7704446 (7.3 MiB) TX bytes:215309 (210.2 KiB)
eth1 Link encap:Ethernet HWaddr 00:11:3B:0A:42:DC
inet addr:192.168.150.30 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:3bff:fe0a:42dc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27116 errors:0 dropped:0 overruns:0 frame:0
TX packets:48666 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7171014 (6.8 MiB) TX bytes:4353054 (4.1 MiB)
Interrupt:58 Base address:0xa000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1201 errors:0 dropped:0 overruns:0 frame:0
TX packets:1201 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:3069384 (2.9 MiB) TX bytes:3069384 (2.9 MiB)
On racnode-2,
[root@racnode-2 ~]# /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:76:B2:EB:27
inet addr:192.168.1.92 Bcast:192.168.15.255 Mask:255.255.240.0
inet6 addr: fe80::216:76ff:feb2:eb27/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:104567 errors:0 dropped:0 overruns:0 frame:0
TX packets:1424 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8546472 (8.1 MiB) TX bytes:197747 (193.1 KiB)
Interrupt:169 Base address:0x6200
eth1 Link encap:Ethernet HWaddr 00:11:95:1F:02:09
inet addr:192.168.150.20 Bcast:192.168.150.255 Mask:255.255.255.0
inet6 addr: fe80::211:95ff:fe1f:209/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:27495 errors:0 dropped:0 overruns:0 frame:0
TX packets:50130 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6697016 (6.3 MiB) TX bytes:4488642 (4.2 MiB)
Interrupt:177 Base address:0x8100
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:1203 errors:0 dropped:0 overruns:0 frame:0
TX packets:1203 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2857788 (2.7 MiB) TX bytes:2857788 (2.7 MiB)
Here I will use network interface eth0 for the public network and eth1 is for the private network. So, on racnode-1 IP address on public network will be 192.168.1.91 and IP address on private network will be 192.168.150.30. Similarly on racnode-2 IP address on public network will be 192.168.1.92 and IP address on private network is 192.168.150.20
3)On each node in the cluster, determine a third IP address that will serve as a virtual IP address which must not use currently in the network and its subnet must be same as on public network. I determine virtual IP address for racnode-1 as 192.168.1.95 and on racnode-2 as 192.168.1.96
4)After completing the network configuration, the IP address and network interface configuration should be as follows,
Node(Hostname) Node Name Type IP Address
racnode-1 racnode-1 public 192.168.1.91
racnode-1 racnode-1-vip virtual 192.168.1.95
racnode-1 racnode-1-priv private 192.168.150.30
racnode-2 racnode-2 public 192.168.1.92
racnode-2 racnode-2-vip virtual 192.168.1.96
racnode-2 racnode-2-priv private 192.168.150.20
5)In all nodes modify /etc/hosts file so that they contain the host IP addresses, virtual IP addresses, and private network IP addresses from both nodes in the cluster, as follows,
[root@racnode-2 ~]# vi /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.1.91 racnode-1
192.168.1.95 racnode-1-vip
192.168.150.30 racnode-1-priv
192.168.1.92 racnode-2
192.168.1.96 racnode-2-vip
192.168.150.20 racnode-2-priv
6)Verify the network configuration by using the ping command to test the connection.
ping -c 3 racnode-2
ping -c 3 racnode-2-vip
ping -c 3 racnode-2-priv
ping -c 3 racnode-1
ping -c 3 racnode-1-vip
ping -c 3 racnode-1-priv
Note that in this stage ping to the virtual ip address will fail until after Oracle Clusterware is installed and running. But ping to your private and public network address should work.
Configure Shared Storage in Oracle RAC installation
An Oracle RAC database is logically or physically shared everything database. All
datafiles, control files, PFILEs, and redo log files in Oracle RAC environments must
reside on shared disks. This is so that all of the cluster database instances
can access them.
IN that shared disk you can use the following file storage options for Oracle RAC database.
1)ASM: Which stands for Automatic Storage Management. And oracle recommends ASM.
2)Oracle Cluster File System (OCFS): OCFS is available for Linux and Windows
platforms, or a third-party cluster file system that is certified for Oracle RAC.
3)A network file system: This is not supported on AIX, POWER, or on IBM zSeries-based Linux.
4)Raw devices
In this post I will show how to configure RAW device to install oracle RAC.
Before discussing it is good to know the difference between single instance database and RAC instance database.
Oracle RAC databases differ architecturally from Oracle RAC single-instance Oracle
databases in that each Oracle RAC database instance also has,
At least one additional thread of redo for each instance
An instance-specific undo tablespace
To configure RAW storage device as a shared storage in RAC database first you have to think about clusterware part. Oracle clusterware has two components OCR and voting disk.
Before you install Oracle Clusterware, you will need to configure 5 raw partitions,
each 256 MB in size, one storing the Oracle Cluster Registry (OCR), another one for duplicate OCR file on a different disk, referred to as the OCR mirror, and three voting disks.
And then If you plan to use raw devices for storing the database files, you will need to create additional raw partitions for each tablespace, online redo log file, control file, SPFILE and password file.
Below is the procedure of how to configure RAW device.
Step 1:
Be sure that you have storage box purchased that is shared on both nodes.
Now to see available shared disk on your system as a root user on racnode-1 use,
# /sbin/fdisk -l
It gave me output like,
.
.
.
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdd doesn't contain a valid partition table
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sde doesn't contain a valid partition table
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdf doesn't contain a valid partition table
Since I have sdb, sdc, sdd, sdf, sde disks on my storage box so all five are shown.
Step 2:
Now as a root user on racnode-1 create two raw partitions 256 MB in size for the OCR and its mirror, and three partitions 256 MB in size for the Oracle Clusterware voting disks. To store first OCR partition I use sdb for this. So, I use
# /sbin/fdisk /dev/sdb
After issuing it,
Use the p command to list the partition table of the device.
Use the n command to create a partition.
After you have created the required partitions on this device, use the w command to write the modified partition table to the device.
Here is the example of creating my first 256 raw partition of sdb disk.
The bolded one is input by me.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-19581, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-19581, default 19581): +256M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode-1 ~]#
Similarly configure other 256MB partition from rest of the disk. I created raw 256MB partition from /dev/sdc for OCR mirror. From the three disks /dev/sdd, /dev/sde and /dev/sdf create each 256MB partition for voting disk.
That is,
#fdisk /dev/sdc
#fdisk /dev/sdd
#fdisk /dev/sde
#fdisk /dev/sdf
After you configure your all 5 disks on your system your fdisk -l output will look like below.
[root@racnode-1 ~]# fdisk -l
Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 1275 10241406 83 Linux
/dev/sda2 1276 3723 19663560 83 Linux
/dev/sda3 3724 6145 19454715 83 Linux
/dev/sda4 6146 9729 28788480 5 Extended
/dev/sda5 6146 8440 18434556 83 Linux
/dev/sda6 8441 9205 6144831 83 Linux
/dev/sda7 9206 9727 4192933+ 82 Linux swap / Solaris
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 32 257008+ 83 Linux
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 32 257008+ 83 Linux
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 32 257008+ 83 Linux
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 32 257008+ 83 Linux
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdf1 1 32 257008+ 83 Linux
Step 3:
As the root user on racnode-1, edit the /etc/sysconfig/rawdevices file and add the mappings for the raw devices used by Oracle Clusterware.
[root@racnode-1 ~]# vi /etc/sysconfig/rawdevices
#OCR Devices
/dev/sdb /dev/sdb1
/dev/sdc /dev/sdc1
#Voting Disk Devices
/dev/sdd /dev/sdd1
/dev/sde /dev/sde1
/dev/sdf /dev/sdf1
Step 4: If you are using RHL4 then as the root user, on the node racnode1, enable the raw devices so that the mappings become effective at the operating system level using,
[root@racnode-1 ~]# service /etc/sysconfig/rawdevices start
If you are using RHL5 then you will get
[root@racnode-1 ~]# service rawdevices start
rawdevices: unrecognized service
Step 5:
In the racnode-2 inform the OS of partition table changes. This is done by partprobe program that informs the operating system kernel of partition table changes, by requesting that the operating system re-read the partition table.
#/sbin/partprobe /dev/sdb
/sbin/partprobe /dev/sdc
/sbin/partprobe /dev/sdd
/sbin/partprobe /dev/sde
/sbin/partprobe /dev/sdf
This forces the operating system on the other node in the cluster to refresh its picture of the shared disk partitions.
Step 6:
On RHL4 for each node make an entry in /etc/sysconfig/rawdevices and use following command,
service rawdevices start
Step 7:
As the root user, on each node in the cluster, enter commands similar to the following to set the owner, group, and permissions on the newly created device files:
chown root:oinstall /dev/raw/raw1
chown root:oinstall /dev/raw/raw2
chown oracle:oinstall /dev/raw/raw3
chown oracle:oinstall /dev/raw/raw4
chown oracle:oinstall /dev/raw/raw5
chmod 640 /dev/raw/raw1
chmod 640 /dev/raw/raw2
chmod 640 /dev/raw/raw3
chmod 640 /dev/raw/raw4
chmod 640 /dev/raw/raw5
datafiles, control files, PFILEs, and redo log files in Oracle RAC environments must
reside on shared disks. This is so that all of the cluster database instances
can access them.
IN that shared disk you can use the following file storage options for Oracle RAC database.
1)ASM: Which stands for Automatic Storage Management. And oracle recommends ASM.
2)Oracle Cluster File System (OCFS): OCFS is available for Linux and Windows
platforms, or a third-party cluster file system that is certified for Oracle RAC.
3)A network file system: This is not supported on AIX, POWER, or on IBM zSeries-based Linux.
4)Raw devices
In this post I will show how to configure RAW device to install oracle RAC.
Before discussing it is good to know the difference between single instance database and RAC instance database.
Oracle RAC databases differ architecturally from Oracle RAC single-instance Oracle
databases in that each Oracle RAC database instance also has,
At least one additional thread of redo for each instance
An instance-specific undo tablespace
To configure RAW storage device as a shared storage in RAC database first you have to think about clusterware part. Oracle clusterware has two components OCR and voting disk.
Before you install Oracle Clusterware, you will need to configure 5 raw partitions,
each 256 MB in size, one storing the Oracle Cluster Registry (OCR), another one for duplicate OCR file on a different disk, referred to as the OCR mirror, and three voting disks.
And then If you plan to use raw devices for storing the database files, you will need to create additional raw partitions for each tablespace, online redo log file, control file, SPFILE and password file.
Below is the procedure of how to configure RAW device.
Step 1:
Be sure that you have storage box purchased that is shared on both nodes.
Now to see available shared disk on your system as a root user on racnode-1 use,
# /sbin/fdisk -l
It gave me output like,
.
.
.
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdd doesn't contain a valid partition table
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sde doesn't contain a valid partition table
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdf doesn't contain a valid partition table
Since I have sdb, sdc, sdd, sdf, sde disks on my storage box so all five are shown.
Step 2:
Now as a root user on racnode-1 create two raw partitions 256 MB in size for the OCR and its mirror, and three partitions 256 MB in size for the Oracle Clusterware voting disks. To store first OCR partition I use sdb for this. So, I use
# /sbin/fdisk /dev/sdb
After issuing it,
Use the p command to list the partition table of the device.
Use the n command to create a partition.
After you have created the required partitions on this device, use the w command to write the modified partition table to the device.
Here is the example of creating my first 256 raw partition of sdb disk.
The bolded one is input by me.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-19581, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-19581, default 19581): +256M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode-1 ~]#
Similarly configure other 256MB partition from rest of the disk. I created raw 256MB partition from /dev/sdc for OCR mirror. From the three disks /dev/sdd, /dev/sde and /dev/sdf create each 256MB partition for voting disk.
That is,
#fdisk /dev/sdc
#fdisk /dev/sdd
#fdisk /dev/sde
#fdisk /dev/sdf
After you configure your all 5 disks on your system your fdisk -l output will look like below.
[root@racnode-1 ~]# fdisk -l
Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 1275 10241406 83 Linux
/dev/sda2 1276 3723 19663560 83 Linux
/dev/sda3 3724 6145 19454715 83 Linux
/dev/sda4 6146 9729 28788480 5 Extended
/dev/sda5 6146 8440 18434556 83 Linux
/dev/sda6 8441 9205 6144831 83 Linux
/dev/sda7 9206 9727 4192933+ 82 Linux swap / Solaris
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 32 257008+ 83 Linux
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 32 257008+ 83 Linux
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 32 257008+ 83 Linux
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 32 257008+ 83 Linux
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdf1 1 32 257008+ 83 Linux
Step 3:
As the root user on racnode-1, edit the /etc/sysconfig/rawdevices file and add the mappings for the raw devices used by Oracle Clusterware.
[root@racnode-1 ~]# vi /etc/sysconfig/rawdevices
#OCR Devices
/dev/sdb /dev/sdb1
/dev/sdc /dev/sdc1
#Voting Disk Devices
/dev/sdd /dev/sdd1
/dev/sde /dev/sde1
/dev/sdf /dev/sdf1
Step 4: If you are using RHL4 then as the root user, on the node racnode1, enable the raw devices so that the mappings become effective at the operating system level using,
[root@racnode-1 ~]# service /etc/sysconfig/rawdevices start
If you are using RHL5 then you will get
[root@racnode-1 ~]# service rawdevices start
rawdevices: unrecognized service
Step 5:
In the racnode-2 inform the OS of partition table changes. This is done by partprobe program that informs the operating system kernel of partition table changes, by requesting that the operating system re-read the partition table.
#/sbin/partprobe /dev/sdb
/sbin/partprobe /dev/sdc
/sbin/partprobe /dev/sdd
/sbin/partprobe /dev/sde
/sbin/partprobe /dev/sdf
This forces the operating system on the other node in the cluster to refresh its picture of the shared disk partitions.
Step 6:
On RHL4 for each node make an entry in /etc/sysconfig/rawdevices and use following command,
service rawdevices start
Step 7:
As the root user, on each node in the cluster, enter commands similar to the following to set the owner, group, and permissions on the newly created device files:
chown root:oinstall /dev/raw/raw1
chown root:oinstall /dev/raw/raw2
chown oracle:oinstall /dev/raw/raw3
chown oracle:oinstall /dev/raw/raw4
chown oracle:oinstall /dev/raw/raw5
chmod 640 /dev/raw/raw1
chmod 640 /dev/raw/raw2
chmod 640 /dev/raw/raw3
chmod 640 /dev/raw/raw4
chmod 640 /dev/raw/raw5
Configure Shared Storage in Oracle RAC installation
An Oracle RAC database is logically or physically shared everything database. All
datafiles, control files, PFILEs, and redo log files in Oracle RAC environments must
reside on shared disks. This is so that all of the cluster database instances
can access them.
IN that shared disk you can use the following file storage options for Oracle RAC database.
1)ASM: Which stands for Automatic Storage Management. And oracle recommends ASM.
2)Oracle Cluster File System (OCFS): OCFS is available for Linux and Windows
platforms, or a third-party cluster file system that is certified for Oracle RAC.
3)A network file system: This is not supported on AIX, POWER, or on IBM zSeries-based Linux.
4)Raw devices
In this post I will show how to configure RAW device to install oracle RAC.
Before discussing it is good to know the difference between single instance database and RAC instance database.
Oracle RAC databases differ architecturally from Oracle RAC single-instance Oracle
databases in that each Oracle RAC database instance also has,
At least one additional thread of redo for each instance
An instance-specific undo tablespace
To configure RAW storage device as a shared storage in RAC database first you have to think about clusterware part. Oracle clusterware has two components OCR and voting disk.
Before you install Oracle Clusterware, you will need to configure 5 raw partitions,
each 256 MB in size, one storing the Oracle Cluster Registry (OCR), another one for duplicate OCR file on a different disk, referred to as the OCR mirror, and three voting disks.
And then If you plan to use raw devices for storing the database files, you will need to create additional raw partitions for each tablespace, online redo log file, control file, SPFILE and password file.
Below is the procedure of how to configure RAW device.
Step 1:
Be sure that you have storage box purchased that is shared on both nodes.
Now to see available shared disk on your system as a root user on racnode-1 use,
# /sbin/fdisk -l
It gave me output like,
.
.
.
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdd doesn't contain a valid partition table
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sde doesn't contain a valid partition table
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdf doesn't contain a valid partition table
Since I have sdb, sdc, sdd, sdf, sde disks on my storage box so all five are shown.
Step 2:
Now as a root user on racnode-1 create two raw partitions 256 MB in size for the OCR and its mirror, and three partitions 256 MB in size for the Oracle Clusterware voting disks. To store first OCR partition I use sdb for this. So, I use
# /sbin/fdisk /dev/sdb
After issuing it,
Use the p command to list the partition table of the device.
Use the n command to create a partition.
After you have created the required partitions on this device, use the w command to write the modified partition table to the device.
Here is the example of creating my first 256 raw partition of sdb disk.
The bolded one is input by me.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-19581, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-19581, default 19581): +256M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode-1 ~]#
Similarly configure other 256MB partition from rest of the disk. I created raw 256MB partition from /dev/sdc for OCR mirror. From the three disks /dev/sdd, /dev/sde and /dev/sdf create each 256MB partition for voting disk.
That is,
#fdisk /dev/sdc
#fdisk /dev/sdd
#fdisk /dev/sde
#fdisk /dev/sdf
After you configure your all 5 disks on your system your fdisk -l output will look like below.
[root@racnode-1 ~]# fdisk -l
Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 1275 10241406 83 Linux
/dev/sda2 1276 3723 19663560 83 Linux
/dev/sda3 3724 6145 19454715 83 Linux
/dev/sda4 6146 9729 28788480 5 Extended
/dev/sda5 6146 8440 18434556 83 Linux
/dev/sda6 8441 9205 6144831 83 Linux
/dev/sda7 9206 9727 4192933+ 82 Linux swap / Solaris
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 32 257008+ 83 Linux
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 32 257008+ 83 Linux
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 32 257008+ 83 Linux
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 32 257008+ 83 Linux
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdf1 1 32 257008+ 83 Linux
Step 3:
As the root user on racnode-1, edit the /etc/sysconfig/rawdevices file and add the mappings for the raw devices used by Oracle Clusterware.
[root@racnode-1 ~]# vi /etc/sysconfig/rawdevices
#OCR Devices
/dev/sdb /dev/sdb1
/dev/sdc /dev/sdc1
#Voting Disk Devices
/dev/sdd /dev/sdd1
/dev/sde /dev/sde1
/dev/sdf /dev/sdf1
Step 4: If you are using RHL4 then as the root user, on the node racnode1, enable the raw devices so that the mappings become effective at the operating system level using,
[root@racnode-1 ~]# service /etc/sysconfig/rawdevices start
If you are using RHL5 then you will get
[root@racnode-1 ~]# service rawdevices start
rawdevices: unrecognized service
Step 5:
In the racnode-2 inform the OS of partition table changes. This is done by partprobe program that informs the operating system kernel of partition table changes, by requesting that the operating system re-read the partition table.
#/sbin/partprobe /dev/sdb
/sbin/partprobe /dev/sdc
/sbin/partprobe /dev/sdd
/sbin/partprobe /dev/sde
/sbin/partprobe /dev/sdf
This forces the operating system on the other node in the cluster to refresh its picture of the shared disk partitions.
Step 6:
On RHL4 for each node make an entry in /etc/sysconfig/rawdevices and use following command,
service rawdevices start
Step 7:
As the root user, on each node in the cluster, enter commands similar to the following to set the owner, group, and permissions on the newly created device files:
chown root:oinstall /dev/raw/raw1
chown root:oinstall /dev/raw/raw2
chown oracle:oinstall /dev/raw/raw3
chown oracle:oinstall /dev/raw/raw4
chown oracle:oinstall /dev/raw/raw5
chmod 640 /dev/raw/raw1
chmod 640 /dev/raw/raw2
chmod 640 /dev/raw/raw3
chmod 640 /dev/raw/raw4
chmod 640 /dev/raw/raw5
datafiles, control files, PFILEs, and redo log files in Oracle RAC environments must
reside on shared disks. This is so that all of the cluster database instances
can access them.
IN that shared disk you can use the following file storage options for Oracle RAC database.
1)ASM: Which stands for Automatic Storage Management. And oracle recommends ASM.
2)Oracle Cluster File System (OCFS): OCFS is available for Linux and Windows
platforms, or a third-party cluster file system that is certified for Oracle RAC.
3)A network file system: This is not supported on AIX, POWER, or on IBM zSeries-based Linux.
4)Raw devices
In this post I will show how to configure RAW device to install oracle RAC.
Before discussing it is good to know the difference between single instance database and RAC instance database.
Oracle RAC databases differ architecturally from Oracle RAC single-instance Oracle
databases in that each Oracle RAC database instance also has,
At least one additional thread of redo for each instance
An instance-specific undo tablespace
To configure RAW storage device as a shared storage in RAC database first you have to think about clusterware part. Oracle clusterware has two components OCR and voting disk.
Before you install Oracle Clusterware, you will need to configure 5 raw partitions,
each 256 MB in size, one storing the Oracle Cluster Registry (OCR), another one for duplicate OCR file on a different disk, referred to as the OCR mirror, and three voting disks.
And then If you plan to use raw devices for storing the database files, you will need to create additional raw partitions for each tablespace, online redo log file, control file, SPFILE and password file.
Below is the procedure of how to configure RAW device.
Step 1:
Be sure that you have storage box purchased that is shared on both nodes.
Now to see available shared disk on your system as a root user on racnode-1 use,
# /sbin/fdisk -l
It gave me output like,
.
.
.
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdd doesn't contain a valid partition table
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sde doesn't contain a valid partition table
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/sdf doesn't contain a valid partition table
Since I have sdb, sdc, sdd, sdf, sde disks on my storage box so all five are shown.
Step 2:
Now as a root user on racnode-1 create two raw partitions 256 MB in size for the OCR and its mirror, and three partitions 256 MB in size for the Oracle Clusterware voting disks. To store first OCR partition I use sdb for this. So, I use
# /sbin/fdisk /dev/sdb
After issuing it,
Use the p command to list the partition table of the device.
Use the n command to create a partition.
After you have created the required partitions on this device, use the w command to write the modified partition table to the device.
Here is the example of creating my first 256 raw partition of sdb disk.
The bolded one is input by me.
[root@racnode-1 ~]# /sbin/fdisk /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.
The number of cylinders for this disk is set to 19581.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-19581, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-19581, default 19581): +256M
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@racnode-1 ~]#
Similarly configure other 256MB partition from rest of the disk. I created raw 256MB partition from /dev/sdc for OCR mirror. From the three disks /dev/sdd, /dev/sde and /dev/sdf create each 256MB partition for voting disk.
That is,
#fdisk /dev/sdc
#fdisk /dev/sdd
#fdisk /dev/sde
#fdisk /dev/sdf
After you configure your all 5 disks on your system your fdisk -l output will look like below.
[root@racnode-1 ~]# fdisk -l
Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 1275 10241406 83 Linux
/dev/sda2 1276 3723 19663560 83 Linux
/dev/sda3 3724 6145 19454715 83 Linux
/dev/sda4 6146 9729 28788480 5 Extended
/dev/sda5 6146 8440 18434556 83 Linux
/dev/sda6 8441 9205 6144831 83 Linux
/dev/sda7 9206 9727 4192933+ 82 Linux swap / Solaris
Disk /dev/sdb: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 32 257008+ 83 Linux
Disk /dev/sdc: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdc1 1 32 257008+ 83 Linux
Disk /dev/sdd: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdd1 1 32 257008+ 83 Linux
Disk /dev/sde: 161.0 GB, 161061273600 bytes
255 heads, 63 sectors/track, 19581 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 32 257008+ 83 Linux
Disk /dev/sdf: 91.2 GB, 91268055040 bytes
255 heads, 63 sectors/track, 11096 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sdf1 1 32 257008+ 83 Linux
Step 3:
As the root user on racnode-1, edit the /etc/sysconfig/rawdevices file and add the mappings for the raw devices used by Oracle Clusterware.
[root@racnode-1 ~]# vi /etc/sysconfig/rawdevices
#OCR Devices
/dev/sdb /dev/sdb1
/dev/sdc /dev/sdc1
#Voting Disk Devices
/dev/sdd /dev/sdd1
/dev/sde /dev/sde1
/dev/sdf /dev/sdf1
Step 4: If you are using RHL4 then as the root user, on the node racnode1, enable the raw devices so that the mappings become effective at the operating system level using,
[root@racnode-1 ~]# service /etc/sysconfig/rawdevices start
If you are using RHL5 then you will get
[root@racnode-1 ~]# service rawdevices start
rawdevices: unrecognized service
Step 5:
In the racnode-2 inform the OS of partition table changes. This is done by partprobe program that informs the operating system kernel of partition table changes, by requesting that the operating system re-read the partition table.
#/sbin/partprobe /dev/sdb
/sbin/partprobe /dev/sdc
/sbin/partprobe /dev/sdd
/sbin/partprobe /dev/sde
/sbin/partprobe /dev/sdf
This forces the operating system on the other node in the cluster to refresh its picture of the shared disk partitions.
Step 6:
On RHL4 for each node make an entry in /etc/sysconfig/rawdevices and use following command,
service rawdevices start
Step 7:
As the root user, on each node in the cluster, enter commands similar to the following to set the owner, group, and permissions on the newly created device files:
chown root:oinstall /dev/raw/raw1
chown root:oinstall /dev/raw/raw2
chown oracle:oinstall /dev/raw/raw3
chown oracle:oinstall /dev/raw/raw4
chown oracle:oinstall /dev/raw/raw5
chmod 640 /dev/raw/raw1
chmod 640 /dev/raw/raw2
chmod 640 /dev/raw/raw3
chmod 640 /dev/raw/raw4
chmod 640 /dev/raw/raw5
Subscribe to:
Posts (Atom)
