SEARCH DOCS
info central: your site for Collage technical info
  CASSATT.COM   INFO CENTRAL
ACTIVE RESPONSE 5.0 TOPICS BLUEPRINTS TROUBLESHOOTING DOC INDEX


 

TOC
Prerequisites
The blueprint steps
Step 1: Reserve a node from the free pool to use as an image host
Step 2: Install and configure the operating system

Step 3: Perform Oracle 10g RAC pre-installation procedures

Step 4: Capture the image
Step 5: Create the tier
Step 6: Personalize the image instances
Step 7: Allocate nodes and activate the tier
   
know-how:

Oracle 10g RAC: Capturing an Image and Creating a Tier

Intended for use with Cassatt Active Response V5.0.

This blueprint walks through preparing the Oracle 10g Real Application Clusters (Oracle 10g RAC) release 2 (10.2) software to run in the Cassatt Active Response environment. If you haven't read Understanding Blueprints for Deploying Applications, read that first for background and context.

Oracle 10g RAC is a complicated application. You should have a good understanding of the Oracle 10g RAC application and you should carefully review the Oracle installation documentation before following the procedures in this blueprint.

Prerequisites

checkmark

Determine the policy settings for the Oracle 10g RAC image and the tier. To understand how to determine policy values, see Understanding Tier Configuration and Personalization.
checkmark

Application node hardware requirements:

  • Application nodes with local disk, remote power control, and at least two NICs to satisfy max nodes for the tier

    Consider any other tiers in your domain that also require these attributes. For more information about application nodes, see Understanding Hardware: What Works Best with Cassatt Active Response.
  • If you are installing a 64-bit version of the OS, and you have not already done so, follow these steps to set a custom attribute to identify x86_64 servers and ensure they are the only nodes allocated to run the image:
    1. Identify all of the x86_64 nodes.
    2. Add a custom attribute to each of those nodes that indicates the nodes can be allocated to run images that require a 64-bit-capable archictecture. Refer to the online help topic entitled "Using custom hardware attributes to specify hardware for your software" for information.
    3. When you capture the Oracle image, assign the same custom attribute to the image as you assigned to the nodes.
    4. When you create the Oracle tier, the custom attribute is assigned by default. Do not remove the custom attribute.
checkmark Image host requirements: If you are installing a 64-bit version of the OS, select a node with the x86_64 custom attribute.

Software requirements:

  • Oracle 10g Real Application Clusters (Oracle 10g RAC) release 2 (10.2)
  • Red Hat ELAS 3 or 4

Networking requirements:

  • IP addresses for 2 times max nodes available in the primary network you will use for the Oracle tier

    If necessary, add a new network to Cassatt Active Response using the Controller.

    In the Controller, reserve 1 times max nodes IP addresses from the primary network for use as virtual IP addresses. Oracle 10g RAC uses virtual IP addresses to accommodate failover within the Oracle tier. You will assign these addresses during personalization.

    See also, Network Addresses: Calculating Requirements.

  • A network IP address space for the private interconnect Oracle 10g RAC requires

    Add the network to Cassatt Active Response using the Controller. You will assign the address space as a dedicated network during tier creation.

    Do not assign this network to any other tier in Cassatt Active Response.

  • If you will be adding networks and have not yet added switches or VLANs, you must do so before adding tiers.

top

The blueprint steps

This blueprint follows the standard high-level blueprint steps.

Step 1: Reserve a node from the free pool to use as an image host

  1. From the Controller, click the Free Pool.
  2. Select a node to reserve.
  3. Select Node Actions > Reserve for Image Capture.
  4. Note the node's IP address, which you will use when you install the OS and capture the image.

Step 2: Install and configure the operating system

Follow instructions to install and configure ELAS 3 (32-bit only) or 4 (32-bit or 64-bit): Red Hat ELAS.

Red Hat Enterprise Linux 5 is not supported for this blueprint.

If you are using Red Hat ELAS 4, install the following RPM:

libaio-0.3.103-3.i386.rpm

Step 3: Perform Oracle 10g RAC pre-installation procedures

Prerequisites

  • Path to the Oracle database data files and Oracle Quorum files.
  • The oramkdir.sh script,available for downloading at Cassatt info central. Put the script in /etc/init.d
  • Download the Cluster Verification Utility (CVU) or copy it from the CD.

    • Follow the instructions from this link to unzip the download file:

    http://updates.oracle.com/unzips/unzips.html

Follow these steps:

  1. Create mount points for the Oracle database files on the NAS; if these file paths are NFS, mount them with the noac option:

    mkdir -m777 /quorum

    mkdir -m777 /ora_data

    mkdir -m777 /imagelib
  2. Create entries in /etc/fstab for /quorum, /ora_data, and /imagelib.

    Refer to Checking the Network Requirements in Section 2, Pre-Installation Tasks, of Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide, 10g Release 2 (10.2) for Linux for assistance with the network buffer size parameters rsize and wsize.

    Format as in these examples:

    name of the NAS:/mount point/apps/oracle/data /ora_data nfs
    rw,hard,nointr,rsize=32768,wsize=32768,tcp,noac,vers=312
    name of the NAS:/mount point/apps/oracle/quorum10G2 /quorum nfs tcp,noac
    0 0
    name of the NAS:/mount point/imagelib nfs tcp 0 0

    3. Cassatt Active Response comments out /etc/fstab entries for NFS-mounted file systems during image capture. The entries are commented out to avoid an NFS mount failure should the NFS shares be restricted. You will uncomment the /etc/fstab entries on each node at personalization time (Step 4).

  3. Use the following pre-installation instructions from Section 2, Pre-Installation Tasks, of Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide, 10g Release 2 (10.2) for Linux. You will perform steps not listed here during personalization.
    1. Understanding and Using Cluster Verification Utility
    2. Logging in to the System as Root
    3. Creating Required Operating System Groups and User

      Skip the last step, Configuring SSH on All Cluster Nodes. You will configure SSH during personalization (Step 4).
    4. Configuring the oracle User's Environment
    5. Checking the Hardware Requirements
    6. Checking the Software Requirements
    7. Configuring Kernel Parameters
    8. Checking the Hardware and Operating System Setup with CVU
    9. Checking the Operating System Requirements Setup with CVU
    10. Identifying Required Software Directories
    11. Identifying or Creating an Oracle Base Directory
    12. Creating the Oracle Clusterware Home Directory
  4. Turn on the oramkdir script to create the /tmp/.oracle directory:

    chkconfig oramkdir.sh on
  5. Create the oraInventory parent directory in /var/opt/oracle/:

    mkdir -m777 /var/opt/oracle
  6. Create the Oracle Enterprise Management Agent directory:

    mkdir -m777 /var/opt/oracle/agent
  7. Define the environment variable in /home/oracle/.bash_profile:

    ORA_OEMAGENT_DIR=/var/opt/oracle/agent
  8. Add a link for tmp/oracle in /var/tmp:

    ln -s /tmp/oracle /var/tmp/oracle

top

Step 4: Capture the image

In this step, you will capture the base image from the image host using the cccapture command—an interactive script. Although you can change the values later, it is most efficient to supply valid values now.

Follow these steps:

  1. Log into the active control node as root.
  2. Enter the following command to start the capture of the base image from the image host:

    /opt/cassatt/bin/cccapture
  3. Answer each prompt.

    The next table lists the suggested values for the Oracle 10g RAC blueprint; unless you have site-specific reasons, accept the default values for other cccapture prompts.

    For names, descriptions, and other discretionary fields, use values that are meaningful to you. In the image name, do not use spaces or characters (for example, /) special to Unix-based file systems such as Linux.

    Specifying the applications and versions contained in this image is not required for this blueprint, but you may wish to do so for tracking purposes.

    For file system selection: selecting unneeded file systems wastes space but is otherwise harmless.

    4. At this prompt...

    Enter...

    Notes
    Enter the image type (linux,vmhost,windows,solaris): linux  

    Do any of the applications in the image require personalization (additional configuration once the image is booted)? [n]

    y

    		  

    Do your applications need write access to any directories in the image other than /etc, /dev, / var, /tmp, and /root? [n]

    y

    		  

    Enter the absolute directory pathname:

    /home/oracle

    		 

    Are there any additional directories to which applications
    included in this image will need to write? [n]

    y

     

    Enter the absolute directory pathname:

    /opt

     

    If a tier using this image must always run on a specific set of nodes,
    custom attributes can be specified for the tier. These same attributes
    must then be specified on each node that will be reserved for the tier.
    The Cassatt system will then match on these attributes to ensure that
    those nodes are always allocated for a tier with this image.

    Do you want to specify any custom attributes for tiers using this image? [n]

    		 y or n If you are installed a 64-bit version of the OS, specify the x86_64 server architecture custom attribute
    Enter an attribute value for nodes to use: x86_64attribute  

    Enter the hardware architecture for this image

    (ia32): [ia32]

    		 ia32 Select "ia32" even when your image requires a 64-bit-capable server. A custom attribute is required to ensure allocation of 64-bit-capable servers.

    Do you want to specify the hardware requirements for a tier using this image? [n]

    y

    		  

    Enter minimum amount of RAM (MBytes) per node for this image (128, 256, 512, 1024, 2048, 4096, 8192): [no minimum]

    1024

    		  

    Enter minimum number of network interface cards (NICs) for this image: [0]

    2

    		  

    Enter maximum number of network interfaces for this image (>= 2): [2]

    4

     

    Does this image require a local disk? [n]

    y

     

    Enter the minimum size in Gbytes for the required local disk: [0]

    disk size

    Cassatt Active Response uses this value in determining which nodes to allocate to tiers running this image. Use the smallest size you believe will provide adequate space for swap and temp (no smaller than 6 GB). Nodes with smaller disks will not be allocated to run this image.

    Does this image require local swap space? [n]:

    y

     

    Enter the required amount of swap space (in Gbytes) [8]:

    2

    The space remaining on the local disk after swap partitions are created is assigned to local temp space.

    Does this image require local temp space? [n]:

    y

    		  

    Should this image be installed on the local disk (allowing the node to boot locally)? [n] y

    		 y or n See Alternatives to NFS: Image on Local Disk for details.

    Monitor via ping?

    y

     

  4. In the Controller, release the image host back to the free pool by selecting the node using the checkbox, then select Node Actions > Release from Image Capture.

top

Step 5: Create the tier

Follow these steps to create and configure the tier. Cassatt Active Response prepopulates tier values with the parameters you set during the image capture. If necessary, you can change them during tier creation.

  1. On the left navigation pane, click Domain or Tiers to display the Tier List.
  2. On the Tier List page, click the New Tier button (on the right).
  3. On the Properties page, set the properties for the Oracle 10g RAC tier. The next table lists the suggested values; unless you have site-specific reasons, accept the defaults in other fields.

    At this prompt...

    Enter...

    Notes

    Email Alerts

    On

     

    Email On:
    Tier Critical
    Tier Non-Critical

     

    Select all choices

    Reboot on failure

    Off

     

    Automatic move to maintenance pool

    On

     

    Harvest priority

    Low

    		  

    Allow harvest

    On

     
  4. On the Requirements page, set the hardware requirements for the tier.

    At this prompt...

    Enter

    Notes

    Operational Target

    TargetNodes

    		  

    Minimum Nodes

    MinNodes

    		  

    Maximum Nodes

    MaxNodes

    		  
  5. On the networks page, select a primary network.
  6. Select the network you added to Cassatt Active Response for use as the Oracle private interconnect and choose the Dedicated radio button.
  7. On the IPs and Hostnames page, assign IP addresses or host names, or accept the default settings.
  8. You can close the Creating Tier Progress page and continue with the next section.

top

Step 6: Personalize the image instances

In this step you'll complete Oracle 10g RAC pre-installation and install the Oracle 10g RAC software.

On the tier page, select the Oracle 10g tier using the checkbox, then select Personalize from the actions dropdown menu.

Cassatt Active Response boots the maximum nodes for the tier.

Prerequisites

  • The following OracleMetaLink problem report (available to registered MetaLink users):

    Doc ID: Note:316583.1
    Subject: vipca Fails Complaining that Interface is not Public

Most steps in this section are performed on each application node; a few are performed on the control node or on a selected node designated for use with the Oracle Universal Installer. Pay close attention to be sure you perform each step on the correct node or nodes.

  • The following OracleMetaLink problem report (available to registered MetaLink users) can be helpful for troubleshooting Oracle cluster problems:

    Doc ID: Note:259301.1
    Subject: CRS and 10g Real Application Clusters

Follow these steps.

  1. On each application node, log on as root and uncomment the /etc/fstab entries for /quorum, /ora_data, and /imagelib.
  2. On each application node, while logged on as root, mount the required file systems:

    mount /imagelib
    mount /quorum
    mount /ora_data
  3. Use the following instructions from Section 2, Pre-Installation Tasks, of Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide, 10g Release 2 (10.2) for Linux.
    1. Creating Required Operating System Groups and User

      Begin where you left off in "Perform Oracle 10g RAC pre-installation procedures" with the last step, Configuring SSH on All Cluster Nodes.
    2. Checking the Network Requirements

      Follow the guidance in the Cassatt Active Response–specific notes column in conjunction with the Oracle installation document. Perform these steps logged on to the control node as root.

      3. Oracle procedure (truncated)

      Cassatt Active Response–specific notes

      IP Address Requirements

      Cassatt Active Response automatically assigns the public and private IP addresses. You will assign the VIP addresses in the next section.

      Configuring the Network Requirements

      1. Add a line for each VIP address/hostname to the /cassatt/etc/hosts file on the control node. Use addresses from outside the Cassatt Active Response DHCP range.
        • For the hostnames, append the suffix -vip to each node name, for example, ora0029-vip.
        • In the file, add the VIP addresses/hostnames at the bottom of the first section, as shown in the example that follows. Do not put any comments after the hostname, because Cassatt controls any address with an appropriate comment following the node information.
      2. Copy the /cassatt/etc/hosts file to /etc/hosts:

        cp -p /cassatt/etc/hosts /etc
      3. Update the dns server:

        /opt/cassatt/sudo/bm_named hosts


      Example /cassatt/etc/hosts file

      # Do not remove the following line, or various programs
      # that require network functionality will fail.
      127.0.0.1 localhost.localdomain localhost
      10.0.140.3 pea1.pea1.lab.msp.cassatt.com pea1
      # Following lines adding manually for VIP connection in oracle
      10.0.140.12 ora0019-vip
      10.0.140.14 ora0020-vip
      # # cassatt_collage
      # Following lines generated by Cassatt Collage # cassatt_collage
      # Do not modify. # cassatt_collage
      # # cassatt_collage
      ...
      10.0.140.110 Pool0011 # cassatt_collage
      10.0.140.113 collage0014 # cassatt_collage
      10.0.140.107 collage0015 # cassatt_collage
      10.0.140.111 collage0016 # cassatt_collage
      10.0.140.112 ora0019 # cassatt_collage
      10.1.141.2 pri0017 # cassatt_collage
      10.0.140.114 ora0020 # cassatt_collage
      10.1.141.3 pri0018 # cassatt_collage
      10.0.143.254 PowerDevice1 # cassatt_collage
      ...

    3. Checking the Network Setup with CVU

    If you are using a non-routable address, such as 10.x or 192.x, the vipca portion of this test will fail, as will the test as a whole. The test nonetheless provides valuable information. See the OracleMetaLink problem report listed in the prerequisites.

  4. Install the Clusterware Home on the node you selected earlier for use with the Oracle Universal Installer. Follow the installation steps from Section 4, Installing Oracle Clusterware, of Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide, 10g Release 2 (10.2) for Linux.
    • Log on as the Oracle user and, in the /home/oracle/.bash_profile, make sure the Oracle Home is set to the Cluster Oracle Home. The cfs parameter allows a shared Cluster Ready Oracle home on the NAS.
    • Continuing as the Oracle user, run the installation command:

      clusterware/runInstaller -cfs -formCluster
    • At the inventory directory and credentials screen, make sure to specify the oraInventory parent directory you created earlier in this document (/var/opt/oracle).
    • Log in as root and, before running the $CRS_HOME/root.sh, edit the $CRS_HOME/install/rooconfig file to replace all occurrences of the string K96 with the string K16.

      This change reorders the shutdown process, so that each time you shut down the node, Oracle is shut down before the NFS directories are unmounted—which is required for Oracle to shut down smoothly in the Cassatt Active Response environment.
    • If you are using a non-routable address, such as 10.x or 192.x, vipca will fail on the final step of database creation. This is because Oracle checks the public connection to make sure it is a routable address. As a workaround, log in as root and run vipca manually as described in metalink note: 316583.1 (refer to the prerequisites). Make sure to use the same netmask when you run vipca as you used for the Cassatt Active Response installation.
  5. Install Oracle Home from the same node used in step 4. Log in as the Oracle user. Follow the installation steps from Section 5, Installing Oracle Database 10g with Real Application Clusters, of Oracle® Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide, 10g Release 2 (10.2) for Linux.
    1. Change the /home/oracle.bash_profile so the Cluster Ready Home is different from the Oracle Home.
    2. Comment out the line that reads: export ORACLE_HOME=$ORA_CRS_HOME, as shown in this example:

      # export ORACLE_HOME=$ORA_CRS_HOME
    3. Execute the profile with . ./.bash_profile:

      database/runInstaller -cfs
  6. When you are finished, return to the Controller, select the tier, and select "Personalization Complete" for the tier.

top

Step 7: Allocate nodes and activate the tier

Follow these steps to allocate nodes and activate the tier.

  1. From the Tiers table on the Domain page, select the checkbox for the Oracle tier.
  2. From the Tier Actions dropdown menu, select Allocate Nodes. Click Allocate Nodes on the confirmation page.

    Cassatt Active Response allocates nodes and displays the Tiers table.

    3. If the Oracle 10g RAC tier depends on another tier, you cannot activate the Oracle tier until after you create and activate the tier Oracle depends on.

  3. Again, select the checkbox for the Oracle 10g RAC tier.
  4. From the Tier Actions dropdown menu, select Activate. Click Activate on the confirmation page.

    Cassatt Active Response starts the operating system and the applications on each node in the tier.

This completes the procedure to create and deploy an Oracle 10g RAC image.

 
 
© Copyright 2008 Cassatt Corporation : All rights reserved. Privacy | Legal | Feedback