VMware ESX Server 3.0.1 and 3.0.2: Capturing an Image and Creating a Tier
Intended for use with Cassatt Active Response Premium Edition and Data Center Edition V5.1.
This blueprint walks through a method for preparing the VMware ESX Server 3.0.1 and 3.0.2 software to run in the Cassatt Active Response environment. If you haven't read Understanding Cassatt Active Response Virtualization Manager and Understanding Blueprints for Deploying Applications, read those first for background and context.
VMware VirtualCenter operations, such as migration with VMotion, are not allowed in the Cassatt Active Response environment. Why not? VirtualCenter operations conflict with Cassatt Active Response automation. Using a tool other than Cassatt Active Response to affect VM deployment to physical machines or to start or stop VMs could cause service disruptions as these changes are overridden by Cassatt Active Response.
Prerequisites
 |
Hardware requirements:
|
|
Image host requirements: choose a server with the same minimum local disk size and RAM as the server with the least resources that will run the ESX Server image. Make sure the image host meets these requirements:
- At least one local disk
- At least 1 GB of physical memory
- Console access
- CD-ROM drive
- If the guest nodes will run a 64-bit version of the OS, select a node with the supported x86_64 custom attribute.
The image host does not need an HBA for this blueprint, even if you will use FC SAN for virtual disk storage. |
|
Software requirements:
- VMware ESX Server 3.0.1 or 3.0.2
- VMware Virtual Infrastructure Client
|
|
VM requirements: Determine the RAM requirements and disk sizes required for each VM. Refer to the blueprints for your applications for more information. |
|
SLA: This blueprint uses a node-based SLA type to maintain your desired number of virtual application nodes. To understand SLAs, see Understanding
Tier Configuration and Personalization. |
|
Network Requirements: For the primary network, allocate IP addresses for two times maximum nodes for the host tier plus VMs (in addition to the node IP address, VMware ESX server needs its own IP address for each physical node). For more information, see Network Addresses: Calculating Requirements.
IPv6 is not supported for primary networks.
The next diagram shows the baseline network configuration for ESX Server. The ESX Server software uses the host tier's primary network (and only the primary network) for managing VMs. In the simplest configuration, the VMs use the same primary network as the host tier uses. Alternatively, the VMs can use any other network that is available with your product edition.
If you have Data Center Edition, and plan to use VLANs other than the VLAN of the primary network, see the sidebar About networks for use with VMware ESX Server. Using VLANs with VMs introduces new networking possibilities, but it is important that you understand its limitations.
Make sure each network has adequate IP addresses free to accommodate maximum nodes for the tier and the VMs that will use the network. |
|
Storage requirements: Determine your storage solution for guest virtual disks. Cassatt Active Response creates (Linux) or copies (Windows and Linux VMDK files) a guest virtual disk when a virtual application node is activated in a guest tier. Because VMware ESX Server requires a local copy, the copy in the image matrix by itself is insufficient for ESX VMs. You can host guest virtual disk copies either in the Cassatt Active Response image matrix, or on external NAS, FC SAN, or iSCSI SAN.
- The storage option is set at the tier level: all virtual application nodes that are generated by a single VM host tier must use the same storage solution.
This ensures that every time a virtual application node boots, it has access to the same virtual disk copy, with all the data it generated/modified on previous boots.
- You can combine ESX images/tiers that use different storage options in the same Cassatt Active Response environment. If you do so, make sure to be rigorous in assigning custom attributes to all virtual application nodes and VM guest tiers. For more information, read about multiple storage locations and virtual application node allocation.
- Unless you use the Cassatt Active Response image matrix, you'll need to have the storage already configured and have connection data available to implement this blueprint.
- For both FC SAN and iSCSI SAN: allocate a SAN zone for the VM host tier to use.
- The zone can comprise a single LUN or many LUNs. If the zone contains multiple LUNs, they will be aggregated into a single VM file system for this blueprint.
- Only physical nodes used by ESX tiers in a single Cassatt Active Response domain should have access to the FC SAN zone. Allowing access to the zone from outside this Cassatt Active Response domain could result in duplicate virtual disk names or other conflicts that could disrupt storage operations.
- All application nodes for the VM host tier you are creating in this blueprint must use the same SAN zone; other VM host tiers can use the same SAN zone or different SAN zones.
 |
FC SAN only: If you choose FC SAN, watch for this FC SAN icon throughout the rest of the blueprint for special FC SAN-only steps and notes. |
| |
Follow these steps to prepare to use FC SAN to store guest virtual disks:
- Compose an HBA key for the VM host tier that indicates the SAN zone the tier will use.
- In the free pool, set the HBA key on a sufficient number of nodes with HBAs to fully allocate the ESX tier, plus any spares in case of failure.
- Configure the SAN switch to allow access for the WWN of each HBA from step 2 to the zone you allocated for the tier.
|
|
|
Cassatt Active Response settings: Turn off Automatic Move To Free Pool: In the Cassatt Active Response Controller, select Discovered Pool > Properties > Automatic Move To Free Pool > Off.
Why? Read about multiple storage locations and virtual application node allocation. |
top
The blueprint steps
This blueprint differs somewhat from the standard high-level blueprint steps because VMware ESX Server does not require a separate operating system.
Step 1: Reserve a node from the free pool to use as an image host
- From the Controller, click the Free Pool.
- Select a physical node to reserve.
You can differentiate between physical and virtual nodes by looking at the ID; virtual node IDs are appended with _vnodeN, for example, node003_vnode1.
- Select Node Actions > Reserve for Image Capture.
- Note the node's IP address, which you will use when you install the software and capture the image.
top
Step 2: Install VMware ESX Server
Use this information in conjunction with the installation steps in the VMware ESX Server installation guide (available from http://www.vmware.com/support/pubs/vi_pubs.html). Perform all of these steps at the console and not remotely.
- For the disk partitioning option: select Recommended, and remove all current disk partitions. Change the automatic partitioning settings as follows:
- Set the / partition to a fixed size of 3072 MB.
- Add a new /tmp partition and set the size according to this formula:
1024 MB + largest VM disk size you will configure
For example, if you plan four VMs for the image, with disk sizes of 4096 MB, 3072 MB, 6144 MB, and 6144 MB, set the /tmp partition to 1024 MB + 6144 MB = 7168 MB
- Change other partitions as needed to make these figures available.
- For the boot option, use the default, "From a drive"
- For the network settings, use these values:
DHCP or Static IP |
Select Static IP |
IP |
Select an unused IP address from within the Cassatt Active Response network but outside the Cassatt Active Response–administered contiguous IP address range |
Subnet mask |
Use the value from the control node installation |
Gateway |
Use the value from the control node installation |
Primary DNS |
Use the IP address of the control node |
Secondary DNS |
Leave blank |
Hostname |
Make up a host name |
| VLAN ID |
Leave blank |
| Create a default network for the Virtual Machines |
Check the box |
top
Post-installation steps
- Enable root logins via ssh: edit /etc/ssh/sshd_config to change "PermitRootLogin" to "yes."
Restart the service as follows:
/etc/init.d/sshd restart
- Connect to the image host via ssh and log in as root.
- Obtain the latest patches and updates from VMware, http://www.vmware.com/download/vi/, and install according the the VMware documentation.
- If you want to use SNMP for monitoring, set the snmpd service to start at startup as follows:
chkconfig snmpd on
Install the Virtual Infrastructure Client and set up licensing
Follow the instructions in the VMware ESX Server installation guide to install the Virtual Infrastructure Client on your workstation, and set up host-based licensing.
Add virtual machines
Use this information in conjunction with the VMware Basic Administration guide (available from http://www.vmware.com/support/pubs/vi_pubs.html) to create new virtual machines.
Need help determining how many VMs your application nodes can run concurrently? See the sidebar Staging VMs on the image host.
- For the virtual machine name, it is handy to include the amount of RAM for the VM (for example: 550mb) so later in the display list you can verify that you’ve created the correct number of VMs with the correct memory sizes.
- Cassatt Active Response requires each VM to be in its own directory, for example:
/vmfs/volumes/VolumeName/VMName/VMname.vmx
If more than one VM is in the same directory, Cassatt Active Response is unable to accurately inventory the virtual application nodes.
Formulate a naming convention so that each VM is in a different directory, and use that directory structure for each VM you add to the image. Make sure to use only characters that are valid for VMware (alphanumerics, hyphen, underscore, dot, and blank). Do not use parentheses, even though VMware allows them. The following example simply increments by one for each VM:
/vmfs/volumes/VolumeName/vm1/vm1.vmx
- On the networks page, add sufficient virtual NICs for the networks that are required by the applications that will run in the VM (one virtual NIC for the primary network, and one for each dedicated network).
- When you create the virtual disk, you can accept the default size for capacity; Cassatt Active Response ignores the blank disk size. For more information, see the sidebar Local disks for VMware VMs: what you should know.
top
Enable SCSI bus sharing
Follow the instructions in the VMware Administration Guide to set SCSI bus sharing to "virtual".
top
Step 3: Capture the ESX image
In this step, you will capture the base image from the image host using the cccapture command—an interactive script. Although you can change the values later, it is most efficient to supply valid values now.
Follow these steps:
- Log into the active control node as root.
- Enter the following command to start the capture of the base image from the image host:
/opt/cassatt/bin/cccapture
- Answer each prompt.
The next table lists suggested values for VMware ESX Server; unless you have site-specific reasons, you can accept the default values for other cccapture prompts.
For names, descriptions, and other discretionary fields, use values that are meaningful to you. In the image name, do not use spaces or characters (for example, /) special to Unix-based file systems such as Linux.
For file system selection: selecting unneeded file systems wastes space but is otherwise harmless.
At this prompt... |
Enter... |
Notes |
Enter the image type (linux,vmhost,windows,solaris): |
vmhost |
|
If a tier using this image must always run on a specific set of nodes,
custom attributes can be specified for the tier. These same attributes
must then be specified on each node that will be reserved for the tier.
The Cassatt system will then match on these attributes to ensure that
those nodes are always allocated for a tier with this image.
Do you want to specify any custom attributes for tiers using this image? [n] |
y |
Specify any custom attributes needed to define supported hardware as described in the prerequisites. |
Enter an attribute value for nodes to use: |
attribute |
| Do your applications need write access to any directories in the image other than /etc, /dev, /var, /tmp, and /root? [n] |
y |
|
| Enter the absolute directory pathname: |
/opt |
|
| Are there any additional directories that applications will need to write to? [n] |
y |
|
| Enter the absolute directory pathname: |
/boot |
|
| Are there any additional directories that applications will need to write to? [n] |
y |
|
| Enter the absolute directory pathname: |
/usr/lib |
|
| Are there any additional directories that applications will need to write to? [n] |
y |
|
| Enter the absolute directory pathname: |
/usr/sbin |
|
Does this image require local swap space? [n] |
y |
|
Enter the required amount of swap space (in Gbytes) [8]: |
2 |
The space remaining on the
local disk after swap partitions
are created is assigned to local
temp space. |
Enter the minimum size in Gbytes for the required
local disk: [0] |
disk size |
Specify the same disk size minimum you used for the image host, or 6 Gbytes multiplied by the number of VMs, whichever is larger. For more details, see the sidebar Local disks for VMware VMs: what you should know.
|
| Should this image be installed on the local disk (allowing the node to boot locally)? [n] |
y or n |
See Alternatives to NFS: Image on Local Disk for details. |
Do you want to specify the hardware requirements
for a tier using this image? [n] |
y |
|
Enter minimum amount of RAM (MBytes) per node for this image (128, 256, 512, 1024, 2048, 4096, 8192): [no minimum] |
memory |
Specify the same RAM minimum you used for the image host |
Enter the amount of time in seconds that the system should wait for the services provided by this image to start (after this time has passed, the system will attempt to reboot or replace the node): [600] |
1800 |
|
OS Monitoring Options
Specify the operating system monitoring that will be used
for this image (at least one monitoring option must be configured):
Monitor via SNMP? [n] |
y |
|
SNMP Retry Count: [3] |
5 |
|
Monitor via ping? [n] |
y |
|
Ping Retry Count: [3] |
5 |
|
Do not release the image host back to the free pool if you will be capturing a VMDK-based VM image. It is most efficient to keep the same image host reserved with the ESX software and VMs intact. When you have completed this blueprint, simply begin the business application blueprint of your choice using the same image host.
Cassatt automatically configures a script monitor for VM host images. Do not disable or modify the script monitor for this image from within the Controller.
top
Step 4: Create the tier
Return to the Controller to define the VM host tier. Cassatt Active Response prepopulates tier values with the parameters you set during image capture. If necessary, you can change them during tier creation.
- On the left navigation pane, click Tiers to display the Tier List.
- On the Tier List page, click the New Tier button (on the right).
- On the Properties page, set the properties for the VM host tier. The next table lists the suggested values for the VM host tier; unless you have site-specific reasons, you can accept the defaults in other fields.
- On the SLA page, select the node-based SLA type and set as follows.
At this prompt... |
Enter... |
Notes |
Operational Target |
TargetNodes |
Remember that GuestNodes = TargetNodes x VMs. |
Minimum Nodes |
MinNodes |
Maximum Nodes |
MaxNodes |
|
If using FC SAN: Remember that node capacity for this tier is limited by the number of nodes that have:
- HBAs
- HBA keys set for this tier (which you set as a Prerequisite to this blueprint and you set for the image during image capture).
|
- On the Requirements page, set the hardware requirements for the tier. Unless you have site-specific reasons, you can accept the defaults in other fields.
At this prompt... |
Enter... |
Notes |
| HBA |
HBAKey |
|
FC SAN only: select the checkbox and use the same HBA key you set for the application nodes with HBAs in the prerequisites. |
| ESX 3 Storage |
Cassatt Active Response Image Matrix |
|
| External NAS |
When you select this option, enter the NAS address(es) |
| FibreChannel SAN |
|
| iSCSI |
When you select this option, enter the IP address(es) or hostname(s) |
- On the networks page, select network(s) as determined in the prerequisites.
- On the IPs and Hostnames page, assign IP addresses or host names, or accept the default settings.
- You can close the Creating Tier Progress page and continue with the next section.
top
Step 5: Personalize the image instances
Personalize the tier only if you are using FC SAN or iSCSI SAN and you have not already created the file system. If you are not using SAN, or if the SAN LUNs are already formatted, skip this step.
Follow these steps; see also VMware ESX Server SAN Configuration Guide.
Although maximum nodes for the tier are booted in this procedure, only one node administers the file system, which is in accordance with the VMware guidance.
- On the tier page, select the ESX tier using the checkbox, then select Personalize from the actions dropdown menu.
Cassatt Active Response boots the maximum nodes for the tier.
- Select a node to use for the next several steps and log in as root.
- Initialize the SAN disk as follows:
- Get the LUN numbers and device names from your SAN administrator.
- Partition each SAN disk/LUN that will hold guest virtual disks with a single partition of type "fb", for example using fdisk:
fdisk device
p [prints the existing partitions]
d partition [deletes the specified partition]
n [creates a new primary partition]
t [changes the partition's system Id from "83" to "fb"]
w [writes the partition table to disk and exits]
The resulting table is similar to the following:
Disk /dev/sde: 255 heads, 63 sectors, 2757 cylinders
Units = cylinders of 16065 * 512 bytes
Device Boot Start End Blocks Id System
/dev/sde1 1 2757 22145571 fb Unknown
- Find the ESX device name for each disk, for example:
vmkpcidivy -q vmhba_devs
The output is similar to the following:
vmhba0:0:0 /dev/sda [the local disk]
vmhba1:0:20 /dev/sdb
vmhba1:0:21 /dev/sdc
vmhba1:0:22 /dev/sdd
vmhba1:0:23 /dev/sde
- Make a VM file system on one of the devices, for example:
vmkfstools -C vmfs3 device:1
Where device is the device found in the last step, and 1 is the partition number (since you only have one partition), for example:
vmkfstools -C vmfs3 vmhba1:0:23:1
Creating file system on "vmhba1:0:23:1" with blockSize 1048576 and volume label "none".
Successfully created new volume: 45d24a72-505c8b46-c89c-000f1f698a4e
- Add any remaining LUNs to the VM file system you created in the last step, for example:
vmkfstools -Z vmhba1:0:22:1 vmhba1:0:23:1
vmkfstools -Z vmhba1:0:21:1 vmhba1:0:23:1
vmkfstools -Z vmhba1:0:20:1 vmhba1:0:23:1
- Verify the file system makeup, for example:
vmkfstools -P -h /vmfs/volumes/volume/
Where volume is the volume created in step d, for example:
vmkfstools -P -h /vmfs/volumes/4626ec28-770603bc-83cb-000e0c2e73dc/
VMFS-3.21 file system spanning 1 partitions.
File system label (if any):
Mode: public
Capacity 59861106688 (57088 file blocks * 1048576), 59204698112 (56462
blocks) avail
UUID: 4626ec28-770603bc-83cb-000e0c2e73dc
Partitions spanned:
vmhba0:0:0:6
- When you are finished, return to the Controller, select the tier, and select "Personalization Complete" for the tier.
top
Step 6: Allocate nodes and activate the tier
Follow these steps to allocate nodes and activate the tier.
- From the Tiers table on the Domain page, select the checkbox for the VM host tier.
- From the Tier Actions dropdown menu, select Allocate Nodes. Click Allocate Nodes on the confirmation page.
Cassatt Active Response allocates nodes and displays the Tiers table.
- Again, select the checkbox for the VM host tier.
- From the Tier Actions dropdown menu, select Activate. Click Activate on the confirmation page.
Cassatt Active Response starts the Cassatt Active Response services, the operating system, and the applications on each node in the tier, then generates virtual application nodes and lists the nodes in the discovered pool.
If you personalized the tier: The virtual application nodes in the discovered pool were generated during personalization, and experienced power failures when their physical hosts were rebooted. Select the nodes using the check boxes, then select Clear power failure from the actions dropdown menu.
- If Automatic Node Inventory is turned off, select the virtual application nodes from the discovered pool and inventory the nodes.
Cassatt Active Response automatically assigns the custom attribute vmware to the nodes, and it assigns rack and slot values to the virtual application nodes.
Do not remove the vmware custom attribute: Cassatt Active Response uses it to assign VMware virtual application nodes to guest tiers, and cannot do so if the attribute is missing.
Do not edit the rack and slot fields for virtual application nodes: Cassatt Active Response needs the automatically generated values to power virtual application nodes on and off.
|
FC SAN only: Cassatt Active Response assigns the HBA key from the host tier to the VMs. |
- In the discovered pool, set a custom attribute on each virtual application node that indicates the storage solution (Cassatt Active Response image matrix, external NAS, FC SAN, or iSCSI SAN) of the VM host tier. If using FC SAN, a good practice is to use the same wording as the HBA key you used for the ESX nodes and the VM host tier. For more information, read about multiple storage locations and virtual application node allocation.
- If Automatic Move To Free Pool is turned off, move the virtual application nodes to the free pool to make them available to run applications.
- If you are finished creating ESX host tiers, and you prefer to have Automatic Move To Free Pool on, turn it on.
This completes the procedure to create and deploy a VMware ESX Server image. Next, use a blueprint to create and capture an image with your business application and deploy it in a VM guest tier.
When you implement a blueprint for a VM guest tier, make sure you install an operating system that VMware supports. Cassatt Active Response does not support operating system versions on guest tiers that VMware does not support.
top
Troubleshooting
Issues with the ESX VM host tier
See XVM: Troubleshooting VMware.
Issues with virtual application nodes after allocation to tiers
See XVM: Troubleshooting VMs and Virtual Application Nodes .
top
Was this article useful? Tell us what you think.
Email infocentral@cassatt.com.
|
