Understanding Tier Configuration and Personalization

Intended for use with Cassatt Active Response Standard Edition, Premium Edition and Data Center Edition V5.1.

Tiers are the main organizing mechanism in Cassatt Active Response for delivering shared resources (Premium Edition and Data Center Edition). Tiers bring together images, networks, SLAs, policies, and monitoring collectors to provide a business service. This article provides guidance and best practices for:

• Preparing for tier creation
• Configuring tier settings
• Personalizing image instances on nodes
• Testing and tier activation

Preparing for tier creation

Configuring tiers is a balancing act—you must look at all tiers with their applications, all resources, and all SLAs so Cassatt Active Response handles flux to meet your needs. Before creating a tier, consider the following questions for each application running in your data center.

• What is the priority of the tier?
• Does the application impose hardware requirements that are different from the base image hardware requirements?
• How frequently is the application used?
• How many employees need access?
• Is usage steady over time, or does it fluctuate?
• Are usage cycles predictable?
• What level of availability is required?
• What networks do your applications require?

top

Prerequisites

Before creating tiers, make sure you:

• Create images for your business applications. See Understanding Image Creation and Capture.
• Configure networks (if using networks other than the Cassatt Active Response network).
  
  Most applications do not require special networking, but you can use different IP address ranges to group applications or for other administrative purposes. For more information, see Network Addresses: Calculating Requirements.
  
  Cassatt Active Response Data Center Edition supports many common networking scenarios for applications requiring specific network topologies (for example, the private interconnect required to run Oracle 10g RAC). You can also specify networks for a tier when you want to isolate the tier with firewall, or you want the tier to use specific IP addresses. See Understanding Cassatt Active Response Network Manager.
• Check blueprints to understand tier settings specific to your application.

top

Configuring tier settings

This section describes each page in the New Tier wizard, and best practices for configuring tier settings. Although this section follows the page order of the wizard, several of the settings may need to be considered together.

Properties page

The Properties page contains these settings:

• Base image
• Email alerts
• Node failure rules
• Node harvesting

top

Base image

After a base image is captured, it is displayed in the Cassatt Active Response Controller in the Images tab. For example:

You can change base image properties to your heart's content until the point when you allocate nodes or activate the tier. After image instances are running in a tier, many of the base image properties are read-only. So, it's a good idea to refine your base image before activating the tier.

On the first page of the Tier mini-wizard, you assign a base image to a tier using a drop-down menu. At the end of tier creation, Cassatt Active Response creates image instances from the base image. For example:

top

Email alerts

You can choose to get email alerts whenever tiers are in critical and warning states. In both cases, Cassatt Active Response always tries to bring the tier back to normal; however, if your hardware reserves are stretched, and you if you've chosen to manually control some settings, you may need to keep a closer watch on resources. The bottom line is, the more you automate, and the more reserves you have, the less you need to be concerned about monitoring Cassatt Active Response, even when tiers are in critical or warning states.

Use commas to separate multiple email addresses in the Email Address text box.

top

Node failure rules

• Reboot on Failure – Your selection for this setting depends on the software running in the tier. For example, a reboot may be sufficient to restore normal operation when applications fail. Cassatt Active Response cannot automatically reboot nodes that do not have power controllers; select Off when your hardware requires a manual reboot.
• Automatic Move to Maintenance Pool – If you select On, Cassatt Active Response automatically moves failed application nodes from the tier to the maintenance pool. If two nodes fail after running the same image instance, Cassatt Active Response automatically disables that image instance so you can diagnose the problem. Meanwhile, Cassatt Active Response replaces the node—if a replacement is available—to keep the tier working at the operational target level. If you select Off, Cassatt Active Response disables the node, but leaves it in the tier so you can diagnose it with its current image instance.

top

Node harvesting

When resources are limited, nodes fail, or priorities change, Cassatt Active Response handles the unexpected with the Tier Priority and Allow Harvest settings. With Allow Harvest = On, Cassatt Active Response can appropriate application nodes from one tier when another tier is below its operational target (whether through a node failure or because you increased the operational target).

• Tier priority – if you are using node harvesting, you'll need to know your applications and set priorities across all tiers. Consider these definitions:
  
  
  • High – Availability and performance are crucial.
  • Medium – A drop in service level is acceptable to keep the high-priority tiers at optimal performance.
  • Low – Application is nonessential; significant drops in performance are acceptable. Note: Some level of availability is ensured; Cassatt Active Response never harvests tiers below minimum.
• Allow harvest – when a tier requires a node, Cassatt Active Response searches the free pool for a suitable replacement; if an appropriate node is not available in the free pool, Cassatt Active Response looks for suitable nodes in tiers that have node harvesting turned on. Cassatt Active Response then harvests nodes from tiers with:
  
  
  • A lower tier priority than the tier in need, or
  • An equal tier priority if the tier is deactivated, or the tier has more nodes than its minimum.

As you can imagine, the logic to handle every harvesting scenario is a bit more complex. For details on the logic behind node harvesting, see the sidebar Node harvesting logic.

To see an animated node harvesting example (that doesn't require any special software to run), see the sidebar Node harvesting animation.

top

SLA

The SLA page contains two choices: Node-based and Load-based. In brief: node-based SLAs set capacity in number of nodes; load-based SLAs set capacity based on a measure of aggregate load.

Node-based SLAs

There are only a few settings for node-based SLAs, but careful attention must be given to these values:

• Operational target – preferred number of application nodes you want Cassatt Active Response to attempt to start and maintain in the tier.
• Minimum nodes – minimum number of application nodes required for the tier to provide service (the application to function).
• Maximum nodes – number of image instance "slots" to create for the tier.
• Idle timer – deactivates the tier automatically when the tier is "idle" for a specified period of time. The idle timer has several associated fields, which are explained later in this section.

The following graphic shows a tier running in a normal state, with node allocations based on target and image instances based on maximum nodes.

Let's look at each setting in more detail.

top

Minimum nodes: what you should know

The minimum nodes setting is generally the absolute minimum required for the application to function. It can also be used to ensure a level of service that's above the absolute minimum required for the application to function. For example, if you have an agreement with a business unit that you will always supply a minimum level of performance for a set number of users, and you know that six nodes of the type specified for the tier are required to meet that service level, you can set your minimum to six.

Minimum nodes is the simplest capacity parameter to set. You can raise or lower the minimum nodes for a tier at any time.

top

Maximum nodes: what you should know

In general, the number you specify for maximum nodes should reflect the most nodes you foresee using for the tier, given maximum load. Although you could encounter an application with a practical maximum, more likely you will use the maximum nodes to determine the number of image instances for Cassatt Active Response to create for the tier.

top

Operational target: what you should know

The operational target determines how many application nodes Cassatt Active Response attempts to run in each tier at any given time. Cassatt Active Response successfully keeps all tiers at their operational targets—as long as you have sufficient hardware. The Controller lists fully allocated tiers with the normal status.

The Controller also gives a normal status to tiers when you have just raised the operational target, even though new nodes are still being allocated.

Whenever there isn't sufficient hardware to keep a tier at its operational target or to bring a tier to a new operational target, Cassatt Active Response lists the tiers with the critical or warning status. In either case, the tier is operating below its operational target and is not fully compliant with its SLAs.

top

Idle timer: what you should know

The idle timer lets you reduce power consumption when servers are not in use. This might happen at night or over the weekend. The idle timer works in conjunction with scheduled power policies, which turn off servers based on a schedule. Instead of waiting for a scheduled reduction, the idle timer uses monitoring information from the servers to determine when to shut them down. Once the idle timer has deactivated a tier, you'll have to reactivate it manually (using the Activate tier action in the Controller or the control tiers functionality in the Policy Manager) or via a schedule in the Policy Manager. So, if you're using the idle timer for weekend shutdowns for example, you might set up a policy to activate tiers on Monday mornings (see the Policy Manager Users Guide).

The idle timer consists of the following parameters:

• Idle time – the length of time in minutes the tier must be idle (as defined by the conditions) before Cassatt Active Response deactivates the tier.
• Expression – a formula based on monitoring information that represent how to decide when the tier is "idle." Expression formulas are built in the same way as the expressions for load-based SLAs, and load-based SLAs use the same formulas for both purposes. To learn more about writing formulas, read Expressions in the load-based SLA section.
• Threshold – Each expression has a threshold, which determines the expression value at which Cassatt Active Response deactivates the tier. You choose the threshold direction (< or >), depending on the type of value in the expression. To learn more about thresholds, read Thresholds in the load-based SLA section.
  
  The idle timer can have as many sets of conditions as you need. Click the + sign to add another set of conditions.
• Deallocate – returns nodes to the free pool in addition to deactivation.

When a tier with an idle timer set has another tier that depends on it, the idle timer times out as usual, but does not deactivate the tier as long as the dependent tier is active.

top

Load-based SLAs

Load-based SLAs comprise the following fields:

• Minimum nodes – minimum number of application nodes required for the tier to provide service (the application to function). Minimum nodes: what you should know in the node-based SLA section describes best practices; be aware, however, that minimum nodes cannot be 0 for a load-based SLA. This is because there can't be any load to measure if there are no nodes, so there must be at least one node.
• Maximum nodes – number of image instance "slots" to create for the tier. For more information and best practices, see Maximum nodes: what you should know in the node-based SLA section.

For a load-based SLA, minimum and maximum cannot both equal 1. Setting both to 1 would leave no possibility for the SLA to adjust capacity. If you want minimum and maximum to be 1, use a node-based SLA.

While the Controller does not disallow setting minimum and maximum to the same value (other than 1), since you can lower minimum later, it doesn't really make sense to do so for the same reason that minimum and maximum cannot both equal 1.

• Capacity criteria – conditions based on monitoring information that represent how you want to measure load on the application. Each SLA has two kinds of conditions: conditions for increasing capacity—when load is high and more resources are needed—and conditions for decreasing capacity—when load is low and resources are idle.
  
  Each tier can have as many sets of conditions as you need for increasing and decreasing capacity, and each set of conditions consists of several parameters, which are discussed later in this section.
  
• Idle timer – deactivates the tier automatically when the tier is "idle" for a specified period of time. For more information and best practices, see Idle timer: what you should know in the node-based SLA section. Note that, for tiers with load-based SLAs, the idle timer uses the load threshold for decreasing capacity, and it does not kick off until the tier is at minimum nodes.
• Load delay – the amount of time for the service level to adjust after starting a new node before determining whether to start another node or to stop a node.
• Node limit – used to cap the number of nodes assigned to the tier at any given time. The node limit setting is used primarily with power policies. When first creating a tier, Cassatt Active Response defaults the node limit equal to maximum nodes, but you can select any value you want between minimum and maximum. For more information on using the node limit setting in power policies, see the Policy Manager User's Guide.
• Undeployment Constraint – provides an intelligent method of selecting the least productive node for deallocation. The undeployment constraint has several associated fields, which are explained later in this section.

Let's look at the settings for load-based SLAs in more detail.

top

Capacity criteria: what you should know

With load-based SLAs, you determine the service level based on how your application works, using measurable factors like CPU utilization and number of users. For each type of load you want to measure, you set up a condition that Cassatt Active Response uses to figure out how many application nodes are required. Using conditions instead of node counts lets you pinpoint precise service levels no matter what kind of hardware you have.

There are two kinds of conditions:

• Increase capacity – tells Cassatt Active Response to add a node whenever any of the conditions is met.
• Decrease capacity – tells Cassatt Active Response to remove a node whenever all of the conditions is met.

Each condition has 4 parts:

• Expression – tells Cassatt Active Response what to measure.
• Threshold and operator – represents the tipping point for adjusting the capacity level.
• Frequency – determines how often Cassatt Active Response calculates the expression value.
• Period – determines the time span over which Cassatt Active Response averages the expression values.

You can have as many sets of conditions of each type as you need to characterize the level of service the application should provide. To add a set of conditions, click the + sign.

Expressions

Writing an expression requires that you understand how the software in the image is monitored. You configure the monitoring service in the OS or application when you initially install the software and before capturing the image.

The monitoring collectors available in Cassatt Active Response and the instructions for how to set up monitoring for images are described in Understanding Monitoring for Business Applications. Remember that monitoring collectors are pluggable, so you might have protocols available to you that are not listed in that document.

An expression comprises at least one monitored value with its protocol, for example: snmp:cpuUser. A formula can contain standard arithmetic operators, for example: (snmp:cpuUser*100)/snmp:pingTime.

Cassatt Active Response processes mathematical expressions using the standard operator order of precedence.

To decide on an expression, you'll need to consider what monitoring protocols and values are relevant to your application, and how to slice and dice those into a formula that provides the service level you need.

Thresholds

To set the threshold, you'll need to get some measurements of how the value(s) you've selected fluctuate under load. I'm afraid I can't give you a lot of guidance here, but I suggest you experiment with the protocols and values you have available until you seem to be getting the results you want.

When you set the threshold, you'll also set the direction (< or >), which depends on what type of value you are using in the expression. For example, an expression based on free memory would require additional resources when the value dropped below the threshold, and an expression based on CPU usage would require additional resources when the value went above the threshold.

Conversely, an expression based on free memory would have excess resources when the value exceeded the threshold, and an expression based on CPU usage would have excess capacity when the value dropped below the threshold.

Be sure you understand the units when setting thresholds. Some values are counts or measurements, while others are percentages.

top

Period and frequency

Setting the timing parameters for SLA conditions also requires knowledge of how your application is used and how critical it is for Cassatt Active Response to respond quickly to load changes.

Here's a general idea of how it works, which will give you enough information to set your frequency and period (the actual code is much more complicated). Here I'm using an example where the frequency is 30 seconds and the period is 2 minutes, which would generate 4 values for averaging.

1. The monitoring collectors sample each node according to the collection interval you set for the collector when you created the image (here's a reminder). Be aware, however, that the collectors actually collect at the shortest interval set on any image that's active in a tier, so your tier may get collections more often than you specified. If the monitored value has changed for the node, the monitoring collector saves the new value.
2. At the frequency point (every 30 seconds in our example), the SLA function gets the most recent value for each node from the monitoring collector, and runs each value through the condition formula.
3. The formula values for all nodes are averaged together to generate one value for the tier.
4. The new tier value is averaged in with the tier values from the preceding period (which is 2 minutes in our example, so the new value is averaged in with the previous 3 values).
5. Cassatt Active Response then compares the new tier average with the SLA thresholds to determine whether to increase or decrease capacity (by allocating or deallocating a node).

When you first activate a tier, Cassatt Active Response waits a full period to determine load before allocating nodes above minimum. In our example, Cassatt Active Response would start the minimum number of nodes, then wait 5 minutes before allocating additional nodes, even if the threshold was breached during that 5-minute period. When you have more than one condition, Cassatt Active Response waits for the longest period, so conditions with shorter periods are ignored immediately after tier activation.

The period and frequency settings allow you to adjust sensitivity to transient events. You may not want Cassatt Active Response to allocate an additional node for a short-lived load spike. Averaging the condition values smoothes the SLA response, but at the expense of longer response times when load shifts really do merit capacity changes. So, for a smoother response, increase the period. To decrease response time, decrease the frequency and period, but realize there is no point in averaging more often than you collect or calculate values: collection interval < frequency < period.

Therein lies the trade-off, and that's where your knowledge of usage patterns and service level requirements comes into play. The defaults may work for you as is, but these timing parameters allow you fine-tuned control over performance.

top

Load delay: what you should know

The load delay period allows the load to balance out before new starts/stops. Set the load delay to equal:

• The time it takes for your nodes to boot and your applications to be operating normally
  
  Plus
• The time it takes for the load value to stabilize. This may take a while if the load is being averaged over a long period, so the load delay must take the averaging period into account.

Undeployment constraint: what you should know

When an SLA breach calls for a deallocation, Cassatt Active Response uses undeployment constraints to determine which node to shut down.

An undeployment constraint has the following elements:

• Eligibility – a calculation based on one or more node monitored values plus arithmetic arguments that represents how you want to determine which nodes can be deallocated.
• Minimum (or no minimum), maximum (or no maximum) – determines the range of eligibility values; nodes outside the range are not considered for deallocation.
• Order by formula, ascending/descending – determines how Cassatt Active Response ranks the nodes that are eligible for deallocation to select the most eligible.

To select the node, Cassatt Active Response follows these steps:

1. Runs the eligibility a formula against each node.
2. Compares the results for each node against the minimum and maximum thresholds.
3. Runs the ordering formula against each node that's inside the eligible range.
4. Sorts the nodes into either ascending or descending order based on the results of the ordering formula.

Writing the two formulas for an undeployment constraint is just like writing a formula for the SLA itself: you determine which monitored value or values are relevant, and add any arithmetic operators, until your result can successfully evaluate nodes. The goal is to identify the the server that is closest to idle.

An important consideration in devising the eligibility formula is compatibility with the SLA conditions for decreasing capacity. Using different kinds of measurements for the formulas can put the two at cross purposes. For example, suppose you use an SLA condition based on I/O and an eligibility formula based on CPU. When I/O drops below the minimum threshold, Cassatt Active response selects the node with the least CPU to remove—which might be a node that's handling heavy I/O. That would increase I/O disproportionately across the remaining nodes and might even result in a new node allocation.

Depending on your formula, you might need only a minimum or a maximum, but not both. In that case, select the "No Min:" or "No Max:" checkbox, as shown in the next illustration.

top

Requirements page

The Requirements page contains these settings:

• Hardware requirements
• Custom requirements
• Tier dependencies

top

Hardware requirements

Hardware requirements are set on the base image. However, you can set hardware requirements on the tier—which takes precedence over the hardware requirements set in the base image. This feature is used primarily for tweaking a tier. For example, an image may be configured for local disk. When you get real hardware in the tier, you may find that this is unnecessary and choose to turn off local disk for the tier.

top

Custom requirements

Custom attributes are set on the base image. However, you can use custom attribute settings on a tier whenever you want hardware go to a specific tier. For example, say you purchased new IBM hardware that you wanted to go in a specific tier, you would set a custom attribute on the tier and the IBM node.

Cassatt Active Response allocates node with custom attributes last because they have the highest "cost." However, when resources are constrained, Cassatt Active Response does allocate nodes with custom attributes to tiers that do not require custom attributes.

top

Tier Dependencies

Some applications require other applications to be running in order to function properly. For example, imagine an enterprise-level order entry system with a 3-tier application stack: a database, an application server, and a web server. The application server can't do its job without the database, and the web server can't do its job without both of the other two.

You can set tier dependencies in Cassatt Active Response to help ensure that each application has the resources it needs—including availability of other applications. Setting a tier dependency in Cassatt Active Response ensures:

• The tier will activate properly by making sure all of the tiers it depends on are running before starting your tier.
• That neither you nor Cassatt Active Response can disrupt your tier operations by deactivating a tier that your tier depends on.

Cassatt Active Response disallows circular dependencies; that is, if tier "A" depends on tier "B," and you try to set tier "B" to depend on tier "A," Cassatt Active Response displays an error message.

Here's what you can expect to happen when you set one tier to depend on another. The next table uses two tiers as an example: an application server tier which depends on a database tier.

Some application blueprints in the Info Central blueprint library specify requirements for tier dependency settings. You can use blueprints as examples to help you understand when a tier dependency is required or useful.

top

Networks page

If you are using just the Cassatt Active Response network, then that is the only network displayed on this page. Other networks are listed as a result of configuring networks in the Networks > New Networks wizard.

top

IPs and host names page

The IPs and host names page is displayed for each network that you select on the Networks page in the wizard.

The IP addresses and host names that you specify on this page are permanently assigned to the tier's image instances, but temporarily assigned to application nodes when nodes are allocated to the tier.

To assign a site-specific host name (not the same as the node name assigned by the Controller), specify either the host name prefix or the individual name of the node. (Cassatt Active Response displays as many text fields for host names as there are maximum nodes set for the tier.)

To use the "Provide via a file" option, create a file using the following format:

<IP_address>, <host_name>

Example:
10.10.73.9, dogmatic
10.10.73.10, system2
# This is a comment line.

Rules:
Use # for comment lines.
Blank lines are ignored.
Cassatt Active Response checks that the IP addresses are available on the network specified.
If one entry is invalid, Cassatt Active Response does not accept the file.

top

Personalizing image instances on nodes

At the end of the New Tier wizard, Cassatt Active Response creates image instances. Personalizing involves logging into each node in the tier and customizing the image instance for the specific application. If your application does not require personalization (check your blueprint or application), and you do not need to configure any external resources (as load balancers and SANs), you can skip this step.

The following table answers some common questions about personalization.

Managing tiers and images

For help in changing tier settings and image properties after a tier is activated, see Updating Images and Tiers.

top

Was this article useful? Tell us what you think.
Email infocentral@cassatt.com.