Custom Views and Dashboards in vRealize Operations

This post covers a few of the most common questions my customers ask me as I demonstrate what you can do with vROps. I’m going to take you through an example of needing to frequently check the CPU ready % of your VMs – this was my customer’s most recent request, but know that you can make this happen for any metric collected by vROps.

First, we’re going to create a custom view for CPU Ready % by going to Content>Views, then clicking on the green Plus to add a new View.

1-AddView

I named this one “Custom-CPU Ready” and gave it a description.

2-CustomCPUReady

Next, pick what our View looks like. In this case, I want to see all of the data in a list format, so I pick List.

3-Presentation

Now to select the subjects – these are the objects that the View will be looking at. We want CPU Ready % on Virtual Machines, so we pick the vCenter Adapter and scroll down until we find Virtual Machine.

4a-Subjects

 

4b-Subjects

We now need to find the CPU Ready % metric

5-CPUData

Double-click on it when you find it in the list on the left, it will then appear in the Data section. Change the Sort order to descending because we want to see the VM with the highest CPU ready on top.

6-SelectReady

The Availability options let you control where inside vROps the View will be usable. I lef the defaults.

7-Visibility

You now see the custom view in the Views list.

8-CustomView

How can we use our brand new view? We want to see the CPU ready for all VMs in the Production cluster. Go to Environment, then drill down into the vSphere World until you reach the Production cluster. Click on the Details tab, you can then scroll down and find the custom View that we created. Click on it and all of your VMs show up, sorted by highest CPU ready.

9-ShowReady

 

Let’s say this is a metric that you look at at daily or multiple times a day. You can create a custom dashboard so the metric is immediately visible if you’re using vROps Advanced.

To create a new dashboard, from the Home menu, click Actions, then Create Dashboard

1-AddDashboard

Name the dashboard and select a layout.

2-NameDashboard

We want to show a View in the widget, so we drag View over into the right pane.

2a-WidgetList

Click the Edit icon in the blank View to customize it.

3-EditWidget

Click on Self Provider to allow us to specify the Production Cluster object on the left, then select our Custom CPU Ready View on the right and click Save.

4-AddWidget

The dashboard is now ready. The CPU Ready for the Production VMs will now show up in the dashboard.

5-FinalDashboard

Proactive Monitoring Tools

As I meander down the hall to my health care provider for an ultrasound, I think of the severe flu I suffered from two weeks ago. I was at home with a high fever, chills, fatigue, and coughing. At one point, my oblique muscles were very tender from all the coughing, I sneezed particularly hard and my muscles had a spasm. I stood up, light-headed, and then blacked out; my wife told me it was time to go to the doctors.

My doctor gave me a quick diagnosis, he told me that coughing, sneezing or laughing can sometimes place a sudden strain on the autonomic nervous system, which can cause you to faint. He was pretty sure that it was a combination of my severe flu, muscle spasm, and the strain from the sneeze that caused my blackout. But, to be safe, my doctor scheduled an ultrasound and some blood work. He specifically wanted to rule out the possibility of kidney stones, since the pain was in the vicinity of my kidneys. Outside of the seasonal flu, on the surface I seemed healthy, but with technology he could gain greater perspective of any underlying issues.

As I lay on the ultrasound table, they checked my liver, kidneys, stomach, and a few other internal organs. They used the technology to proactively look for any issues that may cause a problem down the road. My doctor was interested in preventative health care, taking measures for disease prevention, as opposed to waiting until it becomes a major health incident and it is a disease treatment.

vRealize Operations Manager is the ultrasound of your datacenter. Without proactive monitoring tools, we can only analyze what is on the surface, which means we typically respond to IT system issues only after there is a major incident. When we have vRealize Operations Manager, it gives us a set of tools that helps us analyze the health, risk, and efficiency of our environment.
We gain greater insight into the infrastructure components that support our business applications. Using the vRealize Operations Analysis dashboard, I can immediately see if there are issues in the Workload, if there are system Faults, or if there is Stress in the environment.

Is capacity an issue now or down the road? And what is the bottleneck that needs to be addressed to ensure there are no issues in the near future?

This is preventative health care for your infrastructure, your IT professionals become the doctors with the tools available to proactively diagnose issues that could plague your infrastructure. Instead of screening for common diseases like hypertension, depression, high cholesterol, and kidney stones; they are looking at CPU demand, memory demand, disk I/O, and network latency.

Preventative activities, for example, right-sizing virtual machines and understanding resource growth trends, become a part of day-to-day operations for infrastructure well-being. IT System Engineers will appreciate the opportunity spend more time at home with their families and working on strategic initiatives, instead of being on the phone at 11:38 pm on a Friday night working on a critical server issue that is impacting a production business application that could have been prevented a week ago.

Recently I wrote a couple of white-papers for the VMware TAM Blog site, they include Right-Sizing Best Practices and vRealize Operations Guide 5.8. Hopefully they can help you down the road of preventative medicine.

vRealize Operations 6.0 – Access Control and Dashboard Management

No doubt, vRealize Operations 6.0 is extremely powerful and provides a vast improvement over previous versions. However, it took me a while to navigate some aspects of the new merged UI, in all likelihood it is probably because of my familiarity with the vSphere and Custom UI in version 5.8. In my humble opinion, sometimes when developing a product we lose sight of the balance of usability and features. Recently, I read the book The Design of Everyday Things by Donald Norman, which I highly recommend; I think it should be standard reading for any product manager. When I use a product, I expect it to be as seamless and easy to use as an Apple iPad. Although my wife might not understand what is being presented in the product, I want it to be easy enough that she can manipulate the interface and have labels with detailed information that she can understand.

To illustrate my point, my youngest child was able to fluently use an iPad when he was 14 months old because it was designed with usability in mind, his favorite application at the time, Tom the Cat.

Now that I have digressed, I am going to explain how to setup access control for different users and manage the dashboards options to provide a custom portal for your application developers, IT operators, or business leadership. This is when vRealize Operations becomes a powerful tool, by focusing on specific criteria for your IT and business partners so they have relevant information from the data center metrics provided.

Access control specifies who (user or user group) can do what (privilege) to what (object). For instance, an individual in your network operations center (NOC) needs the capability to review the performance and capacity of your management virtual machines. Since we are going to be using my lab environment, we would create a local user account, specify the group membership, define his user role, and then select the objects he has access to in vRealize Operations.

Users and user groups can be authenticated from different sources, including vRealize Operations local users, LDAP users, and Virtual Center users. Local users will be delegated to xDB for authentication, LDAP users will be delegated to LDAP for authentication, and Virtual Center users will be delegated to Virtual Center for authentication.

The best practice for access control is to use LDAP users and groups. Keep in mind; vRealize 6.0 does not currently support SSO, that is a roadmap item.

A privilege is a specific access right to perform an action. An example of this would be the action to create a new dashboard. A role is a functional grouping of privileges; PowerUser is a role that includes all privileges except for the ones related to user management and cluster management.

Now lets get started with creating our user account with the PowerUser role and access to the Management Application Group I created in my previous post. To start with, from the vRealize Operations home page click on the Administration link and then select Access Control from the navigation panel on the left.

To create our new user, we are going to click on the green + sign under User Accounts.

We are going to enter the user information; including the User Name, Password, First Name, Last Name, and Email Address. Additionally, there are options to disable the user account, lock the user account, and require a password change at next login. I am going to leave these unchecked for this account.


Next, we are going to assign the Groups we want the account to be a member of. I am leaving the default group of Everyone and adding Power Users.

As mentioned above, roles are functional grouping of privileges; I am going to check the PowerUser role for this specific account.

Click on the Objects tab, I am going to click on Applications in the center panel and then select the Management Application Group which includes two objects, my vRealize Operations Manager virtual machine and my VMware vCenter Server appliance.

Now when I log into vRealize Operations Manager with my jgaudreau account, I can only see the VMware vCenter Server Appliance and vRealize Operations Manager virtual machines under vSphere Hosts and Clusters.

In this next section, we are going to perform the tasks to limit the amount of dashboards visible on the vRealize Operations Manager home page for our new user.

We are going to click on the Content link on the home page.

Make sure you have selected Dashboards in the navigation panel, and then click on the gear icon and click on Share Dashboards.

If you want everyone that uses vRealize Operations Manager to see a particular dashboard, you will include it in the Everyone group. On the Share Dashboards window, select the Everyone group and remove all the dashboards except Recommendations. For our scenario, I just want to have Recommendations showing for all users. Remember, the jgaudreau account I created earlier is a member of the Everyone group and the Power Users group.

Not Grouped, shows all the available dashboards. It provides you with the column count of the dashboard, the number of widgets included in the dashboard, if the dashboard is shared, and if it is a locked dashboard. I am going to drag the vSphere VMs Memory, vSphere VMs CPU, and vSphere VMs Disk and Networking to my Power Users group.

In the diagram below, we recognize the groups are now showing under Power Users.

By default, Diagnose and Self Health are visible on the Home page.

 

To remove these two dashboards, we are going to click on the gear icon and then select Remove Dashboard(s) from Home. You will do this for both Diagnose and Self Health.

Now when I log on to the vRealize Operations Manager Home page with my jgaudreau account, the only dashboard that are displayed are Recommendations (Everyone), vSphere VM Memory (Power User), vSphere VMs CPU (Power User), and vSphere VMs Disk and Network (Power User).

We have created a vRealize Operations web portal for our NOC account that is limited in their ability to view only two objects and only displays four dashboards.

Now, let’s focus back to vCenter Operations Manager 5.8. With the previous version you could create custom dashboards focused on specific application groupings, but you could not limit their ability to access all the objects. This is a significant enhancement in vRealize Operations Manager 6.0 that really helps you deliver only the information required to your IT and business users.

vRealize Operations Concurrent User Connections

When deploying vRealize Operations 6.0, it is important to take into account the best practice for the concurrent user connections. This is an often misunderstood aspect when designing a deployment for vCenter Operations Manager 5.x and also the most complex. There are several factors that go into the amount of concurrent users that can access the product UI. In the prior version, when customers wanted to have more than a handful of users looking at the vSphere UI at a single time, it was recommended to create a custom dashboard with minimum overhead. You would want to limit the number alerts, alarms, and RCA information because it places a heavy load on the database, which can drag down the system. The vSphere UI in vCenter Operations 5.8 was able to handle about four concurrent connections, whereas an optimized custom dashboard in the Custom UI could handle anywhere between 10 to 20 concurrent users. There isn’t a hard set number of concurrent connections, but out of the box for vCenter Operations Manager 5.8 you can expect 10 concurrent connections based on a large vApp deployment.

Below are some of the factors that affect the number of connections:

  • How busy are the collectors?
  • Are dynamic thresholds calculating during user sessions?
  • Are custom dashboards using widgets that are taxing (weather maps, TopN, distribution widgets, ect.)?
  • Is there good communication between the UI VM and Analytics VM?
  • Are the VMs running on separate hosts to reduce contention?
  • How many alerts, alarms, and RCAs are being processed?
  • How many back-end storage IOPs are supporting the vCOps virtual machines?

As you can recognize, there are a lot of elements that determine the overall performance and concurrency. The higher the concurrent connections when the system is under load, longer latency and ultimately unacceptable refresh rate become noticeable in the vSphere UI.

For vRealize 6.0, because of the design changes we discussed in my previous post vRealize Operations – Looking Under the Hood, it is possible to get a higher number of concurrent connections. Right now the maximum best practice is 4 concurrent connections per large data node, which would be 32 concurrent connections for a single deployment. But, like vCOps 5.x, that depends on the dashboards that are being presented and the metrics being collected. If you keep it decentralized, with judicious amount of metrics being collected there is a possibility that you could see a modest improvement in concurrency. For example, if you had a data center in Boston that was collecting 10,000 metrics, which is at the top end of a single vRealize Operations 6.0 data node, and went with the full scaled out deployment of 8 nodes and are prudent with what you expose for dashboards to the users, we should able to reach a much higher number of concurrency. Talking with VMware product management, you could see up to twice the amount of concurrency which be up to 64 concurrent users.

But, there are a couple other design requirements you need to consider. Are you planning to centralize your data using remote collectors with vRealize Operations 6.0? In the previous version; if you had data centers in Boston, Atlanta, and London you would typically setup a vCenter Operations Manager 5.x instance at each site. If you were using a optimized dashboard, you could get 10 to 20 concurrent connections per site. vRealize Operations 6.0 provides you the ability of centralizing that data to a single instance with the use of remote collectors at the other locations. For example, Boston would be my master and data nodes, with Atlanta and London having remote collectors. I have moved away from three UI portals that could handle up 30 to 60 concurrent connections, to a single UI portal that can handle 32 concurrent connections.

Also, if you plan on using the new high availability (HA) feature with vRealize Operations 6.0, that is going to give you a maximum of 4 data nodes.

These are just a few factors you should consider before deploying vRealize Operations.

vRealize Operations 6.0 – New Merged UI

vRealize Operations 6.0 has had a major overhaul, one of the biggest changes is the feel and look of the new product UI; which incorporates the previous vSphere UI and Custom UI into a single console. With this in mind, the new vRealize 6.0 UI is split up into different panels. If you look at the image below, the panel on the left is called the Navigation panel. We have the Center panel, which is where all the data is displayed in the middle of the portal. And finally, at the top, we have all our Dashboards.

At the top of the Navigation Panel, you will recognize five buttons that are links to parts of the product UI. vRealize Operations 6.0 is broken up into five sections. They include the Home link, Alert link, Environments link, Content link, and the Administration link.

We start off in the Home screen, the screen that is displayed when you initially log into the product UI. The Home screen is where your dashboards reside, including any custom dashboards you may create and any custom product solutions you may install (vCenter solution, Hyperic Solution, SCOM Solution). These are the same type of dashboards that we were used to seeing in the vCenter Operations Manager Custom UI at https://(vCenter Operations Manager host)/vcops-custom/; now integrated into a single viewing panel on the Home screen of vRealize Operations 6.0.
In the below diagram, I have clicked on the vSphere VMs CPU dashboard, which gives me the heatmap widgets we recognize from vCenter Operations Manager 5.x. It displays VMs sized by CPU demand % and colored by CPU content %.
You can click on the Dashboard List drop down field, and select a specific dashboard to drill down into specific issues.
The Actions drop down list provides the ability to create new dashboards, edit existing dashboards, delete dashboards, remove dashboards from the menu, and set a dashboard as the default.
Now that we have looked at the Home page, let’s look at what I consider the most fundamental facet of vRealize Operations 6.0, the Environment view. As the shepherds of the virtualization infrastructure, we perform the majority of our work monitoring the health and stability of our infrastructure environment and keeping the wolves at bay.
When you click on the Environment link, it is going to bring you to the portal that displays three different inventory lists in the Central panel. These inventory lists include Group, Application, and Inventory.
You can customize the Groups and Applications tab, but the Inventory tab is a static list of all the objects being monitored by vRealize Operations 6.0. Groups and Applications were available in vCenter Operations 5.x. Groups were found in the vSphere UI and Applications were found under the Custom UI. Now both features have been collapsed into the Environment Overview in vRealize 6.0.
I am going to create a custom Application inventory list for a dashboard that I will create in a future post. Click on the green + sign at the top of the Applications tab. Next, I am going to select Custom for the Add Application type, and then click OK.
If you have created applications in the past, this should look very familiar to you.
On the Application Management page, I am going to give my application the name of Management Application Group. I am going to create a new Tier and call it the Management Tier. Under the object filter tree, I am going to select Object Type and then Virtual Machines. I will then drag the virtual machines I want included in my new application group from the bottom right hand panel to the Tier Objects pane.
After I click Save, the new application group shows up under the Applications tab inventory list.
Application groups provide you the ability to monitor specific application resources, it gives you a more concise view the health, risk, and efficiency of the specific application tier. These types of application inventory groups are vital when making custom dashboards for troubleshooting application issues, or providing a dashboard to your application developers so they can monitor the health and stability of the infrastructure platform supporting the business applications.
To finish off the post, we are going to look at the Navigation pane of the Environment portal. The Navigation pane is broken up into inventory trees. There are tree types (storage, hosts and clusters, networking, ect), tree instances (each vCenter would create an instance of the hosts and clusters tree), branches (relationships, and leafs (objects).

 

Each adapter can create new inventory trees when it is installed, for example if I install the EMC Storage Analytics MGMT pack it can make relationships to hosts and datastores that already exist. An important fact to remember, it is important to install the adapter packs that you have in vCenter Operations Manager 5.8 before you migrate to vRealize Operations 6.0. vRealize 6.0 will not know what to do with the data until you have established the necessary relationships.
If I click on a host, then I get information on the Center panel.
In the picture below, you will notice eight different tabs.
  • Summary tab: Displays active alerts for the object and for any descendent objects.
  • Alerts tab: Displays the active alerts for the current object.
  • Analysis tab: You use the Analysis tabs to evaluate the details about the state of your objects to help you resolve problems.
  • Troubleshooting tab: Helps to identify the root cause of problems that are not resolved by alert recommendations or simple analysis.
  • Details tab: This view provides multiple ways to look at different types of collected data by using trends, lists, distributions, and summaries.
  • Environment tab: The Environment tab shows how objects in your environment are related.
  • Projects tab: With projects, you can create scenarios to add virtual machines and hosts to the cluster and view the impact it will have on your environment to determine the best course of action.
  • Reports tab: Several out-of-the-box reports with the ability to generate fully customizable reports.
That is a brief overview of vRealize Operations 6.0’s new merged UI, there have been several changes to help make it more uniform by merging in the vCenter Operations Manager 5.8 vSphere UI and Custom UI. It takes a few days to get use to the new layout, but it is much easier to use than having separate UI interfaces like the previous version

vRealize Operations 6.0 – Under The Hood

vRealize Operations 6.0 – Under The Hood

As this year comes to a close, I thought I would kick off a new series of blog posts on vRealize Operations 6.0. There has been a significant amount of change in vRealize 6.0. It was re-architected from the ground up with over a million lines of new code. With vRealize Operations 6.0, it merges the functionality of the UI virtual machine and the analytics virtual machine into a single scalable platform.Each install of the software includes the entire stack of components; the UI (product and admin),the collector, controller, analytics, and persistent layer.

  • User Interface: Admin UI and Product UI
  • Collector: Resource data collection initiated by adapters
  • Controller: Determines mapping for data insertion and queries
  • Analytics: Metric calculations, threshold processing, alert generation, and stats
  • Persistence: Each node persists its partition of data to disk

Gone is the postgress DB, it has been replaced by EMC Documentum xDB, which is a high performance and scalable native XML based database that is ideal for data-intensive uses. This is the database that will be used in future product releases; the intent is to have a uniform standard in our product platforms.

Some of the exciting new features in vRealize Operations 6.0 include its scalability and resiliency, uniform UI and functionality, actionable alerts, automated remediation, and user definable views, dashboards, and reports.

There are a few node types with vRealize Operations 6.0; they include the master node, data node, replica node, and the remote collector node.

  • Master Node: The initial, required node in the cluster. The master node manages all other nodes. In a single-node installation, the master node must also perform data collection and analysis because it is the sole node, and the only place where vRealize Operations Managers adapters are installed.
  • Data Node: In large deployments, additional nodes have adapters installed to perform the collection and analysis. Large deployments usually include adapters only on data nodes, not on the master node or replica node.
  • Replica Node: To enable high availability (HA), the cluster requires that you convert a data node into a replica of the master node.
  • Remote Collector Node: Distributed deployments might require a remote collector node that can navigate firewalls, interface with a remote data source, reduce bandwidth across data centers, or reduce the load on the vRealize Operations Manager analytics cluster. Remote collectors only gather objects for the inventory, without storing data or performing analysis.

With the new platform, you can scale up or scale out the environment. When deploying the .OVA file, you have the choice of creating the master node as an extra-small deployment to a large deployment. This option impacts the initial size of the node, if you go with an extra-small deployment it has 2 vCPU and 8 GB of memory and a large deployment has 16 vCPU and 48 GB of memory.  The resources needed for your deployment depend on several factors, you need to understand how large the environment is that you plan to monitor, how many metrics you plan to collect, and how long you plan to store the data. You can find the sizing guidelines on KB2083783.  After the vRealize Operations instance outgrows the existing size, you expand the cluster to add nodes of the same size. This is an important aspect to be mindful of; you cannot mix node sizes, if you start with a small deployment all the additional data nodes will be a small deployment.

A common practice will be to add nodes to monitor an environment, as it grows larger. vRealize Operations can scale out to 8 nodes, which can collect 64 thousand objects, 20 million metrics, and can accommodate 32 concurrent connections.

vRealize Operations 6.0 uses Gemfire Locator for connection information to one or more Controller API GemFire Cache Servers. Gemfire is the glue that logically connects all the layers together. It creates a shared memory cluster and maps the work across the systems. Gemfire remoting provides the map reducing technology, Gemfire caching handles the analytics layer, Gemfire persistence handles sharding of the data.

Let’s look at how data gets into a multi-node vRealize Operations deployment. When a new resource is discovered, it comes into the collector with its defined metrics and properties and is added to the system. An auto-discovery task is created and sent to the controller, the controller routes the new resource to the master node. It then takes the resource kind and sends it to the Global xDB database. Afterwards, a resource cache is created in the analytics region on one of the data nodes.

For new data, after a resource has already been discovered, the metrics information will come in through the collector and go right down to the resource cache in the analytics region, after the threshold processing is completed it is persisted in the FSDB. If there is an alert trigger, that will get persisted in the alarm alert database.

To end the post, I want to talk a little bit about availability; this is with the high availability (HA) feature turned on. When using the high availability feature, it is going to cut your node scalability in half because it requires twice the amount of resources. Instead of having an 8-node deployment, which can handle 20 million metrics, you can deploy 4 data nodes that handles 10 million metrics. The other 4 nodes will be replica nodes.

In the diagram below, you will see there is a primary copy and a secondary copy. At the analytics layer, R1 is the primary copy and R1′ is the secondary copy. It is split up across nodes similar to the process you find with data persistence in RAID arrays. Gemfire is used to decide where the primary and secondary resources are going to go, it balances it out between the nodes. The same happens at the persistence layer, you will notice again, we have the R1 primary resource and there is a secondary R1′ resource on an alternate node.

All the user customizations go into the Global xDB (U1-3), these would be the custom views, dashboards, and reports. That does not get shared out across the nodes.

For the master node, the replica node is used as a backup to handle failover internally. Database replication is used to sync the Global xDB database. The best practice is to put the master and master replica on separate hardware.

In my next post, we will start diving into all the changes that have been made into the UI.