I have been working with Windows to VCSA 6.5 Migrations and wanted to share some steps of the VCSA configuration. Most common task was to add the VCSA to the Active Directory Domain and I’d like to show the steps to connect the VCSA from GUI and command line to the Active Directory Domain.

This is a pretty easy configuration, but we had some issues with the domain permissions when Windows to VCSA migration. So we had to connect the VCSA several times to the domain and here are the steps that anyone can follow.

Login to the VCSA webclient, hope you know that there is no more C# fat client for VCSA 6.5 access. Click on the Home menu and select the Administration

Click on System Configuration

Click on Nodes and you can see the vCenter server name unde Nodes, Select Active Directory and hit Join button to add to the Domain

Provide the details such as domain name, Organizational Unit, Username and Password to connect to the Active Directory

You need to reboot the VCSA to display the domain, without a reboot this change will not be applied. Once the reboot is completed. Domain will display under Active Directory section

Connect to the Active Directory using the Command Line

You can use the Command Line interface to connect the VCSA to the Active Directory here are the steps

Login to the shell, type “shell”

Use below command to connect to the Active Directory

/opt/likewise/bin/domainjoin-cli join [domain] [username]

It will prompt you to type the password, type the AD password for the user account that’s it

Proactive analysis is playing a key role in nowadays modern datacenters. Everybody is looking for options and tools which they can manage and identify the coming issues and outages to their datacenters. Runecast and VMware Skyline are two similar products which you can use to do a proactive analysis and take proactive measures to your mission critical workloads.

Recently, I joined to a webinar about the introduction of Runecast and also I was able to work with VMware Skyline as well. So I thought to try and do a review about these two products, hope my readers can take a great insight about these products after reading my post.

I already wrote an article about VMware Skyline and I thought to give a quick introduction about the features of Runecast Analyzer in this post.

So What is Runecast Analyzer?

It’s a tool which brings issue-prevention solutions to the business and minimize the impact to the complex systems can have by giving you the power of prevention measures along with the proactive analysis.

You can start it with a small OVF deployment and configuration can be varied with Small, Medium and Large scale deployments. You can select it in the deployment and it is obvious that virtual hardware resources can be different according to the configuration.

Once it deployed you can connect one or more vCenters to the Runecast Analyzer and after scanning your environment it will give you the status of your datacenter, it has a great interactive dashboard which customer can see the status of their datacenter against the VMware knowledge base.

I will write another article for the Runecast Analyzer review and I’d like to compare Runecast analyzer and VMware Skyline each other in this post.

Runecast Analyzer:

• Is a software product for end-users
• It has analytical engine with runs on-prem on the user side and evaluates VMware setup conditions. It scans vSphere and vSAN, immediately (1-2 minutes max.) shows detected problems. This can be repeated manually anytime or scheduled
• In addition to configuration and hardware details, Runecast Analyzer performs real-time log analysis and identifies issues in the logs immediately and links them to Knowledge Base articles
• Runecast software comes with an interactive dashboard with real-time issues, alerts and reports
• Works fully offline with no need for internet connectivity
• The software runs as a server on a single VM instance, no agents needed

VMware Skyline: 

• Comes as a service. Only for VMware customers under “VMware Premium Support”
• The end-user doesn’t have any visual dashboard/UI showing detected issues, alerts and risk management of connected vCenters
• VMware Skyline uses a “Collector” appliance which needs to be installed on-prem and it scans users’ systems and send findings to the Skyline cloud for further analysis. VMware personnel analyzes the data and generates a report on what is recommended to fix. The report is then delivered to the end-user for further communication. The end-user can’t control the speed of analysis or a time when the report is returned

Runecast Analyzer:

• Runecast Analyzer is frequently updated with VMware Knowledge Base articles for vSphere and vSAN, and support for NSX-V is under the development. It also covers VMware Best Practices for vSphere and vSAN
• The big advantage of Runecast product is the log analysis. It collects syslogs from vCenters, ESXi hosts and VMs and uses smart analysis to discover Knowledge Base problems and also speeds up troubleshooting by point to the trending log events, which are usually related to VMware problems
• In addition to vendor knowledge sources, Runecast Analyzer includes scanning against known issues in validated social media sources (blog articles, forums). In some cases critical issues appear first in social media before being officially documented by the vendor

VMware Skyline:

• VMware Skyline conducts analysis of a user data collected from the collector in alignment with VMware best practices
• Since VMware Skyline processes the users’ data through their support tier, the extra recommendation can come from that

Runecast Analyzer:

• Runecast Analyzer contains a database of VMware security hardening checks and federal DISA-STIG 6 compliance. The Analyzer scans periodically attached vCenters and reports all violations and passed criteria for both security profiles
• Aside from regular Knowledge Base updates, Runecast delivers special and immediate updates (daily basis) in any kind of severe security risk, such as CPU flaws Spectre and Meltdown, which allows admins to very quickly detect vulnerable assets and patch them properly

VMware Skyline:

• VMware Skyline doesn’t claim any specific security feature or compliancy standards

I have been able to log a support request and get the support from Runecast support, so at this point I’m not able to provide a comment on Runecast support.

My Thoughts

Download, installation and configuration was really easy. Connecting to the vCenter server was not a big deal either. Soon after I connected the vCenter I was able to scan my environment and interactive dashboard displayed all my issues and KB articles on the dashboard. I didn’t want to do a google search and find the relevant KBs it directly gave the KB articles, just a simple mouse click away. Also, this tool works without the internet and offline database can be included.

It provides the issue list as with the severity and I was easily identify the most critical and least critical issues in my environment.

Security hardening also one of the key point here and it gives you a greater visibility against the result with the Pass/Fail status, so it gives greater visibility of the environment security

I haven’t got a chance to log a support request and work with the Runecast engineers so I’m not able to comment on their support assistance.

According to my point of view Runecast is one of the greater tools for the Proactive and predictive analysis in your VMware based virtual environment. There is no this type of dashboard in VMware Skyline and also as a customer we are not able to see the real time status of our datacenter, we have to wait until we get the OSR from VMware engineer.

A side by side comparison of Runecast Analyzer and VMware Skyline from a person who tried both products:https://t.co/EppyYJcjq5

— Runecast (@RuneCastBiz) February 23, 2018

If you are planning to perform an ESXi host upgrade you need to find the compatible drivers for your ESXi host. This post is intend to show you the correct method to find the correct driver for your PCI device. If it is a Cisco UCS host you can find the correct enic and fnic drivers for the respective operating system.

Finding the correct driver is really important for the best performance, use of incorrect or the incompatible driver can lead to a performance impact and your production VM workload can be impacted. So, I think this post is quite important.

VMware Hardware Compatibility matrix also known as VMware HCL can be used to identify the drivers according to the hardware specification. We are going to use VID, DID, SVID and SSID to identify the driver

So what are these parameters?

Login to your ESXi host with the root user and execute vmkchdev -l and you will be able to see the similar output as below

So now go to the VMware Compatibility Guide and enter the values as below, select the search object as the IO Devices and hit Update and View Results to find the results. Select the required Hypervisor version and you will get the driver details page

Driver version and the download link will be there to download the driver make sure to check the Model and Driver firmware is correct for your PCI device

This is my Cisco UCS adapter details in my Cisco UCS Manager Console

So now you can find and download the compatible driver for your PCI device.

Also, you can use esxsfg-info | less command to get the same details for your PCI device, here is a sample output

I have been working with couple of vSphere 6.5 upgrades and I experienced this issue while I was upgrading Dell R630 Host in to ESXi 6.5. Host was not updating and getting failing throwing the below error message.

Actually, I have done host upgrades in many times but I did not get such issues with Cisco UCS blades and other ESXi hosts. But I got this error couple of times with these Dell hosts. I’m not sure whether there is a relation to the vendor but this is how I resolved this issue while I’m doing the ESXi upgrade.

I checked the disk space of the host and I noticed that one of the vfat scratch partition is full. I used “df -h” command to check the disk space to get only the vfat partitions use “df -h | grep vfat“. Here is the output.

I executed “du -h /<my_vfat_volume>” to get the breakdown of the utilization

I removed the “6.5.0” folder inside the packages folder. Executed “rm -rf <folder_name>” to remove the folder

Now located a “vpxa-worker-zdump.xxx” file in the /var/core/ folder removed the dump file. Executed “rm -rf vpxa-worker-zdump.xxx” command to permenetly delete the file [Step 7]. See the below detailed steps.

Again, checked the vfat partition space and now it is 0% utilized.

I started the upgrade and it was succeeded after that. Hope this post will help someone who experience the similar issue while doing the upgrade.

We experienced this issue with the template deployment with the custom specification wizard. The important thing here is we experienced this issue soon after the vSphere 6.5 upgrade. Our Virtual Machine deployment team reached us to find a solution after the upgrade.

Previously, they were using the old C# client to deploy the Virtual Machines and with the change of the vCenter this issue came in to the action. I thought to write this post to share and explain the findings and the steps to resolve the issue.

Error occurred while loading data. Check the log files for details

First thing first what are the permission levels required?

Below permission levels should be fulfilled to use the custom specifications for a Role.

Virtual Machine -> Provisioning -> Read Customization Specifications

Virtual Machine -> Provisioning -> Modify Customization Specifications

Also, the created or role that these provisioning users are sitting at should be applied to the vCenter level, it was added to the Datacenter level and I had to change it to the vCenter level as shown in the below screen capture

After that our provisioning engineers were able to see the custom spec scripts

Still not finished…..

“The public key in the specification does not match the vCenter public key. You have to reenter the password in order to proceed”

This error appeared after fixing the above issue

So we had to update the passwords of the templates, follow below steps to update the template passwords

Click on the “Home” icon and select the “Policies and Profiles”

Select the “Customization Specification Manager” and select the template and click on “Edit” icon

Select the “Administrator Password” and update it in the wizard

Click “OK” to complete the change

After that we were able to use the Customization Specification scripts without any issues

If you found this post as useful please rate the post and share it!

I ran in to this while I was trying to access My Lacie NAS drive from my Windows 10 computer. NAS was directly connected to the network and no access permissions required to access the shared folders. I have set permissions according to the user groups. So anyone who got the access permissions can access the relevant folders and if users don’t have the permissions folders will not be displaying in the explorer.

So I tried to connect this drive as a Network file share to my laptop and I was prompted with this error message on my screen. So I thought to share this post in my blog so anyone who has the same issue able to find this in the internet easily.

This is the prompted error message

So I found that this access is blocking from the local group policy and I opened the Run window (Win + R) and typed gpedit.msc to open the Local Group Policy Editor.

Navigated to Computer Configuration -> Administrative Templates -> Network -> Lanman Workstation and selected the Enable insecure guest logons as shown in the below screenshot

By default it was disabled and I have enabled the policy

Click OK to apply the policy and tried to access the shared network drive, I was able to access My Lacie NAS drive from the windows explorer

If you found this post as useful please rate the post and share it!

This is a bit of strange issue which came after the vSphere 6.5 VCSA migration from Windows based vCenter server 5.5 and 6.0 versions. First of all I’d like to mention that we have tested this migration under couple of scenarios in many times in our test environment and we did not encounter similar issue in one of the tests.  So, we were fully confident with the upgrade and we started this in our Production environment.

In this issue I’m not quite sure whether this is any environment specific issue or not because I was not able to find any online articles on the internet. Also, the opened support request with VMware BCS support is still open. The bad side of this issue is sometimes we lost entire permissions levels and we could not see any added permissions after the completion of the migration, but in some cases we could see the permission but those permissions were not working so we had to re-add them to get the access.

Also, VMware confirmed that there is no other way to restore the permission other than re-add them in the new vCenter server appliance. But, after doing couple of tests we were able to find a work around for this and I’m happy to confirm that this method is now working without issues. Anyone who knows any other method to restore the permissions or any other work around please let me know.

First step was configure existing Active Directory as a LDAP Server Identity Source while we keeping the Active Directory Integrated Identity Source before we start the Migration

Then remove the Active Directory Integrated Identity Source and test the user access. Start the Migration

Once the migration was completed identity sources was similar to this and set the created LDAP identity source as the default identity source

We had to remove the Local Identity source to avoid some connectivity issues and login latency for users.

Once we done these changes migration worked perfectly and we didn’t see any issues after that.

If you found this post as useful please rate the post and share it!

Runecast recently released the v1.7 with VSAN support and it has some greater improvements. The major improvement is VSAN support, and vCenter integrated Web plugin. In this post i’d like to discuss these new features and improvements of this release.

Runecast is a greater solution which can provide the best practises and security hardening measures based on the VMware knowledge base. Customers can have a greater visibility of their datacenters and proactive measures can be taken based on the guidelines and knowledge base articles provided in the dashboard. With the release of v1.7 it has extended its support for VMware VSAN as well.

It has three main added improvements to this release:

Greater visualization of your datacenter with it’s interactive dashboard and it’s widgets, now it has a improved dashboard called VMware Stack View Dashboard.

I’d like to highlight VMware Stack View dashboard as it show you the severity of the issue and where it exist in the layer of your VMware environment. It’s a simple pane of glass to all your vCenter resources.

Runecast Analyzer v1.7 Provides the VSAN best practises and recommendations

VSAN is one of the excellent software defined storage solution provided by VMware which can have greater benefits to its customers with highly performenced and scalable solution. Runecast provide the best practises and recommendations for VMware VSAN against the VMware’s larger knowledge base database. It is critical and should be monitored closely with extra care to avoid the possible future issues which can make a serious impact to your Virtual Machine workloads.

Runecast Analyzer is completely working in the offline mode and does not send any data over the internet, so your data stores in your on-prem datacenter.

Runecast Analyzer v1.7 initial configuration and vCenter Plugin configuration

Runecast Analyzer added another valuable configuration item to this release, vCenter plugin which can see the Runecast analysis status and the recommendations in the vCenter.

Once you deployed the Runecast Analyzer as a generic OVF deployment, you can add your vCenter server to Runecast Analyzer by Add vCenter option. Go to Settings->vCenter Connection and click on Add vCenter

Provide your vCenter details and credentials

You can see the Thank you note after the successful configuration to the vCenter

To install the vCenter web plugin click on the Edit button under Action and select Install plugin option

Click on Install in the next window to install the plugin

Let this installation to complete

You can see the plugin status under settings as Installed

After the successful installation of the plugin, you will be able to see the Runecast option in the Home dropdown menu as well as the Home tab of the vCenter. If you were logged in please logout and login to the vCenter to see the plugin

To setup the plugin click on the icon and you can see Runecast Main View, click on GO TO SETTINGS

Now Runecast API Access Token should be configured to start the connection.

To Setup the API Access Token, go back to Runecast Settings -> API Access tokens and click on Generate API access token

Provide a description , functionality mode and Generate the Access Token

Copy the generated token

Provide the FQDN, Generated Access Token and SAVE SETTINGS

You can see the empty Runecast main view and click on PERFORM SCAN to do a scan of your environment

Wait for couple of seconds and it will pull out the details for you and you can see all these details in your vCenter server view

Another good thing….

Runecast Analyzer has added the Spectra and Meltdown analysis Knowledge Base to its database and you can scan your environment against the latest security vulnerabilities and harden your VMware based environment

My Thoughts

Runecast Analyzer with the new release of the VSAN support is a greater product in terms of proactive analysis of your VMware based virtual environment. There is no doubt of the continuous improvement of the product and adding new features in the future.

From the update point of view if you have the active subscription you have the online auto update but it will work with more secure environment without the internet access using it’s offline database, so your data is completely secure.

If you found this post as useful please rate the post and share it!

Once I’m explaining about Oracle Ravello cloud to one of my friend he asked few more questions and he wanted to know the steps of deploying a virtual machine on the bare metal infrastructure. This feature was recently added to this Oracle cloud platform and I thought to write up an article about deploying a virtual machine on Oracle bare metal infrastructure on my blog.

By eliminating a layer of hypervisor in the middle, HVX is able to provide near-native performance to the running virtual machines. There are many improved features in this bare metal infrastructure and below features can be introduced with this platform.

Broad range of Elastic Compute

• Bare metal servers, bare metal NVMe, VMs.
• Provision in minutes, Pay by the hour

High Performance Storage

• Local NVMe servers
• Dynamically attachable remote Block Storage with consistent general purpose IOPS
• High performance, high durable object  storage

Virtual Private Networking

• Managing your high scale Private IP networks
• IPSec VPN, Fast Connect
• Stateful firewalls, optional internet gateway, load balancing

Industry-leading performance and security with payas-you-go pricing

• Available in < 5 minutes
• Standard: Non-NVMe SSD, 256GB RAM
• High IO: 12.8TB NVMe SSD, 512GB RAM
• Dense IO: 28.8TB NVMe SSD, 512GB RAM

How to Deploy a Virtual Machine on Bare Metal Infrastructure

Steps are quite easy but this capability only limited to three datacenters, definitely it will be available soon in the other datacenters as well in near future. At the time of writing this article it is available on below datacenters:

• US East 5
• US Southwest 1
• Europe Central 3

You have to deploy your application in one of these datacenters

Select the Virtual Machine on your canvas and Click on Advanced Configuration

Change the “preferPhysicalHost” parameter to “true” and click OK

Save the settings of the Virtual Machine

Update the Application and you are all set and your Virtual Machine is now running on the Oracle Ravello Cloud bare metal infrastructure. Power on the Virtual Machine and experience the improved performance.

I recently, started my YouTube Channel for this blog and I have created a small video to add to my Video Gallery.

If you found this post as useful please rate the post and share it!

I was playing around with YouTube and one of my friends asked me how we can get the YouTube API key for our channel for the use of some other work or application. It was quite an easy thing and I explained him the steps to get this and I thought to write an article with the steps and thought to share it with my friends.

I usually blog about Virtualization and cloud related things but I thought to write this article and share it in my blog as I’m using more Google products to my work. I’m a big fan of Google mail, Google Hangouts and obviously YouTube and some other Google Apps. So, this post is for the sharing purpose to all my readers.

To get in to the Google Developers Console click here. You will be asked to sign in to an Google account, so Sign in to your account

Click on Create Project

Click on Create again

Provide a Project Name and click Create

You’ll be redirecting to the Dashboard

Goto APIs & Services -> Library

Click on YouTube Data API v3 

Click on Enable to enable the Youtube API

You will be redirecting to the Dashboard and You can see the Project Name and the API status (enable/disable)

Provide the application method that you are going to calling the API

Select the Type of data that you are going to access (Public or User Data) and click What Credentials do I need? button

You can see Your API Key and click On done once you copied the key

If you found this post as useful please rate the post and share it!

Vembu has reached me and gave me an opportunity to test their flagship product Vembu BDR Suite, and I developed a Lab with a Backup server to test Vembu products. Before we start I’d like to give you the background of the company, company was founded in 2002 and currently serving more than 60,000 businesses all over the world.

Initially, Vembu’s product was around SQL Migrations and then they focused on Backup strategies, now it has been over 10 years and company developed with the more capable backup and disaster recovery solutions. Important thing of their products is the number of solutions provided for the different platforms including VMware, Hyper-V, Image backup, Network Backup, Cloud Disaster Recovery, Online Backup, SaaS and Management and Monitoring tools, also Vembu supports for Applications such as Microsoft Exchange, SQL and Sharepoint and Cloud products such as Google Apps and Office 365 and you can store them in offsite and onsite store or even to the cloud. I think below product download portal will give you a clear insight of the capabilities of these products.

You can access and download these products from Vembu Portal, If you don’t have an account you can sign up for the portal account and one of their agents will reach you shortly.

Once you got access to the portal you can see some greater features in the portal, You can see the licenses usage in the Dashboard, also Products, Licenses, Subscriptions, Billing Address, all invoices, BDR Server information, Purchase history and Profile can be update in the same page.

I don’t have License yet for my BDR Server

So Let’s see how we can configure the Vembu VM backup in the VMware Environment

Vembu VM Backup Installation

To Download the VM Backup Click on the Download under Vembu VM Backup

Click on the Download Installer and you can see the Supported Operating Systems which you can install the BDR Suite

Once You downloaded the Setup run the Installer

Installer will start and click on Next in the Welcome Window

Accept the License agreement and click Next

You can customize the configuration by selecting the checkbox and you can see the default configuration

If you wish to change the configuration you can change the configuration in the next step and click Next to continue

Select the Installation folder

Select the Drive letter for the Storage Repository in the next step

Note: C: Drive is not recommended by the vendor

If you select the C: drive as the Storage repository below warning message will appear on the screen

Also, adding a Shared folder (CIFS/NFS) is also supported and NAS and SAN devices connected using NFS or CIFS supported

You can change the Webserver port, admin username and password in the next window. Change the details according to your requirement and click Next to continue the installation

Click Install to begin the installation

Installation will install the necessary packages to your server and Click Finish to Start the services and open the Web Console (You can select these options from the check boxes)

Once it opens the Web Console type your username, password and click Sign in

Select the Time Zone for your BDR Server and click on Save

Provide a unique ID for your BDR Server Vendor recommends MACHINENAME.DOMAIN name for this because it is an unique id, click on Update the submit the changes

Once you submit the changes you can see the configuration option, I’m going to work with my VMware vSphere Lab so I select Add VMware vSphere Server to add my vCenter server

Provide the vCenter Server details,credentials and port to communicate with the vCenter server, click on Save

You can see the added vCenter server after that in the Console

So I’d like to run a backup and restoration procedure in a different post as this post is going to be a long post. So stay tuned, let’s see how we can backup and restore the Virtual Machines and what are the available option in this backup configuration

My Thoughts

This is was the first experience with Vembu backup and it was quite easy process to start work with the product. I was easily create an account in the Vembu portal and soon after I completed the Sign in process I got an email from one of their agents asking a convenient time for a walkthrough of their products and details under Vembu BDR suite. He was willing to help me to find the best choice for my requirement and the environment.

There were few links to their product configuration guides and procedures. Also, a summary of the best practises and and initial access procedures soon after the installation of the product. There were some useful email contacts for technical assistance and the sales. Also, you can request a support assistant during the evaluation when you click on the download, below widget will appear on your screen and you can easily request assistance by submitting the form

From the Installation point of view it was pretty easy task. Anyone who has the software installation experience can easily install the BDR Suite within couple of minutes, there was no separate database configuration Postgres database is installing automatically in the same server, so I did not want to worry about any Database configuration. Soon after the completion of the installation application was ready to use and Web console opened in my screen. It has a simple and user friendly dashboard for the rest of the configuration.

Adding my vCenter was just few clicks and it was added easily, so far according to my experience it was quite simple easy and straightforward to configure the BDR server. Let’s see what are the options of the Backup and Restoration procedures and how feasible these options in my next post. So, stay tuned!

If you found this post as useful please rate the post and share it!

In this article I’d like to show you the complete steps to configure SNMP v1 in the vCenter Server appliance. I have been closely working with vCenter server Migrations to 6.5 in my clients and had to re-configure the SNMP targets after the migrations, hope this article will help someone who try to configure the SNMP in a vCenter Server 6.5 Appliance.

What is SNMP ?

I assume you already know the meaning of the SNMP. These four letters stands for Simple Network Management Protocol which is collecting and organizing managed devices on IP networks. It can use to send device information from one device to another. Most of the network monitoring tools use SNMP traps to monitor the devices and capture the alarms and alerts.

In previous release of the vCenter server there was a GUI to configure the SNMP servers in the vCenter Server settings, but in VCSA you have to use command line interface to configure the SNMP target servers.

Firstly, enable ssh login to the VCSA

Open a shell console to the VCSA

Type:

snmp.get

To view the current snmp settings of the VCSA, initially there shouldn’t be any configuration in the VCSA

First thing, we need to configure the SNMP community to configure the SNMP in the VCSA, use below format to set the SNMP community to your VCSA

snmp.set --communities <COMMUNITY_NAME>

Type below commands to configure SNMP targets, for one SNMP target configure as below:

snmp.set --targets <IP_ADDRESS_OR_HOSTNAME>@<PORT>/<COMMUNITY_STRING>

You can see the example as below

To configure multiple targets use the command as below:

snmp.set --targets <IP_ADDRESS_OR_HOSTNAME>@<PORT>/<COMMUNITY_STRING>,<IP_ADDRESS_OR_HOSTNAME>@<PORT>/<COMMUNITY_STRING>

Once these targets are configured execute the snmp.get command to view the configuration

To set the SNMP log level use below command

snmp.set --loglevel <DEBUG|INFO|WARNING|ERROR>

Finally, enable the SNMP on the VCSA

snmp.enable

SNMP Test and Help

To send a SNMP Test command use the below command:

snmp.test

To View the SNMP Statistics, use below command:

snmp.stats

You can also view the available options and configurations, type “snmp.” and hit TAB to view the options

To view the available configuration options in “snmp.set” command use below command:

snmp.set --help

EX: This is the snmp.limits + TAB command output to see the Configuration Options

That’s it for now with the SNMPv1 Configuration.

If you found this post as useful please rate the post and share it!

We have been informed by VMware with regard to this issue and this is going viral all over the world after the patch updates comes under KB4088875 and KB4088878. This is basically affecting to the Windows 2008 R2 and Windows 7 Virtual Machines with VMXNET3 adapters. VMware is currently investigating and working in their full scope to find a solution for this serious issue. If you are planning to update your Patches please make possible effort to avoid these patches from your List.

You might be getting similar errors if you have DHCP configured

As per the feedback we are getting from social media some had to upgrade the Virtual Hardware levels and some had to add the IP information again to those affected Virtual Machines. If you come up with anything different to these steps please feel free to comment on this so we can share it with others.

Another workaround to fix this issue was remove the NIC and re-adding to the Virtual Machine, also reinstalling the driver was one of the workarounds, but at this point there is no guarantee of any of these workarounds to fix your issue. Just try and see.

Please keep your eyes on below external links to get the latest updates, I will update the post as I’m getting the information:

Official Update From VMware (as of 15/03/2018):

If you found this post as useful please rate the post and share it!

We had this issue while we were upgrading the ESXi hosts to vSphere 6.0 from 5.5. vCenter server was successfully migrated with the VMware Update Manager and it’s database. We tried to upgrade the host after importing the base ESXi iso and crating the baselines from the beginning. I can confirm that we did not get any error while we are upgrading the VMware Upgrade Manager.

However, we had to follow the below steps and had to reinitialize the database to fix this issue. Make sure to take a backup of the Update Manager Database, your ISO images, offline patches and any host extensions before we start the process.

This was the appeared error while we were trying to upgrade the host.

Before we start the re-initialization of the database, stopped the Update Manager service, opened a command prompt as an administrator and changed the directory to C:\Program Files (x86)\VMware\Infrastructure\Update Manager\ by executing below command.

cd "C:\Program Files (x86)\VMware\Infrastructure\Update Manager\"

Executed the below command to start the re-initialization process ( dot is required after -C and -L switches)

vciInstallUtils.exe -O dbcreate -C . -L .

See my screen capture for the command that I executed

It will run database queries and will take couple of time to complete the task similar output can be seen in the command prompt

After the completion of this command all the baselines, Images and extensions wiped out from the VUM database. Had to manually add them and created the necessary baselines. That’s the reason behind I asked you to save all your VUM data before start the process.

If you found this post as useful please rate the post and share it!

I thought it is great to write this article as this was quite useful tip for VMware admins to find out what are the default values of the vCenter server appliance such as SSO domain, DB user, DB DSN, domain DN and I wanted to know which account used to add the VCSA to my active directory domain while proceeding with the 6.5 upgrade.

So, I used below steps to find these details in my VCSA and I think this might be helpful for all my VMware colleagues.

Another practical situation was, I built couple of test vCenters with 6.5 and used a custom vSphere SSO domain, I couldn’t touch my vCenters for weeks and I started my test again. Bad Situation !!!, I completely forgot my custom test domain and I didn’t take a note of that, what should I do with this at this point?

Use of “Install Defaults Directory”

In vCenter Server Appliance, it has a directory called “Install Defaults” and it contains lots of files with these details and we can easily read those files and find out the required information.

Open a SSH session and login as root. Yes, you need to have the root credentials anyway to check this. Then go to Shell prompt

change directory to /etc/vmware/install-defaults, list the contents of the directory by executing “ls” command you can see a similar output like this and use below commands

cd /etc/vmware/install-defaults
ls

What is my SSO domain?

Read the content of the “vmdir.domain-name” to view the SSO domain using cat command and in the output you can see the SSO domain of your vCenter server

cat vmdir.domain-name

Which AD account used to connect to the Active Directory?

Read the content of the “ad.domain.username” file

cat ad.domain.username

Reading the content of these files provide more information about the vCenter appliance these are just few examples, these is just the output of reading the database related information

Here, I have created a short video to show “How to get the forgotten Single Sign-On domain from VCSA”

I hope you enjoyed this post!

If you found this post as useful please rate the post and share it!

We have been noticing some Virtual Machine unavailability alerts due to a running Virtual Machine level backups. Customers were complaining about the application unavailability and they were getting some alerts saying Virtual Machine is not available or down state at the same time of everyday. This issue appeared due to the FREEZE operation on the file system.

We have tried several troubleshooting steps to find a solution for this and had to spent considerable amount of time on the same issue. Finally, we were able to find a solution after performing the below steps separately for Windows and Linux Virtual Machines.

Note : VMware Tools must be installed and should be up to date.

This was the workaround for Linux Virtual Machines

Connected to the ESXi host which was running the affected Linux Virtual Machine and changed the directory to the Virtual Machine folder.

#cd /vmfs/volumes/<datastore_name>/<Virtual_Machine_folder>

Checked disk.EnableUUID setting is in the .vmx file

/vmfs/volumes/<datastore>/<virtual_machine_folder>/#cat virtualmachinename.vmx | grep disk.EnableUUID

In my case this parameter did not exist, so I had to edited the .vmx file using a text editor, added the disk.EnableUUID parameter and set it as “FALSE”, if the parameter is exist set the value to “FALSE”. (I used “vi” editor and “:wq” used to save the parameter).

Confirmed the parameter addition using the same command

After the completion of the parameter addition logged in to the Operating system of the Virtual Machine and changed the directory to “/etc/vmware-tools/” folder and check whether there is “tools.conf” file is there, if it id not there then create a file. In my case we had to create the file.

#cd /etc/vmware-tools/
#/etc/vmware-tools/ls

To create the file used vi editor

#/etc/vmware-tools/vi tools.conf

Added the below content to the file, save it and exit

[vmbackup]
enableSyncDriver = false
[logging]
vmtoolsd.level = debug
vmsvc.level = debug
vmusr.level = error
vmvss.level = debug
guestinfo.level = error
vix.level = error
vmsvc.handler=vmx
vmbackup.level=debug
vmbackup.handler=vmx

Workaround for the Windows Virtual Machines

If it is a Windows Virtual Machine we had to check the Windows Application logs to identify whether there are errors related to the VSS Writers. In some cases VSS Writers may having some issues and had to exclude the VSS Writers from the snapshot operation. A sample issue can be seen in this Microsoft article, you might be seen something similar to these symptoms.

To find the VSS Writers run a Command Prompt with the Administrative privileges and run the below command

C:\Windows\system32>vssadmin list writers

Read more about VSSAdmin Utility from Microsoft Documentation.

Then opened the “%ALLUSERSPROFILE%\Application Data\VMware\VMware Tools\” location in the explorer

Created a “vmbackup.conf” file, to do that opened a Notepad added the VSS Writer and saved it as a configuration file. See below screen captures for more information

Created the “tools.conf” file in the same location and added the below content

[logging]
log = true
vmtoolsd.level = debug
vmtoolsd.handler = file
vmtoolsd.data = C:\ProgramData\VMware\VMware Tools\vmtoolsd.log
vmsvc.level=debug
vmsvc.handler=vmx
vmbackup.level=debug
vmbackup.handler=vmx
vmusr.level = error
vmvss.level = error
guestinfo.level = error
vix.level = error

You should be able to see something similar to this after the creation of the file

Restarted the VMware Tools Service

After doing these configuration changes to the .vmx file and the .conf file of the affected Virtual Machines this issue was resolved and Virtual Machines are not freezing during the backup operation.

If you found this post as useful please rate the post and share it!

This is just a small tip for the VMware tools installation on a Linux Operating System. If you are installing the VMware tools on a Linux Machine you might be familiar with this error message when you perform the “./vmware-install.pl” command. So, I thought to write this as Tip! for the VMware administrators and I think this might be really helpful for them.

This is the error that can appear on your terminal and I just listed the files in the “vmware-tools-distrib” folder for your information.

For example this is the command for the CentOS:

[root@server vmware-tools-distrib]#yum -y install perl

For more information See the steps to install the tools on a linux system, I wrote this article using a CentOS Virtual Machine

How to Install VMware Tools on Linux Virtual Machine

Mount the VMware Tools installer to the Virtual Machine, Right-Click on the Virtual Machine -> Guest -> Install/Upgrade VMware Tools

Login to the Operating System and follow the below Steps in the terminal:

[root@server]#mkdir /mnt/cdrom
[root@server]#mount /dev/cdrom /mnt/cdrom
[root@server]#cp /mnt/cdrom/VMwareTools-version.tar.gz /tmp/
[root@server]#cd /tmp
[root@server tmp]#ls | grep VMwareTools
#Note the VMware Tools Version, ex: VMwareTools-5.0.0-12124.tar.gz
[root@server tmp]#tar -zxvf VMwareTools-version.tar.gz
[root@server tmp]#cd vmware-tools-distrib
[root@server vmware-tools-distrib]#./vmware-install.pl

Check the VMware Tools status in the Summary Tab

Once You confirmed the status unmount and remove the installation packages from the Operating System for the clean up

[root@server vmware-tools-distrib]#cd #
[root@server]#umount /mnt/cdrom
[root@server]#rm /tmp/VMwareTools-version.tar.gz
[root@server]#rm -rf /tmp/vmware-tools-distrib

If you found this post as useful please rate the post and share it!

Microsoft has released preview of Windows Server 2019 Preview build 17623 and announced that it will be generally available in second half of 2018. At this point you can access the Preview build 17623 from Windows Insider program. Previously, I wrote an article about the Windows Server 2016 at the first release and you can read it here.

Windows Server 2019 Preview build on top of the strong foundation of the Windows Server 2016, according to Microsoft Windows Server 2016 is the fastest customer adapted Server version and they believe Server 2019 will keep same momentum after the release. It also supports the inplace upgrade of Windows Server 2012 R2 and 2016.

Suits for Hybrid Cloud Scenarios in the Customer Environment

One of the main purpose of this release is to support to the Hybrid Cloud migration of the customer workloads, these days customers are extending Active Directory integration to the cloud, synchronizing file servers, and backup in the cloud and this release will perfectly suits for the Hybrid Cloud environment.

Also, one of the other important thing is to work with the Project Honolulu which was released in the Microsoft Ignite 2017. Microsoft believes that Windows Server 2019 and Project Honolulu will help customers to easily integrate Microsoft Azure Services such as Azure Backup, Azure File Sync and disaster recovery to their customers.

Security Concerns

Microsoft has made improved security features to this release in three fold approach – Protect, Detect and Respond. Windows Server 2019 Preview comes with major improvements in these three areas.

Shielded Virtual Machines

Microsoft has introduced Shield Virtual Machines with this release, which means protect virtual machines (VM) from compromised or malicious administrators in the fabric so only VM admins can access it on known, healthy, and attested guarded fabric. Linux Virtual Machines will support as a Shielded Virtual Machine with this release of Windows Server 2019 Preview and Microsoft is extending the VMConnect to improve the troubleshooting capabilities.

Windows Defender Advanced Threat Protection

Added the memory and kernel level attacker activities and abilities to take actions on compromised machines in response to incidents such as remote collection of additional forensic data, remediating malicious files, terminating malicious processes etc.

Windows Defender ATP Exploit Guard

This has the improved host intrusion prevention capabilities, the four components of Windows Defender Exploit Guard are designed to lock down the device against a wide variety of attack vectors and block behaviors commonly used in malware attacks.

• Attack Surface Reduction (ASR)
• Network Protection
• Controlled Folder Access
• Exploit Protection

Encrypted Network

Encrypted Networks that will let admins encrypt network segments, with a flip of a switch to protect the network layer between servers and networks are simple to configure DTLS-based encryption using the Network Controller to manage the end-to-end encryption and protect data as it travels through the wires and network devices between the hosts It is configured by the Administrator on a per-subnet basis. In this case VM to VM traffic is automatically encrypting as packets leave from the hosts.

Extending Clusters with Cluster Sets

“Cluster Sets” is the new cloud scale-out technology which increase the number of Cluster Nodes in the SDDC (Software Defined Data Center). Cluster sets groups the Windows failover clusters. Cluster Sets enable the movement of the Virtual Machines across the Clusters.

No NTLM Authentication for the Failover Clusters

Windows Failover Clusters no longer use NTLM authentication and use Kerberos and Certificate based authentication.

Known Issues:

• In‑place OS upgrade: Domain Controllers. During an in-place OS upgrade, Active Directory (AD) Domain Controllers (DC) might not be upgraded correctly. So, back up any AD DCs before performing an in-place OS upgrade.
• Editing or creating policies for AppLocker can cause the MMC snap-in to crash when generated rules for a packaged app.
• After upgrading the operating system, the AppX database may have corrupted entries, which causes problems for components that use those entries.
• Testing of the Windows core may fail because of a timeout while attempting to load the test libraries.

Downloading and Installation of Windows Server 2019 Preview

Registered Windows Insider users are able to download the preview version of the Windows Server 2019, Use Windows Server Insider Preview Download page to download the ISO file or the VHDX for the Virtual Machine instance, If you are not registered register as a Insider user and share your feedback for the product improvement.

Select the Language and Download your file

Installation doesn’t have any differences just follow the normal installation procedure

It has four selection of the Operating Systems, Windows Server without GUI

Desktop Experience should be installed if you need the GUI

Provide the Password for the Local Administrator Account

Login Screen is similar to the Windows 10 login screen

This is how Server Manager looks like

GUI and Start Menu is similar to Windows 10

System Configuration boot still shows Windows Server 2016 as I mentioned earlier in this post

That’s it for this post, let’s see how we can install the Project Honolulu on Windows 2019 Preview in a different post.

If you found this post as useful please rate the post and share it!

The post Introducing Windows Server 2019 Preview appeared first on TECHCRUMBLE.

Runecast recently released the v1.7 with VSAN support and it has some greater improvements. The major improvement is VSAN support, and vCenter integrated Web plugin. In this post i’d like to discuss these new features and improvements of this release.

Runecast is a greater solution which can provide the best practises and security hardening measures based on the VMware knowledge base. Customers can have a greater visibility of their datacenters and proactive measures can be taken based on the guidelines and knowledge base articles provided in the dashboard. With the release of v1.7 it has extended its support for VMware VSAN as well.

It has three main added improvements to this release:

Greater visualization of your datacenter with it’s interactive dashboard and it’s widgets, now it has a improved dashboard called VMware Stack View Dashboard.

I’d like to highlight VMware Stack View dashboard as it show you the severity of the issue and where it exist in the layer of your VMware environment. It’s a simple pane of glass to all your vCenter resources.

Runecast Analyzer v1.7 Provides the VSAN best practises and recommendations

VSAN is one of the excellent software defined storage solution provided by VMware which can have greater benefits to its customers with highly performenced and scalable solution. Runecast provide the best practises and recommendations for VMware VSAN against the VMware’s larger knowledge base database. It is critical and should be monitored closely with extra care to avoid the possible future issues which can make a serious impact to your Virtual Machine workloads.

Runecast Analyzer is completely working in the offline mode and does not send any data over the internet, so your data stores in your on-prem datacenter.

Runecast Analyzer v1.7 initial configuration and vCenter Plugin configuration

Runecast Analyzer added another valuable configuration item to this release, vCenter plugin which can see the Runecast analysis status and the recommendations in the vCenter.

Once you deployed the Runecast Analyzer as a generic OVF deployment, you can add your vCenter server to Runecast Analyzer by Add vCenter option. Go to Settings->vCenter Connection and click on Add vCenter

Provide your vCenter details and credentials

You can see the Thank you note after the successful configuration to the vCenter

To install the vCenter web plugin click on the Edit button under Action and select Install plugin option

Click on Install in the next window to install the plugin

Let this installation to complete

You can see the plugin status under settings as Installed

After the successful installation of the plugin, you will be able to see the Runecast option in the Home dropdown menu as well as the Home tab of the vCenter. If you were logged in please logout and login to the vCenter to see the plugin

To setup the plugin click on the icon and you can see Runecast Main View, click on GO TO SETTINGS

Now Runecast API Access Token should be configured to start the connection.

To Setup the API Access Token, go back to Runecast Settings -> API Access tokens and click on Generate API access token

Provide a description , functionality mode and Generate the Access Token

Copy the generated token

Provide the FQDN, Generated Access Token and SAVE SETTINGS

You can see the empty Runecast main view and click on PERFORM SCAN to do a scan of your environment

Wait for couple of seconds and it will pull out the details for you and you can see all these details in your vCenter server view

Another good thing….

Runecast Analyzer has added the Spectra and Meltdown analysis Knowledge Base to its database and you can scan your environment against the latest security vulnerabilities and harden your VMware based environment

My Thoughts

Runecast Analyzer with the new release of the VSAN support is a greater product in terms of proactive analysis of your VMware based virtual environment. There is no doubt of the continuous improvement of the product and adding new features in the future.

From the update point of view if you have the active subscription you have the online auto update but it will work with more secure environment without the internet access using it’s offline database, so your data is completely secure.

If you found this post as useful please rate the post and share it!

The post Runecast Analyzer v1.7 with VSAN Support and Web Plugin appeared first on TECHCRUMBLE.

I recently attended a webinar with Runecast and it was about how to Mitigate Spectre and Meltdown in VMware Environments.  In this post, you can see the steps to Mitigate Spectre and Meltdown in VMware Environment using Runecast Analyzer. Previously, I wrote an article about Spectre and Meltdown and the VMware recommendations to mitigate this Spectre and Meltdown vulnerabilities, you can read it here.

In a VMware Virtual environment protection should be start from all layers starting from Management to core components.

Mitigation should be staring from vCenter server to the ESXi hosts, Runecast analyzer will scan the environment against the VMware knowledge base and provide the recommendations.

Runecast Analyzer Appliance 1.7.3.0 – Update Appliance

If you are planning to mitigate Spectre and Meltdown vulnerabilities using Runecast Analyzer make sure to have the latest version of the Analyzer. Sometimes you may already seen the update available notification in the top panel.

To update the  Runecast Analyzer login to the appliance VAMI interface type “https://<IP_ADDRESS_OR_FQDN>:5480” on the browser and use the credentials provided in the Runecast Analyzer user guide.

Go to update Tab and check whether you can see the “Available Updates” notification and the appliance version, otherwise click on “Check Updates” button. To install the updates click on “Install Updates” button

It will prompt to install the updates click on OK to start the update.

Check the version again in the VAMI interface after the update

After login to the Runecast Analyzer you can see the updated information in the notification panel

Run an analysis against the VMware environment and you can see the Spectra and Meltdown issues under issue list

Perform the Analysis

Click on the issue it’ll provide the VMware response for the Spectra and Meltdown, KBs and detailed mitigation steps under details tab

Affected components can be seen in the “Findings” tab

One important thing is it will show the complete updates and communications regarding the Spectre and Meltdown updates. Follow the instructions and the updates in order to mitigate the vulnerabilities in the environment. Runecast will give you the best instructions and updates for your environment.

Use vCenter Web Client

Runecast Analyzer has a vCenter Web Client plugin and you can find my previous article about installing Web Client plugin to the vCenter can be found here.

Plugin can be used to scan the environment and issue can be seen in the “Details” tab

Under “Affected objects” displays the ESXi hosts subjected to the vulnerability

Change Runecast Analyzer Update Settings to Automatic Updates

To enable the automatic Updates, login to the VAMI interface. Go to “Update” tab and Select “Settings“. It has three main options for the updates. Select according to the requirement.

Runecast Analyzer updates may include KB/Best Practice/Security hardening updates, application component updates or appliance OS updates.

If Runecast Analyzer has the internet connectivity it can use the default repository for the updates. If there is a proxy requirement add the proxy configuration under network settings.

If you found this post as useful please rate the post and share it!

The post How to Mitigate Spectre and Meltdown in VMware Environment with Runecast Analyzer appeared first on TECHCRUMBLE.