Upgrade 2012 R2 Cluster to 2016 using Cluster OS Rolling Upgrade

mirrored from : http://www.altaro.com/hyper-v/upgrade-2012-r2-cluster-2016-using-cluster-os-rolling-upgrade/

Starting with Windows Server 2016 we get a new, nice feature which makes the cluster upgrade job a lot easier and cheap. And to be honest, it was about time. In previous server versions if we wanted to upgrade a Windows cluster we had to build a new one with at least one host in it, then migrate the virtual machines, reinstall the old hosts with the new OS then attach the hosts to the new cluster. With server 2016 it’s a little bit easier but we still have some work to do, like re-configuring networks, Hyper-V settings, storage, etc. At least we don’t need to get downtime for our virtual machines and don’t need extra hardware.

Now that I’ve mentioned all the goods, it’s time for the bad ones also, which are actually just one: we need to have at least Windows server 2012 R2 running in the current cluster. This means that if our Hyper-V cluster is running 2008R2 or 2012 for that matter, we will need to use the old method to upgrade the cluster. To make it even easier, I have copied a nice upgrade process workflow from Technet which shows the steps that need to be followed in order for the upgrade to be successful. Before we move forward make sure you have sufficient capacity on the cluster to maintain the workload since the upgrade requires removing one node at a time from the cluster. Also, make sure that the running Hyper-V nodes are compatible with server 2016, or Microsoft support will hung up on you.

Stopping Cluster Aware Updating and backing up data

The first task is to backup the cluster database and the workload data. Now I’m not going to show this here since it’s out of the scope of the article, but all you have to do is backup the system state of your Hyper-V servers, then the virtual machines that are running on them using your favorite backup software. The next step will be to check if Cluster Aware Updating (CAU) is running or not. If it’s running, it needs to be stopped, so the cluster nodes are not paused, drain, or rebooted by this service. From the Administrative Tools menu click the Cluster-Aware Updating icon and take a look if there are servers in pending or updating state. If they are click the Cancel Updating Run option to stop the process.

PowerShell can also be used to check this, if you are a PowerShell savvy. The command needs to be executed from one of the cluster nodes, else the -ClusterName parameter needs to be specified followed by the cluster name.

If CAU is running on the cluster, it needs to be stopped first, then continue with the upgrade. Again, if you are doing this from a remote station, you need to add the -ClusterName parameter followed by the cluster name.

Removing Hyper-V nodes from the cluster

Now open the Failover Cluster Manager, go to the Nodes menu, right-click one of the nodes and choose Pause > Drain Roles. During the draining operation all the virtual machines that are sitting on the node(s) will be migrated to other nodes in the cluster. Depending on how many and how big the virtual machines are, the operation can take a while or it can be done in several minutes. When it’s done, the host will have a Paused status.

You can pause more than one node, but it all depends how many resources will be left in the cluster. If you know for sure that all your virtual machines will run smoothly with just two or three nodes online, then go ahead and pause the rest of them. This way the upgrade will be faster since you can install server 2016 on more that one server at once.

The next step will be to remove or evict the node from the cluster. For this, right-click it, and choose More Actions > Evict. Answer Yes on the warning that pops-up.

As before, we can use PowerShell to do all this. To drain the roles from a cluster node and put the node in a pause state, use the bellow PowerShell line. If the command it’s executed remotely you will need to use the -Cluster option followed by the cluster name.

The last command is to evict the node from the cluster is:

Install the Hyper-V role on the new server

Now we are ready to put server 2016 on the node that we evicted from the cluster; a fresh install, not an upgrade. After the install make sure you have all the drivers in-place, storage adapters, server it’s patched and joined to the domain. Also, configure the storage and network adapters so you have good connectivity and see the LUNs. Install the Hyper-V and the Failover Cluster roles on the 2016 server using either the GUI or PowerShell.

Install the Hyper-V role using PowerShell

If you are using PowerShell remotely like I am here, you need to add the -ComputerName option followed by the remote computer name in the command line.

Install the Hyper-V role using the GUI

For those that are using the GUI it’s very easy. Open Server Manager and check the Hyper-V box from the Server Roles screen

then from the Features screen check the box next to Failover Clustering.

Creating the Virtual Switches on the 2016 Hyper-V server

Since we’ve lost everything during the fresh install, we need to create back the Virtual Switches for our Hyper-V server. They need to be named exactly as before, and they need to be connected to the exact same network adapters. If by mistake you attach a Virtual Switch to a different network adapter, traffic will not go in the right direction and you will have problems configure this box in the cluster, later on.

Connect to the Hyper-V server using the Hyper-V Manager console, and from the Actions pane click the Virtual Switch Manager link.

In the window that pops-up create your switches as before by clicking the Create Virtual Switch button.

Joining the 2016 Hyper-V server to the cluster

Now that all our connections and configurations are done and verify on our 2016 Hyper-V server it’s time to add this box back in the cluster.

Add a node to the cluster using the GUI

Open the Failover Cluster Manager and connect to the cluster. Once connected, right-click the Nodes folder and choose Add Node.

Type the server name in the box then click the Add button. Once the server is verified click Next and finish the wizard using the defaults.

In a few minutes we should have our 2016 server running inside a 2012 R2 cluster.

Add a node to the cluster using PowerShell

We can also join the node to the cluster by using the bellow PowerShell command. If you are doing this remotely, this time you don’t have to put any special parameter, just the ones you see here.

Now if we try to connect to the cluster from a 2012 R2 server we might get the an error message:

The cluster to which you are attempting to connect is not a version of the cluster supported by this version of Failover Cluster Manager.

To be honest, I wasn’t expecting this since the cluster is running in mixed mode. What this means for us is that from now on we need to connect to the cluster from our 2016 Hyper-V node, in order to manage it. You can go ahead and migrate some of the workloads to the new node.

Upgrading the rest of the nodes to server 2016

It’s time to take care of the rest of the nodes in the cluster. Redo the steps I mentioned above for the rest of the servers until all of them are running server 2016 and are joined to the cluster. This should happen with no downtime for the virtual machines or for the users to notice something. In the end we should have a nice and healthy cluster running the latest operating system.

Updating the cluster functional level

Right now our cluster runs in mixed mode, which is a compatibility feature just so we as administrators be able to do the transition from server 2012 R2 to 2016 without downtime. It is not meant to run forever this way. As soon as all the nodes in the cluster are upgraded to server 2016 the cluster functional level needs to be updated also. Once this is done, there is not turning back, we can’t add 2012 R2 nodes to this cluster anymore. Before updating the cluster functional level, make sure all the nodes in the cluster are up and running. If you are good-to-go, open a PowerShell window and run the bellow command:

Summary

As you can see this feature is great and it helps a lot in our workloads, but it still needs a little bit of polishing. It has some limitations in the sense that only 2012 R2 clusters can be in-place upgraded but I guess this is the Microsoft way of pushing clients to use the latest OSs. Go ahead and test it, implement it and if you have any questions feel free to use the comments area.