Step-By-Step: Active Directory Migration from Windows Server 2003 to Windows Server 2012 R2

With the end of support for Windows XP, Office 2003 and Exchange 2003 now upon us, April 8th 2014 to be exact, attention now turns to Windows Server or specifically Windows Server 2003.  End of support for Windows Server 2003 is currently slated for July 14th 2015 and a great many organizations still utilize said offering as the cornerstone to their infrastructure to this day.  One question to keep in mind though is that with the move on the client end to Windows 8.1, will Windows Server 2003 or 2003 R2 be robust enough to properly enable employees and provide secure access to the plethora of devices in a world now dominated by BYOD?  Alternatively an IT administrator might ponder on the further enablement invoked via the deployment of Windows Server 2012 R2 should it be deployed in said organization.  Lets explore this possibility via the cornerstone of access enablement offered via Windows Server by investigating the evolution of the Active Directory offering now found in Windows Server 2012 R2

As you may know, Active Directory provides authentication and authorization mechanisms as well as framework from within other related services that can be deployed. As an LDAP compliant database, it commonly contains the most used objects such as users, computers, and groups organized into organizational units or OUs by any number of logical or business needs. Group Policy Objects or GPOs are then linked to OUs to centralize the settings for various users or computers across an organization. Part of the quandaries that IT professionals face is taking advantage of nuances provided in Active Directory in newer server offerings such as Windows Server 2012.  As detailed in Pierre's post, "Windows Server 2012 Active Directory – What’s New?", Active Directory provided in Windows Server 2012 R2 is provided impactful enhancements.  Yet some organizations choose not to migrate due to reasons of uncertainty.

This Step-By-Step has been created to assist with that uncertainty and provide guidance for IT professionals looking to migrate their organizations Active Directory offering from Windows Server 2003 to 2012 R2.

Transferring the Flexible Single Master Operations (FSMO) Role

1. Open the Active Directory Users and Computers console on your new Windows Server 2012 R2 computer.
2. Right click your domain and select Operations Masters in the sub menu.
3. In the Operations Masters window, ensure the RID tab is selected.
4. Select the Change button.
   
   
5. Select Yes when asked about transferring the operations master role.
6. Once the operations master role has successfully transferred, click OK to continue.
7. Ensure the Operations Master box now shows your new 2012 R2 Windows Server.
8. Repeat steps 4 to 6 for the PDC and Infrastructure tabs.
9. Once completed, click Close to close the Operations Masters window.
10. Close the Active Directory Users and Computers window.

Changing the Active Directory Domain Controller 

1. Open the Active Directory Domains and Trusts console on your new Windows Server 2012 R2 computer.
2. Right click your domain and select Change Active Directory Domain Controller... in the sub menu.
3. In the Change Directory Server window, select This Domain Controller or AD LDS instance.
4. Select your new 2012 R2 Windows Server.
   
   
5. Click OK to continue.
6. Back in the Active Directory Domains and Trusts window, hover over the Active Directory Domains and Trusts found in the folder tree on the left hand side to ensure the server now reflects your new 2012 R2 Windows server.
7. Right click Active Directory Domains and Trusts found in the folder tree and select Operations Manager... in the sub menu.
8. In the Operations Master window, click Change to transfer the domain naming master role to the 2012 R2 Windows Server.
9. When asked if you are sure you wish to transfer the operations master role to a different computer,click Yes.
10. Once the operations master is successfully transferred, click OK to continue.
11. Click Close to close the Operations Master window.
12. Close the Active Directory Domains and Trusts console.

Changing the Schema Master

1. Open a command prompt in administration view on your new Windows Server 2012 R2 computer.
2. On the command prompt window, enter regsvr32 schmmgmt.dll and hit enter.
3. Once completed successfully, click OK to close the RegSvr32 window.
   
   
4. Close the command prompt.

Add the Active Directory Schema Console from MMC

1. Open a MMC console on your new Windows Server 2012 R2 computer.
2. Click File > Add/Remove Snap-in...
3. In the Add or Remove Snap-ins window, select Active Directory Schema and click the Add >button.
   
   
4. Click OK to continue.

Change the Schema Master

1. In the same MMC console, right click Active Directory Schema and select Change Active Directory Domain Controller... in the sub menu.
2. In the Change Directory Server window, select This Domain Controller or AD LDS instance.
3. Select your new 2012 R2 Windows Server.
4. Click OK to continue.
5. A warning will appear stating that the Active Directory Schema snap-in in not connected. Click OK to continue.
6. Hover over the Active Directory Schema folder in the folder tree to ensure the new Windows Server 2012 R2 computer is shown. 
7. Now right click Active Directory Schema and select Operations Master... in the sub menu.
8. In the Change Schema Master window, click Change to transfer the schema master role to the 2012 R2 Windows Server. 
9. When asked if you are sure you wish to transfer the schema master role to a different computer,click Yes.
10. Once the schema master is successfully transferred, click OK to continue.
11. Click Close to close the Change Schema Master window.
12. In the MMC, click File > Exit.
13. When asked to save the console, click No.

 Once completed, open the Active Directory Users and Computers console to verify that the Active Directory database successfully replicated to your new Windows Server 2012 R2 computer.  Be aware that the database replication may take some time depending on the number of objects in Active Directory.

Removing the 2003 Windows Server from the Global Catalog Server

1. Open Active Directory Sites and Services on your new Windows Server 2012 R2 computer.
2. Expand the Sites folder, then the Default-First-Site-Name folder, then the Servers folder.
3. Expand both listed servers. One should be your new 2012 Windows Server and one should be you 2003 Windows Server.
4. Right click NTDS Settings found under your old 2003 Windows Server.
5. In the sub menu, select Properties.
6. Under the General Tab, unselect Global Catalog and then click the Apply button.
7. Click OK to continue.
8. Close the Active Directory Sites and Services window.
9. Verify that your new 2012 R2 Windows Server is running the FSMO role by opening the command prompt in Administrative view and running the following command: Netdom query fsmo.
10. In the Network and Sharing Center, be sure to change the Preferred DNS server to match the Alternate DNS server, then delete the IP address listed under the Alternate DNS server should it currently be pointed to the old 2003 Windows Server.

All that's left is to demote the old 2003 Windows server by first adding the new 2012 R2 Windows Server as the Primary DNS, followed by running DCPROMO to demote the old 2003 Windows server.

Unable to install Microsoft Windows Server 2012 into a VMware ESXi 5.x or Hyper-V virtual machine with HP provided media

Symptom

When users try to install Microsoft Windows Server 2012 using HP provided media into VMware vSphere ESXi 5.1 managed virtual machine or under Microsoft Hyper-V, the Windows Installer will pop-up a window with the following message:

Debug Assertion Failed! Program: x:\tools\BIOSCHECK64.exe File: c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\atlmfc\include\atlsmpstr.h Line:112 Expression: nRefs !=0 For information how your program can cause an assertion failure, see the Visual C++ documentation on asserts. (Press Retry to debug the application)"

Users can click on the Ignore button to continue with the installation.

Affected configurations

The system is configured with at least one of the following:

• Microsoft Windows Server 2012, any service pack
• VMware ESXi 5.0 Update 1, Update 2
• VMware ESXi 5.1, any Update

This tip is not system specific.

This tip is not option specific.

Note: This does not imply that the network operating system will work under all combinations of hardware and software.

Workaround

It is recommended to use the retail media rather than HP provided media to install Microsoft Windows Server 2012 into a virtual machine to prevent the warning.

If it is necessary to use HP provided media to install Windows Server 2012 to a virtual machine, the user can do the following:

• For Hyper-V, the user can just ignore the warning message and go on with the installation
• For VMware, the user can just ignore the warning message and go on with the installation or follow the instructions below to configure the virtual machine to reflect host System Management Basic Input/Output System (SMBIOS) structure to fix the issue.

Either of the following two (2) methods can do the work:

• Edit the virtual machine's .vmx file to contain the following line:

SMBIOS.reflectHost = "true"

Hyper-V High Ping Latency

Had an interesting issue today, an insanely slow, brand new server….. or so I thought. First a bit of background on the client, they are fairly large, with over 400 client access devices to maintain, not including server, network equipment etc. to support this the client has 3 servers, one purchased a year and the oldest one thrown out, keeping all devices in warranty and with modern powerful equipment to keep things running, in addition to this there are two other servers that are replaced every three years, these are treated differently as these are speciality servers, and only do one task.

So, building a new server for a client, in this case a HP ML350p gen8 with Windows Server 2012 R2 STD running as a Hypervisor, nothing special in that, I do this exceedingly regularly so it has become more of a routine build for me. What got me with this one though was during testing I was getting insanely high ping latency, not only to the virtual machines from the network and vice versa, but also from the hypervisor to the machines and vice versa. Pings to other virtual machines on others server, on different LAN segments were all responding normally in <1MS

My first thought was there was something wrong with the virtual machines and that I had butchered something in the migration, but as they worked on other hypervisors without delays, that knocked that one on the head. Then I thought network location issue, but that does not make any sense due the fact that pinging from a hypervisor to guest does not go across a physical network, so it has to be the brand new server.

Ok, so what’s new about this server, well its got newer Processors, greater memory, faster HDD’s with larger capacity’s, basically it was more a case of what wasn’t different to the last server. Not going to go through the whole process of troubleshooting, but basically it was to do with the NIC’s, fine now what about them is it. After trial and error, and of course every techs most important tool, Google I came across the issue what is it…



THE ISSUE IS VMQ or Virtual Machine Queuing inside the Broadcom NIC drivers as shown below, disable this and the issue clears instantly





Pings and other indicators are now back down to <1ms which is what I expected to see in the first place.

Hardware effected by this was as follows

HP ML350p gen8
Server 2012 R2 Standard
onboard Broadcom Quad Port NIC

High Ping Latency in Hyper-V Virtual Machines

While working on a Hyper-V 3.0 deployment recently, I noticed that all of the virtual machines running on all newly built hypervisors were showing an unhealthy high ping latency trying to ping anything on the local network. Pinging the same destinations from the hypervisors resulted in perfectly reasonable 1ms ping times, but pings from inside the virtual machines varied from 1ms all the way past 200ms.

I started digging, thinking initially that it was just a clock synchronization issue that we saw on AMD as well as Intel chips, specifically on HP DL servers, in Windows Server 2003 days. The issue is not very well documented but one of the few Microsoft articles that attempts to explain it here. Solution to that issue is to add /USEPMTIMER switch to the boot.ini file. It used to fix negative ping times on AMD chips as well as lengthy ping times on Intel chips.

Windows Server 2012 does not have a boot.ini file obviously, so the way to add this switch is to issue the following command in elevated command line or PowerShell:bcdedit.exe /set USEPLATFORMCLOCK on


(This explains the right side of the screenshot above – boot configuration is shown “before” and “after” running this command).

Unfortunately, applying this configuration at the hypervisor level and rebooting the server made no difference to ping times from VM (guest partitions).
Virtual Machine Queues

Virtual machine queues were introduced in Windows Server 2008. The purpose of this feature was to improve network performance of virtual machines receiving a lot of inbound traffic, by providing a more direct access to the hardware NIC. To quote directly from Microsoft:

“When VMQ is enabled, a dedicated queue is established on the physical network adapter for each virtual network adapter that has requested a queue. As packets arrive for a virtual network adapter, the physical network adapter places them in that network adapter’s queue. When packets are indicated up, all the packet data in the queue is delivered directly to the virtual network adapter. Packets arriving for virtual network adapters that don’t have a dedicated queue, as well as all multicast and broadcast packets, are delivered to the virtual network in the default queue. The virtual network handles routing of these packets to the appropriate virtual network adapters as it normally would.”
Resolution

When the hypervisors were built, VMQ features of the network card on the physical host were enabled to achieve better VM network performance. Through disabling performance features such as VMQ, TCP Chimney, and Receive Side Scaling, it turned out that VMQ was the root cause for high ping latency. As soon as VMQ was disabled at the parent partition’s NIC driver level, VM ping times got down to a steady <1ms reading.

The hardware/software affected by this issue:
HP DL 360p Gen8 server chassis
4-port 1Gbps Broadcom NIC (HP Ethernet 1Gb 4-port 331FLR Adapter)
Windows Server 2012 Datacenter Edition
Broadcom’s NIC driver dated October 26 2012, version 15.4.0.17 (link to HP site)

Bottom line, always perform at least basic QA of your builds before putting them into production. Some hardware (and more likely drivers) may cause performance issues instead of providing a performance boost - especially in early releases.

Reset Administrator Password on ILO

ESXI

Thanks to HP drivers, ILO can be configured directly inside ESXi without the need for a server reboot: going into the command line (locally or via ssh) you can move to /opt/hp/tools, and there you will find the commandhponcfg. This tool can be used to configure ILO:
first, write down the actual configuration of ILO. The easiest way to do it is by saving in an XML file:

1	/opt/hp/tools # ./hponcfg -w ilo.xml

2	HP Lights-Out Online Configuration utility

3	Version 4.0-10 (c) Hewlett-Packard Company, 2011

4	Firmware Revision = 1.81 Device type = iLO 2 Driver name = hpilo

5	Management Processor configuration is successfully written to file "ilo.xml"

then, edit the XML file and configure your desired parameters:

01	<!-- HPONCFG VERSION = "4.0-10.0" -->

02	<!-- Generated 8/6/2012 11:11:4 -->

03	<RIBCL VERSION="2.1">

04	<LOGIN USER_LOGIN="Administrator" PASSWORD="password">

05	<RIB_INFO MODE="write">

06	<MOD_NETWORK_SETTINGS>

07	<SPEED_AUTOSELECT VALUE = "Y"/>

08	<IP_ADDRESS VALUE = "10.123.183.197"/>

09	<SUBNET_MASK VALUE = "255.255.255.0"/>

10	<DNS_NAME VALUE = "esx-ilo"/>

11	<DHCP_ENABLE VALUE = "N"/>

12	<DOMAIN_NAME VALUE = "domain.local"/>

13	</MOD_NETWORK_SETTINGS>

14	</RIB_INFO>

15

</LOGIN>

16

</RIBCL>

Note: you will not need to edit everytime all the parameters, it will be enough to add the lines you want to change. But, you will always have to add the username and password line to authenticate the changes you are doing.load the new configuration into the ILO, it will take some time to complete:

1	/opt/hp/tools # ./hponcfg -f ilo.xml

2	HP Lights-Out Online Configuration utility

3	Version 4.0-10 (c) Hewlett-Packard Company, 2011

4	Firmware Revision = 1.81 Device type = iLO 2 Driver name = hpilo

5	<INFORM>Integrated Lights-Out will reset at the end of the script.</INFORM>

6

7	Please wait while the firmware is reset. This might take a minute

8	Script succeeded

Finally, ILO will automatically reset to load the new parameters, and you will be able to reach the ILO via web interface and login into it. Also, ESXi will show you the new ILO configuration:

WINDOWS 

1. I installed SNMP because it was a preprequesite for HP Insight Management Agents.
2. I dowloaded and installed the HP Insight Management Agents.
3. I then downloaded HP Lights-Out Online Configuration Utility.
4. I ran into NTVDM errors trying to run the file so I just used WinRAR to extract the contents into C:\hp\ilo. I also extracted the zip file contained within the initial archive.
5. I then downloaded the HP Lights-Out XML Scripting Sample for Windows (Linux users can download the files in tgz format here or here.) extracted it and found the file I was looking for --



<ribcl VERSION="2.0">
<login USER_LOGIN="Administrator" PASSWORD="boguspassword">
<user_INFO MODE="write">
<mod_USER USER_LOGIN="Administrator">
<password value="newpass"/>
</mod_USER>
</user_INFO>
</login>
</ribcl>


6. Using notepad, I opened up the sample file and modified it slightly. Initially, I just removed the LOGIN and ran the file but HPONCFG gave me a syntax error. I then added it back and gave the Administrator a bogus password. Apparently, the LOGIN line is required for syntax reasons but it is not actually processed.
7. Next, I opened a command line and changed directories to C:\hp\ilo and typed the following:
HPONCFG.exe /f Administrator_reset_pw.xml /l log.txt > output.txt

8. I opened up Firefox, navigated to my iLO machine and voila! I was able to login as Administrator.

If changing Administrator's password seems too scary, you can also add another user with administrator privileges. You can then login as that user and change the Administrator password via the web console. Use the following code, suited to your liking:



<ribcl version="2.0">
<login USER_LOGIN="Administrator" PASSWORD="boguspass">
<user_INFO MODE="write" >
<add_USER
USER_NAME="Chrissy"
USER_LOGIN="Chrissy"
PASSWORD="mynewpass">
<reset_SERVER_PRIV value = "Y" />
<admin_PRIV value = "Y" />
</add_USER>
</user_INFO>
</login>
</ribcl>