Virtualizing Your Domain Controllers without getting fired!

Please pretty please do not just hit the button and P2V/ColdClone/HotClone/Copy your Windows Server Domain Controllers, regardless if they run Windows Server 2000/2003/2008 etc.

In best case You accomplish to virtualize your domain controllers, wich You could have done with a few simple steps just as easily with out any danger.

In worst case You render your Domain Controllers useless, create several other problems and hickups in your infrastructure, not limited to complete production halt and at least several hours of pain and horror trying to get everything back and running!

Personally I have nothing against virtual Domain Controllers, usually best practice is not to run all kinds of other software or services on a Domain Controller, plus the need to have multiple Domain Controllers for redundancy will quickly add alot of boxes doing very little. Virtualizing some or all of these Domain Controllers, will put better use of ressources and still keep the box seperate from other services. Dont forget to change time synchronisation settings in the w32time service, vmware tools and ntp servers in the ESX's, but thats another story.

One of the big problems with doing a clone of a Domain Controller, is that if you get problems, you will not notice them untill it is too late. The domain controller will seem to function and work with clients, but it will actually have stopped replicating with all other domain controllers, because it has detected that it has been copied. The result is an inconsistent domain with client records not being updated, they will slowly stop working depending on what domain controller they get in contact with, untill everything goes dead. If you have then virtualized ALL domain controllers, You will be left with 1-3 months of changes going down the tube together with your damaged Domain Controllers. Dont forget to take a full backup of at least 1 Domain Controller before starting your cloning!

So what happens when things go bad?

  • First of You might get problems, but no event log entries - aarrggh try and detect that!
  • If a Domain Controller replicates data after being cloned, it will acknowledge what information it has replicated to the other Domain Controllers. In effect they know what the cloned Domain Controller knows. If the Cloned machine is then turned on, with older information, the other Domain Controllers will refuse to give it the information - after all they know it has allready gotten it! This will create a missing gap of information potentially creating big problems. It is usually refered to as USN Rollback and is a common symptom of a Hot Clone or a Domain Controller that was cloned but the original got Turned On after the cloning. More info here http://support.microsoft.com/kb/875495/
  • If a Domain Controller detects disk signature changes, it will put it self in isolation and refuse replication. Basicly it has detected it has been copied and to avoid replicating wrong information to others it isolates it self. It still keeps on running and serving users, but since it can not replicate, it does not replicate important information like password changes, machine information, etc.
  • Microsoft does not support cloning of Domain Controllers - your on your own!
  • VMware does not support cloning of Domain Controllers - your still on your own!

VMware have more pain and death information about cloning an existing domain controller here http://kb.vmware.com/kb/1006996

How do we avoid all this pain and death? Here is a couple of ways You can safely virtualize your Domain Controllers.

  • My prefered way. If it is just a Domain Controller (It should be), why not just create a new virtual server from scratch and DCPROMO the server up to a new Domain Controller, and DCPROMO down your old server and decommission it? Safest, easiest way of doing things. (dont forget to move FSMO & GC Roles)
  • Now imagine You have a server full of other services as well, and for some reason You feel it is just not worth it doing one from scratch (Yes you can copy DHCP databases, shares, DNS, etc. from one server to another!), well then do this - Make sure You have another Domain Controller running including a Global Catalog server, move any FSMO roles away from the domain controller to another server, then DCPROMO it down to a regular server. ColdClone the server. Turn off the physical server (never reintroduce it to the network!). Turn on your virtual server, DCPROMO it back to a DC and move any FSMO/GC roles as needed. Done!
  • You only have one server, it is full of stuff (i.e. SBS?). You could just clone it hope for the best and cry if it fails... Or set up a temporary Domain Controller on a new (virtual?) server (yes it is possible to have multiple domain controllers in a Small Business Server setup - but only 1 SBS), replicate the domain, create a full backup, backup and restore the database.. up to you, but I would not recommend it. Whatever You choose here, make sure the physical server is never turned on after cloning, dont change disk sizes, and create a full backup before you start! Basicly your physical server will be your best backup, but it is not enough to ensure no problems will happen!

I know some people say, well it worked when I did it.. It is like saying I do not need RAID on my servers storage, I have not had a Disk failure ever! When You have the problem, it doesnt matter how many times it worked, you have the problem!

So a quick check list of do's and dont's

  • Do a full backup first (at least active state!)
  • Do NOT do HOT-CLONES!
  • Do have more than one Domain Controller
  • Do NOT turn on the physical server again - ever - after cloning it
  • Do clone your server, while de-promoted and promote after cloning again
  • Do NOT clone ALL your Domain Controllers at the same time, leave at least one physical for 3 months
  • Do create new virtual Domain Controllers to replace old physical
  • Do NOT change disk sizes or types during a clone
  • Do check event logs after cloning to check for problems
  • Do NOT use normal time settings on virtual Domain Controllers
  • Do look up best practices for virtual Domain Controllers time settings

I hope I saved someone a couple of hours of pain 🙂 If not You read this far - leave a comment to encourage more info in the future!

- Sole Viktor

63 Responses to “Virtualizing Your Domain Controllers without getting fired!”

  • Val:

    Hi Sole,

    I am building a DR based on Vmware. I have p2v most of the servers and even tested them off DR and all working fine, thanks to replicating SAN boxes. So the environment is a mix of physical (prod) and virtual (DR).
    Would you advise p2v of the 2x DCs and exporting the copies to DR. I know disc sig changes and IP address changes might cause issues. But is this something that can be done.
    I have tried building a new VM as another DC in the DR but the dcpromo process fails to complete with errors.

  • Christophe:

    Hi Sole,
    thanks for the article, it’s really exhaustive even if it’s from 2010. I’m asking the same question as Steve, in my infrastructure I got two DC, one primary(with dhcp, dns ,print) – one secondary(DC DNS, WDS) their on two different physical server. The next step is to virtualize all my infrastructure. For testing this, I will have a Lab environnement completly separated from the DC network, like Steve, even the internet providers are differents. Can I use one of my backup of my server to copy them on the virutalized environnement ? Or is it better to make a cold copy of the server ? I’ve thinked that to test to remount an image from my backup could be a good test.

    Thanks for your response and your time
    CN
    (sorry for my bad english …)

  • Chris:

    How in God’s name does someone not just build a virtual domain controller and migrate roles?

  • NM:

    HELP! not sure if this is still monitored, but if the network has only one domain controller and one were to HotClone it before the physical domain controller had its power turned off and then on, would there be any issues caused by this alone?

    what if the HotClone was powered on briefly while the physical domain controller was running disk checks during boot up, but turned off before the physical domain controller finally booted up?

  • Tom Rogters:

    Ok, I just found this information today. Yesterday I used DISK2VHD to create a VHDX of my secondary Domain Controller. I immediately shutdown the physical secondary DC and removed it from the network. How do I know if I have problems? Everything seems fine. My primary DC is physical and has all FSMO roles and is a GC. Should I just DCPROMO down the secondary virtual DC and leave it as my CA, and build a new Virtual DC from scratch? I made a FULL Active System State backup of the secondary DC a few days ago, but the original physical secondary DC has NOT been connected back to the network and will not. Am I just paranoid now after reading this? A year or so ago I put this 2nd DC into VM Ware with VM Ware’s P2V util. I removed it from being a virtual later on and went back to the physical box, and have been going for over a year with no issues. I just went back to a VM for DC2 yesterday. how should I proceed? Do I need to contact Microsoft? TIA

  • Kevin:

    OK unfortunately I come to this page because it has happened to me and I am in trouble.
    Had trouble with VM and had to switch to a clone and like you said it looked like it was working fine on the outside. Users and computers all ok.
    It stopped communicating with the other DC’s. I have tried promoting another DC but I can’t. I tried changing the operations Master but I can’t.
    Any ideas on how to get out of this?

  • Casey:

    My plan is to virtualize our environment, but the problem is I don’t want anything to conflict with my current DC. So maybe you guys can assist, this is my first time building a domain controller so bare with me. I walked into a shit storm at the new company I just joined. I plan on demoting our current Win2k3 server after building our new AD from scratch and promoting it as the primary domain controller, obviously I need to get this done before Win2K3 support ends, so I am on a time crunch. I have a new 2012 server, but am trying to figure out how I can build it offline without conflicting with the current domain controller. Should I setup a small network on a hub with the current DC’s configs, build AD, and deploy it on our internal network? What are your suggestions? I got an idea of what all I need to do, my main question is, can I do it offline, and if I can….how do I go about it?

  • Sole:

    Sorry, I am not going to answer any more questions to this post, as it is quite old.

    For you who is considering to virtualize your DC’s.
    The easy solution: do new virtual servers and dcpromo them to DC OR dcpromo your DC server down, P2V, dcpromo it up again.

    For you who made a mess of your Active Directory by converting live servers. If you want to be sure, you have to roll back to one of the backups or physical servers, before your virtualization. And dcpromo down all other servers, so you only have the one from “before”, then dcpromo up new DC’s. This way you will have the clean DC from “before” and all others will learn from it. Can’t really be more specific without going into the individual situation and then it would be on the clock.

    -Sole

  • Dirk:

    Hi,

    I get, that installing a new server is the best way to go, but that way, you end up with a DC with a new name& ip address, because no 2 servers can have the same name and of course not the same ip.

    Or am I missing something?

  • Arash:

    Tnx a lot from bottom of my heart. Your advices helped me to get into big problems.
    Tnx a again.

  • I have a DC with special software running. I need to migrate it from VMware to X. I planned on removing ADS role as FSMO and all items in catalog has been moved to another server. Then proceed to V2V and re-add the ADD role after. Is this an acceptable route to do?

  • Sole:

    Sorry about that 🙂

  • Sole:

    I believe so, removing the AD role, doing what you need, and then adding it, sounds like a good start.

Leave a Reply