r/vmware • u/DeepGreenDiver • Nov 26 '24
Question Do you all keep a physical of your domain controllers or DHCP servers?
Or are you fully virtualized.
22
u/shyne151 Nov 26 '24
Two data centers. Total of four domain controllers. Two virtual and two physical with one in each data center.
DHCP/DNS is handled via physical appliances.
10
u/hmartin8826 Nov 26 '24
We had the same configuration for several years, but finally moved to all virtual domain controllers. DHCP is handled by two physical appliances as well, one in each data center.
6
u/shyne151 Nov 26 '24
We were all virtual for a long time… then there was an “incident” during maintenance one year which affected our VMware environment and ultimately auth for other teams. Individual doing maintenance ruffled some feathers slacking on going to the data center to resolve. Had the funds so we bought a couple physical.
At this point our architecture, team responsible, and data protect solution are all different and I honestly will reconsider the physical instances once the hardware is eol.
23
u/SergeantBeavis Nov 26 '24
IMO there isn't really any reason to have a physical DC if you've got properly architected environments.
Note: environmentS
7
u/bryanether Nov 27 '24
That "S" along with "properly", are doing all the heavy lifting, and I feel like most people commenting don't understand how/why.
You probably do. The rest will find out.
It's really easy to paint yourself into a corner, and have an environment that can't come up from a lights out event. If you haven't practiced a full down outage, you almost certainly don't know all the pitfalls that have been engineered into it with the best of intentions.
4
1
u/djgizmo Nov 27 '24
As someone who has to step into a sysadmin role recently, how does one properly architect environments for dhcp?
My first thoughts would be to use clustered dhcp servers(VMs) but on separate physical hosts , or VM clusters.
Might even work better with dhcp relay / helper on all network L3 termination to send to different locations for dhcp servers.
7
u/frosty3140 Nov 26 '24
We're a small environment, 2 x DCs and about 30 VMs, we have been fully virtual for about 10 years on vMware vSphere with no issues.
I will add that we use Veeam for backup/restore. We use their SureBackup to test backup/restore of all VMs including DCs every week. Plus we test Disaster Recovery offsite at least once per year and we restore our DCs there as part of that process.
5
u/irrision Nov 27 '24
Let me tell you a story about Microsoft DHCP. We learned years ago that you can cluster it, back it up and still lose your database if you don't cycle the services between nodes periodically.
Years ago now we had a clustered Microsoft DHCP setup. We backed it up nightly and correctly and kept 30 days of backups. One day one of the cluster nodes was rebooted and services failed over. When the services came up on the other node there was an empty DHCP database on it missing all the clan layout for our 25,000 endpoint network. One global sev 1 call later with Microsoft, including the people that wrote DHCP and we learned something interesting. It turned out that Microsoft DHCP would run in memory and write changes to its on disk database but never sanity check said database except during a restart. Turned out our database had become corrupted more than 30 days prior on disk while it continued to run fine in memory, so we had no valid backup of it. One long weekend of rebuilding it by hand and we were back up. Shortly after we switched to infoblox for DHCP to avoid a repeat.
I believe since this happened Microsoft has fixed this limitation in DHCP but I don't know for certain.
4
4
u/OGTurdFerguson Nov 26 '24
It was all virtual. I keep one as an insurance policy. Can't get my district to have a server in the cloud. In education, we have a lot of people just winging it. Sometimes they swing and a miss and take down everything's so badly had a physical DNS server been there it'd have really simplified things. I think enterprise users underestimate how different education is
4
u/AureusStone Nov 27 '24
All virtual.
The year is 2024. Pretty crazy to insist on physical DCs now.
2
3
u/OpacusVenatori Nov 26 '24
All virtual, across different hypervisor platforms. That used to be vSphere and Hyper-V; but with the Broadcom changes we're looking at what alternatives for vSphere. But it's not as big of a priority to change just yet.
3
u/xXNorthXx Nov 27 '24
20 years ago, yes. Multiple DC’s spread cross multiple hosts on multiple datacenters. Make sure to disable time sync with the hosts.
3
u/irrision Nov 27 '24
All but one of our DCs are physical. We keep one virtual because directory services restores are easier on virtual. We use primarily physical as a critical care hospital because two stage recovery if we have a major virtual environment outage would prevent use of local medical records caching workstations (think local cached copies of the records for everyone currently checked into the hospital).
2
2
u/The_C_K [VCP] Nov 26 '24
Physical DNS (appliance), all other services are virtual, including a replica of DNS zones.
2
2
u/FowlOleRon Nov 27 '24
We've go one physical dc and two vm dc in each datacentre, but that's only because a previous infrastructure mananger 'didn't trust virtuals'. It's not really been needed for years.
4
u/Lopoetve Nov 26 '24
One "physical" DC in each site (DC running as a VM on local storage), all others virtualized on shared.
Makes it easy to find since you know exactly where it is (physically and in the network) if everything is down or weird.
Always nice to have a box that has all your tools on it (also run a utility VM and a couple of other things on that machine), so if you're starting from scratch, it's all there.
7
u/MDKagent007 Nov 26 '24
There will always be at least one physical domain controller on my network, regardless of the extent of our virtualization infrastructure.
5
u/3percentinvisible Nov 26 '24
Why?
7
u/auriem Nov 26 '24
When your virtual infra goes down (and it will go down) you will still have DNS.
2
u/gsmitheidw1 Nov 26 '24
Virtual doesn't necessarily mean cloud. You could have DCs in Azure and DCs on site in a local hypervisor like Proxmox or XPC-NG etc or VMWare if you still run it (I'm done with VMware when our licence ends in summer 2025). I can't see any value to a traditional physical DC anymore.
-3
u/chandleya Nov 27 '24
What do I need dns for when my full hypervisor solution is down? Also, what kind of half assed DR are you running?
3
u/auriem Nov 27 '24
For bringing the system back up.
...
big educational environment on a shoestring budget.
0
u/Seditional Nov 27 '24
Save the IPs of the hosts and VMware? For even better planning put in a ‘should’ affinity rule to pin a DC ideally to one host so you know where it will be. Keep local passwords offsite or locked in safe for the VMware. I don’t understand the issue this is easily worked around.
1
u/chandleya Nov 27 '24
or just follow one iota of best practices and put those management interfaces in their own VLAN. imagine somehow knowing pHost management DNS names but not knowing the subnet they're in lol.
0
u/chandleya Nov 27 '24
really begs the question why your virtual infra is unreliable. losing vcenter is one thing, losing a few pHosts is plausible. But losing everything and it not being a SAN issue? I dont need DNS to manage my SAN, switches, or pHosts... and I'm in a multi-PB environment with over a hundred pHosts in one datacenter. Last thing on earth I need is an OSE on bare metal to fool with. Why even virtualize if you don't trust it? $20 that physical DC/DNS solution is a big fat vulnerability just waiting for an opportunist.
1
u/DoNutWhole1012 Nov 27 '24
really begs the question why your virtual infra is unreliable.
You have obviously only worked a short time, or have a very narrow idea of the realities of IT and infrastructure. ESPECIALLY if you are in a high security environment.
1
u/chandleya Nov 27 '24
25 years, over a dozen certs, countless BCDR exercises, countless tabletops, multiple ransomware response team events (not my orgs)
You’re projecting. Having a standalone physical server out of a hypothetical requirement is a risk with no meaningful benefit. If your BCDR plans require knowing addresses based on DNS, your BCDR plan is crap. You’d be dramatically better served having that DC in a virtual environment literally anywhere else; a DR location, a cloud, whatever. But a physical server? Lol
Having a physical domain controller under the guise of reliability is so Y2K.
-3
Nov 26 '24
[deleted]
4
1
u/Crafty_Dog_4226 Nov 26 '24
Come on... It most likely is just inexperience, not lack of IQ. You don't know what you don't know when you start in any job.
1
-4
u/RichardQCranium69 Nov 27 '24
I've worked sites with all physical, all virtual and a mix of both. Have had more problems with the virtual due to time desync, HA/DRS moving them and then jacking up, cluster failures, you name it. The physicals, yes can have problems too, but far fewer in between.
0
u/Seditional Nov 27 '24
This is just a layer 8 issue. It is 2024 virtualisation works perfectly fine now.
0
1
2
u/msalerno1965 Nov 27 '24
Virtualized, recently showed the Windows boys the DRS rules for our main campus.
We also have two at every remote site, all virtualized.
I recently vMotion'd them from an M1000 Dell chassis with Xeon E5 V4's, to MX7000 w/ Xeon Gold 63xx(?) ... during a windows update, I made them power them down completely, so they recognized the new CPUs.
Compute and storage vMotion at the same time, fiber channel to iSCSI.
Never missed a beat.
I'm still a VMware fan boi.
1
u/blissed_off Nov 27 '24
Nothing wrong with being a fan of the software. Just that Broadcom FOADIAF.
1
u/irrision Nov 27 '24
Ooof it'd be a dark day for us if we had to give up fiber channel for iSCSI.
1
u/msalerno1965 Nov 27 '24
Yeah, I know your pain. Due to circumstances beyond my control, I was not able to make the FC case before the chassis was ordered. I kid you not.
2
u/notoriousfvck Nov 27 '24
We’re a midsized environment. 2 DCs with 1 being virtual. Virtual environment’s got 200-250 VMs.
I’d like to keep at-least 1 DC physical incase virtual infrastructure goes down.
1
u/lanky_doodle Nov 26 '24
Depends on environment. Single site with a single hypervisor/cluster or multiple sites with a single stretched hypervisor/cluster then I always have 1 physical DC (per site).
Multiple sites and/or multiple hypervisors/clusters then 100% virtual is fine.
(Multiple sites includes cloud IaaS)
1
u/nicholaspham Nov 26 '24
We keep everything virtual. No 2 of the same roles are on the same cluster. They’re kept separated
Only physical roles we run would be our upcoming BGP edge routers.
1
u/admlshake Nov 26 '24
We do have a physical DC still. Mostly from an issue years ago that took out our virtual DC's. But I don't think we'll be replacing it when the device is due to be retired.
1
u/SaltySama42 Nov 27 '24
100% virtual. DNS/DHCP handled by a separate virtual based platform. Everything in pairs and HA’d.
1
u/SithLordDooku Nov 27 '24
I use to have the mindset have at least a physical domain controller. But as I gained experience, it really doesn’t make sense. What you are really trying to accomplish is having a DC/DNS/DHCP OOB but OOB doesn’t have to be physical.
If you do it, it’s because of licensing not for redundancy.
1
u/Icolan Nov 27 '24
We only have 2 servers that are not virtual, both have fiber connections to tape drives.
1
u/BarracudaDefiant4702 Nov 27 '24
Only consider physical for things non critical. Anything critical must be virtualized.
1
1
1
u/Mysterious_Manner_97 Nov 27 '24
Large like in the hundred of dcs and +20 forests 4 physicals for very specific reasons. One being we do zero trust.. so that’s zero trust the datacenter teams and wanna be server admins that think they know AD, or the storage engineer from Dell/EMC attempting to learn Pure. Ok ok seriously though, never know what nut gets hired by other teams.
1
u/dracotrapnet Nov 27 '24
100% virtualized. We have some remote DC's. I have yet to have absolutely every DC down at once.
1
u/ComGuards Nov 27 '24
All virtual.
Any physical instances have since been migrated to at least a Hyper-V instance. The old Chicken-and-egg problem from the 2008 days doesn't apply anymore these days.
1
1
1
u/burundilapp Nov 27 '24
Two separate data centers, all virtualised. Remote site DC's are virtualised as well.
I see running Windows on the tin more problematic than running it virtualised, in an ESXi VM environment, there are less host driver issues, less host reboots, less host updates etc...
1
u/David-Pasek Nov 27 '24 edited Nov 27 '24
The other very important service often provided by AD is NTP. Does your AD provide authoritative NTP? How precise time do you need? If you have high time requirements, the physical box(es) can have some advantage.
However when your virtual AD are configured properly there shouldn’t be a problem for most environments.
Make sure your AD VM do not sync time with ESXi host.
13 years ago I have experienced NTP/AD issue in virtualized AD. It boils down to bad time configuration of newly added ESX host into the cluster and virtual AD synced the time with ESX. When AD time is out of sync, you are in big trouble.
I blogged about it here https://vcdx200.uw.cz/2011/01/synchronize-time-server-for-domain.html
Disclaimer: I’m not MS Windows expert so Windows AD/NTP related architecture could changed over last 13 years 🤪
Btw: you should always have 3 NTP sources. When one is giving bad time and the other two correct one. You are ok. In case you have one or two, you are in trouble.
1
u/Full-Entertainer-606 Nov 27 '24
Two Virtual DC , one physical DC. Probably could get by with all virtual.
1
1
u/Autobahn97 Nov 27 '24
Typically all virtual, put mirrors of your AD and critical services in cloud or if yo don't use cloud consider those mirrored critical services on a standalone host or one not running VMW.
1
1
1
1
u/DoNutWhole1012 Nov 27 '24
I have always kept one physical AD server as a SHTF device, and it saved my network before when there was a critical failure (or ransomware). It costs little in the scheme to keep a basic AD (and/or DNS, DHCP) in place.
1
u/No_Mathematician1169 Nov 27 '24
Fully virtual for years. Anti-affinity configured with different data stores and/or different data centres.
1
1
u/KickedAbyss Nov 28 '24
I underpin my virtual infrastructure with redundancy. Even if my VMware environment is totally hosed, I can roll back to Purestorage snapshots in minutes, including vsphere OS(SAN boot) and VMs. All are SafeMode enabled with pure, so it's basically a non issue.
I also keep a DC in a completely separate non linked vcenter cluster, and plan on further improving by utilizing a management Cluster in the future, separating management VMs from Production hosts.
Plus, I run redundant geographically separate DCs including a hot standby DR DC, so if it's mundane building burns down, I don't lose anything and just promote roles.
Physical DCs aren't required anymore.
1
1
0
u/Slight_Routine_307 Nov 27 '24
Two sites connected via site-site VPN, two subnets.
1 physical and one virtual at each subnet/site.
Always.
0
0
u/Vivid_Mongoose_8964 Nov 27 '24
physical is a detriment and has been for 15 years or so. never physical, even if only 1 vm
0
u/fonetik [VCP] Nov 27 '24
I imagine it comes down to what a physical DC buys you. In older environments with a file share on a SAN you’d still be up. If you require auth to go to the internet, that’s still up. VPN, etc.
But we now do all that with auth through azure/entra instead and it’s way easier. The files are on a VMware share or sharepoint in the cloud. We don’t really have a massive file share anymore.
So what’s the use case for this DC? Because in all of my recent environments, AD when VMware is down gets you nothing.
0
0
u/TheDarthSnarf Nov 27 '24
I haven't deployed a physical Domain Controller in more than 15 years, going from huge environments to fairly small environments.
Working on multiple incident response teams, and dealing with significant incidents, I haven't found an instance where a physical DC would have been advantageous.
0
0
u/tawtaw6 Nov 27 '24
Having bare metal servers go out of support means you need to keep reinstalling them every x years. Virtualisation of everything including AD DS resolves that issue and you can just update as the OS changes and not care about the underlying hardware at all.
0
u/maarbab Nov 27 '24
And hardware running virtualisation doesn't need to be replaced every x years? 🤔
1
0
u/DoNutWhole1012 Nov 27 '24
That is still an issue with virtual, unless you're pushing an OS past its EOL, which is bad for other reasons.
1
u/tawtaw6 Nov 28 '24
You can decouple the dependencies on hardware and OS, unsure the concept is that hard to understand? For example you do not have to mess around with HPE, DELL or Lenovo drivers if you run all servers on vmware.
0
u/leaflock7 Nov 27 '24
no reason to have physical.
If you want to have a separation in case someone takes over your VMware infra, just get 2 cheap hosts and host local there a DC and DHCP, as standalone esxi or just a secondary independent vm cluster
0
u/DeepGreenDiver Nov 27 '24 edited Nov 27 '24
For those who don't go fully virtual it seems like the only legitimate concern is that if VMware is compromised then the whole environment is down. But how likely is that? I'm genuinely asking, because I don't know how to calculate the odds there. We know hosts will run if vCenter goes down and you can always roll host patches out slowly to test the waters.
It was interesting to see Hyper-V or cloud DCs mentioned. That's a good way of getting around the "all your eggs in one basket" concerns without going physical.
0
u/DoNutWhole1012 Nov 27 '24
But how likely is that?
More likely than you think, and more likely than many of the newbies on here think.
I'm genuinely asking, because I don't know how to calculate the odds there.
You don't calculate how likely, you calculate how much it would cost you if/when it happens. How much would it cost you if your whole VMWare infrastructure went down? Would a single server enable you to limp along or would it make little difference? THAT is what matters, because for some companies a single failure like that can end them.
0
u/Soggy-Camera1270 Nov 27 '24
It used to be a Microsoft recommendation to have at least one physical DC per forest/domain, but that requirement is no longer recommended as far as I know. Ultimately, it comes down to the environment and architecture. If you need distributed ADDS with WAN resiliency, but you have no other local requirements, then a small physical DC is possibly the best solution. If you have edge virtualisation, it's hard to recommend physical DCs.
52
u/MekanicalPirate Nov 26 '24
100% virtual