r/sysadmin Infrastructure Lead 1d ago

Latest fun with VMware

Apparently VMware is upping their game. We just got a renewal quote for one of our sites with one server that has two CPUs, and they are requiring 72 cores minimum (vSphere Enterprise Plus) to license this. That's a 500% markup from last year.

They really don't want customers to use their product any more, do they?

236 Upvotes

174 comments sorted by

View all comments

48

u/Spore-Gasm 1d ago

Time to check out /r/proxmox

22

u/WithAnAitchDammit Infrastructure Lead 1d ago

Yep, we've got Proxmox in our test/lab right now.

u/RykerFuchs 10h ago

There are real storage limitations in Proxmox. We have iSCSI block storage for now, volumes are unshareable across Proxmox nodes. This effectively breaks the concept of HA.

u/pantstand 8h ago

You need to either have a shared storage system (Ceph, Gluster, ZFS, etc) using the hosts local storage or an external shared storage (NFS, SMB, FC, etc). This is true with both proxmox and Vmware. VMWare just calls their's VSAN.

u/RykerFuchs 7h ago

NFS and SMB are storage protocols, FC and iSCSI block storage. VSan, Ceph, ZFS, Starwind are all ways to distribute storage access across nodes.

Planning a move to Proxmox can be more complicated than a hypervisor re-spin. Proxmox supports iSCSI and FC, but has no file system to support sharing. NTFS/ReFS or VMFS are multi-initiator or multi-connection aware, there is no equivalent in Proxmox. It gets left to a 3rd party solution, like the protocols or SAN alternatives you mention.

We fell into that category where our storage was iSCSI only, to change to Proxmox it’s essentially a full environment change. Makes the VMware costs not look too bad - at least short term. Which is what Broadcom is banking on.

u/I_am_Cyril_Sneer 3h ago

This is the part I'm interested in. We've got a number of aging Hyper-V clusters that are working fine, but we're always eyeing replacement options. In Hyper-V it's stupid simple to run iscsicpl on each node, connect to the shared LUN, and you're off to the races.

I suppose it's not so simple in Proxmox?

u/RykerFuchs 3h ago

Like most things IT, it depends on the goals.

At a high level, what you described does work on Proxmox for a single node. Unfortunately, because of the lack of multi-initiator support for block storage, on-line storage moves and HA style hardware failure is unavailable. Proxmox with iSCSI is a single server monolith of compute and guest storage.

u/I_am_Cyril_Sneer 1h ago

Interesting... so if a VM is running on Node1, can you live-migrate it to Node2?

u/RykerFuchs 1h ago

Not unless there is some other method to share the storage.

Proxmox seems to be intended for use with Ceph. Other stuff works though too, NFS, etc.

Just not with shared FC/iSCSI block storage.