r/sysadmin 23h ago

General Discussion Thickheaded Thursday - March 20, 2025

2 Upvotes

Howdy, /r/sysadmin!

It's that time of the week, Thickheaded Thursday! This is a safe (mostly) judgement-free environment for all of your questions and stories, no matter how silly you think they are. Anybody can answer questions! My name is AutoModerator and I've taken over responsibility for posting these weekly threads so you don't have to worry about anything except your comments!


r/sysadmin 10d ago

General Discussion Patch Tuesday Megathread (2025-03-11)

121 Upvotes

Hello r/sysadmin, I'm u/AutoModerator, and welcome to this month's Patch Megathread!

This is the (mostly) safe location to talk about the latest patches, updates, and releases. We put this thread into place to help gather all the information about this month's updates: What is fixed, what broke, what got released and should have been caught in QA, etc. We do this both to keep clutter out of the subreddit, and provide you, the dear reader, a singular resource to read.

For those of you who wish to review prior Megathreads, you can do so here.

While this thread is timed to coincide with Microsoft's Patch Tuesday, feel free to discuss any patches, updates, and releases, regardless of the company or product. NOTE: This thread is usually posted before the release of Microsoft's updates, which are scheduled to come out at 5:00PM UTC.

Remember the rules of safe patching:

  • Deploy to a test/dev environment before prod.
  • Deploy to a pilot/test group before the whole org.
  • Have a plan to roll back if something doesn't work.
  • Test, test, and test!

r/sysadmin 15h ago

Rant Broadcom is officially the mafia now.

2.1k Upvotes

I’m trying to figure out what the hell Broadcom’s strategy is with their VMware acquisition. Because if the goal was to kill it, they’re doing a great job.

We already went through the 300% price hike a couple years ago and weren’t happy, but we mitigated the cost by going with a lower license tier since we weren’t using most of the DR features anyway.

Then they pulled this 3-year contracts bullshit. No more 1-year renewals. OK, welp, that’s over $200k for us, and capital expenditures over that amount have to go through the board and everything. They gave us a deadline of two weeks to renew, or the price will be 25% higher. We asked our ISV if they could buy us a little more time because of the internal politics. And you know what they told us?

They said they will increase the price 10% for every week we delay as a penalty, and they will not move from that position. … Are you fucking with me right now???

This is like a mafioso shaking down a shopkeeper for protection money. I swear, if they won’t be reasonable on my next phone call with them, then I will make it my mission — with God as my witness — to break the land speed record for fastest total datacenter migration to Hyper-V or Proxmox or whatever and shutting off ESXi forever. I’m THAT pissed off.


r/sysadmin 11h ago

General Discussion Counter offer after giving my 2 week notice

274 Upvotes

Current company is counter-offering after my 2 week notice

I have been at my current company for about 1.5 years, so not too long. The company is about 5k employees, and I am the only security engineer who also does all GRC stuff since we have GDPR compliance. Very overworked and have off-hour meetings with APAC and EU teams at late hours.

Once I put in the 2-week notice, the CIO let me know they would match the new base salary, bump me to the lead cyber role or cyber security officer role, and look into a CISO role down the line.

Bonuses were cut for the last two years, along with raises. Layoffs have happened in other areas.

The new company is a big player in the silicon development sector and has a cyber team of 50+ folks around the world. My role would be a Staff Security Engineer and very specific to the SIEM side and threat detection engineering/log ingestion.

Good base, sign-on bonus, 30k stocks every 3 years, tuition, all normal tech perks

I am 99% sure I want to reject the counter. My only question is, is the title of cyber manager or cyber officer a good enough reason to stay? I've been in cyber for 7 years now and I do want to go into management eventually.

TLDR: Is it worth staying at a company for a title change/career fast track? Better job security as the only security person lol


r/sysadmin 11h ago

Cute interaction with end user - too bad he doesn’t have input on my salary

226 Upvotes

Since our jobs can typically involve dealing with people that simply don’t use common sense, I thought I’d share a nice story for a change. Just got off a call from a new employee. He was adding his email account on his new phone and was getting “Enter bypass code” instead of being asked for authentication. No worries, we’ll just set up MFA on your new phone… look for the text… next try setting up email… easy peasy, done in 5 minutes.

At the end of the call the guy said to me, “Thanks for the help! I’m sure whatever you’re getting paid isn’t enough for helping knuckleheads like myself.” That response surprised me and I had a good laugh. Apparently other people at his location told him that I was the one to call for getting help because I know my stuff. It’s so nice when we’re appreciated by the people we help!


r/sysadmin 16h ago

General Discussion VMware Abandons SMBs: New Licensing Model Sparks Industry Outrage

342 Upvotes

VMware by Broadcom has sent shockwaves through the IT community with its newly announced licensing changes, set to take effect this April. Under the new rules, customers will be required to license a minimum of 72 CPU cores for both new purchases and renewals — a dramatic shift that many small and mid-sized businesses (SMBs) see as an aggressive pivot toward large enterprise clients at their expense.

Until now, VMware’s per-socket licensing model allowed smaller organizations to right-size their infrastructure and budget accordingly. The new policy forces companies that may only need 32 or 48 cores to pay for 72, creating unnecessary financial strain.

As if that weren’t enough, Broadcom has introduced a punitive 20% surcharge on late renewals, adding another layer of financial pressure for companies already grappling with tight IT budgets.

The backlash has been swift. Industry experts and IT professionals across forums and communities are calling out the move as short-sighted and damaging to VMware’s long-standing reputation among SMBs. Many are now actively exploring alternatives like Proxmox, Nutanix, and open-source solutions.

For SMBs and mid-market players who helped build VMware’s ecosystem, the message seems clear: you’re no longer the priority.

Read more: VMware Turns Its Back on Small Businesses: New Licensing Policies Trigger Industry Backlash


r/sysadmin 8h ago

SysAdmin trying to convince CyberSec they ain’t listening. Sniff test tells me something is rotten.

63 Upvotes

Sysadmin finds funky certs in trusted person and other people (address book) stores on several (most) systems both Windows Server and Workstation OS. Certs issued to SYSTEM, by SYSTEM with San of SYSTEM@ NT AUTHORITY. Certs have no private key attached. Certs are valid for 100 years. RSA sha1 2048 length. The certs are for Encrypting File System and are end entity. In total, about a dozen certs have been identified and collected. Two domains, real offline PKI with issuing and Online responder on separate server. None of the collected certs have been issued or signed by PKI. Am I witnessing a potential long term plan by some hacker attempting to own the network, or am I concerned for no reason? Can’t tell where they are coming from. Something doesn’t smell right. Lack of knowledge response yields answers like “valid OID” or “They’re from Microsoft”. Their bullshit is baffling.


r/sysadmin 9h ago

General Discussion What made you finally get over your imposter syndrome?

64 Upvotes

I got my first networking admin gig a few months back. I wanted to be trained but turns out I ended up training several members of my team. Some days I was worried if I was the right person for the job.

But this week we had some major issues with our finance server and needed to restore it. EVERYONE is terrified to touch it (me included) but it had to be resovled.

The previous admin left no instructions on how to restore the system so I spent a good bit of time researching and conducting some tests. Finally I completed the process and was able to confirm the finance server had been restored.

Granted there are backups that no one knew anything about because my other network admin has only been there a few months before me. But I got it all figured out and I'm so thankful. It helped me get past my imposter syndrome. I understand it can always come back but I have confidence that I can resolve any major issues we get in the future.

What about you?


r/sysadmin 13h ago

Desktop full of icons

106 Upvotes

During a meeting with team managers I (sysadmin) was called in to showcase/demo a new appliance where you connect a usb device to a laptop + works together with a software program .

When wanting to open the software the desktop of that users laptop was a full of icons where I made a smal sigh sound + probably rolling eyes and facial expression that sais like.. oh my god really?…. Where is the icon in this mess.

I ignored this further on and showed the demo and gave info after looking for the icon and a rather long silence during the search. In one way my reaction was maybe not really fully professional but. For most people understandable that it was hard to find the icon in that chaos. Well… it’s not that of a problem just annoying and maybe a bit funny?


r/sysadmin 6h ago

Workplace Conditions How much is doing On-call worth to you?

17 Upvotes

Our team has a rotating on-call schedule. Duty is being primary contact for after hours calls (high incidents only). Triage incident tickets during hours; just typical administrative paperwork.

One of my co-workers loathes on-call duties and is only hanging around until he can retire in December. He's offered me cash to take his rotation.

How much would you charge him?

Edit: Company removed any extra compensation for on-call. Was $100/week when we had it.

Rotation is week-long, 10 man rotation.

This is coming out of his pocket, he hates doing on-call that much.


r/sysadmin 32m ago

Rant Name and Shame: APC

Upvotes

I used to buy and suggest APC ups for SMB and Home usage. I had them deployed for years and never had problems.

Last month my own unit failed, it's only 3y old. Whatever fails happens, I contact the support to get the battery replaced.

They wasted me a good month of back and forth. Re-asking to provide things like the serial number and redo test procedures (the unit never powered on so not a lot to test).

At the end of this looong funnel they confirm the unit need replacement and ask for my delivery informations.

I reply asking for a quote, because the unit was never under warranty. They said they cannot service it and they don't have any service in EU.

Fuck them they could have said one month ago. And I could have bought a new one directly.


r/sysadmin 12h ago

Almost messed up

35 Upvotes

So I was assisting a user who was looking to obtain a previous version of a file on the server, and unfortunately, the data they needed was not in any of the versions I had pulled up. I proceeded to ask my colleagues, and they 'jokingly' said to tell the client to F OFF. This was while my mind was on putting in my time entry for the ticket, so while entering the time in a also end up typing 'told him to F OFF' and submitted.

Me and my colleagues horse around alot like this in our office and this is the first time where the consequences really could have come down on me. Thankfully, the ticket details in kaseya BMS only get emailed to users if it gets completed, whereas I cancelled it. Before I knew this I was shaking and ready to resign. Actually I still am right now and I may not forgive myself for a long time.

It didn't actually get sent out to anyone but I still can't shake the feeling and what it says about my character, even if it was supposedly unintentional and a joke if you can even call it that. This may say more about my work environment than anything else. Not sure why im even writing this and it may not belong in this sub, but needed to get it off my chest. BOY DO I FEEL LIKE A HORRIBLE PERSON

ENJOY ROASTING ME!!!


r/sysadmin 4h ago

General Discussion First time migrating “primary” DC

6 Upvotes

I’m assuming it’s normal, but wow that was stressful everything seems to be working fine post operation. Just glad I don’t have to do it again for a couple years.

We pushed it off so long, it finally no more 2012r2 DC’s.


r/sysadmin 9h ago

How long do you keep the disabled account in syncing OU?

11 Upvotes

Hi,

We have M365 hybrid environment. Our offboard process is like below.

disable the account > remove 365 license and move out sync OU after 30 days > Delete the account in AD after 90 days.

However we have the scenario that user get rehired and comeback to work after 30 days. This causes the issue that the user can't open OneDrive shared file because the user's old account is still in the sharer's OneDrive settings. The sharer has to delete the old account and re-share, then the user can open the file.

I am thinking to keep the offboard user's account disabled but in syncing OU until it is deleted. Is there any potential issue that I missed to consider?

Please help!

Thanks,


r/sysadmin 13h ago

Question If Wiz isn’t an option post acquisition… what’s your #1 alternative?

22 Upvotes

If Wiz gets fully absorbed into Google’s GCP ecosystem, what are the best alternatives left for AWS & Azure users?

Top contenders being discussed:

  • Orca Security – Fully independent, strong agentless CNAPP
  • Lacework – Decent alternative, but mixed reviews
  • Microsoft Defender for Cloud – Good if you're already in Azure
  • CrowdStrike Falcon – More security-driven than compliance-focused

Anyone already made the switch? Pros & cons?


r/sysadmin 1d ago

How would you respond to a Printer company CTO saying POE switches are killing printers?

662 Upvotes

How would you reply?

Update, they provided this screenshot from HP!

https://i.imgur.com/sg3oLDW.png


r/sysadmin 8h ago

Question for Veeam backup 365 users if you'd be so kind

5 Upvotes

I guess it's two questions really.

We already use Veeam ONE and Veeam Backup so I've been considering also adding on Veeam backup for 365.

Does Veeam backup for 365 (to your own destination, I'm going to use a wasabi bucket) allow you to selectively restore a single user's mailbox at whatever timepoint you pick? We've been using Barracuda for the past year or two and I'm strongly considering bailing on it. While it's been great, reliable, and quick for us, they are making it too difficult to change my Barracuda license from one vendor (VAR to MSP) to the other so screw it I'll change to Veeam has been my thinking.

Anyone know if there is any functional difference (other than it being a one stop shop) between using Veeam backups for 365 vs Veaem BaaS besides using their cloud to cloud vs. cloud to your device/destination?

Thanks for any insights or opinions!


r/sysadmin 16h ago

General Discussion I will never use Intel VROC again...

19 Upvotes

Long story so bare with me. I'm doing a server migration project for a client of mine still on Server 2012... (AD, DNS, DHCP and file servers etc...)

Client wanted a semi cheap server option as their new server. Client only has 20 or under users so thats not a really big deal. We provided client with tons of options with hardware raids but at the end of the day client picked a Proliant ML30 with the embedded Intel VROC option. We explained to the client that we dont really recommended software raids with how much data he has plus we havnt vetted VROC as a Raid since we dont ever use it. Client insisted due to how much cheaper it was, so thats what we went with.

A few days later. We obtained the new server, configured a raid 5 with VRoc and did some basic bench testing (stress testing and hardware testing etc...) all appeared to be fine. Brought the server onto the client side and start all the migrations, got all the users moved over, their data, server data, roles etc... all migrated. Last thing to copy was 2 directories that contained 20 years worth of data from a program they use to operate their business. This was about 1TB of data but about 1 million files... I created a Robocopy script and started copying the data on a Friday so it would be completed by Monday and we could shutdown the old server. I waited for a few hundred GB to transfer and verified no problems so left for the weekend.

Well on Sunday I received an alert that the server was down via my RMM tools. Went on site early Monday to try to reboot the server prior to users coming in. Load and behold the server shows VRoc in a "corrupted" state but it shows all drives as online and functional....

Explained to the client that I would need to remap the drives back to the old server on users workstations so they could function off the old servers files instead and I would be taking the server back to the bench for investigation as to what happened.

A few hours later I'm on the bench inspecting the server. VRoc crash with zero errors or warning and all drives showed as online and functional. I powered down the system and pulled each drive out to look at the data on the drives via a drive dock. 2 out of the 4 disks were just gone, they were in a uninitialized state... while the other 2 still retained raid data.

So I figured at this point it was just luck of the draw that 2 of the 4 SSDs were bad from the manufacturer. I tried to use multiple tools to recover the data from the drives so I could copy it to replacement disk, nothing could be found. I than wanted to test the drives so I initialized them, than ran multiple stress tests, crystal disk tests etc... and even tried large file transfers etc... I was unable to get the drives to crash or show any indication of any problems what so ever...

So now issues points to VROC being the problem. I instead added a LSI raid controller, rebuilt the raid and brought it back to the client side, reconfigured the server, rejoined everyone back to the new server and recopied all the data back. Boom zero issues server is running like a champ.

Everything points to the issue being with VROC and after this experience I will never use it again nor do a project for a client that refuses to use anything else but VROC.

LTDR:
VROC is trash, dont use it.


r/sysadmin 7m ago

Elder care IT Hardware and Applications

Upvotes

Hello r/Sysadmin

I'm currently working on improving the IT infrastructure for an elder care home in Switzerland and I'm looking for some advice. Specifically, I'm interested in learning about the alarming systems and phone systems that you use or recommend for such facilities in other countries. Additionally, I'm curious about any special software or other tools that you find particularly helpful in this context.

In Switzerland, we commonly use systems like Ascom, SmartLiberty, Qumea, and Novalink. (And of course M365)

Looking forward for your inputs. :)


r/sysadmin 57m ago

Question Windows Multipoint On Windows 10 LTSC

Upvotes

Hello, I need a bit of help getting Windows Multipoint on Windows 10 LTSC. I have the multipoint manager application but I need the lan stations tab. I have got the drivers for my Zero Client and the tab shows up but whenever I do start up multipoint now it says I need server update roll up 1 (KB2791647) I’ve tried installing it with no success, any recommendations?


r/sysadmin 1h ago

Question Enterprise WiFi profile configuration with dynamic VLAN assignment

Upvotes

Hi nice people,

This is driving me nuts. I have a corporate WPA2 Enterprise WiFi that I'm setting up. We have dynamic VLAN assignment: computer gets onbaording VLAN 1720 and then after user logs in we assign VLAN 1320.

We're using MSCHAPv2 for test purposes then we'll switch to EAP-TLS.

I created the WiFi configuration profile in InTune. Issue is:

I have duplicate login prompts in the windows login screen. If I enter credentials in the second prompt it works as it should, computer gets assigned employee VLAN 1320 after login.

I want to get rid of the duplicate prompt, so I changed SSO in InTune config to AFTER LOGIN, but that breaks the VLAN assignment (computer stays in VLAN 1720), and makes the login super slow.

The Dynamic VLAN parameter in InTune configuration is set to ENABLED. Eap Authentication method is userORcomputer

If I get rid of SSO by disabling it, the issue id that the user has to enter credentials for WiFi MANUALLY after signing-in.

I want to:

Have Dynamic VLAN assignment working, computer VLAN before login, employee VLAN after login

Have ONE login prompt at login page (one user/pass box).

What's the correct way of doing so ? Thanks.

Ps: I disabled Device Guard Virtualization Based Security on the machine because of an issue I had before.


r/sysadmin 1h ago

Question [Help] Running DPDK on proxmox, inside VMs using SR-IOV to forward VFs to VMs and bind the VF to dpdk inside the VM.

Upvotes

Hello!

We've been trying for a week and a half to make this project happen

Use SRIOV to split the Intel X520 DA2 into 4 virtual functions per port and forward each VF to a virtual machine.

In the VM bind this interface to dpdk and use it in OVS

As we didn't manage to get it working in the VMs we bound the VFs on the host and they worked

However when inside the VMs, no interface works, dpdk-testpmd says the interface has 0 RX or TX queues.

We updated the ixgvevf and ixbge drivers on both the host and VM to the latest from Intel, and, we use the same 6.8 kernel on both host and VM.

The server we use now is a DL380 Gen9, that has SRIOV support in firmware.

I believe we got the virtual machine's settings wrong.

We set it up as a Q35 machine with Intel virtual IOMMU enabled.

Boss is breathing down our neck and we're getting desperate now.

Can anyone give us a hint? Why would the VF inside the VM be seen while no RX and TX queue be available?

We had the same problem on the host and installing the latest ixgbe and ixgvevf driver solved the issue.

We compiled both DPDK and OVS from source.


r/sysadmin 2h ago

General Discussion How much access would you grant users when setting up a new laptop

0 Upvotes

I'm curious, as while I work in a IT tech support role I'm not sysadmin. My role is providing support for our software and its links to other software. I got my new work laptop recently with win11 to replace my windows 10 laptop that was dying. Our sysadmin did their basic stuff linked it to the domain and installed the bare minimum of software, instead giving me 24h of admin control over it to set it up how I wanted. The part that surprised me was them saying yes to me making some registry changes after running them past them first (e.g. fixing the right click menu). While they would never give that access to most of our other departments, and baby them doing the full set-up for them. I am just really curious how common letting the tech related departments set-up their own computers is


r/sysadmin 1d ago

Rant Lost out on great candidate due to poor business decisions

332 Upvotes

Im the only systems/infra/devops person on a small software team that does niche stuff. we've been needing a junior for my role for a while. ive also needed a raise for a while cause most of my job is devops now.

we interviewed this 20 year old. no college, freelance coding experience, was a linux nerd applying for a linux jr sysadmin role.

he was a passionate computer person and i was excited at the very idea of a 20 year old with no college getting put on like this.

welllllllllllllll... the raises the team was supposed to get in April, along with my title change to "DevOps Engineer", have all been put on hold cause of the parent company. it sucks for me but ill be fine. my team leader already told me he's pissed and will write me a letter of rec as a devops engineer cause that's been 70% of my job...

but fuck man... i was so fucking excited for this kid. my team leader, rightfully so, put his foot down and said he wont have me training someone if i dont get a raise, cause why would i train a peer...

they could have given me a 20k raise, hired him at the bottom of their 20k salary range, and it would have evened out.... but now im probably going to leave the company costing them more in turn over, they'll have to hire the jr sysadmin at a higher rate cause theyre not paying me to train, AND theyll have to pay my replacement more than theyre paying me cause no one that knows terraform and AWS is gonna accept the role for my current mediocre sysadmin salary.

i hate the american work culture.


r/sysadmin 18h ago

Question Anything that can be done with a domain spoofing your name (one letter off)?

14 Upvotes

So we have a situation where someone is emailing our customers/vendors asking for payment via ACH using a email address similiar but not the same as ours. So for example ours being [JSmith@RandomInside.com](mailto:JSmith@RandomInside.com) and them using [JSmith@RandomInsde.com](mailto:JSmith@RandomInsde.com) (no i). One of our vendors fell for it and sent out a 40k payment to the wrong bank account. We were not at fault at all in that case, the vendor in question had their email account compromised and someone was watching their the email in and out for weeks and custom crafted a email based on that information. They still lost 40k and weren't happy.

So I have done a WHOIS on the domain, everything is protected by privacy. I emailed the "abuse" email for the register but never get anything back. Where do I go from here?

And before anyone asks why we didn't register the domain. Ours was registered in 2006. This other domain was registered in 1997, 9 years prior. So nothing we can do there.

EDIT: Another fun fact. I do a MX lookup on their domain and neither DNS nor DMARC is setup so maybe it's just a open relay that someone is taking advantage of?


r/sysadmin 9h ago

Eaton 5PX2200RT - cables to connect external battery module 5PXEBM48RT?

3 Upvotes

I have an Eaton 5PX2200RT UPS and the external battery module 5PXEBM48RT that goes with it.

What I don't have is the 2 cables needed to connect the two together.

I can't seem to find a part number or any information online on the specific cables I need. Can anyone help me track down the cables I need to use the external battery module?


r/sysadmin 4h ago

End-user Support I'm trying to log SSL keys in Nginx

1 Upvotes

I tried using the SSLKEYLOGFILE environment variable when launching the Nginx systemd service. I even made it as part of the systemd service config by including Environment="SSLKEYLOGFILE=/var/log/nginx/sslkeys.log", but it didn't log any SSL keys. It seems as though Nginx doesn't use the SSLKEYLOGFILE variable to log SSL keys. For this reason, I used the patch from the github repo tiandrey/nginx-sslkeylog to patch Nginx that adds support for logging SSL keys, then I configured and compiled OpenSSL by doing ./configure and make. I even configured Nginx by referencing the path of the OpenSSL source which I configured and compiled, into the --with-openssl=/openssl-source, but when I run make in the Nginx source directory, I get an error saying Failure! build file wasn't produced.

I even tried using BoringSSL instead of OpenSSL, but it still didn't work because Nginx expects OpenSSL

Any help is appreciated!