NHacker Next
- new
- past
- show
- ask
- show
- jobs
- submit
login
▲Tesco moving 40k server workloads off VMware amid Broadcom's abusive conduct<span class="_source_ka9gd_36"> (arstechnica.com)
Rendered at 15:27:15 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
The Broadcom business model (outside the chip business) had been pretty well known, and they don’t really hide it.
They are tech bottom feeders. They find large businesses with a decent moat and free cash flow but are in long term decline (and wasting cash trying to find something new). They buy them, cut development, support and marginal products. Raise prices and squeeze as much as they can.
They are a PE firm
You're right that most shares are held by institutions (~80%), but that typically reflects the fact that most share ownership by individuals/companies goes through intermediaries (401k, fund investments, ETF etc.). Most of this institutional ownership is just asset managers, insurance, banks etc. taking their cut before passing returns/loses through to the end risk holder. The average institutional ownership of companies in the S&P500 for example is also ~80%.
None of this takes away from the point that Broadcom is absolutely run like a PE firm as the original commenter noted.
Not surprising given the CEO was appointed by KKR/Silverlake 20 years ago.
https://finance.yahoo.com/quote/AVGO/holders/?p=AVGO
So a public(ly-traded) private equity firm. :)
For any non Uk people, it’s the largest supermarket in Uk. Combination of large stores and smaller high street convenience stores.
(2nd largest was owned by Walmart who sold it recently to private equity and so now it’s saddled with debt and being ruined…).
So many well-known UK companies have been sunk by debt interest on loans taken out to acquire said companies.
By all means use the companies to secure loans, but the liability should be on the books of the parent companies not the companies being acquired!
There have even been cases where the companies have been effectively asset-stripped by "sell and lease back" of property, leaving the companies a shell of their former selves with no meaningful assets, so as soon as there are any unexpected headwinds they collapse.
I did a big work trip to the UK a couple years back with over 100 people. I tried to explain meal deals and nobody believed me. Then our people basically stripped the meal deal shelves of the tesco express beside our hotel.
Also, as a foreigner who lives over there, I think they are... sad? I'm surprised they got a positive reception from your coworkers. For me they are a backup and a failure to do something more interesting.
Most cities will have local sandwich options as well near major office districts, but they might not be as cheap.
Both are way ahead of the USA.
I imagine the positive reception is because the US doesn't have them so it's kind of novel.
Considering a £ is more then a €, supposedly at last - it doesn't sound like a good deal to me
It's also a dense enough country that they will be at most a day or two old.
https://www.tesco.com/shop/en-GB/products/262308005
The "main" has expanded to Huel, salads, wraps, sushi, even hot food
The "snack" can be more than crisps: small bags of fresh chicken, 2 boiled eggs, small sushi pack, gyozas etc
The "drink" includes quality smoothies, acceptable vending machine coffee etc
Meal deal value maximizing is the whole game lol. There are also lots of healthier options if you choose carefully
In certain Sainsbury's you can get hot food as the main such as a small green curry or chicken goujons, and wedges or hash browns as the side
But the price creeps up £0.50 practically yearly. I think it's £5.50 already in Sainsbury's
It's better to view it as a cheaper alternative to eating at a restaurant rather than somehow saving money compared to bringing in leftovers. People who think £5.50 a day for lunch is saving money versus cooking themselves are delusional
I ended up flying back home with some oxford shirt from the Tesco, and it's really cool (vneck pull over - tie - shirt sets were sold out with my size unfortunately)
Another was a Java SE licensing change that went from around $1k per instance, of which we had about 5. Mind you there is little to no maintenance support provided here. The increase was to $5.25 per organizational employee per instance, whether they used the instance or not - of which we had 100k. The choice was obviously a simple one.
I can only assume very few organization stay on the ride for those kinds of changes, but obviously they must - but why?
Typically you're looking at a year or two of discovery, audits and planning, another year or two to cover the main transition, and then up to five years of mopping up.
There are other near-ubiquitous vendors (eg. Microsoft and Cisco) who manage to be tolerated as annoying rather than outright abusive. I guess they take a slightly different view of how hard to squeeze their customers.
then oracle cut costs on next gen exo-data stuff and agreed to waive some license costs this time and bam just doubled down on them again. ugh.
They lived in fear of something slipping through the net. So print servers were switched off because they contained an embedded Oracle JRE. And deployment pipelines that used Hashicorp's Packer had to be rewritten to eliminate the VirtualBox plugin (despite it not being used). Office coffee machines were looked at with suspicion.
Every vendor had to be queried, every piece of software had to be tested and have appropriate controls put in place. There were pre-emptive audits and endless compliance procedures.
There was so much work involved that any cost savings must have been fairly minimal.
Or vendors just abiding by contracts they've already signed!
If you have a contract that says they can change the price at any time, and you're a business, that's on you.
Broadcom’s marketing for Proxmox is extremely effective.
"Although it was written with VMware as the source in mind, most sections should apply to other source hypervisors as well."
https://pve.proxmox.com/wiki/Migrate_to_Proxmox_VE
Our MSP refuses to consider ProxMox because “we need to support it”… but are happy as clams to throw me outrageous HyperV labor costs.
They’re literally putting me in a position where I either need to fire them because they refuse to use an open source solution and hire people that can read code… or fire them because they want 50,000 to move 15 VMs over to HyperV.
I want an MSP that isn’t scared of things outside of Microsoft.
Show me the uk website.
Oh what’s that? There isn’t one you say…
There is a lot that goes on that the media and the professionals involved don’t speak about.
Phone, email or visit the court registry.
Phoning will get you the national Contact centre, emails may or not any not be replied to… court registry needs an appointment…
It’s all very nuanced. Until you know how it all works, you don’t know
The filings by the parties are made public in the USA and other countries. So you can actually see the arguments and evidence.
Magistrates court trials are not recorded so no transcripts. So a malicious prosecution is easy to pursue.
When one files for a transcript in courts that are recorded, the transcript is first sent to the Judge and the judge is allowed to change anything they like, with no transparency. They can also refuse a transcript if they feel like it.
I’m not going to respond any further to you as you for some unknown reason are talking about things that you do not understand…
Have a good day.
The only exceptions are when the judge imposes reporting restrictions or anonymisation (unlikely in a non-criminal case as it's meant to protect the identities of complainants or sensitive witnesses) or for ex parte actions (which are very limited, eg. to cover an emergency injunction to prevent imminent destruction of evidence).
Magistrates don’t deal with what you term petty offences, their powers have been steadily increasing over time. And indeed any conviction has serious affect on a person even after spent.
The rest of your post is also that of a person with very basic understanding of the matters. So not sure why you commented.
That's horrific
Im confused as to why you think you know better
What is a VMware alternative, that isn't compatible with backup software? I'm guessing it's not nutanix?
Interesting times.
As a sysadmin of Proxmox, I do not see how it can scale to 40k VMs. The Proxmox folks themselves have seen "~24" nodes in a cluster (theoretical support is higher), so you'd probably need a lot of clusters for 40k:
* https://forum.proxmox.com/threads/proxmox-with-48-nodes.1746...
For such a size (and sticking strickly with open source), XCP-ng could be an option, or OpenStack. In the closed source space, Nutanix.
As of 2021, CERN had 35k instances/VMs in their OpenStack implementation:
* https://superuser.openinfra.org/articles/scaling-bare-metal-...
Why? Honest question, what leads to that Kind of size and why cant it use NAS shares or SAN disks for most of that data? Kudos on the migration!
When I get back to work I will finish the samba configuration of the ceph cluster front-ends to replace those elephants.
Having tried azure local before (it seems magical but the more you use it the worst it gets, update failed for no apparent reason on only some supposedlyidentical nodes, the sdn was atrocious to deploy and was manageable from wac only), I would recommend proxmox over it anyday.
If you don't have linux expertise on hand and have traditional FC based storage, I would recommend something else, probably nutanix if your budget is big enough.
I’ve worked with a few major US grocers on very similar projects (some hardware only refreshes and one VMware to HyperV/Azure Local migration).
OpenShift Virtualisation or whatever it’s called for the virtualisation part of VMWare.
Used to do those migration in a previous life.
Have lots of customer who run it and would echo your same positive review.
Prism Central has definitely gotten better with the UI since the earlier days. I still prefer Prism Element in some cases, but overall it all works pretty well.
We use HYCU for backups and while I was really skeptical about it in the beginning, it is absolutely solid in a Nutanix environment. Overall we are happy with Nutanix.
It's all based around open source projects virt-v2v and Migration Toolkit for Virt, and the typical target is OpenShift Virtualization.
There are various zero-copy options if you're using specific storage. In the best case the downtime for each guest can be as little as a few minutes. If the storage stars don't align then it can take a few hours per VM (but conversions happen in parallel, dozens or hundreds at a time).
[I don't have any specific knowledge about where this Tesco account is going. We have plenty of competitors. Everyone is dining at the Broadcom trough right now. Broadcom's "strategy" is absolutely baffling to me.]
Edit: Almost forgot that I gave a 5 minute lightning talk about it: https://pretalx.com/devconf-cz-2024/talk/SN93LG/
I know plenty of Enterprise customers who cannot move easily and just renewed 3 year VMware licenses for their cluster at insane rates. They are planning on moving but I'd be shocked if they complete it. $LastCompany had VMware footprint I know will be very difficult to move off, deployments, monitoring, backups were all dependent on VMware. There are plenty of US Government entities who are not even considering it at this time.
Also, Broadcom has slashed expenses so I wouldn't be shocked if profit margins are crazy. This article: https://www.theregister.com/software/2025/03/07/bulk-of-big-... indicates over 1 Billion additional revenue per quarter
If you look deeper into the migration article, it's pointed out that they are already facing migration challenges. I wouldn't be shocked if 3 years later, there are some workloads still running on VMware, you can't easily get them off and just renews insane licensing cost for much smaller hardware footprint.
What about the long term? Who care, massive money made and they can use that to keep going.
(As an end-user sort of person, I get a strong smell of Bladerunner from this kind of thing, where you can see old PCs in the background on top of decks with cables running out of them).
Usually the story is they're running something like CNC control which originally ran on baremetal, then got virtualized onto VMware when the hardware died (possibly using VDI to make it appear on a terminal close to the machien), and it's still doing the same thing effectively today.
It’s not growing in any meaningful way relative to other technology businesses.
And number of those has nowhere to go but down too. There is no growth in either of those, because everyone who will at some point try to get rid of them. Not all, not immediately, but the ultimate trajectory is down.
I understand that this is normal but I've never understood it.
If all the containers are running the same company's applications (so they don't care about security boundaries between them), what's the difference between having all the containers under the same kernel vs separate kernels?
Note: if you want to conflate “containers“ with an entire job management and scheduling system (“k8s”) then you’re not actually talking about the current target customer for VMware.
None of those matter in the slightest with containers. Why would you need to reallocate hardware resources when the containers can run on another piece of hardware? You would snapshot the relevant storage, not the whole OS and kitchen sink.
VMs as an intermediary between hardware and containers is just a waste of resources - both directly (RAM, CPU, storage to run a useless OS with no benefit) and indirectly (all of those VM's OS needs maintaining and patching).
It's basically a hold over from the olden days of "everything is a VM".
Sure, most of what we do is very cattle and we could run on bare hardware but why not take advantage of easy to add flexibility.
Your "machine for testing" would be a container. Decommissioning a container is easy. Decommissioning a physical machine only happens when it's obsolete.
The future is Snickers!
Also known as incompetence. Broadcom's business model is public. Their plans for VMware were public from when the acquisition was announced.
Those companies had years to plan how to get rid of everything VMware. Instead they paid through the nose to postpone the inevitable for a few years.
If one believes that they intend to get new VMware customers, or that they intend to have more than single-digit numbers of customers on VMware ten years from now, I can see how that might make their strategy baffling.
They appear to have made a lot of money doing what they're doing, so it looks to be working quite well for them... regardless of what the public or their former customers think about it.
Lots of orgs have been documenting their moves to KubeVirt over the past year or so. There's KubeCon video recordings on the youtube channel from Amsterdam with lots of this kind of stuff, especially from european end users.
One thing I find consistent is orgs are also looking at the whole stack, this is just another major component of digital sovereignty.
Disclaimer: work for CNCF on this but worked on the first version of VMWare Tanzu so every announcement in this space is interesting lol.
He talked about "Broadcom lies.."
Could very well just have been that last stubborn server they just never got round to!
With the new $35 billion dollar AI / XPU deal, Broadcom is looking to ditch the legacy customers and move to the new shiny AI billions.
Good on them for sticking it to Broadcom.
What is wrong with system designers these days? Are they designing or just selling?
The days of manually setting up servers in hyperscaling-environments are long long gone.
Example: Your GitLab CICD needs Runners. They are dynamically requested "somewhere in our cloud somwhere in the world" and then spun up and configures fully automatically. No human touches this stuff anymore.
Its ok for Amazon to do it since they paid for the physical machines anyway and they want to dogfood their AWS services, it does not make sense for someone who rents compute and licenses.
Maybe each self-service checkout runs a VM, or each staffed checkout.
They wouldn't want to rely entirely on a remote database to be able to sell products.
I worked with a Danish retailer with +3000 store in ~50 countries, and even adding their webshop on top and they were closer to 200 (maybe 300) servers (most VMs). Then you need the ActiveDirectory, office IT, all that stuff, with redundancy and it adds up quickly... but not to 40K.
What I will say that people forget is that production might be 8 beefy VMs, you still need to replicate that to a number of test systems, staging environments and so on. So a 8 node production cluster because maybe 24 servers when accounting those other environments.
I can see how you hit 40k pretty quickly.
That's just for the operations of the local stores and we're now at probably >25,000 VMs and we've only touched the retail locations. We still haven't addressed logistics locations, corporate offices, stuff to manage their customer-facing applications and websites, etc.
When I first saw 40,000 VMs I too thought it was a bit excessive, but when you're an org wit several thousand locations that you want to be somewhat self-sufficient things add up quickly!
a) you migrate in increments, so even if your migration needs to run old and new to compare, you don't need to do it for everything at once.
b) you probably have some slack, and you can make slack by packing tighter during migration.
c) you probably have some amount of regular hardware refresh. Retaining the old hardware a bit longer can get you more headroom for migration.
d) some servers can probably take an extended maintenance outage during conversion.
e) depending on everything, you might be able to get short term capacity from cloud or short term leases.
There's almost certainly some automation around migration. Some of it might even work.
Have a plan, make progress... even if you don't migrate everything by the date, you'll have done a lot and reduce the broadcom bill.
The actual hardware servers are clustered together into pools of resources. The pools are where the VMs live. The bigger the new pool becomes, the faster you can empty the old one. So the migration starts very slowly, ramps up quickly, and then tapers off.
For that half you are migrating, you are essentially operating without redundancy. If these are serious production workloads, the tradeoff is not as simple as you make it seem.
Normally, if one server on a production cluster goes down, the other members of that cluster seamlessly will take over. This is where the extra capacity comes in. You don't migrate the workload to another cluster. You just lose overhead capacity. If you lose too much then you start migrating parts of the workload to the failover. Not the entire thing.
You usually don't have to use your redundant cluster at all until it's time to rebuild the failed cluster. You might pick one of these spare clusters you keep around for redundancy to migrate all or part of the production workload to while you fix the production cluster.
When doing a big migration you take a percentage of your redundancy and convert it to the new environment. This is your staging environment. Once it is capable of doing work, you slowly grow it out and shrink the old environment at the same time.
Plenty of time to schedule a migration.
But it still took duplicate set of HW and I couldn’t imagine doing it without a lot of IaC and automation in place (plus physical space, power and cooling)
Like I said, I specialize in that. My career has been doing hard infra stuff other people won't or can't.
They are completely destroying their customer base for these products.
In some cases you can do zero copy conversions, so downtime can be done in a few minutes, but it relies on the customer have very particular storage configurations (NetApp basically). In other cases there can be significant downtime that needs to be scheduled. I worked one case where the customer shut down several production lines over a number of weekends so we could convert the workloads. (Everything was meticulously planned, along with fallbacks that thankfully we did not need to use.)
Some things you don't convert at all. Databases generally get replicated at DB level to new hardware. Single-purpose appliances need to be reprovisioned by going back to the vendor and asking for a KVM equivalent.
Then there's all kinds of craziness, like we had customers who rolled their own backup solutions where we had to add special cases to the software to detect and ignore the backup partitions. Or people running Windows 95 or RHEL 3 (for real!) where there are no virtio drivers and we don't certify the hypervisor so it requires support exceptions. At this point people have been using VMware for nearly 30 years, there's all kinds of crazy legacy.
Don't think about how hard it is to migrate a VM to a new provider. Think about how hard it is to:
* Get procurement to sign off on a new vendor
* Guarantee that your ISO compliance standards can be met under the new regime
* Make sure that GDPR requirements are met during any data transfer process to the satisfaction of your legal team
* Get the old infrastructure team and the new infrastructure team coordinated enough to be able to plan a migration without downtime
* Mollify the consultants that the CEO's friend said he should hire
* Analyse the migration plan to death to derisk it while at the same time be unable to actually evaluate it small scale due to the points above
I know nothing about Tesco, but sometimes ops cultures lack the skills or mandate to successfully switch tech stacks.
> procure alternative solutions with reduced functionality
meaning VMWare is still basically the only option if you need something that works out of the box. Hopefully this changes in the mid term as other customers migrate away.
The competition is compelling, actually. Red Hat OVE, Nutanix for those who want support, and Proxmox is emerging as a possibility in the ent space.
I read "reduced functionality" as they married themselves to something specific and non-portable, like oh, pick a card from VMware - NSX networking, VSAN storage, maybe something in Tanzu, and that phrase reflected their difficulty escaping the lock-in quickly enough. (This was all speculation)
I've negotiated a lot of contracts and renewals. I've been threatened twice - Oracle, and then Broadcom. We had perpetual licenses, but that didn't matter, according to them we were out of compliance and as a "courtesy" they delayed sending C&D as a precursor to suing us - this was the intro meeting call. There was no budging on price, and they actually priced the cheaper alternative we could have considered ("VVF") at like a 0.1% discount from their core "VCF" product, I think as a fuck-you. It was a great time, our reseller and I shared a drink over that one.
>"Broadcom’s recent $1 trillion valuation is largely related to Broadcom’s expectations of AI"
Who needs paying customers when you have AI?
Possibly they’ll do enough brand damage that it turns out to be a negative ROI, but for now they’re printing money.
Even better if you can charge a mildly high license fee for 20 years first and then jack it up to something outrageous and still have customers who just can't drop you.
migrating to quarkus won't save you either - since it's IBM on the other hand.
if only other ecosystems could catch up to Java/JVM solutions.
any attempt at milking spring-boot will lead to forking it into OpenBoot or something
If AI survives, we’ll see inflated costs drive companies back to hiring actual human beings to do the work.
At EDB we’ve forked Greenplum from last OSS into WarehousePG, added over a dozen customers with petabytes of data, and hired a few dozen specialists. We have an extension for Lakehouse connectivity based on DataFusion (with optional offload to Spark including GPU acceleration) to read/write Iceberg. And we have a lot planned for the next version, which you might infer from the name: WarehousePG 19.
[0] https://github.com/warehouse-pg/warehouse-pg