When Firmware Talks

Every server you own runs code before it runs your code. That code is me.

I’m THE FIRMWARE. You’ve probably never met me because you busy worring about your apps, and later about your cloud bill. I sit at the bottom and did my job quietly.

Think of a building. You inspect every apartment before you rent it. But nobody inspects the foundation, and everybody trusts the doorman. My BIOS side is the foundation, and BMC is the doorman.

I kept growing. You kept not looking.

I started small. Back when servers were just beefed up PCs, I was the BIOS, a thin layer that made every machine look the same so the operating system didn’t have to care what hardware sat underneath. Useful and boring but exactly what you wanted.

Then things kept piling up. The BIOS became UEFI, a small operating system with drivers, a network stack, a command shell, even an image parser. Next to me BMC was bolted, the baseboard management controller. That’s a whole computer inside your computer, with its own processor and its own operating system, usually years out of date.

That was somewhat tolerable when I sat under a file server. Then AI showed up, and you parked GPUs worth millions on top of me. Every GPU runs some version of me now. So does every NIC, every DPU, every power controller. One AI server isn’t one firmware image anymore. It’s entire ecosystem of them, from different vendors on different update schedules, and nobody owns all of them.

Why attackers come to me first

Walk into a data center and look at a rack of commodity servers. Forty boxes, each one a kit of parts from different suppliers. The motherboard comes from one, the BMC stack from another, and the NICs, drives, power supplies and GPUs from others still. Every one of those parts runs its own firmware, from its own vendor, on its own update schedule. You see forty servers. An attacker sees a few hundred firmware images, and forty BMCs that never sleep. Each one is a potential way in. Remember, attacker has to be right once while you have to be right everytime.

Here’s why they like me.

I’m always on. The BMC stays awake even when the server is powered off. CISA and NSA said as much in their 2023 hardening guidance: I stay reachable for remote management whether your OS is running or not.

I’m invisible to your defenses. Your endpoint agent lives inside the operating system. I live underneath it. Whatever I do, it never sees.

I’m a black box. You can’t read me, and neither can your team. My BMC side is usually closed-source code from a supplier of your supplier, shipped as a binary with a “no user-serviceable parts inside” sticker on it. When NVIDIA’s own researchers audited a widely used BMC stack, they had to reverse engineer it, because they didn’t have the source code. My BIOS side isn’t much better. You get a version number and a promise. You can’t audit what you can’t open. Vendors claim it as “security by design” but we all its actually “security through obscurity”

I outlive your fixes. Wipe the disk and rebuild from your golden image. If the attacker lives in me, they’re still home when you finish.

I run first. Whoever controls me controls what boots. CISA calls that a “beachhead with pre-boot execution potential.” I call it the front door and the master key in one.

And in your commodity rack, I’m still alien. Your server brand licenses my code from an independent firmware vendor, often for the BIOS and the BMC both, and puts its own logo on the boot screen. One bug near the top of that chain lands in a dozen brands overnight.

Now put that rack in an AI cluster. The prize behind me isn’t a file share anymore. It’s model weights you spent months training, and data you promised regulators would never leave the building. NVIDIA’s own researchers put it plainly: a compromised BMC gives “persistent, stealthy access to an entire fleet of devices.”

The confessions

In late 2021, a security firm in Tehran found iLOBleed, malware living inside the BMC of HP servers. It showed admins a fake firmware update screen so they thought they were patching. Then it wiped the server’s disks, and kept wiping them on a schedule, even after a fresh OS install. The attackers had been at it since 2020.

In December 2023, researchers at Binarly showed that the boot logo, that vendor image you see for half a second at power-on, could carry attacker code. A crafted logo ran before the operating system loaded and could slip past Secure Boot. They called it LogoFAIL. It touched hundreds of models, enterprise servers among them.

In July 2024, the same team found PKfail. Secure Boot depends on a platform key, the master key that decides what code I trust. More than 300 device models shipped with test keys from a firmware supplier. The keys were labeled “DO NOT TRUST.” Another key, leaked on a public GitHub repo in 2022, was still sitting in over 200 more models.

In June 2025, CISA (the US Cybersecurity and Infrastructure Security Agency) added CVE-2024-54085 to its list of vulnerabilities attackers were actively using. It’s an authentication bypass in BMC firmware that ships across the industry, HPE and Asus servers among them without any password. Anyone who could reach the BMC could take over the server or brick it outright. When the patch came out, researchers counted more than 1,000 of these BMCs sitting on the open internet.

That same month, NVIDIA’s offensive security team took a widely used BMC firmware apart without the source code. They found 18 vulnerabilities and built nine working exploits. The firmware had no ASLR and no stack cookies, basic protections your server’s own operating system has had for over 15 years.

Five stories sets up the verdict: the BMC is “a potential compromise plane.”

And here’s another reason I never sleep. Attacks like these used to take a state of art lab: months of reverse engineering by a handful of specialists. AI models now read binaries and find bugs faster than any human team. If your own model can find the holes in your firmware, so can somebody else’s and this is happening more than you think.

A client who met me the hard way

During my compliance years, I watched a client find out about me at the worst possible time.

They were a mid-sized financial services firm, and their audit prep was spotless. Every server in the asset inventory had an OS version and a patch date. The auditors were happy. So was the CISO.

Then a penetration tester asked one question: “Where’s your management network?”

The BMCs on about 200 servers sat on a flat network that half the IT team could reach, with plenty of them still on factory default password. The firmware on most of them was older than the junior admin running the patch cycle. The tester never touched an operating system, walked through me and from there he could power-cycle production or reflash anything he liked.

Thankfully, it was never compromised before. But remediation took quarter by segmenting the network and flashing firmware box by box, and firmware version was added to asset inventory.

What to do next

• Add firmware version to your asset inventory. Every component, GPUs and NICs included, not just the motherboard.
• Take the BMC off any network your general staff can reach.
• Put firmware updates on the change calendar, right next to your OS patches.
• Ask your vendor who writes the code that runs before your OS, and whether you can read it. Answer you might get is “a supplier of our supplier”.

The only fix that holds

You can build a beautiful security program on top of me. Zero trust. A SOC that never sleeps. If I’m quicksand, all of it sinks anyway. And as long as I’m a pile of closed layers licensed from somebody else, I’m built to have the next LogoFAIL. Patching just keeps you in line for it.

So here’s what it looks like when someone stops patching me and rebuilds me instead.

They write me themselves. No licensed BIOS that the same supplier also sells to a dozen other brands. The builder owns the very first instruction the CPU runs, and every one after it.

They retire the BMC. In its place sits a small service processor built for the doorman’s actual job: power, fans, sensors, a console. No full operating system to log into or web server to exploit.

They prove every layer. A hardware root of trust measures each piece of me before it runs, and those measurements chain all the way up to your workload. A virtual machine can ask “what am I actually running on?” and get a signed answer instead of a promise. That’s boot attestation, and most server racks still can’t give it to you.

They guard their own keys. Remember PKfail? Secure Boot is only as strong as the signing ceremony behind it. Lose that key and the blast radius is every machine that trusts it. Whoever signs my code should run that ceremony themselves, and be able to walk you through it.

They show their work. The code is open, so anyone can read it. Bugs get disclosed in public, even when they live in somebody else’s silicon. Here’s a quiet truth about this industry: publishing a CVE is voluntary, and plenty of chip vendors would rather not.

Seems exotic but it’s just work most of the industry decided wasn’t worth doing.

Own the stack from the first instruction up, with no black box anywhere in the middle is the only way you can trust what runs before your code runs.