I’m sitting at my desk, third coffee in hand, still trembling after my visit to DefCon—with all its jiggery-pokery hardware hacking. But let’s tamp our flights of fancy down a little bit. Today, I’m going to tell you about one of my favorite experiences – when our Network Operations Center fixed a manufacturing behemoth’s uptime issues down to just about zero. This one is a good reminder that sometimes — old-school grit with modern tech is golden.
I began my dark journey in 1993 as one of you, as a network admin — back in the day when if we wanted to duplex voice and data on PSTN we had to jack with a multiplexer. And yes, I observed that Slammer worm slamming into networks like a charging ram. Fast forward, and I now co-own PJ Networks — one of the (if not the) most established of cybersecurity services businesses. Here’s a story about what can happen when you mix decades of experience with relentless problem-solving.
MegaMach is a national producer of industrial components both in the United States and abroad. They rely extensively on an 24/7 IT infrastructure. Which, you know, has historically been anything but that. MegaMach was experiencing monthly network downtime of about 4 hours before they came onboard with us and were risking severe production down-time and loss of profits.
But here’s the kicker: their current monitoring system was reactive! That is, alerts arrived only when things stopped working. They were “flying blind”, relying on antiquated hardware and could not easily pinpoint a cause for ongoing issues. No wonder their uptimes managed to hover down around 99.5%—and while that might sound all right in some contexts (manufacturing ain’t one of ’em?) It’s like a car that sputters every 10 miles on a cross-country drive. Not acceptable.
And if you believe there is no cyber security angle here, think again. We saw aging firewalls with out-of-date firmware and unsafe levels of segmentation. To be honest, if I didn’t interfere, the flies were eventually going to have a ransomware nightmare — one I’ve witnessed too many times on far too many factory floors.
By the time MegaMach approached us, the company required more than a few band-aids. We performed this work, however, as surgeons and strategists. Here’s a quick rundown:
I recall our lead engineer saying: “This is not about tech, it’s about putting tools in place so the entire business is working better.” And he was right. Unfortunately, we’ve all been burned before by the company whose NOC is an opaque box that just rings bells instead of delivering actionable intelligence 24 hours a day.
We staffed a 24/7 NOC with experienced pros, people I’ve worked with for years, a few of whom I’d handpicked because I’ve seen them sniff out even the faintest of oddities on the network.
Some notable steps:
Here’s the best part. MegaMach after PJ Networks NOC in the last 6 months of support:
The head of operations at MegaMach told me later, “You fixed what we thought was unfixable. Our production lines not only go; they roar — and we sleep better at night.” Feedback like that keeps me operating on all cylinders, even after decades on the job.
So what can you take away?
If I could go back to those heady days of pounding networks with NSTSulines I’d say to myself — “Stop worrying about speed, start worrying about resilience” Today, that adage still applies to cybersecurity and network operations.
If you run a manufacturing or industrial network and downtime is “part of the game” — it doesn’t have to be.
I’m not going squishy here — I honestly think cybersecurity and network management are the new engine of industry. The stakes are this high. I’ve survived the worm anarchy of Slammer, seen banks excitedly accept zero trust architectures (I helped three of them do so last year alone), and tango’d on the bleeding edge of hardware hacking at DefCon. The basics are — ironically — stubbornly basic.
Good visibility. Smart segmentation. Vigilant patching.
I mean, of course, huge AI-powered security solutions can be cool, but sometimes they’re just a bloated use of tech. For those of you who asked me at DefCon about how we do it, well I say, get a good NOC and some real expertise. Where PJ Networks excels.
Oh — and a quiet note on passwords, before I go. Please, no more making users memorize complex passwords they aren’t able to. Instead, provide them with honest multi-factor auth and good behavioral analytics. That is security that does have its uses.
Last, but not least — give credit where credit is due. MegaMach has granted us permission to have published these information, and it’s their openness that made this achievement happen.
Thanks for reading. Now excuse me while I go pour my fourth cup of coffee. See you at the next blog post, where I’ll most likely just return to griping about router misconfigurations or what have you.
— Sanjay Seth
Founder, PJ Networks Pvt Ltd
The Future Of CyberSecurity Digital Forensic and Data Recovery Expert. CyberSecurity Consultancy 1993 CyberNetworking.co.uk Networking Information Security.