Raziel Posted July 10, 2022 Share Posted July 10, 2022 I've been trying to figure out just *what* Rodgers did that fucked their entire network for 15 hours. Like, what you need to do to cause a nationwide outage is pretty specific. 1 Link to comment Share on other sites More sharing options...
tbarrie Posted July 10, 2022 Share Posted July 10, 2022 12 minutes ago, Raziel said: I've been trying to figure out just *what* Rodgers did that fucked their entire network for 15 hours. Like, what you need to do to cause a nationwide outage is pretty specific. My supposition since early yesterday afternoon has been "Something sufficiently stupid that they can't even figure out a good way to spin it." On a technical level - yeah, no idea. 1 1 Link to comment Share on other sites More sharing options...
J.T. Posted July 12, 2022 Share Posted July 12, 2022 On 7/10/2022 at 1:31 PM, Raziel said: I've been trying to figure out just *what* Rodgers did that fucked their entire network for 15 hours. Like, what you need to do to cause a nationwide outage is pretty specific. "caused by maintenance in our core network" = we fucked up. 2 Link to comment Share on other sites More sharing options...
JLSigman Posted July 12, 2022 Share Posted July 12, 2022 1 hour ago, J.T. said: "caused by maintenance in our core network" = we fucked up. Absolutely. This is 20 years ago, but we had an outside contractor come in to DHEC and try to upgrade our email server and it died for a week because they didn't actually test the upgrade on the stuff we were running. 1 Link to comment Share on other sites More sharing options...
Technico Support Posted July 12, 2022 Share Posted July 12, 2022 Everything is broken: "What the fuck are we paying IT for?" No outages, everything is great: "What the fuck are we paying IT for?" 1 4 Link to comment Share on other sites More sharing options...
J.T. Posted July 12, 2022 Share Posted July 12, 2022 (edited) 53 minutes ago, Technico Support said: Everything is broken: "What the fuck are we paying IT for?" No outages, everything is great: "What the fuck are we paying IT for?" When I worked for the Japanese, they loved the sound of silence. That means that everything is operating as it should because people are working hard in the background to make sure bad shit does not happen. Now that I work for the Department of the Army, they take the Quarrian from Mass Effect approach to silence. The last thing they want to hear is quiet. Artillery should be booming and shit. If there is no noise, something is wrong. Edited July 12, 2022 by J.T. 1 1 Link to comment Share on other sites More sharing options...
Raziel Posted July 12, 2022 Share Posted July 12, 2022 2 hours ago, J.T. said: "caused by maintenance in our core network" = we fucked up. I read that as "someone deployed a new config to every main device and it was either missing a line, or there was a license" that needed to be applied first (because missing the license disables the commands and makes something needed NOT work)." and they needed techs to go out to every individual device and roll the change back 1 Link to comment Share on other sites More sharing options...
Tarheel Moneghetti Posted July 12, 2022 Share Posted July 12, 2022 (edited) 1 hour ago, JLSigman said: Absolutely. This is 20 years ago, but we had an outside contractor come in to DHEC and try to upgrade our email server and it died for a week because they didn't actually test the upgrade on the stuff we were running. Same. In the early 2000’s, my employer had our service provider come in and upgrade the internet and cable. Service provider didn’t test internet and cable connections when they were done, so they didn’t realize that the entire building was a buffering nightmare. Wiring in the building was shoddily done by builder and couldn’t handle the upgrades. The service provider ended coming back and rewiring a lot of the building for free because it was the only way to fulfill the contract and the contract didn’t allow them to charge extra for the work. Edited July 12, 2022 by Tarheel Moneghetti 1 Link to comment Share on other sites More sharing options...
Technico Support Posted July 12, 2022 Share Posted July 12, 2022 About time to start that "IT people bitching about the IT field" thread. 1 1 Link to comment Share on other sites More sharing options...
Robert C Posted July 12, 2022 Share Posted July 12, 2022 2 hours ago, J.T. said: Now that I work for the Department of the Army, they take the Quarrian from Mass Effect approach to silence. The last thing they want to hear is quiet. Artillery should be booming and shit. If there is no noise, something is wrong. The sound of quiet when there isn't supposed to be quiet is bad stuff. I did an internship at Motorola during undergrad. One day I was trying to grab the signal processor chips I was supposed to be testing, which were stored on a rack next to one of the production testers. Specifically, they were right next to the big, red button on the side. It was amazing how quiet everything got when all those fans shut off at the same time. Those things aren't supposed to shut down, ever. Took em about three hours to get it up and running again. Would've been worse if the Teradyne techs hadn't happened to be on site that day. Not fun. 1 Link to comment Share on other sites More sharing options...
J.T. Posted July 12, 2022 Share Posted July 12, 2022 56 minutes ago, Technico Support said: About time to start that "IT people bitching about the IT field" thread. I would mostly bitch about my Luddite end users. 2 Link to comment Share on other sites More sharing options...
Technico Support Posted July 12, 2022 Share Posted July 12, 2022 Just now, J.T. said: I would mostly bitch about my Luddite end users. PEBCAK error. Link to comment Share on other sites More sharing options...
J.T. Posted July 12, 2022 Share Posted July 12, 2022 (edited) 2 minutes ago, Technico Support said: PEBCAK error. We will not even discuss how many sim accounts I had to unlock today because I was out on leave and mother fuckers cannot remember their passwords for shit after a long holiday break. Edited July 12, 2022 by J.T. 1 1 Link to comment Share on other sites More sharing options...
J.T. Posted July 12, 2022 Share Posted July 12, 2022 (edited) 1 hour ago, Robert C said: The sound of quiet when there isn't supposed to be quiet is bad stuff. I mean, yeah. if I did not hear the whir of a dozen blade servers in the CIO or if the chiller went dead, I'd definitely freak out. I expect a little white noise, but Army dudes are not happy unless there is activity all around them at all times. Harmony is bad. If there is no problem to solve, they will try to invent one. You cannot go up a grade if all of your promotion paperwork says "No issues at this station." Officers try to pass this paranoia off onto us DACs and we resist with all of our might. There are certainly a few things that keep me up at night, but I'm not up all night worried about shit every night. Edited July 12, 2022 by J.T. Link to comment Share on other sites More sharing options...
Raziel Posted July 12, 2022 Share Posted July 12, 2022 2 hours ago, Robert C said: The sound of quiet when there isn't supposed to be quiet is bad stuff. I did an internship at Motorola during undergrad. One day I was trying to grab the signal processor chips I was supposed to be testing, which were stored on a rack next to one of the production testers. Specifically, they were right next to the big, red button on the side. It was amazing how quiet everything got when all those fans shut off at the same time. Those things aren't supposed to shut down, ever. Took em about three hours to get it up and running again. Would've been worse if the Teradyne techs hadn't happened to be on site that day. Not fun. Never, EVER, touch that big red button. We had a white glove delivery driver reach for that button on the wall thinking it was the big, yellow button that released the maglocks on the doors from the inside, even though the yellow button had a "PRESS TO EXIT" sign above it. That was the last time we saw that driver in our site. Link to comment Share on other sites More sharing options...
Raziel Posted July 12, 2022 Share Posted July 12, 2022 3 hours ago, Technico Support said: About time to start that "IT people bitching about the IT field" thread. 90% of my bitching would be about Sales, PM's, or Executive Management. 3 Link to comment Share on other sites More sharing options...
Ramo2653 Posted July 13, 2022 Share Posted July 13, 2022 11 hours ago, J.T. said: We will not even discuss how many sim accounts I had to unlock today because I was out on leave and mother fuckers cannot remember their passwords for shit after a long holiday break. I want to laugh, but I had this problem when I took my big 2 week vacation last August and we switched our login system and I didn't take any notes on my new password I was down from 8:30 to 1 that day. At least I got to read the board on my personal laptop while I was on hold. 1 Link to comment Share on other sites More sharing options...
J.T. Posted November 17, 2022 Share Posted November 17, 2022 Though it has long since been patched, here is a live demonstration of the thread's namesake for those of you who have never seen the exploit in action. KRACK Attacks: Bypassing WPA2 against Android and Linux - YouTube Link to comment Share on other sites More sharing options...
Raziel Posted November 18, 2022 Share Posted November 18, 2022 That would be one reason we try and account for any rogues we find when doing wifi validation/predictive walks when designing. Link to comment Share on other sites More sharing options...
J.T. Posted November 21, 2022 Share Posted November 21, 2022 (edited) You white hats and lovers of Mr. Robot should get a kick out of this. Here are 4 episodes of Dave Bombal's podcast featuring hacks from Mr Robot demonstrated by everyone's favorite grey hat, OccupyTheWeb: Hack like Mr Robot // WiFi, Bluetooth and Scada hacking - YouTube SMS spoofing and Raspberry Pi Scada hacking - YouTube Warning! This is how cars are hacked. Just like in Mr Robot. - YouTube Hacking cell phones like Mr Robot - YouTube Yes, those were real live cracks on the show. So realistic in fact that CERT issued a white paper for the FBI cell phone hack. Edited November 21, 2022 by J.T. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now