Rule Zero Tech Support |
Posted: 20 Jan 2021 03:57 PM PST It's an average post-holiday weekday afternoon at my work, which means that I am mostly hanging out, answering various questions from our first line of contact, clarifying department rules and updating procedures. I am also checking up on various ancient work orders gathering dust in our queue, usually for lack of customer or management response, and indiscriminately nuking the ones that I can close due to lack of customer response while auto-repeating a stage in an ongoing Arknights event. Just your usual Tier III slow day help desk stuff. An email chime rings. Now, my emails normally don't have audio alerts, except for a select group of very high ranked people, who need urgent Tier III level attention to address their pressing problems, such as plugging in a monitor. Among the team, I'm the one on actual emails and calls that afternoon, so I pop it open. Huh, it looks like our CEO is trying to join a conference via specific app, but it's just not working. Oh, and the conference started half an hour ago and they need someone to come up "in the next two minutes". Call me crazy, but if I have an important conference coming up using a program that I have not tested before, I might call IT out before the conference starts. As I enter CEO's office one walk up the stairs later, I discover that he's got a whole setup going with a smartphone he is using for this clamped in with actual proper hardware, good lighting, the whole thing. Frankly I am impressed and relieved: this specific conferencing software doesn't play well with our firewall sometimes, but if it goes out over the smartphone wifi that's way easier - they have their own, much laxer rules. "So can you tell me exactly what is wrong?" I ask CEO after exchanging extremely brief pleasantries. He is in a hurry, and so am I since I forgot to restart my Arknights stage before I headed up, so I figured we both have things we'd be doing that don't involve somebody making polite small talk for half an hour. "My conference app is not working", he replies, waving at the phone which does indeed appear blank. The app is up, but other than the meeting name it's just basically blank. You can hear people on the other end, but no matter if I click camera or mic buttons, nothing happens. A suspicion forms in my head. "Okay," I say, "drop the session and start it again with me looking over your shoulder. Let's go step by step." Everything goes perfectly fine, the app joins a meeting, he types out a name and -- why the everliving fuck is he clicking Don't Allow to every prompt that comes up? "Why are you clicking Don't Allow to every prompt that comes up?" I ask the CEO very diplomatically. "Oh, I was told I shouldn't let apps access stuff on my phone." "If you don't let this conferencing app access your microphone or camera, it will not be able able to transmit anything using your microphone or camera." "Oh, is that how it works?" "Yeah. Let me reset your permissions in the settings...here you go. Camera and audio feed. You are live." "Wow, you are great! I think they've been trying to solve my issue for over a year now with this phone app, and you fixed it in two minutes!" "Haha, no problem, you have a good day" I say as another horrible suspicion forms in my head. Back at my desk, after restarting my Arknights stage (you gotta have priorities), I start digging in the call logging system. Sure enough, here is a work order sent directly to our networking team, bypassing all normal channels (me) from 14 months ago, highest priority, four different techs all going back and forth about how our CEOs phone does not permit video and audio traffic from this conferencing app over our WiFi network. Vendors have been contacted, entire network closets torn apart and put back together, multiple wifi modems and APs replaced. There have been thousands of dollars and close to a thousand man-hours put in by people with six-digit salaries trying to fix this elusive issue. All because none of these Senior Network Engineers ever heard of Rule Zero: Don't Trust The Customer. As a Tier III tech, I have the ability to hijack assignments and make sure that everybody involved gets a message when I close the work order. This one was particularly satisfying to close, with a solution description of "Customer was denying permission to access phone resources to the app. No actual network issue is apparent. See Work Order #" So yeah. This is why the first thing I teach any techies I train is always Rule Zero: Don't Trust, and Also Verify [link] [comments] |
Posted: 20 Jan 2021 06:39 PM PST So the recent rule zero post on disallowing access reminded me of this fun one. Me: me, myself, and I. L2 helpdesk tech thoroughly sick of BS at this point L1: L1 tech with enough years of tech support that the questions asked should not be asked. We are working on building new hardware because I swear everyone's laptops started dying at the same time when we went WFH and we are constantly behind trying to keep up (between tea spills and cats I'm ready to pull my hair out) L1 tech lives relatively close to the office, so we asked him to go in and build recently. Afterwards he's supposed to get all the laptops back on at his home, connect to network and I'll remote in so we can tag team finalize custom settings. L1: "WiFi isn't working on any of these laptops..... neither is mic.... or speakers. This is weird." (queue panicked garbled L1 calling me talking a million miles a minute that everything is on fire.) Me: "walk back through the build runbook, any settings not match?" L1: "I swear I followed it exactly." Me: *sigh* "hardwire one for me please." *remotes in* After some poking around I determine that as far as this computer is concerned there is nothing plugged in for peripherals to this motherboard. "Hey, L1, can you please get eyes in the BIOS (yes I know it's technically UEFI these days, the text and my years still say BIOS, people understand what I mean) and check the I/O ports?" L1: grumbles and complains and finally pulls up the BIOS. "oh.... uhm, there is this I/O section with everything disabled except for the hard drive. Camera, WiFi, everything..." Me: "great.... can you please re-enable everything? remember when you were first confused in the BIOS back in the office and you started talking over me? You were supposed to be disabling extra BOOT devices! Not all the I/O ports. If you disable the motherboard from talking to the camera/wifi/etc.... those devices are essentially unplugged and it can't talk to them..." L1: "OH... is that what that means!?!?" Me: "There is a reason that the runbook with the settings I asked about DOESN't disable those things... don't touch BIOS settings that you don't know what they mean next time, and don't tell me something matches the documentation without re going through the documentation and confirming" (yeah, he's annoyed with me for that one... but...) [link] [comments] |
They wanted more emails, which they also wouldn't read Posted: 20 Jan 2021 05:41 PM PST I was the tech lead on a project that included a "bridge" component that served as an active sync/gateway for data between two different systems. During normal operations the bridge monitored the systems and as certain data elements were updated, it would translate them into the corresponding objects in the other system. It was our solution to allow the customer to use their preferred systems that (otherwise) didn't talk to one another. There was a customer liaison who was ridiculously hard to work with. Arrogant, opinionated, and they always covered up their deficiencies by throwing other people under the bus. I'm sure most of us have dealt with this type before. They were never incorrect or wrong; it was always somebody else's fault. We'll call this person "GT" So I'm working away, being productive, when I get a call from a manager, saying I'm needed on a conference call with GT right away. I grab my laptop and head down to the executive conference room, joining a call already in progress. They announce that I'm there and GT starts ripping into me. "The bridge is down! What's going on?" "It's down for routine maintenance. It will be down for another hour and a half while we upgrade the software." "Why wasn't I told about this?" "An email went out, as per our standard operating procedure, announcing this." "Well, I didn't get it!" (Pulling up my email client on my laptop) "Please refer to the email dated (a month earlier), titled "System Bridge will be offline for scheduled maintenance on (today's date)" which is addressed to you, the program manager, and the QA lead. <choked noises from the other end> "I get so many emails, how am I supposed to track all of them?" "It is marked "Urgent"" (the email system lets you set priorities) "Well, that's not good enough. I want you to notify me when this is going to happen." In my sweetest, most innocent voice.... "So, you want me to send ANOTHER email, because you don't read the emails I already send? I don't see how that's going to make things better." "Figure something out!" they shouted, and I'm looking around the room on my end, and most of the managers are trying hard not to laugh. I went back to work, shaking my head. God is great, beer is good, and people are crazy. [link] [comments] |
You are subscribed to email updates from Tales From Tech Support. To stop receiving these emails, you may unsubscribe now. | Email delivery powered by Google |
Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States |
No comments:
Post a Comment