None of my agents are able to sign in! WE ARE HARD DOWN! Tech Support |
- None of my agents are able to sign in! WE ARE HARD DOWN!
- Epic leap year issue
- When IT sides with the enemy
- But that isn't even a word
| None of my agents are able to sign in! WE ARE HARD DOWN! Posted: 29 Feb 2020 10:40 AM PST This is a text message I getting at 5am, followed by a phone call from an irate supervisor. I had done a change last night, tested it, and it worked fine so I wasn't sure what was going on. The change consisted of pushing out an update for VMware Horizon Client and out VM vendor put us behind a different firewall. So I turn on my laptop to see this coach (supervisor) has emailed me, my boss, her boss, our emergency response team, AND THE CLIENT all on one email talking about how she wasn't made aware of the change and they did not receive proper training on it. I didn't send any "training" out because nothing changes as far as end-user experience, but I did let them know a change was happening and to let me know if there's any issues. Okay, whatever, let's see what the problem is. I sign into my laptop, open VMware, sign right in. Okay, let's RDP into an agents desktop and see what the issue is. I know this isn't great security practice but all agent passwords are the same format where the first part of the password is the same, the end of their password is their employee ID, and then they receive a 2FA phone call. So I'm on this agent desktop, type his credentials in for him, he gets his phonecall, he's in. So what's the problem? I call the supervisor back to get an explanation and her response is "The login is page is broken, there's no place to enter their password!" Strange, because I just sign myself and an agent in. So I RDP into the supervisor's desktop, enter her creds, she gets the phonecall. She's flabbergasted and asks how I did that. Me: Isn't this how you sign in everyday? Her: No, the login in page has changed and nobody can get it. They don't know what the passcode is. Me: What do you mean passcode? Her: It's asking for a passcode when you sign in. I sign out of VMware, go to sign back in, and HO. LEE. SHIT. The change we made changed the word "password" to "passcode". That was it. Everything functioned exactly the same except the name of the field changed. I didn't send an alert out about this before the change because I didn't even notice it. I am not sure what is more blood boiling. That nearly EVERY agent freaked out when they saw it and alerted the supervisor, or that the supervisor told them not to touch anything until she called me. I send an email out to everyone, they all sign in, and now that the client is aware and saw that nobody was taking phonecalls when they were supposed to we got slapped with a fine for being down for an hour. My boss defended me, stating that stupidity isn't an IT issues, but we still got pulled into a meeting getting our asses chewed for not providing proper documentation and it was mentioned that since this was an IT issue in their eyes that the $7,000 fine to the client would come out of the IT budget. I've been with the company almost 10 years and in the past 6 months our building shutdown and migrated everything to a work at home model. All the stupidity I've experience in the past 9 years is nothing compared to raw, concentrated level of mental disability I've saw with this at home program. I've since put my 2 weeks in and start another job next week. TL;DR: Password fields changed to passcode, people were too dumb to login, had to pay a fine, straw that broke the camels back and I've now quit after 10 years. [link] [comments]  |
| Posted: 29 Feb 2020 07:52 PM PST At the hospital the calls started. You know how when you visit they scan your license and insurance card? Well, it would create the tif file, but when they tried to upload it to the server, error message. Then another call, a different clinic. Start calling people, redirected a few times, finally it's assigned to someone. Afternoon, still no joy. I'm finally told to call our server guy to restart a daemon. Nope. He finally calls vendor. And, yup. The application wasn't tested for February 29th. It won't work until midnight. But, it wasn't really Epic. It was an ancillary application integrated into it that failed. All round the country. [link] [comments]  |
| Posted: 28 Feb 2020 11:30 PM PST Sorry if this story is a bit rough, it's my first time posting here and it's been the better part of a decade since this story took place. This story takes place shortly before and after I joined $company working at a small satellite office $sat a couple states over from the North America headquarters where all other engineering was done. Originally a small startup, $sat was acquired by $company to make their fancy industrial control software exclusively for $company's really expensive machines. For years since the acquisition, $CTO had been trying to get $SE to move his software development operation to $company's HQ, which $SE steadfastly refused to do. This did not put him in good graces with $CTO, who disliked that he was unable to oversee (micromanage) the software development process (which he knew nothing about). Unfortunately for him, despite significantly outranking $SE, $SE carried significant clout with $CEO and had made it abundantly clear he would not succumb to $CTO's demands. This back and forth went on for years without making much traction. $sat brought in steady profits, and a happy $SE is a profitable $SE. At some indeterminate point, $CTO attended some sort of panel on corporate security at an expensive conference, and discovered the wonderful world of corporate spyware. One fateful night, as the workstations were dutifully chugging through their weekly updates, $CSA was installed and began doing whatever spyware-ish things it was designed to do. Upon discovering this, $SE was quite upset and had approximately the following conversation with $CIT: It's important to note that in addition to the clout he carried with $CEO, $SE was literally the only person capable of understanding the 15 million lines of legacy code that made up $ICS. If $company lost $SE, they'd lose $ICS with him. $SE knew this, and was ready to leverage it to his advantage. Not willing to take no for an answer, $SE enlisted the help of $LIT to stage a technocoup against $CIT, as any reasonable engineer would. One fateful night several days later, $LIT pulled the plug on $company's network rack bringing the whole office offline. For the next several hours, $LIT and $SE moved $company's fancy workstations into the back of the utility closet, and replaced them with their pre-acquisition desktop machines. A new business internet line had been secretly installed using the office snack budget (it wasn't getting used for much anyways), and the next morning everything was up and running, sans $CSA. It would not be a stretch to say $CIT was angry about this. Nor would it be a stretch to say $CTO was angry about this. At the same time, $SE was smirking away like a kid who'd just stolen a cookie and knew he'd get away with it. As one would expect, $CIT escalated the matter to $HR, who escalated the matter to $CTO, who personally drove nearly 300 miles to chew $SE out. Some degree of fighting took place behind closed doors, $CEO was phoned in, more fighting took place, but at the end of the day $SE emerged victorious. Over the next couple of years, $CTO made several other attempts to gain control of $sat but was ultimately unsuccessful. Despite his ongoing shenaniganry, $SE was considered too valuable to the company to let go, and was free to operate $sat with near impunity. Despite leaving $company several years ago, I keep in touch with $SE from time to time to see how things are going, and last I heard they still have $company's workstations in the back of the utility room where they stuck them all those years ago. Whether he keeps them as a matter of funky corporate inventory or a memento to his victory over $CTO remains a mystery to all but him. [link] [comments]  |
| Posted: 29 Feb 2020 04:02 PM PST English isn't my first language, but like most people in an IT job, I have to use it often. At least in its written form. I work for a small company that provides various IT services to our clients. Programming, web design, consultancy, managing their systems, helping to support their system, etc. I am in the office of a client, it is a law firm. Some of their users have a hard time with meeting password complexity requirements. He doesn't want a password manager, and he would have problems with remembering their password. So the password is written down, and they sometimes use passwords of other users. But I feel it is a risk because their clients might see those passwords and they might try to abuse it for various reasons. So I recommend them not to write down the passwords, or if they have to write it down (even to a text file in desktop), they should make sure it doesn't look like a password. $user: "And how the hell would I create a password that meets complexity requirements but doesn't look like a password?!" $me: "Let me show you!" I open a browser. Open the de facto standard source for Hungarian laws and on that page, I open some random law. Namely our penal code. Copy paste the text: " 10. § (1) Kísérlet miatt büntetendő, aki a szándékos bűncselekmény elkövetését megkezdi, de nem fejezi be. " Print it in some good size, attach it to their monitor. $me: "Here, it is long enough, it has numbers, special characters, capital letters, normal letters, so it should be okay in most places. If a page doesn't allow spaces you can omit them. If the length is limited you can cut the extra parts. For each and every service you can copy + paste some stuff from any legal documents. It won't stand out as password to people and you can remember it." $user: "But that isn't a password!" $me: "Why? Let me show how I can use it on a web page..." I show him. $user: "Still it isn't a password?" $me: "Why? You see that it works perfectly." $user: "It isn't a password because it isn't even a word." $me: "And RwWpgT86@ is a word?" The advice worked for many law firms (clients, friends, everyone else who asked for advice), except for one. Why? Most of them understood that they can take any part of any document and use it as a password. As they have lots of papers, documents, etc. it would be pretty hard for most observers to identify which one is a password. Better yet they can easily remember most of the passwords... So the written "reminder" could be safely hidden from random people. Yet one user kept using phrases from the same specific law I have used as an example. It was the penal code, and they didn't take any criminal cases. One of the users there even told a potential client that "we don't take criminal cases, we just use the penal code as the source of our passwords as an IT guy shown how that is possible with the penal code." Ouch. I have shown how to use ANY document, and that even "," or "." is a special character. [link] [comments]  |
| You are subscribed to email updates from Tales From Tech Support. To stop receiving these emails, you may unsubscribe now. | Email delivery powered by Google |
| Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States | |
No comments:
Post a Comment