Overheard a troubling support call from my coworker... Tech Support |
- Overheard a troubling support call from my coworker...
- Never Trust The Guy That's Leaving
- Mercury's position should not effect your backup procedures
- Trying to have a meeting by only brining external monitor where files are kept!
- The kayaking incident
- You know what they say about eyesight and computers.
- The Cabinet of Horros - Part II: The Cleaners
- What's a beamer
| Overheard a troubling support call from my coworker... Posted: 27 Jul 2018 11:04 AM PDT For my first post to this sub, I've got a pretty short one. I do a mix of development/testing/support/odd jobs at a really small company that makes voice controlled applications for warehouses. Got a call from one of our more... challenged customers just now, and here's what I overheard from my coworker who answered... Coworker: after listening to the other end in silence for a minute or so "...oh... so you deleted the database...?" Those of us not on the phone listening: "Well that doesn't sound good..." pause Coworker: "oh... you deleted the entire directory??" Us: "Oh shit... that's really not good..." pause Coworker: "You deleted the whole server?!?" Us: "WTF?!?" He's helping them through the issue, but this is just par for the course for this customer. I've had to handle several after hours issues for them where they call to report that they made drastic network changes without notifying us and wonder why things stop working... [link] [comments]  |
| Never Trust The Guy That's Leaving Posted: 28 Jul 2018 01:58 AM PDT Another story to go along with my tale of the broken installation. The big difference with this tale is that I take an awful lot of the blame for what went wrong. My own stupidity and not making time for the simple stuff left us pretty far up the creek without a paddle in sight. But please do remember, before you judge me too much, that as skilled as I am, I was simply used as a firefighter whenever those above me screwed the pooch. I had little knowledge of the infrastructure as that information was kept from me. Some backstory...
Anyways. Upon handing his notice in, he started to edit these email settings so they mailed me directly. I very quickly set up a Distribution List for such things and asked him to send them there. MISTAKE 1 - I definitely should have done this myself. Why, will become apparent pretty soon. $ITM "It'll mail you anytime it has an error or fails a backup" $TR "Not for successful backups?" $ITM "No, there's too many jobs and it'll spam you and you'll just ignore them." MISTAKE 2 - I know, I can hear you all screaming at me. I should have gone and changed that. Let's assume now that everything was running smoothly and I wasn't fighting a constant slew of dumpster fires on a daily basis just to keep things running for the 4 weeks between $ITM leaving and where our story starts. A 6th form student ($ST) rushed into the room - on a friday afternoon of all the times he could - and kick starts just a shower of awfulness. $ST Mr TR, I have lost my work! I've dealt with this particular student before. He's a decent kid and hasn't been one to BS me in the years I've been there so I forego the usual crap and we jump straight into it. I log on to the backup server and start the process of running a restore. Except, there's no backups for the last five weeks. None. Zilch. Zero. S***! Right, there has to be an explanation, lets have a look. **$TR "**The server's being a pain in the ass. scribble down the file name and I'll recover it for you once I've slapped it about a bit." It doesn't take long for me to realise that the problem is that the backups have been failing, daily, for a full month! S***! S***! Not just the student data server. Finacial data, database servers that include information we are legally obliged to hold onto for years and ensure we have backups of. We are in a right mess. I am in a right mess. S***! S***! S***! I could honestly see my job going out the window at this point. I dig. I read through logs of jobs that I should have had mailed to me and discover that the repository has completely run out of space. I dig a little deeper and it appears that it has been doing this for months and $ITM has "cured" it by either manually deleting old backups or extending the datastore it's using as a repository. Oh crap! The datastore... My worst fears come to life when i see that no only has the volume for the Backup Repo filled to capacity. every other volume is throwing alerts for low space and everything is over-provisioned. S***! S***! S***! S***! Surely I should have gotten emails about this, too? Now I'm in a rabbit-hole of errors I need to find and I'm yet to recover this poor kid's work. At this point, it's time to fess up and go see someone and tell them about the prolem(s) before someone else does.
Meetings with financial directors and head teachers and department leads soon follow and I have to explain over and over again the issues and stand there and say I have no idea why I didn't know they were happening. The first of my three prepared envelopes doesn't feel like an appropriate excuse for a f*** up this monumental. The thing about this story is, there was no magical fix that brought everything back to life. It was a near impossible situation. I sat with the student and recovered what I could from the locally stored files on the last computer he was logged in to (thank god the profile wipe on logout wasn't working) and instead of losing weeks of work he only lost a couple of hours. I had a little bit of goodwill left over from $ITM retiring and me keeping things going and I managed to keep my job. But not without being hauled up in front of a committee of governors and having to explain why i wanted to spend tens of thousands on a new backup system with a separate repository. As well as a few grand on increasing RAM on all of our hyper-visors that were running an entire academy's worth of servers on barely enough memory to keep the place functioning. It was an absolute nightmare few weeks that ended with a checklist being created for whoever was in first everymorning to go through to make surethings were still standing. and to warn us of any incoming issues. And the problem with the emails? Wanna take a guess? $ITM can't spell. [servicedesk@thisschool.com](mailto:servicedesk@thisschool.com) became [servicedeks@thisschool.com](mailto:servicedeks@thisschool.com) and was copy-pasted into everything and never checked. TL:DR Don't rely on other people to do the work for you. Ever. [link] [comments]  |
| Mercury's position should not effect your backup procedures Posted: 27 Jul 2018 09:25 AM PDT After seeing a post about mercury starting to go into retrograde it reminded me of a story. Some years back I used to work at a company that supported a few dozen small/medium law firms. One of the daily tasks we had was to verify backups ran just fine the night before and make sure the next tape was in for that evening's run. One particular small client has this strange woman working there as a paralegal that was in charge of swapping the tape. She was one of those people that was into all that new age nonsense, crystals, astrology and the whole nine yards. When I first started working there I kept chasing and chasing her to put in tapes with no response. Finally I voiced my concerns to a coworker she told me that the woman only put in the tapes when mercury was in retrograde because it effects technology. WTF? So as you can imagine that eventually was a major issue for them. They lost a bunch of data and the last tape they had to go back to was months old. I think they had paper copies of a lot of the documents but every time they needed to do a quick edit on the document they were sitting there hand typing the document over so they can revise it with a few quick edits. This was before good OCR software was out there so it ended up being quicker to retype than to scan, OCR and fix the messed up scanned document. [link] [comments]  |
| Trying to have a meeting by only brining external monitor where files are kept! Posted: 27 Jul 2018 06:42 AM PDT This incident took place in 2015. I was "tech support" for the US Army in a deployed environment (Afghan). Hanging out in my office one day when one of the 1LT's came up to me needing help with a question. Her setup at her desk is one laptop connected to two external monitors. She states that "she has a meeting in about 30 minutes and she needs help projecting her power-point onto the big TV's in the conference room, but her files aren't showing up on the TV!" I tell her no issue I can help with that even though hooking up a HDMI to a laptop is pretty much something a child can do. I walk over to the conference room with my buddy (he just wants to get out the office). I'm looking at the setup and notice she has the HDMI going from the TV and its connected to one of her external monitors which is plugged in and powered on. So I'm looking around for her laptop assuming its somewhere in sight. After a few minutes I notice the laptop isn't there and ask where is it? She then states "No I don't need the laptop my power-point along with all my other files were saved onto the external monitor" My co-worker immediately understood what she was trying to say and busted out laughing. I was still puzzled on what she was trying to tell me. I asked what did she mean? She said "I always dragged my files from the desktop to this monitor so all my files are on this monitor, so I brought it to hook it up to the TV!" I then realized why he was laughing so hard, which btw he was still laughing extremely hard. I just stood there in shock being that this LT always like to mentioned she was a West Point grad. Instead of wasting my time explaining it I just went to her office, grabbed the laptop and did what needed to be done. She still didn't see why I needed to laptop when the files were on the monitor. After it was done I walked with my buddy back to the office and we laughed for the rest of the day. These are the people leading todays troops folks. [link] [comments]  |
| Posted: 27 Jul 2018 02:36 PM PDT This story happened about a year ago when I did level 1 tech support at a software company. I got a call on the general support line and started a screen share with the customer, we'll call him $Kayak. $Kayak, as he's pulling up some files on his computer: "Sorry if I'm slow to respond, my sister's kayaking today and she's keeping me updated" $Me: "Oh kayaking, how fun! I wish I could go kayaking" $Kayak: "Oh, no...I said she was in a car accident...she's keeping me updated." I proceeded to apologize profusely, and thankfully he said it was a minor accident. My coworker overheard the whole thing and started laughing profusely as I turned bright red. TIL 'kayaking' sounds like 'car accident'. [link] [comments]  |
| You know what they say about eyesight and computers. Posted: 27 Jul 2018 05:48 PM PDT I'll preface this by saying my family and extended family's eyesight is bad. Ranging from literally all my uncles and aunts having reading glasses, to two cousins with colorblindness, to another who already has macular degeneration before they're 24. So I'm clearing up tickets that got lost in the tsunami-of-tickets-from-summer-intern-hires - one just needs two monitors at his desk. I bring two and install them. The user says the right monitor is tinted blue. I'm perplexed. It doesn't seem tinted to me. I spend about thirty minutes with him jiggling with the monitor settings - hot, cold, contrast, etc, but the user still sees blue - and he gets his manager to back it up. Now, I'm going to be a rude noob POS here, but with this company, we have an issue with everyone trying to get custom monitors, either from IT, or bought from [insert retailer here] using their division's budget - and I can't blame them, because who wants a sad 22-inch [RHYMES WITH PEMOVO] when you can have a 32 144hz [RHYMES WITH THE SPANISH PRONUNCIATION OF 'JESUS'] monstrosity, especially if you work with editing manuals or design and work in a wing where half the cubicles have them? I, being a noob, assumed they wanted one of the spare fancy monitors in lieu of the two monitors. (This wasn't too long after I got caught between an exec and her boss nearly get into a shouting match over buying the fancy monitors, while I was trying to install said monitors.) Yes, I'm a dolt for thinking this, because he didn't say anything about wanting the new fancy monitors. I'm weirded out and tell my mentor, who goes with me. She reports it's blue, too, and gives me a look. (Note: we both wear glasses.) We replace it with one that was significantly less blue. I found out why the monitor looked blue and had crappy resolution, as well - one of the monitor brands we use has a new version of one of their models that looks exactly the same as the old one until you plug it in, and it has things like "actual HD support from Apple", "doesn't look like hot garbage if you set it to a higher resolution", and 'doesn't have a weird blue wash to everything'. And, of course, I've got a hot date with the ophthalmologist and the opticians. [link] [comments]  |
| The Cabinet of Horros - Part II: The Cleaners Posted: 27 Jul 2018 07:08 AM PDT Following my adventures with the old server this will be the second part of my story fixing the unfixable. Everything written happened a few years ago and I have left behind the workplace for (somehow related) other reasons. The first part can be found here. A week goes by. The summer heat stresses the poor little server in its smeltering hot room even more but it is still holding up. The sensors say 70°C and I get a little nervous.
And I was on my way, knowing I won't be thanked for any work I would do, but saving the company would be nice. First thing I established a fallback for the AD, file server, printer, everything with our head office in another country. With our 100mbit connection this would not be ideal but it would keep us working (besides sending a printing job a thousand kilometers to another country and back to the printer next to you is grotesque). This took a great deal of discussion if this was really necessary (yes) safe (it's your VPN, tell me) or efficient (hell no) but in the end I had a fall-back if (or when) the server goes belly-up. Next I gathered the two unlucky interns and started cleaning up the storage cabinet. That was honestly a bit entertaining. The office was relatively new and they must have just crammed all the stuff in there that nobody could decide to throw away. Empty printer cartidges worth 5 years of heavy printing (and 75€ in recycling - bought the interns lunch and ice cream), old tech and stuff I just recycled no questions asked and so on. Sometimes $boss came along and saw the piles of garbage.
Of course I can and I will. Nobody needs 47 spare VGA cables (yes I counted and the whole office was rocking HDMI or DP since I started taking over). But the best part were the documents. The whole company was active in a heavily regulated field. By that I mean third party audits, really high market entrance thresholds and documentation for every little thing. That documentation was in this small room. With lots of paper. And printer toner. And a small in size but considerable fire hazard. So as long as this little dragon slept in there I kept all of that stuff somewhere else.
The whole ordeal felt like emptying the Tardis. Heaps of stuff the same size as the room were moved. Finally we reached the methaphorical bottom and another surprise awaited us. Customer prototypes (classified of course) and a complete CAD workstation worth a couple thousand just sat there and nobody seemed to miss them.
So I proceeded to dump these sometimes bulky prototypes on the desk of $boss just (to annoy him) make sure that they are taken care of. Eventually my intern moving crew got the new, very exciting task of calling the former customers and inquiring where to send their devices. You would not believe how difficult it is to get companies to take back their own property. So after lots of moving and cursing ends this chapter of my epic quest to establish some kind of IT infrastructure that deserves this name and soon™ there will be a story about how I unleashed to Kraken to shorten a discussion about remote support. And that little server was still going strong and got a little relieve when I put a huge ventilator in the doorway to the room so it would not just spontaniously combust... [link] [comments]  |
| Posted: 27 Jul 2018 08:24 AM PDT For almost 2 years I have been working in IT support for a college and a lot of the teachers are not tech-savvy but this particular incident takes the cake. Our story begins with a call. Me- M Teacher- T M: Hello <name of college>, <my name> speaking how can I help you. T: Hello this is <teachers name> in <classroom number>, the beamer in the classroom is not turning on and I'm teaching a class soon! What can I do to fix it? <I was a bit puzzled as the college doesn't own anything called a beamer and I'm not sure if that is even a real thing.> M: That is odd I have never heard of a beamer, is it owned by the college? T: Yes it was your department that set it up. M: Can you describe what the "beamer" looks like? T: <Teacher describes what sounds like one of our projectors.> M: Do you mean a projector? T: Yes that might be it. M: Would it be ok if I come and have a look. T: Yes that is fine but be quick. I hang up and create a ticket for the issue then head over to the classroom. 3 minutes later I arrive at the classroom. The teacher shows me the "beamer" and can confirm it is, in fact, a projector. After checking the projector and it's cables I find out that the plug socket for the projector is switched off. I switch the plug socket on and the projector starts to light up and work. Later on that day I spoke to some of my colleague in IT support and none of us can work out where the teacher got the name beamer from except for the fact that the projector beams an image of the computer screen onto the interactive smart board. [link] [comments]  |
| You are subscribed to email updates from Tales From Tech Support. To stop receiving these emails, you may unsubscribe now. | Email delivery powered by Google |
| Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States | |
No comments:
Post a Comment