The server is underwater Tech Support |
- The server is underwater
- Storage guy meet old server a tale of IT TomFoolery
- Hit my breaking point today
- Works better when plugged in
- ...and then we waited to go to prison
- It Blew Up
- Image Resolution
| Posted: 01 Mar 2022 03:47 AM PST So I used to work for an MSP that specialized in school tech support & I worked on the help desk. I answer the phone as usual & the following conversation takes place. IT = me U = Customer IT- hello this is (blank) how can I help? U- We have no internet & the server is underwater. IT- sorry sir, could you repeat that, the line isn't great. U- the server is underwater IT- ah okay so the server is suffering from water damage? What exactly happened? How extensive is the damage? U- the area flooded & so has the school. The server is partially submerged. Can you help at all? IT- right okay, theirs not much I can do from my end I'm afraid. I'll raise this to the area manager & they'll be in contact to arrange a visit. The server is powered of correct? U- no, it's still running & has power. IT- okay sir, I suggest if possible you shut the power to it if you can do so safely. Not much to add but it's one of the calls I had that caught me so off guard [link] [comments] |
| Storage guy meet old server a tale of IT TomFoolery Posted: 28 Feb 2022 06:38 PM PST long time reader first time poster here. I have 25 years experience (yea I'm old) as a Dba. I'm now what most would consider a tier 3 support as in call me when all hope is lost. I currently work for a large global shipping company. As happens the powers that be have decided it would be cheaper to have support done on a global level rather than a local level. So while our servers are here in North America SAN support it half way around the world. Not really all that great when you know you have to actually have to physically be on the server. Now you all are thinking, what your not using virtual servers? In this case no. See the servers in question were originally made back in about 1999/2000. The software on them that our database uses was released the year I had my second kid. Said second kid is turning 21 in 6 months. To say this stuff is old is an understatement. We do not breath on this system, do not look at it and we do not updated it or reboot it. In other words DO NOT TOUCH IT. Today I start to get messages from our home grown monitoring (there is no monitoring system left that can monitor this old stuff) that the databases on that server are down. I was hopeful we were decomming them and no told me. Alas, nope they are down because they can no longer write to their files. Uhm that's weird. Contact the only admin that can work on these old things and he is like either it's an HBA issue or someone has yanked the disks. Oh joy my day is about get fun. About an hour later ye old admin tells me the following: The SAN team in country half around the globe got an error on one of the drives so they got jr SAN guy here in North America to go in and replace the storage cables on the old as heck server. Jr pulled the cables from the live production server because SAN team told him there would be no outage. There is even a sign on the server that says DO NOT TOUCH. SAN team on other side of globe told him to ignore that sign and replace the cables. Yep Jr pulled storage cables from a prod server during the day without letting the admins know or doing any paperwork. The old server is now requesting it's final vacation and refuses to come up. Said old server does not have an ILO either so any and all reboots are blind unless the admin drives 2 hours to the DC to attach a crash cart to it. For you youngins, attach a keyboard and monitor directly to the server. We have no back up plan if this server has decided to take its final nap. I feel no remorse for the people using this app on this sever as they have been told for the better part of 10 years to get off it and they refuse because upgrading will be too expensive. We cannot even eBay for parts because this thing is so old. Moral of the story here if it says do not touch DO NOT TOUCH IT and contact someone who isn't half way around the world to see if you should touch it. [link] [comments] |
| Posted: 28 Feb 2022 06:15 AM PST Quick background... I work for a very large computer company that ships servers worldwide. I am in charge of RMA/RCA and will try to help customers online to avoid returns. Last month, customer orders a series of systems with an older OS which is close to EOL even after being spoken to about this at length. Once systems are installed the customer decides to update BIOS to a non approved version and breaks the logging process. Fan sensors no longer reporting. Customer calls in ticket on Friday. I spent all weekend creating a patch to fix the issue. This morning, customer refuses to install the patch or cold boot the systems and wants to replace all servers at my companies cost. It's only 9:00 am and I am already done for the day. [link] [comments] |
| Posted: 28 Feb 2022 06:38 PM PST Happen couple months ago, I went to a client site to fix issue with wireless network and realize that the printer in one of the department was unplugged from the network. I fixed it, one of the staff members who i shall call Jane thanked me saying it was down for a while and now she can finally print. Few weeks later, get an email from Jane saying the department can't print anymore. I checked the printer, realized there was an ip conflict and fixed her issue. Few days later Jane's departmemt manager who I shall call Karen sends an email saying that staff cant print and the printer always having issue and they want a new printer. I responded letting her know the previous issue was that network cable was unplugged and the second time was an ip conflict and nothing is wrong with printer but I will look into it. Contacted the staff to see what the issue was... Printer wasnt installed on the laptop. Installed, now they can print One month later Jane email they can't print because it low on toner. My company sent the toner to them. Month later Jane sends email again they can't print. Then Karen emails saying this always happening with that print, that it is not working and demands it be replaced with a new one and says how my company only patching a broken printer that needs to be replaced. I again explained that nothing wrong with the printer and again point out why they couldnt print with the past issues. I again look into it. Apparently printer out of ink and no one installed the new toner that we sent them a month ago. I had one of their staff do it an printer was operational again. Few weeks pass by Jane sends emails to Karen telling her they cant print. Karen send email demanding printer be replace and that it constantly breaking down. I pulled up my previous email chain with the past issue and resolution, forwarded that to her letting her know nothing is wrong with the printer. I called to speak with Jane but was informed no one was in at the moment in that department. Decided i'll try back later. Another manager at Karen company who i shall call Karena email my boss agreeing with Karen demanding printer be replaced and this was an ongoing issue. I replied to the email letting Karena and my boss know that i tried reaching Jane or anyone in the department but no one was available. Also included my previous email chain showing nothing was ever wrong with the printer At some point during the day my company sent a copier tech to check the printer. Tech sent his update to one of our sales rep who then emailed Karen and Karena letting them know problem was fixed but forgot to remove the attachment pic of copier tech ticket. I read the notes in the pic and i quote "nothing is wrong with the printer, it works fine but works even better when plugged into the power outlet" [link] [comments] |
| ...and then we waited to go to prison Posted: 28 Feb 2022 09:31 AM PST
Our shop had a sister store (the 'parent' store) in a nearby city. We rarely ever saw staff from the other store, and more rarely saw the owner. In the one year I worked there I saw the owner twice. That said, we did a lot of business for a semi-rural county and we sold lots of whitebox PCs to local businesses. We also sold lots of software. Lots and lots of software. One day in late spring several cases of software showed up along with the proof for next week's newspaper ad. 200 copies of Office 97 Professional and according to the ad it's on sale starting the next week for....$219?!? HOLY GUACAMOLE!!!! I bought one. The manager bought one. All the employees bought one. Our friends and family....it was a ClippyTM lovefest! The ad hit on Tuesday (IIRC) and we sold every copy by Friday. We figured it was a one shot deal but we called up to the mothership and asked if any more were coming and to our amazement more were already enroute. We got another 200+ copies, the sale ran for one more week, and at the end of that week they went to regular price of... $249?!? 👀 We were told the boss got a really good deal. That lasted for months. Our 'sale' price became the normal price. We had a bank call up and order 150 copies. Another local business ordered 200. For the first two or three months we sold out almost every week then it slowed down to a steady 10-20 copies a week. With that volume of sales you expect the occasional call of 'hey I'm trying to reload my Office and it isn't reading the discs.' For the average single copy owner we would tell them to swing by and we'd give them another copy we burned if they brought us their Office 97 box with the license key. But then one day we got this call.
Some days pass.
Tech 1 relays the story to me. So I ask the other two techs and lo and behold we've all had quite a few of these calls over the last few weeks. I called the parent store and talk to the purchasing manager. He tells me they just found out there was a big run of bad discs pressed and they'll send us several cases of new Office discs to replace the bad one. Okay.... Two days later the company van shows up driven by the parent manager and unloads SIX (6) LARGE cases of all new retail Office 97. I think there were 144 units per case.
As businesses start bringing in their copies (or more usually having us come one site and swap them out), Junior Tech decides to pop open one of the new boxes for some reason.
Shows me Office 97 CD with a visibly off center laber and you can see the ink dots on the label from the printing process.
Epilogue So after <local manager> had a few calls with <parent manager> and the owner told us to not worry about it, some decisions were made by some people at the parent store to collect lots of documentation and make a couple of other phone calls. Apparently the owner had connected with some guy who could get software at a really good price and thought this was a great way to make a fast buck. I heard a couple years later via the rumor mill the owner was somewhere in Central or South America. I believe there were at least two warrants out for him. [link] [comments] |
| Posted: 01 Mar 2022 07:55 AM PST This happened a few years ago, when I was still help desk at my client, a government contract. I had the usual sorts of calls from our users (99% internal, with the occasional call on our outside line from others who used apps that the client supported). One day, I get a call from a user who used the kind of terminology you sort of expect from the non-tech savvy. $User: My computer blew up. Naturally, I assumed that something had crashed or locked up or whatever, so as I prepare to go through the list of questions, they continued-- $User: I heard a "poof" and smelt smoke, and now it's not working. $Me: Oh! So it literally blew up. We shared a laugh, because they understood what my immediate assumption was. I got their info, sent off the ticket, and then had a fun story to share with friends and coworkers. It's been a few years, so I don't recall the specifics, but I believe it was down to a hard drive shorting or otherwise failing in a really bad manner. [link] [comments] |
| Posted: 27 Feb 2022 05:50 PM PST I work for msp level 1, 2, 3 because they expect us to do it all and recently I heard my boss tell someone, that I handle level 4 issues. Not sure what 4 does but I don't get pay enough for this, but seriously, can someone tell me what's L4? Anyways, one day user calls, one of the newer tech answers. Few minutes into the call she gets fustrated and ask if I can intervine. I take the call and ask whats the problem. Caller explains she got a picture that she is trying to print full page but it's not taking up the full page I remotely connect to pc, have her show me the picture she wants to print. I open word, blow it up full page and send it to print. Caller then ask if she have to edit it everytime to print full page and that she use to just open the picture, was able to select different sizes like wallet or full page and it would do it automatically but the application didn't do it anymore So i double click pic and windows photo app opens, options are there to select different sizes. I try wallet works fine, i select full page image doesnt expand to full page. I check image properties, resolution something like 267x577. I explain to here becauee resolution is so small when she selects full page it will only blow it up to max size of the actual pic resolution. Caller not understanding starts ranting she was able to do it before. I again explained it to her, i even went on google, downloaded a high res image something like 3000x2000 and demonstrated it to her that it does do full page for high res image but not for low res and she will have to edit it and blow up low res images. She again argues she was able to do it and starts putting blame on us saying after we did whatever to there systems it doesnt work anymore. Confused for a moment about what she talking about, then remembered that her company had win7 pc which were recently replaced for win10. So i explained to her computers were replaced she using win10 now and in win7, photo app may have done that and blown it full page also making it pixellated but now win10 only blows image up to max res when full page selected. She then insist she wants to go back on win 7. I told her no she can't She then repeats her question why picture doesnt show full page. I again explain the resolution issue and demonstrated again with another high res i again downloaded that it works but for low res, she will have to manually blow up to fit full page. She continues to go on about "this is the way" she been doing it before and don't see why she have to manually edit anything and blamed us for changing the system. I put phone on mute and start banging it on the desk At this point the new tech is laughing at me. I then unmute and start speaking in a loud no BS tone to the caller. Asked her to show me were she got the image from. She pulls up an email with several picture attachment all with low res downloaded from internet. Simply told her all will have the same issue and that whenever she downloading pic from Internet in the future, download high res image and the way win10 photo app handles the pic is by design and that there is nothing else i can do for her and to have a nice day Moment she said ok, i ended the call [link] [comments] |
| You are subscribed to email updates from Tales From Tech Support. To stop receiving these emails, you may unsubscribe now. | Email delivery powered by Google |
| Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States | |
No comments:
Post a Comment