Posts by pirxhh

BOFH: Don't threaten us with a good time – ensure it

Here on the old continent, car liability insurance is mandatory, to protect the public from you operating a potential deadly piece of machinery.

Comprehensive or theft insurance is not, as the public cares very little if you have to take the bus after causing an accident with said piece of sh^Hteel. You may decide to take the risk of losing your car, and the lower it gets in value, the less attractive comprehensive insurance becomes. But if you should injure a cyclist, the insurance company will pay (and, should you have been intoxicated, may recover their losses from you). The victim should not have to suffer from your bankruptcy.

Linux admin asked savvy scientist for IT help and the boffin blew it

A bank branch had work done to rewire one of the offices with 10base-T. The contractors used a nice diamond core driller to get down to the "IT room" in the basement.

Gravity happened: The core fell down into the basement. The contractors called it a day (hey, it's pub o'clock and now the IT guys can put in their wiring over the weekend.)

What they had not seen (maybe on purpose): The falling core had damaged a sewer line that promptly flooded the terminal concentrator underneath. The IT guys were understandably p***ed by the condition, dunnikindivers were called, and the whole branch was closed for a week for decontamination and repairs.

Yes, your network is down – you annoyed us so much we crashed it

Re: Ways to encourage payment

We had a client who was rumoured to be in financial dire straits.

We offered our services on a prepayment basis but also gave them a small discount (10% or so) for that, so everybody could save face. That was over ten years ago, and they are still a loyal customer.

Compression? What's that? And why is the network congested and the PCs frozen?

I once worked on a project for an insurance company.

You could tell it was "old economy" (when "new economy" was A Thing) by that the in-house physical mail distribution was often faster than email (the former was darn efficient while the latter suffered from some error in the bowels of Lotus Notes on mainframe, IIRC).

So when things were moderately urgent, I wrote the data on one of the individually numbered, signed-for 3.5 inch floppy disks, turned on the "I have mail to pick up" light, and within minutes the mailman collected said disk and brought it from my 20th floor office to IT on 3rd floor. When they were really urgent, I took the service elevator myself.

Re: "Shared Cloudy Thing"

Never underestimate the capacity of a truckload of tape cartridges (or HDDs)

Techie took five minutes to fix problem Adobe and Microsoft couldn't solve in two weeks

Re: Windows Update still resetting things

Clippy got resurrected/zombified as Copilot...

Quickest fix: Turning up the brightness that was turned all the way down by giving the monitor a light dusting.

The NorthStar terminals had brightness and contrast buttons the size and depth of a quarter (coin), and only part of the rim was sticking out. So one wipe --> totally dark screen.

I got "caught" by a user to diligently copy a stack of diskettes... by using a photocopier.

After she recovered from the ROTFL attack, I explained that it was the easiest way to document which serial numbers (aka licenses) went to which office location (this was in the 1990s, when fax machines were a Thing and digital cameras... weren't)

After we fix that, how about we also accidentally break something important?

Unless offshore, where not wearing gloves and walking on any kind of stairs without using the handrails are fireable offences.

Re: Carry spares

I do cyber security stuff for a living, often on offshore rigs. My kit is compact (as the space on the helicopter is limited) but pretty comprehensive. I know that I can get safety gear onboard if needed (don't get me started on each drilling company seems to require a different brand of impact gloves), but I don't rely on anything else being available as the next shop is... not exactly nearby.

Re: hmm , am I a hoarder

I do the same but have to pack a bit more diligently, as I do most in-city trips by bicycle (and longer trips by train if I can), so the kitchen sink rarely fits into my bags...

Client tells techie: You're not leaving the country until this printer is working

Re: African customs

A few years back, I was on a (non-IT) mission to visit some ports in northern African countries. On immigration, the queues were long and slow moving. When it was our turn, the guy stared at my passport and the plethora of visas therein and told me to wait. And wait. And wait...

Then my phone rang. With an apologetic smile, I took the call. The voice at the other end asked me where I was, as they were waiting to pick us up. I explained the situation, they asked to please hand the phone to the immigration officer - whose demeanor very suddenly changed. Within seconds, we had our passports stamped and were ushered through where the secretary of state for transport had been (less and less) patiently waiting for us. The drive to the ministry escorted by police on motorcycles with flashing lights felt a bit surreal.

Disruptive weather....

Once, I got stuck on a Friday afternoon in Munich. A lot of snow almost shut down the airport; I was very lucky that I was able to catch a jump seat on probably the last plane to make it to my destination that day. I was working for the IT arm of an airline at the time, meaning I had the security clearance to fly on the spare seat in the cockpit. I learned a lot about airport procedures that day, plus quite some vocabulary that my fairly prim mum would had disapproved of.

BOFH: I get locked out, but I get in again

Re: PFY's responsibilities

I had done this an *ahem* few years ago, as a terminate-and-stay-resident program for DOS.

Hooked the keyboard interrupt, and when keystrokes were close to each other (i.e., someone typing fast) introduced random key swaps.

When the victim^H^H^H^H^H^Huser took notice and typed slowly, all was normal. Speed up, and the fun began.

Hilarity ensued.

Work for you? Again? After you lied about the job and stole my stuff? No thanks

Re: The Reason So Many People Automatically Hate *All* Managers

Early in my career, I had a very good manager.

He had only a very limited knowledge of the subject matter (IT), but he was good at networking and managing his peers and the ones above him. Who also knew that he (marketing major) was not too technical, so he could always say "I'll check with my technical people and get back to you" without losing face.

His successor was way more technically adept (and a decent if not great manager), but he sometimes felt pressured to agree to something that would require a well-oiled time machine to achieve.

Twilio reminds users that Authy Desktop apps die in March – not in August

If you self-host with using the Vaultwarden back-end software, you're all set.

Re: Found a silver lining

When you self-host using Vaultwarden as your server software, you get a fully open source solution and have most of the premium feature you're likely to want, such as sharing vaults.

That's what I do for the family; I have access to my mum's utilities, insurers etc. should it become necessary but not her various fora, knitting club and the like.

New year, new bug – rivalry between devs led to a deep-code disaster

Re: The real lesson...

It also has its place in crypto, where you want to make sure that different code paths take the same time. Compilers may optimize too well in that case; you don't want minimal time but constant time (to avoid leaking information about keys).

Standards-obsessed boss ignored one, and suffered all night for his sin

It may well be - remember "Xeroxgate" about ten years ago? Most large Xerox scan/print/copy machines habitually swapped characters, due to an over-eager data compression algorithm. So you could never be entirely sure that your scans were accurate or if the scanner had changed some sixes into perfectly aligned, clear eights.

Xerox first claimed that it only happened at some settings, as explained briefly on page 16odd of the manual. Still, it later was proven that it happened at any setting - and, as the issue was a few years old, many archives of scanned data (and hard photocopies, as those were effectively scanned and printed) could not be trusted any more.

Workload written by student made millions, ran on unsupported hardware, with zero maintenance

A part I designed as a student intern made it (after full qualification, of course) into a family of commercial airliners. They'll still be putting those in when I'll retire in a few years. Gives me warm, fuzzy feelings whenever I travel on one of those birds.

Scripted shortcut caused double-click disaster of sysadmin's own making

Re: It is mandatory to have a Oh No Oh Second experience

Stercus, stercus, stercus, moriturus sum!

GNU Pterry

PEBCAK problem transformed young techie into grizzled cynical sysadmin

Re: Plausible...

My first computer had 2 *Kilo*bytes.

Designed from schematics in a Zilog Z80 data book and hand-threaded and soldered. Surprisingly, it worked. Programmed in hand-assembled machine language (Z80 was easy, very orthogonal instruction set).

As a pupil in the late seventies, I wanted a computer but had more time than money. Learned a lot from building that thing.

Tools for end users

I had numerous calls/texts from my elderly mother that her bank / pharmacy / email / WiFi were broken.

Troubleshooting was always a bit tricky...

Early May, I had an epiphany and built her a "traffic light" type thingy - a few WS2812 adressable LEDs driven by an ESP chip, built into an Ikea picture frame.

Now, my first question is "which lights are green and which are red?"

No. 1 is WiFi access works, #2 is the router responds, #3 is Google (as a handy stand-in for "the internet"), #4 is our e-mail server, and #5 is the control box for the PV panels on her roof.

Best mother's day present (for me :-) ever!

Criminals go full Viking on CloudNordic, wipe all servers and customer data

Re: Where are the backups?

TBH, any sort of financial liability would not help the clients much.

The provider is likely to be bankrupt from an incident like this (so the final payout will be negligible), and in any case, the legal system will take too long to save the clients' business should they rely on the data.

It's a bit like insurance against a meteorite strike: They may pay your heirs a princely sum, but you'd still be dead.

Re: "their own backups as a contingency"

Nowadays, there's quite a number of companies for which "on-site" is not a meaningful term at all. They don't own/rent any permanent premises, all employees are remote/traveling/working from home. In that case, cloud-first makes sense - but it requires assurances that the data will, in fact, be there when needed.

Some (too many?) believe in vendor assurances and contracts. Sure, you can sue the provider, but what good will it do? A massive hit like CloudNordic's will likely wipe out the provider, making any lawsuit not only too late to save your company but probably fruitless.

Re: Why is it the company's responsibility to make backups of the customer's data?

Yes.

My irreplaceable data lives on my PC, with an hourly backup (using restic, not a mounted file system) to a local TrueNAS. That in turn is backed up to my brother's server, about 400km away, again using restic over ssh.

All cloud stuff lives in Nextcloud on a rented VPS, with hourly backup to my server - so if the VPS is lost I will lose time and may lose a few hours of emails and such, but nothing essential.

My company, on the other side, has joined the cult of Azure. The best I can do is to manually back up anything I deem important from my laptop to a portable SSD, that I keep disconnected (and encrypted with Bitlocker to go).

The choice: Pay BT megabucks, or do something a bit illegal. OK, that’s no choice

Re: Similar language problem on Windows 1 0

It helps to memorize the ASCII codes of the most-needed ones (like @=64, \=92, |=124, and so on) - then you can type ALT+(ASCII code on the numeric keypad). Works everywhere (sadly, not on Linux) regardless of keyboard layout.

Security? Working servers? Who needs those when you can have a shiny floor?

Re: Clean keyboards

Great idea!

We use tiny keyboards with a touchpad, about the size of the venerable Nokia communicator, for our lab machines (mostly Intel NUC and Raspberry Pi). They are wireless (dongle stored inside). I'll remember taking one for every user support task (not my official job any more but still happens).

Re: Clean keyboards

Yes indeed.

My personal keyboard is an original IBM MF-II. The (outer) keycaps are easy to detach; they regularly go into a sock and then in the (30°C wash).

At work, we have a shared desk concept; we all got personal bluetooth keyboards and mice so it's only my own dirt I'm touching.

Cunningly camouflaged cable routed around WAN-sized hole in project budget

Re: I knew of a company

Yep, did that with WiFi cards and antennas built fom Pringles cans.

At home, I made a deal with my neighbours' son: He got Internet (drilled a hole for a Cat5 through his and my wall) in exchange for mowing my (tiny) lawn.

Airline puts international passengers on the scales pre-flight

Ask Bob Pearson (captain of the "Gimli Glider" :-)

Nah, I see them a lot. Not at really small airports like Puerto Williams, Chile (where the "building" is one container next to a gravel runway), though.

Re: Good.

A few years back, I took a flight from Perth to Learmonth, Australia. At the check-in counter, I was weighed, as was my carry-on - this was for the connecting helicopter flight to a drilling rig. Usullay it's done at the heliport, as they need to assign seats so that the aircraft is well balanced.

Seriously, boss? You want that stupid password? OK, you get that stupid password

Root can always change the enforce_for_root, set their desired password, and set it back.

Of course there *may* be an audit trail, or SE_LINUX rules may actually prevent even root from changing that file, but still...

Re: Just the one letter?

The (mostly) safe way I use, being bitten by that quite frequently:

Letters - all the letters without Umlauts, but exclude A, Q, W, Y and Z - these are in different positions on AZERTY/QWERTY/QWERTZ keyboards. If you may encounter a Turkish keyboard, also exclude I as Turkish has a un-dotted I in the space

Numbers - use the number ad for those

Special characters - use + - * / (always in the same position on the number pad)

This should still give you enough reasonably safe characters to choose from, especially when assigning initial passwords (when you can't be 100% sure what keyboard layout your victim, er, client uses.

Defunct comms link connected to nothing at a fire station – for 15 years

Re: Paper Label Fail

The Brother P-Touch ones are better; they use a thermal ribbon. The Esselte blacken the sticker itself, like old fax paper, and tend to get rather brownish over time, especially on warm surfaces (or at least used to, I've switched to Brother 20-odd years ago).

What's up with IT, Doc? Rabbit hole reveals cause of outage

Re: un-neutered male

Just stick toothpicks where cats tend to do their unwanted business. Seems to work well as they fear for the 8ntegrity of their delicate bums.

Re: Ouch

My GF at the time had a pet rabbit.

After a lost cable, we tried rubbing its successor in Tabaso - no dice.Then I found an old shower hose with a metal spiral covering. Ru Ning a ne cable through that finally discouraged the little rascal (I needed to replace the plug, obviously)

User was told three times 'Do Not Reboot This PC' – then unplugged it anyway

Re: Content

The Cretaceous-Tertiary boundary (often spelled K/T boundary) is associated with one of the most investigated mass extinction events, i.e., the end of the dinosaurs. The age of the K/T boundary is currently estimated to be about 66 million years based on absolute dating methods.

Re: Content

I still use one, built October 1992.

Re: Not Worth an On-Call Story, But ...

LOTO tags are designed as they are for a reason.

Warning - do not operate (or unplug, in this case)

Locked on ...

This tag may be removed by ... only

The reason is, of course, "I thought it didn't apply to me"

Re: Content

I one had to explain by phone what a slash is...

Early in my career (just after the C-T boundary), I was in need of telling a user in some other part of Europe to type //HC into his terminal and hit Enter. After 15 frustrating minutes of trying to explain this arcane ritual, I sent them a fax.

Cleaner ignored 'do not use tap' sign, destroyed phone systems ... and the entire building

Re: Concrete dust = Kryptonite

I'm a bit OCD on backups - all really important stuff lives in Nextcloud on a VPS somewhere, synced to my (or the relevant family members') PC/phone. The Nextcloud is backed up hourly to my home server and my brother's home server, some 400km away.

Regular stuff lives on my PC, with an hourly backup to my home server, which is in turn backed up to my brother's and vice versa. So everything exists in at least 3 copies in at least two locations. (The backup is never mounted as a drive, so your run-of-the-mill ransomware would not too easily get to it, and all files are versioned.)

Maybe overkill, but drives are cheap.

Server broke because it was invisibly designed to break

Tax men

Years ago, I ate at the tax office's canteen about once a year. It was incredibly bad - a basement room, greasy unappetizing food staff calling Methuselah young man... the works.

It was so bad it had to be intentional, lest the taxmen be pleasantly sated and forgiving after lunch. Can't have that - lads need to keep their edge by nurturing low-grade ulcers, apparently.

Two signs in the comms cabinet said 'Do not unplug'. Guess what happened

Re: Don't forget mischief

I'd suggest lightly dusting the hard-to-press swich with a suitable dye. The culprit will be known and can be publicly crucified, defenestrated or otherwise disciplined.

Re: Not only two signs...

I like thst it is marketed for dementia care. Applies to a not too smsll portion of Manglement (and the workforce overall, really), as far as vital circuits are involved...

Boss broke servers with a careless bit of keyboarding, leaving techies to sort it out late on a Sunday

Re: The cycle is complete.

Usually, I'd just disconnect the switch from the mainbord. I haven't had a genuine need for a reset button on a PC in about this century.

For ESPs, Arduinos and their ilk sure, but not on PCs.

Re: "an on-prem email server"

Is that when the instant ionization of anyone using that non-word fails?

To make this computer work, users had to press a button. Why didn't it work? Guess

Re: Bad design

My Dell laptop had the paint coming off the left shift key, resulting in a fairly bright keyboard backlight "spotlight".

I should have just taped it over, but warranty! So I had Dell fix it. Big mistake.

First time the guy came to exchange the keyboard. Afterwards, it felt a bit wobbly, but okay, maybe that's expected. Only that the thing bluescreened several times during the following week. Crash dumps were inconclusive. Once it even complained abot the hard sisk (M.2 SSD) gone missing and refused to boot, but recovered after a bit of prodding.

Back to IT it went, and I got a loaner. Their tests did not show anything wrong.

Dell came again, a scant two weeks belatedly, and fixed it again.

Turned out the keyboard was very loose, and the case screws were loose too - except the one that was missing altogether. Our IT guy tightened everything, and it has been stable for two weeks now.

My theory is that the technician, and I'm using that term loosely, did not tighten the screws sufficiently on the first visit, causing any slight warping of the case to slightly disconnect the SSD.

Re: If I have to look in the manual (absolute last resort of course) it's a really bad design!

I used to have a Nokia VT220 compatible terminal next to my bed. Nice for late-night/early-morning system maintenance and email.

Nowadays, my DIY alarm clock (also ESP8266 based) can be controlled via Alexa and FHEM. It will slowly dim up the LED lighting, starting 20 minutes before I'm supposed to get up.

Needlessly complext technology? You bet.