Hope your holiday was horrid: You botched the last thing you did before leaving

Monday is upon us once again and The Register hopes that when you arrive at your desk, all is well. We offer that sentiment because we use the first day of the working week to bring you a fresh instalment of "Who, Me?" – the reader-contributed column in which you confess to making mistakes, and explain how you survived them. …

[Michael H.F. Wilkinson]

"That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

You never know when that iomega zip-drive complete with several zip disks, or that 8" 128 kB floppy disk labelled CP/M 2.0 will come in handy

[Sam not the Viking]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

In our loft, I still have the original packing-boxes for equipment that is long gone. Some of them contain a single lead which was probably vital to function......

[Anonymous Coward]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

Same here - boxes of old software packages returned (c/w manuals) to their original packaging, boxes of assorted leads and adapters, old drives (including at least one Zip), old computers (including a ZX81 and a maxed out Acorn Electron), several old PDAs (including a Psion II, Psion 3a, and a Palm Tungsten), several laptops and a couple of iMacs. A year or so ago I emailed several computer museums around the UK, offering it to them for free. One asked for one of the Electron modules but I didn't want to split it up. Another said they were interested in taking the lot but, when I tried to contact them to arrange delivery (I even offered to hire a van and deliver it on an 800 mile round trip), they didn't reply. It's all still in my attic gathering dust. I expect it will all go to the local tip when I pop my clogs (or my wife decides we're going to move house and downsize).

I've even got a trunk containing all the UK printed versions of Elektor magazine - I started subscribing from issue 1 and only stopped when they stopped paper issues. Again, more for the recycling centre one day.

[MiguelC]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

I still have a large box full of 8 bits ISA cards, amongst them a Soundblaster 2.0, an analog TV receiver and a firewire adapter - recollection of the function of many others is, if not lost, at least kept in a box for storage :)

[Icon: Windows 2.0 user]

[mike.dee]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

Yellow Ethernet thin cable works perfectly to make cables for your hi-fi and brag about you have a special cable too. If you have time you could also braid three cat3 utp cables and make another esoteric hi fi cable for loudspeakers at a very low cost.

[Antron Argaiv]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

Thick yellow cable is essentially RG-214 and works great for low loss HF feedlines. Have some feeding my 40m inverted vee as I type. Bonus: crimp on N connectors.

[chivo243]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

When Apple brings back Firewire, I'm set!

[Frank Bitterlich]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

So, you think I shouldn't throw out my 1st gen iPod yet?

[What? Me worry?]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

I still use mine. It connects through a 400 to 800 Firewire cable to a Firewire to Thunderbolt dongle to Thunderbolt to USB-C dongle to a new M5 MacBook Air. Music was ripped to external drive ages ago and still functional in  Music app. Haven't bothered getting other apps or tools as it still works for what I want. :)

[ChoHag]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

Not until the Mark 1 Human Ear is obsolete and we have to upgrade to the latest version.

Digiear?

It won't have a 3.5mm jack, I can tell you that much.

(Note: the Mark 1 Human Ear doesn't have any kind of jack either, no matter how much the socket looks like 5mm; don't even try)

[CrazyOldCatMan]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

So, you think I shouldn't throw out my 1st gen iPod yet?

My previousx2 car (Honda FR-V) had a specific iPad connector for the stereo. So, if you buy one of those, possibly.

(I've still got one of the 2nd gen iPods, a Sony Minidisc recorder (with discs) plus various cheapie MP3 players. And a Sony CD Discman)

[GlenP]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

I have to confess I did have a couple of clearouts recently so my tech stockpile is much reduced. The ZIP drive went, as did the ZX81, but I've still got a USB floppy drive just in case!

[CrazyOldCatMan]

Re: "That Box Full Of Old Tech You Should Probably Have Thrown Out But Kept Just In Case"

Or even that Apple Newton that I bought, believing the hype, back in the halcyon days of the mid 1990s.

And the Nokia Communicator? There *must* be a use still! (It was actually useful - connected by Bluetooth to my Sony phone, it gave me (very slow) internet access away from home.)

But even I have realised that all the Sun Sparc 1 pizza boxes with attending cruft (SCSI-1 drives etc etc) were not really useful any more and it got recycled in accordance with the WEEE rules (honestly! I assume that's what our local tip does with the pile of old computers that's always in one of the open shipping containers)

[Giles C]

Not me but the same….

Came in one morning and one segment of the network would not connect. This was about 15 years ago

Now the windows admins had set a profile which basically locked all machines if they couldn’t see a domain controller so this one was going to be hard.

It took a good while searching but eventually I found someone had attached a server running whatever the Citrix thin client manager was to the network and it was happily giving out ip addresses with itself as the default router whilst getting its own gateway as itself.

Due to the locked down machines it was difficult to track what the problem was in the first place.

Once I had found the machine and removed it everything went back to normal.

This is when we started moaning about locked down machines. Our windows admins had set the machines to first run through a group policy disabling all “admin” type functions, and then for people such as myself who needed to get to lower level commands to do our job, it would then apply another policy to remove all the restrictions it had just applied - this took about 10 minutes from turning a laptop on to being able to do anything……. They also foisted Microsoft direct access as a vpn client which could take up to 20 minutes to log you in great if you get called out on a urgent problem.

[Jou (Mxyzptlk)]

Re: Not me but the same….

Now with 16 times the amount of RAM, 16 times the CPU power, 16 times faster NVME storage, and still not faster.

[Caver_Dave]

Re: Not me but the same….

I have to put in a request to IT whenever I want to install a program or change a setting on my new laptop. (The one it was replacing was OK, but out of warrantee.)

A pain in the ass, becomes much larger when you realise I am in the UK and IT are West coast US.

Took over a week of sending multiple requests per day, before I had the new laptop configured to do nearly as much as the old one (there were some things that they categorically would not let me do as they have just one set of rules that cover the intern in accounts and us expert developers! e.g. for network testing I have to compile on the laptop and copy everything I need to my test network on a USB stick, rather than using a segregated network device. It means that I can no longer single step through code that I am developing - my managers just tell me to bill any inefficiency to IT - but it is a constant irritation and waste of time.)

[CrazyOldCatMan]

Re: Not me but the same….

Came in one morning and one segment of the network would not connect. This was about 15 years ago

My first techie job, once I'd given up any pretensions of being a programmer was DOS/Windows/Token Ring network support.

I managed to disconnect an entiure floor. The main closet on the ground floor had inter-floor connectors and I tried to 'optimise' the setup. And, in doing so, collapsed the ring to just within the network closet.

Came out into the office to hear about 100 IBM PS/2s sat there, clicking away as they vainly waited for a token that was never going to arrive.

Straight back into the closet, put it back to exactly how it was, came out, mumbled something like "must have tripped over a cable".

[Giles C]

Re: Not me but the same….

When working on a token ring network, we had reports of a slow connection, I found the cable and started pulling it from the cabinet to find the other end (this is the type 1 stuff). Well we pulled it and pulled it and got back to the doorway to reception (about 50 metres) but at this point the cable was still connected at both ends in the cabinet. Why some idiot had got a 100m patch cable and plugged it between two ports about a metre apart I don’t know but the segment speeded up after that. (It had been running a dumb terminal and was swapped to a pc)

[Yet Another Anonymous coward]

Been there done that, but it wasn't me

Poor WiFi in the breakroom so I gave the pimpliest of PFYs a spare home WiFi router I had in the aforementioned pile and told them to try and set it up as a WiFi extender/access point.

Naturally they managed to enable DHCP

Since this was a startup running on wet string and no support it took a while to work out what and why was happening

[Tim99]

Re: Been there done that, but it wasn't me

When I did this stuff, I had a simple rule. If adding a Wi-Fi Access point with a device that had a router option, always, always, set it up in bridge mode. If you still think you want to set it up in any other mode - stop - go for a beverage of your choice; and when you come back, it is hoped that your brain is now working, then set it up in bridge mode.

[Korev]

Re: Been there done that, but it wasn't me

Too many of those "beverage of choice" and enabling the DHCP server would seem like a good idea!

[Korev]

Re: Been there done that, but it wasn't me

So it was you!

I had to deal with the fallout from someone doing this a few years ago. We never found the culprit...

[Harry WWC]

dhcp woes

I was a teacher (at the time 'Acting Head Teacher') in a TAFE (Tech College) here in Oz.

Early one afternoon I have one of the guys who supports the college network infrastructure come to me and ask if I knew of a rogue dhcp server in our area. I did not.

We did eventually track it to one of the computer labs where the students had gone to lunch, and had been building virtual networks (a couple of servers and a client or two) using vmware-workstation and were (if they had followed instructions) meant to have all those on their internal virtual network.

One student had a habit of skipping steps and in this instance had set their windows ad / dhcp server virtual machine to attach to the real network (bridged), and started merrily handing out dhcp addresses to all and sundry. we tracked the machine down via the switch it was connected to, and as we couldn't log in, I decided to press and hold the power button.

one of the other teachers said "you can't do that, he'll lose any unsaved work!" I responded that I didn't care and that this was affecting machines across the campus. I left a note for him to come and see me after he returned from lunch.

we had a quiet chat about reading and following instructions - and not putting rogue dhcp servers on the network.

[Doctor Syntax]

Re: dhcp woes

"he'll lose any unsaved work!"

That made it an excellent learning opportunity.

[ttlanhil]

Re: dhcp woes

"you can't do that, he'll lose any unsaved work!"

Minor concern in the circumstances - but you could also just unplug the network cable (I assume you can access it)

But also... If you lock your machine and go to lunch with unsaved work?

Another learning opportunity - those of us who've worked on less reliable systems have learned those lessons...

[Jou (Mxyzptlk)]

Re: dhcp woes

Why not unplug network cable(s)? Power down the switch, if dedicated for that room? And why is the computer lab not really separate? OK the last one is a typical today thinking, 20+ years back less so. But the first two would have been a valid solution here.

[Wally Dug]

Holiday Wind Down

For a start, never make any changes, deletions or additions on a Friday - unless you are working the weekend as normal - and if you are on holiday next week, then holiday wind down basically starts on Monday of that week and do not make any changes unless you really need to. That way, you can be assured that your systems are running satisfactorily (to the best of your knowledge) so that you can relax and be blameless when you are away. Always remember Murphy's Law and always CYA.

[Doctor Syntax]

Re: Holiday Wind Down

"if you are on holiday next week, then holiday wind down basically starts on Monday of that week "

That means that by Friday afternoon there's nothing much to do which in turn means that it's a good time to play with random bits of discarded kit.

[ttlanhil]

Re: Holiday Wind Down

Yes, but not attached to the network!

Or, at least, not on the main network

If you have a separate network for testing old kit that might do unexpected things - sure

[Excused Boots]

Re: Holiday Wind Down

But Shirley you are aware of the old mantra;

every organisation has a network for testing old kit or random shit......

and some even have a separate production network as well!

[The Oncoming Scorn]

Re: Holiday Wind Down.

"That means that by Friday afternoon there's nothing much to do which in turn means that it's a good time to call POETS Day."

FTFY - No Charge, I'm buggering off back to the UK on Saturday for a week, so I will be looking for a discreet slip out the door on Friday afternoon.

[Giles C]

Re: Holiday Wind Down

Hmm nice In theory, but I had last week off (as it is a bank holiday in the uk I am back tomorrow)

Friday morning spent isolating a problem with a line card in a chassis

Then debugging a dhcp problem, testing new wireless configuration and Friday afternoon spent writing a handover email which came to 2500 words due to the number of projects I am working on before getting hit with a load of questions about 4:30.

Didn’t help that I got hit with a norovirus of similar on Monday / Tuesday which screwed up my schedules somewhat….

[KittenHuffer]

Re: Holiday Wind Down

I once started a job where the interviewer (the Mangler I would be working for) was very loose with the truth concerning flexible working hours. This led to me finding something else and handing my notice in less than 5 weeks later.

I was still made to work my entire notice period, and they still had me compiling code changes half an hour before I left on my final day!!!!

[An_Old_Dog]

Not Over a Vacation, But ...

As a tech at a large organisation, I was sent out to install an Ethernet card into a PC. This was something I'd done there many times.

Rhe secretary waved me into the empty office with the kit.

The PC was an IBM Microchannel architecture machine, which was a new thing, as was the 3Com MCA card.

The docs were excellent, and I quickly had the card installed, and fairly-quickly configured and connected to our network.

Looking at the labels on the three floppy diskettes, I saw one was a card-testing program. Wanting to be careful and thorough, I thought, "I should run this." I found the relevant spot in 3Com's manual explaining how to use the (DOS-mode) program, and followed the directions.

The program came up, put up a nice screen display ... and then nothing happened. No completion message, no error message, nothing.

After a minute and a half, I Control-C'd it, got a DOS prompt, and went back to the manual.

I re-checked the steps, did them again, and ... nothing.

About half a minute later, the phone rang, and I answered it. It was the head of Networks calling. "What the hell are you doing?!"

"I've installed an Ethernet card and am running 3Com's test program."

"Stop doing it, and don't do it again! That card is bombing the network with malformed packets!"

"Yes, sir!"

On the up side, the card worked perfectly with our DOS-based networking software (TCP/IP and IPX/SPX, for Novell Netware and DOS or Windows 3.X).

[Strahd Ivarius]

Re: Not Over a Vacation, But ...

and then he turned the page in the manual and saw the last lines of the testing procedure:

"how long did it take for the network team to phone you and ask you to stop doing the test?"

[TimMaher]

Re: “last lines”

I thought it would be “This page intentionally left blank.”

[ttlanhil]

Re: “last lines”

Shouldn't do that on the actual page - it's not blank then!

"The previous page was intentionally left blank. The next one was an accident by big Jim."

[The Oncoming Scorn]

Re: “last lines”

The 3.5 pages of blank sheets, that at the 2/3rds down from the top of the next page with text with a great big STOP (Icon) & text stating "DO NOT PRESS ENTER AT THIS STAGE".

The consequences of this meant that "ENTER" had already been pressed out of force of habit & the server had to be reimaged from scratch.

This was undesirable at 12.30AM give or take how the deployment was proceeding in a banks server room.

[FirstTangoInParis]

Re: “last lines”

Just like microwave ready meals that say '800W for four minutes'. Then a bit later 'halfway through, stir'. Oh, already done 4 minutes, never mind.

[NetMage]

Re: Not Over a Vacation, But ...

Reminds me of the day we discovered the hard way that HP decided to add a scan the network for printers feature to their LaserJet Windows driver when the full package was installed. Since our network was 10.0.0.0/8 and flat, and their was no rate limiting on the scan, it flooded the local area completely off the network.

[Excused Boots]

Re: Not Over a Vacation, But ...

The moral of the story being;

a) Never have an HP printer on your network.

b) If you absolutely have to have an HP printer, never ever install the full spyware package, just the drivers and set the thing up by hand

[Doctor Syntax]

Re: Not Over a Vacation, But ...

c) Unless you have an HP printer that goes back to the days when the HP Way was still remembered and it connects via a JetDirect box in which case you're probably OK.

[CrazyOldCatMan]

Re: Not Over a Vacation, But ...

it connects via a JetDirect box

I've got a couple of those in the "box of random IT tat" in the garage.

Really must take the box to the dump.

[breakfast]

Re: Not Over a Vacation, But ...

Clearly when the testing program said it ran in DOS mode and you were naïve to assume it meant the command line interface rather than "Denial Of Service."

[hugo tyson]

Marketing made the setup more "friendly"

Problem is, thick people removing meaning, to make things "easier to use" and "more friendly".

The question *meant* "Should I provide DHCP service?" not "Do you use DHCP?"

The latter is ambiguous, since, as the user found out, the answer "Yes, we do use, and are already using DHCP, so don't you go interfering" is also the [YES] button.

[Bebu sa Ware]

DHCPing and ARPwatch

A beer to each.

When you have the custody of a network to which the great unwashed of tertiary education having unfettered access these two sidekicks were indispensable.

Whether it was an old wifi router with 4 or 5 lan ports brought from home or a tethered phone bridged on to the lan, a DHCP service was invariably active.

Students were normally guilty of sheer gormlessness but staff could apply considerable talent and originality to produce truly magnificent cockups.

[ttlanhil]

Re: DHCPing and ARPwatch

"Students were normally guilty of sheer gormlessness but staff could apply considerable talent and originality to produce truly magnificent cockups."

That's the benefit of experience! Kids these days don't understand just how much more creatively their seniors can mess things up

[Doctor Syntax]

Somewhere langishing on a shelf, unconnected and unpowered, is a WiFi gadget with just a single RJ45 and power as its only sockets acquired 2nd hand (no docs) to use as a bridge on the home network.

With only one network connection it wasn't exactly the sort of thing to set up as a modem/router/gateway device. Nevertheless it defaulted to being a DHCP server - after all, why not?

[Excused Boots]

Being a bit of a resident greybeard, I do seem to recall that back in the days (cough...cough...mumble...mumble decades ago), some Apple Mac computers had a habit of enabling a built-in DHCP server when connected to a network, and then hilarity ensued.

Much more recently, well two years ago, at a client of mine, which is a managed office complex, some of the admin staff started claiming that ‘the internet is broken’.

OK, I happened to be on site, checked a machine, yes it had odd IP and gateway addresses, Windows machines, so ran ifconfig /all to find the address of the rogue DHCP server, managed to find its MAC address and then interrogated the switches to find which port it was attached to.

Down in the basement, the maintenance guy had decided he needed more ports and had helpfully brought in some random home router - and yes, you can work out what had happened. Now obviously, all of the various tenants were on their own VLANs, isolated from each other and would not have been affected, but still!

I recall being ‘not too pleased’ with the maintenance guy and made my displeasure quite plain - which on reflection may not have been a good idea as he was about six foot five and built like the proverbial brick shit-house - but I was pissed off. It appeared that he had tried to connect this example of ‘shadow IT’ to a different switch port but had been foiled by my administratively disabling and all ports which are not connected. So he connected it to the port for his own PC, which luckily was untagged on only one VLAN so it didn’t affect the rest of the organisation.

Now the odd thing is, his own PC, connected directly to this foreign router, carried on working with the ‘right’ addresses, whereas a staff PC one floor up failed.

DHCP - it’s a fucking black-art isn’t it?

[David Hicklin]

> Now the odd thing is, his own PC, connected directly to this foreign router, carried on working with the ‘right’ addresses, whereas a staff PC one floor up failed

Its a while since I messed with networking but that could be that his PC had a long enough lease time to stay as it was, whilst the staff PC was just switched on and had gone looking for an IP address.

[cosmodrome]

Some of the better home routers (Fritz boxes, for example) allow you to assign permanent addresses to specific clients while keeping DHCP up for everyone else. They also allow mixing static and dynamic IPs.

[blu3b3rry]

Gotta love DHCP servers

At $~WORKPLACE they get used fairly heavily for running software development setups. Everything gets flashed with a PXE ROM and then boots from a host PC running a DHCP server loaded with whatever tweaked software binaries they need.

The original plan didn't foresee IT wanting the host PC's set up with standard corporate Windows 10 installs, and also required them added to the company domain with domain logins. As a result they were hooked to the company network to allow this to all work.

This was found to be a very convenient way to run development setups, so naturally it wasn't long before the hardware engineers found out about this nice useful tool and started doing the same. Naturally they weren't as careful/understanding of the hazards of a DHCP server.

Ever since there has been the occasional network borkage thanks to a engineer applying less than the required amount of diligence when setting one up.

[billdehaan]

Rogue DHCP server? Ban the operating system!

I worked for a multinational which was mostly Windows based, although some there was some Linux development (I was one of them), and a few remote site deployment types had Linux laptops for certain data analytics, as well.

One day, one of those site people came to home base for some training, and while there, he plugged his laptop into a network port. It turned out his laptop was running a DHCP server because his was the gateway machine when in the field. In a properly secured environment, a node running an unapproved server like that would be blocked, but this company was not like that. Once they found the offending laptop and identified it as being a Linux machine, a VP sprung into action. In classic shoot the messenger fashion, he declared that Linux was the culprit, and that the downtime caused by Linux was in the tens of thousands of dollars. Invoking his authority as VP of Procurement, he formally declared that Linux was officially banned from the company, henceforth.

For those of us working on Linux applications there at the time, we just shrugged, and told our management that we wouldn't be able to deliver, what with this new edict. It was most amusing watching horrified executives discover that a VP had ordered put a stop to three major projects a week before delivery. They tried to tell people to ignore it, but everyone was in malicious compliance mode, and no way were they refusing a direct edict written by a senior VP.

"If he's stupid enough to ban Linux without knowing what he's talking about, what's he going to do to people who disobey him?" was the thinking. Management spent almost three weeks before they were able to get Linux usage formally removed from the list of termination offences.

As for "the last thing before leaving for vacation" events, I have two. Both were a site administrator. And yes, the same guy.

The first was an upgrade of VMS for the Vax. He got the tapes just before he went on vacation and decided to upgrade it before leaving. He finished the job around 4:30pm on the Friday. He logged in, things looked good, and went away for three weeks. Although he'd restored the data disk with all the projects, he'd unfortunately, neglected to re-enable user logins. He could log in, as administrator, but no one else could. The company came in to work on Monday morning and discovered their Vax was effectively offline, and would be for the next month.

The second was two years later, where he made some upgrade to the hardware rack configuration. Naturally, you install new hardware before going away for a month, right? It was some classified overclocked imaging system that ran hot. As in really, really hot. It came with a secondary unit, basically a refrigerator. However, the refrigerator made a hell of a lot of noise, and the administrator thought the heat wasn't that much, so he turned it off. Then he locked the system room, and went on vacation.

At 4am, alarms went off, and security staff went to the system room. They couldn't get in, and the doorknob was hot. They had to call a locksmith in to drill through the door so they could get into the room. Supposedly the room temperature was 42C. One of the flimsy plastic chairs in the room was apparently warped by the heat already. I don't know if they turned off the imaging system, turned on the refrigerator, or brought in buckets of ice, but they got the room down to bearable levels in about an hour.

No, I don't know how that administrator managed to not get fired, but I suspect that being a drinking buddy of the CEO may have been a factor.

[Excused Boots]

Re: Rogue DHCP server? Ban the operating system!

"No, I don't know how that administrator managed to not get fired, but I suspect that being a drinking buddy of the CEO may have been a factor.”

I will suggest an alternative explanation based on the first issue. The VAX was offline for a month - which implies that the organisation did not employ anyone else who could fix the issue, or had the knowledge or technical skills. Which is not good, suppose said admin falls under a bus, does the company go bust because they can’t function any more?

Now you would have thought they might have learnt a lesson, but the second incident would indicate that, maybe not.

Yes OK being a drinking buddy of the CEO might well be a factor, I think a bigger one is the company’s failure to understand risk, and mitigate against it.

[billdehaan]

Re: Rogue DHCP server? Ban the operating system!

implies that the organisation did not employ anyone else who could fix the issue, or had the knowledge or technical skills

Actually, the organization had people who could fix it. Many, in fact. Pretty much any one of them would have been an improvement over the administrator we had.

However.

The organization in question was a defence contractor, and it was a secure building. And given their Kafka-esque security policies, that meant approving another person for system access would be required, and that took months, literally. By the time that approval was completed, the original administrator would have returned months ago.

Now, why didn't they approve a backup anyway? Because that would have to be approved by the CEO. Who... was friends with the current administrator.

As for risk, the company did "cost plus" R&D work for various military and three letter acronym organizations. That meant that for every dollar the company spent, it recouped something like $1.15 from the customers. The customers, being funded by taxes, were not terribly concerned with cost overruns, they were expected. The company didn't lose money by having the Vax offline for a month, they just charged it to the customer. So there was no incentive at all to fix things.

If you remember back to the 1980s and 1990s when everyone was talking about the Pentagon paying thousands of dollars for hammers and toilet seats, this is one of the reasons why.

[Anonymous Coward]

Just before Christmas

Just before I went of leave we rolled out a Windows NT 4 Service Pack (3 IIRC - was a long time ago) overnight, everything went OK except at remote site, where the remote link went down in middle of systems updating, which left 50 or so machines behaving erratically and if rebooted crashed.

Reason the remote link went down - due to differences in the 64k lines modems and the 128k line modems we upgraded to, which I had missed in docs, not setting the control lines meant the equipment would occasionally switch the modem into test mode due to the data on line (would likely not been an issue if not encrypted), shutting the link down.

[Flightmode]

Many jobs ago, we upgraded our Novell stock from 3.12 to 4.11; and at the same time we deployed new Compaq servers across the country. This usually meant getting the hardware delivered to the site and racked by local staff, then one from my team travelling to site and setting up the Netware software, bringing it online and finally upgrading all the local PCs with NDS-capable clients. This was usually pretty a cushy two-day job as there would only be a dozen or so clients per local office.

And then you'd inevitably get a text message from one of your colleagues when you were on the train home, asking if you'd remembered to 'set reply-to-get-nearest-server off' when cutting over to the new box, as the rest of the users were complaining about an extremely slow network. By design, we wanted all client and such client tasks to be handled by the beefiest 4.11 servers in our central data center that had been installed first; but of course, any new servers we installed in satellite locations were slightly newer and completely un-loaded, so they responded faster - even if they were two router hops (and likely at least one ISDN link) away. It was a simple enough change - just change a single config line and then kick all the (non-local) clients off the server (let's face it, that's going to be a server reboot) - but whoever went on site always forgot it (yes, including me). The issue wouldn't be observed immediately; it would take a few people calling in from different regions before whoever was on helpdesk duty caught on.

[Albert Coates]

Server reboot a little drastic?

But Shirley, just Load Monitor, select Connection Info, select offending clients and press delete. Much the same thing with ConsoleOne. Pretty much the same with 3.11, except use Rconsole. (Ex Novell Instructor)

[I could be a dog really]

There are 10 sorts of people - those who have accidentally put a rogue DHCP server on a network, and those who have never touched any networking kit. Or alternatively, those who have, and those who haven't ... yet.

For extra giggles, Microshaft have made it even more interesting by making their DHCP client and server non-RFC compliant ... well because they can. The two work well together, but really don't like being mixed with RFC compliant networks. But I digress. Anyway, the Windows DHCP client is really really "sticky" - it really likes to keep the same address and will do so unless explicitly told not to. So when you stick a spare router in as a 4 port switch, the Windows clients generally carry on talking to the server that gave them their existing IP address. And this will carry on for weeks or months, until one day the server is down for whatever reason - and only then does the network break. This makes it "interesting" figuring out what it was you (or one of your colleagues) did wrong as no-one recalls touching the network recently.

[CrazyOldCatMan]

For extra giggles, Microshaft have made it even more interesting by making their DHCP client and server non-RFC compliant ... well because they can

I'm de-Windowsifying my home network (1 file server/DC/DHCP/DNS server already removed) and have a replacement RPi5 configured to do DHCP & DNS, all ready to do (service is disabled at the moment).

At some point I'm going to do a full network shutdown, disable DHCP on the 2 Windows servers left (one VM, one little Lenovo Mini) and enable the Pi.

And hide under the sofa cushions until everything has connected.

(Especially the air-source heat pump and the Solar PV management.)

[Luiz Abdala]

Well, VPN, you know... can lock you out of your own router.

I needed to fiddle with the DHCP on my home router.

But it wasn't for the life of me letting me login into it.

Turns out my VPN was on, making my pc to appear as an external entity to the router, which promptly slammed the door on my face.

Learning opportunity, and the test of router being locked down tight to prevent ANYTHING to connect to it, even if this something is on the internal interface, not the WAN port (which was routed throught the wan port because vpn, but still) was completed succesfully.

[Doctor Syntax]

Re: Well, VPN, you know... can lock you out of your own router.

Had the situation the first two lines describe. It turned out that the ISP in their ?wisdom had decided that the ISP supplied router should be only managed by them remotely and either changed the admin password or locked down LAN ports by way of a S/W "upgrade". Note ven a factory reset could fix it. Apart from being seriously pissed of, not to say disadvantaged by this I was even more cross to discover that the thing could be managed from the WAN. It only lasted long enough for me to get hold of a something better.

[CrazyOldCatMan]

Re: Well, VPN, you know... can lock you out of your own router.

It turned out that the ISP in their ?wisdom had decided that the ISP supplied router should be only managed by them

You used the ISP supplied router? How.. cute.

(In my Zen-migrating-to-CityFibre upgrade I did plug their own router in just to check that the problem I was having [1] wasn't a line problem. It wasn't)

[1] The 'use your own router' doc hadn't been updated to say 'if you are on CityFibre, you need to set your VLAN to 911'. Took about two minutes of conversation with a (proper) techie at Zen to realise what the problem was. And another two minutes for me to work out how to do it in the new-to-me OpenWRT. Worked like a reliable thing on reliable day since.

[vogon00]

DHCP? I can go lower than that

Many moons ago, when carrier-grade VoIP was becoming a thing, QOS for voice calls was a hot topic.

My group were tasked with coming up with a policy for managing QOS for both signalling and RTP media our setup. I forget exactly why we used it, but someone bought in some software (Supplier's name withheld) that monitored/probed most of the nodes in the network, came up with a list of choke-points in real-time and messed around with OSPF costs to make sure media didn't transit a congested path. There were 3 separate machines running this as a service, all on the same local Ethernet segment.

The vendor's gurus duly arrived to install & commission their bandwidth manager package....and couldn't get it to work at all. Local instances seemed to be running OK, but the topology it discovered was complete tosh, continually changed, and their logs were full of protocol/message error warnings. They'd' been at it for three days without success when I happened to notice some MAC addresses moving between switch ports when they should have been immobile.

Turns out they had copied the config of a 'reference' system elsewhere and based each of 'our' machines on that config, changing IP addresses as required. However, they forgot to change the config lines that over-rode a local Ethernet's burned-in MAC with a 'forged' one....so there were 3 machines on the same segment/L2 domain with identical MAC addresses.

Faces wend a bit red when I asked if these 'mobile' MAC addresses were theirs:-)

[I could be a dog really]

Re: DHCP? I can go lower than that

Oh yes, duplicate MAC addresses are real fun - mostly because it's so rare (few people ever fiddle with them outside of VM environments, and vendor's devices should be unique.) So when it does happen, few will recognise the symptoms - I'd probably struggle.

Even if a vendor messes up, it probably won't be noticed, until ...

Some years ago I was talking with someone from a local university. They'd bought "several" thousand new PCs in one batch, from a vendor with a brother called Rodney. They had the sort of problems you've described, and eventually realised that for every 256 MAC addresses, there are 257 PCs. There was an error (probably an off-by-one classic) which meant that each time the LSB rolled over, they got a duplicate MAC address. Probably been going on for years, but few people buy 257 PCs, let alone thousands, so there'd never be duplicates on one network.

[The Oncoming Scorn]

Re: DHCP? I can go lower than that

When I did a placement at Xerox, one of the engineers who got saddled with me, told the tale of going to site, to investigate a issue.

Having fought his way down the M4 to Newport, parking up in the middle of a city center, lugged all his networking kit & tools to site (In the rain), up the final set of lifts & discovered within the space of 5 minutes that the offending workstation MAC address was identical to one in London.

Having identified the problem he then had to make his way back to his car in the pouring rain.

Icon - Getting his mac (Which I don't think he had) this didn't exactly make him a happy bunny to put it mildly on that occasion.

[G.Y.]

name? Re: DHCP? I can go lower than that

What does the name Rodney have to do with this?

[Doctor Syntax]

Re: name? DHCP? I can go lower than that

You need to Trot off and look it up.

[DoctorPaul]

Re: name? DHCP? I can go lower than that

You plonker. Not you, Rodney.

[Robert Carnegie]

Re: name? DHCP? I can go lower than that

A reference to the British television comedy "Only Fools and Horses" ( - Work), in which Derek Trotter and his brother Rodney are usually involved in trying to make money by buying and then selling various products with profound design or implementation flaws.

[Peter Gathercole]

Re: Duplicate MACS? I can go lower than that

Over 30 years ago, I was in a large computer vendor's customer support centre that ran their network over Token Ring (which was unsurprisingly a mandated technology for this company at the time), with the MAUs in a space that we didn't have access to. It was a building shared with other departments, and running the physical network infrastructure was outsourced to another department (we only had control of some external access infrastructure, everything in building was out of our control).

One day, the network breaks on and off over the day, then fixes itself miraculously at the end of the day. Takes out all of the systems on the desks of the support centre staff. Happened three days in a row. Many of the desktop systems were X Stations (this company's name for X terminals), without significant diagnostics, but the few remaining desktop PCs reported that the network was beaconing. We went through shutting down and then pyysically disconnecting systems one by one, assuming that it was a faulty cable or TR card on one of our systems. Could not work out the problem.

Finally worked out between which two systems the problem was happening, and eventually got the building wiring team expert engaged on the problem. Traced the cables back to the network room (which was an absolute mess of MAUs, 3174 and 3274 mainframe terminal controllers with cables knotted a foot deep), and found...

...in one of the MAUs running our network, there was a cable plugged in to a port between the two systems we'd identified. The building network team could not work out what it was as it didn't appear in the wiring database, but it was definitly not one of our desks. Unplugging this fixed the problem.

The networking team told us, after tracing cables for a couple of days (it was such a mess), that it turned out one of the meeting rooms adjacent to our space in the office had been wired in to our supposedly private ring, and a visiting presenter came in several days in a row, and plugged his system with a 4MB/s TR adapter into our 16MB/s ring. Each day he tried to get it working, and left his system attached while presenting, and trying to work out why it couldn't connect to the non-existant site LAN he was trying to use during the breaks, only turning it off when he left for the day.

Got an apology from the wiring team, and the reward from our management was a task to try and re-engineer the ring into two bridged segments, with duplicated internal and external connectivity so if the same thing happened again, it would only affect one segment and only half of the support centre staff. We then had to move every other desk workstation to the new segment. With hindsight, it would have been better to break it into two separate subnets as well as two rings, and route rather than bridge, but we only had limited resources available, and IP routers were at the time expensive (and we had spare PS/2s and TR adapters for a bridge) and routers were not on this company's own product list, and we had nobody in our staff at the time thinking like a network admin.

We should really have asked for CAUs for our network, but as the building was in the process of being decommissioned, that was never going to happen. When we moved, we did have CAUs installed for us, and we also had one entire floor of the new building, so mistakes like this were less likely to happen.

[CrazyOldCatMan]

Re: DHCP? I can go lower than that

Oh yes, duplicate MAC addresses are real fun - mostly because it's so rare

Maybe it's not duplicate MAC addresses causing the issue but connecting more than one ethernet port to the same network in a TrueNAS setup (without some sort of a bond or bridge involved) causes packet storms with what very much looks like duplicate MAC errors..

As I discovered when I migrated to TrueNAS Scale, connected the 4 cables back up and noticed that the switch had gone completely mad. And, even plugged directly into the switch, had about 80% packet loss trying to ping the switch. Pinging anything else? Forget it.

Unplugged TrueNAS box, all normal. Thought "hmm.. might be a bad ethernet cable/port" replaced the cables, started plugging it back in. First cable, all good. 2nd cable, hmm.. network errors. 3rd cable, network unusable.

Unplugged all but one cable, all good again.

So I checked online, found others describing the error. Found I had to configure a bridge anyway (so that the VMs could see each other and the host IP) which seemed to be a cure.

[Peter Gathercole]

Re: DHCP? I can go lower than that

Duplicate MAC addresses are possible with user-definable alternate MAC addresses on a surprising number of NICs. Never quite worked out why people wanted to use these, although I do believe that in the early '90s, North West Water in the UK did, with some very unpredictable problems surfacing as a result.

Actually, I can think of one example where it may be used, in high-availabillity networking, where you could have a alternate NIC taking over from a failed one, right down to the MAC address to prevent delays due to having to age out or replace entries in the ARP cache of client systems. But you would still need to suffer spanning-tree re-train delays.

Actually, aren't alternate MAC addresses also used in Etherchannel port groups?

[JMcKeand]

Microsoft Small Business Server DHCP

Back NT 4 days, SBS differed from standard NT on the DHCP Server. It would quietly stop (little or nothing in the logs - I already knew that it had stopped, tell me why) the service if it detected another DHCP on the network. I think this eventually would be changed by at least pointing out that there was a rouge DHCP, but it never told you anything beyond the existence of the rouge.

[Excused Boots]

Re: Microsoft Small Business Server DHCP

"but it never told you anything beyond the existence of the rouge.”

Other than the fact that it was a shade of red!

Sorry, sorry couldn’t help myself.

But yes you are right, the original error logs on SBS were a bit spartan, but that fact that the DHCP service would simply shut itself down for no obvious reason, was a good indication of their being another DHCP server on the LAN. Finding it was down to you, but the original premise of SBS was a tiny organisation, possibly in one room, so tracking down odd devices might not be too much of an issue.

[Already?]

Check In

On the Friday before a much needed week off that must have also been a school holiday (wife teaches) I offered to complete some necessary but not too difficult still outstanding code changes on the Friday night at home before we scooted off. I duly did them and checked the changes into Git, which we’d recently started using. No one had given us any real Git training and my erroneous belief was that check in makes my changes available on the server and Push promotes them to the live code, so I sent the email - done, all on Git, ready for Pull Request and Push, see you on Monday week - and logged out for 10 days. Only on my return did I discover that my failure to Push meant that my Friday night efforts had been wasted, and that someone else had had to do my changes whilst I was away on the beach.

We had a Git Basics training session soon after, which included how Merge moves changes into the ready for release code.

[Dave559]

Does your organization use DHCP?

This reminds me of a certain something…

DOES YOUR ORGANIZATION USE DHCP?

> yes

WOULDN'T YOU PREFER A GOOD GAME OF CHESS?

>

[J.G.Harston]

hoooo liiii daaayyyy? Sorry, I don't understand that word.

[Xalran]

First rule

First rule I learned once out of all the training courses my first employer saw fit to do (there was a good reason, you don't learn how a public exchange work and how to configure it from scratch) the first day I was sent on a live site for job training...

The senior integrator (which had barely 2 years of seniority over me) told me the first rule when working with live public exchanges :

Fridays are for paperwork.

Absolutely NO WORK on the exchangge will be performed.

And that's twice true for the Friday Afternoons... Bring a book in case there's not ennough paperwork, but hands off the TTY, the PC, and anything interracting with the exchange.

( summarized, in French as : le Vendredi, c'est touche pas p'tit con)

[Anonymous Coward]

Bloody TP-Link

Installed a system for a client. They then decided to extend coverage and (their "IT pro" son) installed some TP-Link powerline extenders.

A couple of weeks later I got a call that the internet wasn't working. All OK to the router from outside, so went on-site to investigate. Allocated IPs not in expected range.

A bit of detective work and it turns out the powerline extenders have a built-in DHCP server that gets turned on when it can't find one on the network (presumably to help with initial setup). Naturally that causes a race condition when you have several of those devices, plus in this case a router that was slightly slower to boot after a power cut.

Reported that back to the "IT pro" son, who denied that these devices have a DHCP server so it couldn't be them. He pointed to the spec sheet, with no mention of DHCP. I sent a screenshot of the configuration page where you can disable that stupid functionality.

Funnily enough, no problems since.

[Excused Boots]

Re. Powerline Extenders.

Anyone ever had that issue where they manage to bridge to the house/company next door - because why would you change the default passphrase on them?

Yes your PC is picking up next doors DCHP server and network config, which explains why when you run a speedtest your ISP appears to be completely different to the one you pay, and also why your internet connection is running at the speed of an arthritic snail!

Go and unplug it, and see what happens?

[Doctor Syntax]

Re: Re. Powerline Extenders.

"why would you change the default passphrase on them?"

The paranoia that comes naturally to any system admin, especially a DBA.

[Anonymous Coward]

I got to work one morning and found I couldn't log in, so I filled up the office coffee filter and waited for our network admin to arrive.

Five minutes later, Chris was at his desk, he also couldn't log in to anything, so he went to the comms room and looked opened the console on the domain server and saw that no computers on site were connected, and the domain controller couldn't connect to any of the other servers. Chris' suspicion was we had rogue DHCP server on site

Once the whole IT staff were assembled, we set off to check all the buildings to see what was connected.

Now I should mention this was a large public school with many boarding houses, and many teaching buildings, many out buildings, even a pub and a shop scattered around the town, all connected back to our office by fibre or point-to-point wireless.

Many hours, and many, many steps later, one of our team found the culprit.

One of our boarding house chefs had brought in an Alexa to listen to the cricket as there was no DAB signal in the house.

He'd not been able to find an empty Ethernet socket for the Alexa (it was an older model with an Ethernet connection) he'd brought in a spare "switch" he found at home. Said "switch" being an old Wi-Fi router with DHCP turned on.

The morning's work prompted Chris to set up a WiFi VLAN to be used by smart devices and we were not troubled by them again...

[GlenP]

Coincidentally...

We have a possible IP address conflict this morning (I don't believe that is the issue) and I was telling my PFY about similar DHCP occurrences, such as the MFD engineer who enabled DHCP on a new device during install (I'm guessing the DHCP was legacy but by then there was no way it should ever have been configured) or, similarly, an IP Phone Switch that did the same.

[goblinski]

Not sure what changes were made to the site, but:

- The bookmarks that I had (painstakingly) figured out for On Call and Who, Me, no longer work. To name them:

https://www.theregister.com/Tag/Who%2C Me%3F/

https://www.theregister.com/Tag/on-call

- A search in the website by On Call or Who, Me returns nothing.

- The different On Call and Who, Me articles are now tagged by their relevant fields (Software, Security), which makes them completely diluted in the rest of the content. For example, this one is under Networking.

If this is to stay, please comment on this, and advise on how to find older articles from these two for future reference. Thanks

[goblinski]

Whew :)

I see they now appeared as their own tags on the top right. Thanks for that.