
We've all been there, I managed to kill a worldwide instance of Oracle with a silly little error on a recurring query. I would tell more or email it but I would be found out.
Ho ho ho! It may be Christmas Eve but Who, Me? stops for no festive season. This time, our weekly reader's column of tech support problems of their own making comes from "Harold", who was a senior managed compute engineer for a major provider of stuff. "I was tasked with a simple project to extract a full list of users in all …
I used to run a site using Server Side includes to look up template files to keep the look across the site (circa 2002 or so). My host had default 404 etc error pages which were write-locked but for some reason I could save over them in place, so I added SSI Includees to make the error pages match those of the rest of the site.
Later on, I rewrote the site in PHP, and since the SSI error pages wouldn't use PHP, and I had learned more in those days, I used a .htaccess page to point the server at new, PHP error pages, retired the old SSI based templates, and never looked back.
Unfortunately, one day, my host deleted the .htaccess file, so the server failed back on the old error pages, which still had SSI include links to template files that no longer existed. They also had the server set up so that on a 404 page not found during an SSI Include request, it would instead include the 404 page.
So we had a 404 page that would try to include a template file, which because it didn't exist, would instead include the 404 page in line, and so on... 404s all the way down, until the server ran out of resources and fell over. For all paying users on the node.
While working as a software developer for a large well know credit reference company I had a phone call one day from the central security team on another site asking me what I was doing. Apparently a particular part of their website was under a massive DOS attack and they'd tracked it back to my computer. All I was doing was accessing the site as an ordinary user, nothing fancy. Not even actively, it was just open in my browser. They asked me to stop whatever it was I was doing, which just entailed me closing the browser window and everything apparently went back to normal. I have no idea what caused the DOS but I can only surmise that there must have been a bug either with the website or with the network setup which triggered when a particular part of the site was accessed from an IP address within the company causing something to get stuck in a high priority loop. Explanations on a postcard to...
There's nothing wrong with the -delete option, you should first run your find with a -print (actually -depth -print since -delete implies -depth for obvious reasons) to get the list of files it will delete and sanity check it before you use -delete. Same thing as with any potentially dangerous command like 'rm -rf' or 'mkfs', you need to be SURE you are doing what you want to do and shouldn't rely on software crutches to do it for you. Failure to do so runs the risk of being El Reg's next "Who, Me?" story :)
Unrelated, but the code for find's -delete option was based on a patch I sent to the Bugtraq security list about 20 years ago. How time flies!
Grrr. Creaping featureism!
The 'correct' use should be:
find <dir> -name -exec rm {} \;
(the starting directory should be required, I don't know when GNU made it optional). I admit that it spawns more processes, but if you're really worried about this, then:
find <dir> -name -print | xargs rm
These work on pretty much all unix-like operating systems whereas the GNU ones don't, which is why they trip off the fingers when I'm working.
Mine is the copy with the SVID in the pocket!
You should have read for context and thought about why I wrote this patch for the Bugtraq security list!
It originated from a discussion regarding attacks where someone who had write access one of the directories being cleaned up (often directories where everyone has write access, like /tmp) could fool that command into deleting ANY file on the system. Both of the variants posted above are vulnerable to this, so I posted a find patch I'd written with the -delete builtin which was not vulnerable.
It isn't creeping featurism if it solves an actual problem.
No. You will find that -print0 is not in the SVID version of find. There is a typo, though, as I missed out the filename!
I take your point about print0, but even though whitespace is actually allowed in filenames in pretty much all UNIX-like filesystems.
If I ruled the world, the set of characters that would be illegal in filenames would be much larger! I really hate when filesystems are shared between UNIX-like OS's and Windows (Samba, NFS etc.), and I see filenames created by Windows including spaces, asterisks and other characters. And that is not to mention when different multi-byte character encoding systems were used, prior to UTF8 becoming dominant.
But then, many people think I'm far too old to make relevant comment about modern computing.
If i'm typing a particularly long command that will delete or otherwise change a lot of files, I'll either not add the sudo rm
at the start, or type zsudo
in case I accidentally hit the enter key before I'm ready.
I'm sure we've all deleted a parent directory when we were intending to delete a subdir.
I once advised a Professor that rather than staying with his very old Sparc box running Solaris under x86 was a better idea.
He bought a new machine and was not very happy as he said it was far slower than his old machine.
I said I couldn't see why but he was the respected professor of computer science.
A couple of days later he admitted there was an infinite loop in his Prolog code and once this was sorted it really was very fast.
Some recursive functions are easy to understand, but hard to execute like the Ackermann function.
Wilhelm Ackermann was a pupil of David Hilbert who was a pupil of Ferdinand von Lindemann who was a pupil of Felix Klein who was a pupil of .... Probably going all the way down to Noah who had just to count to two.
"Ho ho ho! It may be Christmas Eve but Who, Me? stops for no festive season."
Seeing as how this is in the Data Centre portion of El Reg, I'm assuming that the people responsible for pushing out the Christmas Eve article are getting double time for both work and travel, plus time in lieu, plus meal allowance and travel reimbursement and a minimum callout time of four hours...
...and that you've written a script to publish the article automatically while you're downing a few lagers at the pub.
If any of the El Reg ink stained wretches are at the office, (oh heck, even if you're not) Merry Christmas to you all!
This is the loop that never ends;
it goes on and on, my friends...
Some coder started running it, having forgetting to fuzz,
and now it will continue, simply because...
This is the loop that never ends;
it goes on and on, my friends...
Some coder started running it, having forgetting to fuzz,
and now it will continue, simply because...
[loops ad nauseum]
It was a dark and stormy night, and the Captain said to Antonio "Antonio tell us a tale"; so, Antonio told us a tale, and it went like this:
"It was a dark and stormy night, and the Captain said to Antonio "Antonio tell us a tale"; so, Antonio told us a tale, and it went like this:" ..
The version I heard, and have often repeated, goes:
"It was a wild and stormy night, and the Captain said to the First Mate,"Jack, tell us a story". And the First Mate began.
"It was a wild and stormy night, and the Captain said to the First Mate,"Jack, tell us a story". And the First Mate began.
"It was a wild and stormy night
And so on ad duffyouupeum.
several decades ago, almost four, I had to break a disk cluster on a VMS system. this is a job to beware. but it needed to be done to advance reliability. so, since there were more users than space on any single disk, had to divide them. didn't work well to split halfway down the first letter of last names, so I had to count instances and fork them left and right. test went fine. so one night, kicked off the script.
first AM when users started logging in, they couldn't. I couldn't remotely. so I used the admin credentials.
logging went to the system disk because, space.
until there was none.
deleted the log, drove in, job done, lesson learned. should have flipped the logging over the VAXcluster to another machine. learn, grasshoppers.
The ultimate way of wrecking your day, a command where if you forget an option it overloads the cpu.
Happened a good 10 years ago, a colleague was doing some checks, ran a debug but hit enter before the last option, the core switch went into a loop, we realised what had been done, a quick glance towards the computer room, and without a word both of us got up walked down to the core and rebooted it.
Apparently causing a mass panic when someone looked for us and found two empty desks.....
Neither of us work for that company now though.....
Inasmuch that I cocked up, but admitted my mistake when it became clear I'd caused it. (AFTER fixing the issue).
I managed to cock up a 7 way lightswitch arrangement such that one switch acted as a 2 seperate circuits when one of the intermediate (en_US '4 way') switches was thrown one particular way. Very confusing till I spotted the mistake in wiring, and recognized my connection style on the switch.
For my sins, I am an MCT, and once ran a (successful) course on SQL Server Querying for a company.
As with other Microsoft courses, it was all run in a virtual environment, on laptops I brought in, but using a database (AdventureWorks, for those who have met it) which had nothing to do with the actual line of business.
We finished a few hours early, so the delegates asked if they could put their new skills into practice on the real databases. Since they had a nice training room, with plenty of PCs, I didn't see any problem with this, and the IT manager couldn't see any issues either, so they all set up, practicing joins, sub queries, grouping, and all the other fun you can have with SQL.....
...for about 10 minutes, when the DBA came running in and demanded, in no uncertain terms, to know what the f@&k we were doing to his production servers, which had slowed to a crawl under the extra load.
With an accidental DOS from an app i left running overnight. I guess this was late 80's.
It was in Beta at the time and since it was overnight it probably only had 7 users on it.
And hopefully that helped them make it more robust, so all to the good.
This post has been deleted by its author
At a place I worked as a contractor there was a low level server ops guy, let's call him John Smith, and he wrote a very simple script to change passwords. I don't quite know how his script was more efficient that the one line of powershell that it ran but then he was a very low level person, the type who tend to do the stuff that nobody can be bothered to script because they can't script clicking on the next box.
So the script that "John" wrote basically took a username and changed the password. I would hazard a guess that it basically did:
Get-ADUser $user | Set-ADAccountPassword -NewPassword "Changeme123!"
OK, so there were some other options in there as well but you get the gist.
Anyway, "John' runs this script using his privileged account (let's him do desktop stuff like change passwords) and it seems to hang. He goes off to lunch.
At this point it's worth noting that in the line of powershell above if $user is null then Get-ADUser will get every user object in the domain and pipe all of them to the password change applet. It hadn't hung, it was just a little busy.
While "John" was off at lunch the rest of the IT department realised that they were unable to reach stuff, many had locked their desktops while eating their own supermarket "own brand" sandwiches at their desk and when they tried to unlock their desktops found that their passwords were no longer valid.
Later in the incident room it became apparent that "John" was responsible however he not only managed to retain his job but eventually managed to move sideways (and slightly down) into the desktop team.
I took out a Sun Solaris server with a looping bash script. Thankfully it was development, not production. I stupidly appended a line to the logfile I was grepping with the result of finding an entry (rather than a separate file) and, more to the point, used the exact phrase I was grepping FOR. Something akin to '<search term> found on line xxxx".
It took about 5 seconds to blow up like the Hulk and take the box down.