In Search of Lost Time: GNU Grep 3.7 released with fix for 'extreme performance degradation' GNU grep 3.7 has been released with a fix for a bug causing "extreme performance degradation" in certain types of search. This search tool, which looks for character patterns in files, is a core utility on Linux and other Unix-like operating systems. In November last year, a user noted: "I have a use case where I run grep with …
I was a great fan of ack once upon a time, till one day I nearly lost data to it.
https://groups.google.com/g/ack-users/c/oa82NsPqhvo/m/Y2f0RTnY5dEJ
when someone else ran into similar problems and asked for documentation on how ack chooses what files to search and what files to ignore, the author's reply was "There's no English that explains how it works". https://groups.google.com/g/ack-users/c/rmRt92zBUlk/m/R6s85VhhDLoJ
Still being a fan (but thinking hard about why), I wrote it up, https://groups.google.com/g/ack-users/c/kdlaASvikFo/m/1ObiGm1L_yUJ and asked the author to include it in the docs somewhere. His response? "In my copious free time".
Sure this was back in 2009, but it still rankles. I have a long memory for open source authors who deal like this with users.
I would add cut, join, comm, diff and sort, with the occasional tr and tail, pulled together by a good understanding of some of the more obscure parts of ksh/bash.
The tradditional UNIX tool set contains just so many obscure, but often very useful commands if you know them.
It was all just so easy when you could grab the permuted index for the man pages, and just read through to see what commands were there, and then go and read the actual man pages. The GNU Info documentation, even if it were complete, which is isn't, is just not as easy to browse.
Cat, although being one of the first commands most people use (at least if you follow the traditional "learn" route - remember learn?), actually becomes a little redundant once you get to know redirection of I/O streams.
I still do sometimes use cat, especially cat -v (as an alternative to od) and cat -u, and sometimes use cat as a command with sudo or su (something like "sudo cat file | pg" to read a file I don't have read access to as my non-privileged ID) which has advantages over just sudo pg, in that the command you are actually interacting with is running as your ID rather than root, reducing exposure to inadvertent mistakes.
I also sometimes use it with ssh to copy files from one machine to another rather than using scp or sftp. A bit random, but UNIX always has had more than one way of doing things.
I didn't forget tail.
> Cat, although being one of the first commands most people use (at least if you follow the traditional "learn" route - remember learn?), actually becomes a little redundant once you get to know redirection of I/O streams.
On natively-POSIX hosts this is true, but it's worth knowing otherwise that where you're stuck on Windows with "git bash" (MSYS) or cygwin for comfort, there are corner cases where cat (strictly speaking, "useless use of cat") is what Just Works.
In one particular example when I found I couldn't get a manually-edited diff back into a git repository directly with 'patch' recently (known problem with files suddenly having DOS newlines), in tandem with my file being in /var (evidently special in some way my google fu did not help in finding) I ended up doing `cat /var/tmp/foo | git apply -` to get what I wanted!
Being that I work on UNIX and Linux almost exclusively, I couldn't possibly comment on the use of cat on non-Posix systems. I mean, why would you even want to do that!? Just use Linux (I'm being ironic here, I am actually tainted by having to use Windows as an access platform to client systtems that I look after). I'm sure that your git example would work on Linux.
But I would wonder whether "git apply - < /var/tmp/foo" would have worked (even in a cmd session - MS/DOS redirection was similar to UNIX, although implemented differently), although as you say it was to do with DOS newlines, maybe the version of the "cat" command you were using was actually knowledgeable about DOS text files, and automatically stripped off the carriage returns (UNIX uses Linefeed characters as the end-of-line characters, DOS uses CR-LF pairs, and often does not terminate the last line properly).
Hrmm. grep yes. I am less sure of sed and awk.
Little Jonny the sysadmin had a problem. Little Jonny thought he could solve it with sed and awk. Little Jonny had three problems.
More seriously, I find sed and awk fine for simple substitutions and variable outputs from a string but not after that.,
Many people reach for Perl and Python for processing streams of characters. I found that a good understanding of awk has meant that I've never really needed to use either of these other two languages.
Sed is easy for relatively simple substitution, and can get complicated if you try to handle multiple pattern spaces, but can do some amazing things spanning multiple lines and different and conditional pattern substitutions, but I find I have to return to the manual most times I want to do anything complicated. Or do it in awk.
Awk can be treated much more as a traditional language with an input/output cycle with pattern matching built in. But it's very 'C' like, which makes it ideal for us old greybeards who cut our teeth on genetic UNIX back in the early years, when 'C' was what you had to know.
The only time I really switch to Perl is if I need to use OS features that are not really character orientated, like interacting with the networking layers, where there are readily built CPAN modules. And I still can't get my head around a language where indentation defines programming block structure and looping.
Python still seems important enough to keep trying, though.
I would be interested in hearing what you use for things more complicated than "simple substitutions and variable outputs from a string".
If you drop into Perl or Python, then I suggest that the learning curve for those tools was much, much greater than it would have been for awk. But if you know Perl already...
Strangely enough, I remember being first shown Perl back in 1990. One of my colleagues excitedly showed me a file, and said "What do you make of this?"
I said, "Well, there's a bit of awk syntax there, that looks like an associative array reference but in a C structured while loop, and that substitution looks like it is sed, but all these in the same source file?" And he said, "This is a new tool called Perl that tries to be all things for everybody. It takes the best bits from wherever".
Unfortunately, it took some of the worst as well. I think it was just after the release of Perl 3.
This post has been deleted by its author
Umm. Completely redundant "cat" and "tail". O was it your intention to get as many commands in as you could think of.
More succinctly grep --version | sed "/grep (GNU grep)/s:3.6:3.7:
By the way, the colons are not a mistake. Read your sed and ed man mages. Very useful if you have slashes in the strings you're trying to manipulate.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
