back to article In Search of Lost Time: GNU Grep 3.7 released with fix for 'extreme performance degradation'

GNU grep 3.7 has been released with a fix for a bug causing "extreme performance degradation" in certain types of search. This search tool, which looks for character patterns in files, is a core utility on Linux and other Unix-like operating systems. In November last year, a user noted: "I have a use case where I run grep with …

  1. Tom Chiverton 1 Silver badge

    I'd vote for ack as a good replacement

    1. Sitaram Chamarty
      Thumb Down

      ack? NAK!

      I was a great fan of ack once upon a time, till one day I nearly lost data to it.

      https://groups.google.com/g/ack-users/c/oa82NsPqhvo/m/Y2f0RTnY5dEJ

      when someone else ran into similar problems and asked for documentation on how ack chooses what files to search and what files to ignore, the author's reply was "There's no English that explains how it works". https://groups.google.com/g/ack-users/c/rmRt92zBUlk/m/R6s85VhhDLoJ

      Still being a fan (but thinking hard about why), I wrote it up, https://groups.google.com/g/ack-users/c/kdlaASvikFo/m/1ObiGm1L_yUJ and asked the author to include it in the docs somewhere. His response? "In my copious free time".

      Sure this was back in 2009, but it still rankles. I have a long memory for open source authors who deal like this with users.

  2. James 47

    I've always got by with fgrep

    1. PerlyKing Silver badge
      Headmaster

      RTFM

      fgrep is the same as grep -F, as used in Meyering's example in the article.

  3. Dr_N Silver badge
    Go

    grep, sed & awk

    With a touch of shell scripting they're all you'll ever need to quickly solve most everyday problems.

    1. Peter Gathercole Silver badge

      Re: grep, sed & awk

      I would add cut, join, comm, diff and sort, with the occasional tr and tail, pulled together by a good understanding of some of the more obscure parts of ksh/bash.

      The tradditional UNIX tool set contains just so many obscure, but often very useful commands if you know them.

      It was all just so easy when you could grab the permuted index for the man pages, and just read through to see what commands were there, and then go and read the actual man pages. The GNU Info documentation, even if it were complete, which is isn't, is just not as easy to browse.

      1. 2+2=5 Silver badge

        Re: grep, sed & awk

        There are some useful modern tools to add to Peter G's list, particularly gron which converts JSON into a 'one value per line with a full path' format making it easily grep-able.

        1. anothercynic Silver badge

          Re: grep, sed & awk

          Ooooooooooooh! That's a new one for me!

          Thank you!

          *adds to arsenal of tools*

      2. Skiron

        Re: grep, sed & awk

        You forgot cat and tail.

        1. Peter Gathercole Silver badge

          Re: grep, sed & awk

          Cat, although being one of the first commands most people use (at least if you follow the traditional "learn" route - remember learn?), actually becomes a little redundant once you get to know redirection of I/O streams.

          I still do sometimes use cat, especially cat -v (as an alternative to od) and cat -u, and sometimes use cat as a command with sudo or su (something like "sudo cat file | pg" to read a file I don't have read access to as my non-privileged ID) which has advantages over just sudo pg, in that the command you are actually interacting with is running as your ID rather than root, reducing exposure to inadvertent mistakes.

          I also sometimes use it with ssh to copy files from one machine to another rather than using scp or sftp. A bit random, but UNIX always has had more than one way of doing things.

          I didn't forget tail.

          1. William Towle
            Boffin

            Re: grep, sed & awk

            > Cat, although being one of the first commands most people use (at least if you follow the traditional "learn" route - remember learn?), actually becomes a little redundant once you get to know redirection of I/O streams.

            On natively-POSIX hosts this is true, but it's worth knowing otherwise that where you're stuck on Windows with "git bash" (MSYS) or cygwin for comfort, there are corner cases where cat (strictly speaking, "useless use of cat") is what Just Works.

            In one particular example when I found I couldn't get a manually-edited diff back into a git repository directly with 'patch' recently (known problem with files suddenly having DOS newlines), in tandem with my file being in /var (evidently special in some way my google fu did not help in finding) I ended up doing `cat /var/tmp/foo | git apply -` to get what I wanted!

            1. Peter Gathercole Silver badge

              Re: grep, sed & awk @William

              Being that I work on UNIX and Linux almost exclusively, I couldn't possibly comment on the use of cat on non-Posix systems. I mean, why would you even want to do that!? Just use Linux (I'm being ironic here, I am actually tainted by having to use Windows as an access platform to client systtems that I look after). I'm sure that your git example would work on Linux.

              But I would wonder whether "git apply - < /var/tmp/foo" would have worked (even in a cmd session - MS/DOS redirection was similar to UNIX, although implemented differently), although as you say it was to do with DOS newlines, maybe the version of the "cat" command you were using was actually knowledgeable about DOS text files, and automatically stripped off the carriage returns (UNIX uses Linefeed characters as the end-of-line characters, DOS uses CR-LF pairs, and often does not terminate the last line properly).

      3. big_D Silver badge

        Re: grep, sed & awk

        I use grep and tail probably the most from those listed.

    2. Roger Kynaston Silver badge

      Re: grep, sed & awk

      Hrmm. grep yes. I am less sure of sed and awk.

      Little Jonny the sysadmin had a problem. Little Jonny thought he could solve it with sed and awk. Little Jonny had three problems.

      More seriously, I find sed and awk fine for simple substitutions and variable outputs from a string but not after that.,

      1. Anonymous Coward
        Anonymous Coward

        Re: grep, sed & awk

        You would be surprised how many silicon chip designs have required sed & awk to get them into the fab over the years. Just sayin'.

      2. Peter Gathercole Silver badge

        Re: grep, sed & awk

        Many people reach for Perl and Python for processing streams of characters. I found that a good understanding of awk has meant that I've never really needed to use either of these other two languages.

        Sed is easy for relatively simple substitution, and can get complicated if you try to handle multiple pattern spaces, but can do some amazing things spanning multiple lines and different and conditional pattern substitutions, but I find I have to return to the manual most times I want to do anything complicated. Or do it in awk.

        Awk can be treated much more as a traditional language with an input/output cycle with pattern matching built in. But it's very 'C' like, which makes it ideal for us old greybeards who cut our teeth on genetic UNIX back in the early years, when 'C' was what you had to know.

        The only time I really switch to Perl is if I need to use OS features that are not really character orientated, like interacting with the networking layers, where there are readily built CPAN modules. And I still can't get my head around a language where indentation defines programming block structure and looping.

        Python still seems important enough to keep trying, though.

      3. Peter Gathercole Silver badge

        Re: grep, sed & awk @Roger Kynaston

        I would be interested in hearing what you use for things more complicated than "simple substitutions and variable outputs from a string".

        If you drop into Perl or Python, then I suggest that the learning curve for those tools was much, much greater than it would have been for awk. But if you know Perl already...

        Strangely enough, I remember being first shown Perl back in 1990. One of my colleagues excitedly showed me a file, and said "What do you make of this?"

        I said, "Well, there's a bit of awk syntax there, that looks like an associative array reference but in a C structured while loop, and that substitution looks like it is sed, but all these in the same source file?" And he said, "This is a new tool called Perl that tries to be all things for everybody. It takes the best bits from wherever".

        Unfortunately, it took some of the worst as well. I think it was just after the release of Perl 3.

    3. wallyhall

      Re: grep, sed & awk

      Including Tetris... (a friend and old colleague wrote this): https://github.com/csBlueChip/BAShTris

      Absolute insanity!

    4. Anonymous Coward
      Anonymous Coward

      Re: grep, sed & awk

      grep, sed and awk, I always thought of that process as being similar to using a meat grinder and a sausage stuffer. Incredible tools for so many tasks involving text files.

  4. John Robson Silver badge
    Thumb Up

    Interesting...

    Enjoyed the link to the "why is gnu grep fast" mailing list...

  5. sitta_europea Silver badge

    Thanks!

    Until I read this article I didn't even know what version I was using. I probably use grep at least a hundred times a day but I'm still only on versions 2.27 (Debian Buster) to 3.3 (Raspbian).

    1. HildyJ Silver badge
      Linux

      You'd think

      For such a vital utility, you'd think that distros would try a bit better to keep it up to date.

      1. Doctor Syntax Silver badge

        Re: You'd think

        You might think the versions in distros based on Debian are a bit dusty. That's because Debian likes to let the dust settle on tweaked versions - for reasons such as the events in the article.

  6. Anonymous Coward
    Anonymous Coward

    So.....I'm using 3.6......

    $ rpm -qa |grep grep

    Result: grep-3.6-2.fc34.x86_64

    So....I suppose Fedora will eventually catch up with version 3.7.

    1. Anonymous Coward
      Anonymous Coward

      Re: So.....I'm using 3.6......

      grep --version

      grep (GNU grep) 3.6

      1. W.S.Gosset Silver badge

        Re: So.....I'm using 3.6......

        $ grep --version | grep grep

        grep (GNU grep) 3.6

        1. W.S.Gosset Silver badge

          Re: So.....I'm using 3.6......

          $ grep --version | grep 'grep.*grep' | cat | tail

          grep (GNU grep) 3.6

          1. Androgynous Cupboard Silver badge

            Re: So.....I'm using 3.6......

            > grep --version | grep 'grep.*grep' | cat | tail | tr 6 7

            Upgrade complete

            1. Peter Gathercole Silver badge

              Re: So.....I'm using 3.6......

              Umm. Completely redundant "cat" and "tail". O was it your intention to get as many commands in as you could think of.

              More succinctly grep --version | sed "/grep (GNU grep)/s:3.6:3.7:

              By the way, the colons are not a mistake. Read your sed and ed man mages. Very useful if you have slashes in the strings you're trying to manipulate.

              1. Peter Gathercole Silver badge

                Re: So.....I'm using 3.6...... @Me

                Oops. Missed a set of quotes at the end. Not caught in the cut.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021