back to article The quest for faster Python: Pyston returns to open source, Facebook releases Cinder, or should devs just use PyPy?

Facebook has released Cinder, used internally in Instagram to improve Python performance, while another faster Python, called Pyston, has released version 2.2 and made the project open source (again). Python is the world's second most popular programming language (after JavaScript) according to some surveys; but it is by no …

  1. spireite

    You don't always need speed

    If you're writing an API, then ordinary Python does the job. Saving a ms or two hardly matters.

    1. A Non e-mouse Silver badge

      Re: You don't always need speed

      Premature optimization is the root of all evil.

      1. Blackjack Silver badge

        Re: You don't always need speed

        The goal is for the thing to be easy to code with, if you want speed there are other options.

        1. spireite

          Re: You don't always need speed

          I was asked to consider writing my code in Rust, or in Go.... frankly, I couldn't see the point of doing that when Python did the job perfectly well.

          1. _andrew

            Re: You don't always need speed

            Many years ago I thought so too, and wrote an exploratory project in Python (a compiler, of sorts). It was so excruciatingly slow that I re-wrote it in another cross-platform dynamic interpreted language that happened to have a good JIT (Racket scheme) and was completely happy ever after. Python is OK for what it is, and as a general purpose wrapper for fast C code it's quite excellent, but it has some dynamisms that are particularly egregious for attempts to go fast, after the fact.

      2. Kevin McMurtrie Silver badge

        Re: You don't always need speed

        As this article states, optimization after choosing Python is it's own realm of hell. Imagine all the insane architecture hacks thrown into the system before actually concluding that writing your own Python runtime is the next logical step.

        I've seen companies build expansive nightmarish Python optimization architectures for back-end code that would be just a few pages of ordinary Java, C++, or even Go.

    2. Anonymous Coward
      Anonymous Coward

      Re: You don't always need speed

      BASIC, Pascal, Java, Python are all baby languages that some people never outgrow. That's what drives them from interpreters to compilers towards compiler optimization.

  2. vincent himpe

    let's run an interpreted language on a virtual machine that runs on another virtual machine and shoehorn that onto another virtual machine that runs on an operating system... that may sit in another vm ...

    Gimme back my 8k floating point basic interpreter that ran directly from ROM. Those were the days... i'm getting old. we need 5 billion transistors and 500 megabyte of code to print hello world.

    1. Tim99 Silver badge

      Yes, and 8 gigabytes of frameworks to write it...

  3. ST Silver badge

    Oooooh, Python is slow!

    And bloated too!

    Who knew?

    In other news: water is wet.

    1. two00lbwaster

      Re: Oooooh, Python is slow!

      Slowness is relative and poor programming can create exceptionally slow code. Multi ms hops to a DB on another server, after already spent multi ms searching a cache only to miss, then waiting for the DB to curn out a result and send it back to you is probably always going to be slower than the python code that you're executing.

      1. ST Silver badge

        Re: Oooooh, Python is slow!

        > [ ... ] waiting for the DB to curn out a result and send it back to you is probably always going to be slower [ ... ]

        Funny how Python is slow and bloated without accessing any DB in the backend.

        Next excuse?

  4. sreynolds

    Flogging dead horses

    If you need speed from the outset then bite the bullet at the start and us a man's language. None of this interpreted, JIT or other crap.

    1. wobbly1

      Re: Flogging dead horses

      Pah! call those real languages? try TI 990 machine code. Particularly entered through 16 push buttons, 1 bit at a time.

      1. Ken Hagan Gold badge

        Re: Flogging dead horses

        Pah! Call that a real language? Get yourself one of those FPGA things and write your own hardware. *Then* program it in assembly language.

  5. thames

    C and then some

    "Python performance is not always a problem, since library developers have followed Van Rossum's advice and written performance-critical code in C. "

    If you absolutely, positively need performance above all else, then there aren't really many useful alternatives to C. The accepted practice is to write the the application in Python and to then benchmark it. If performance meets your targets, then you're done.

    If it doesn't meet your targets then use the benchmarks to guide your optimizations. Optimize in Python first. For those bits that still don't meet your target, then write them is C.

    Python is designed to interface well to C. The reason why CPython persists in remaining the dominant implementation is because it interfaces so well to C. If you are doing something typical in Python, chances are there is a widely used library which does it, and if performance is critical it will be written in C.

    There is also increasing use of Cython for these sorts of optimizations. Cython is an extended version of Python which translates Python to C which you then compile as a C extension. It's a good answer for those who feel allergic to C.

    1. Charlie Clark Silver badge

      Re: C and then some

      then there aren't really many useful alternatives to C.

      Depending on the domain, you may find plenty of Fortran and Lisp fans out there… ;-)

      PyPy does a fantastic job of optimising Python byte code and has over time achieved quite astonishing results. Unfortunately it doesn't play well with Cython, which is a great alternative for those occasional hotspots in libraries.

      But the general drive to shoe-horning types in Python so that compilers can optimise should be resisted: code becomes bloated and unreadable and thus difficult to maintain. And then it turns out that YAGNI but it ticks a load of boxes, which is great for the project managers.

      Where Python does need to improve is on multiple CPUs and, due to the way Python works, this is hard™ but there have been significant improvements and investments, due to the ecosystem built around Pandas and Numpy.

      1. thames

        Re: C and then some

        The main driving force behind adding optional type hints into Python has nothing to do with compilers. It's there strictly because people flogging IDEs want it so their static analysis algorithms in their IDEs can work better at code completion. The original "unofficial" version was ported in from an IDE by the maker (I can't remember which one), and the current "official" one was created to get rid of that one with something better. I was a case of "well if you're going to do it anyway we might as well make sure that it gets done properly".

        Cython have their own type declaration system, but it's oriented just towards giving the C compiler more information. It isn't part of the Python language itself though, and Cython isn't backwards compatible with Python. This is why you add the type declarations as part of the final optimization process after you are satisfied your Cython program is working correctly.

        Personally I do however think that the Cython approach is the one most likely to produce the greatest performance improvements, although I wouldn't do it exactly the way they did. The main issue with performance is actually related to how dynamic the language is. This requires constant run time type checking and all the baggage that goes with that. It lets you do things in far fewer lines of code than most other languages, but it comes at a price. If you could selectively sacrifice some dynamic features on a method by method basis to automatically generate C extensions then you could have the best of both worlds. There are some third party add-ins such as Numba which do this for numerical processing, but I think that much better could be done if it were part of the language.

    2. vincent himpe

      Re: C and then some

      i still cannot understand why C is such a popular language. C was designed for a register based processor (pdp-11 is a register based architecture) . x86 is NOT a register based architecture. The damn thing fits like pliers on a pig. C on ARM (or 68k or powerpc) produces much better and denser code than on x86.

      1. thames

        Re: C and then some

        The reason that C is such a popular language is because it does what it's intended to do well and it's ubiquitous. It's not what I would choose if I were designing a language, but it's there and I have reconciled myself to it. I wouldn't use it for everything, but then I don't believe there is any language which is best at everything, which is why I prefer to use Python for somethings and C for others, as well as other languages where they are appropriate.

        As for x86, its assembly language is such a dog's breakfast I doubt that any reasonable language would fit it well. It's just layer upon layer of more "stuff" added in no logical or consistent fashion, SIMD operations being a good example.

      2. MacroRodent Silver badge

        Re: C and then some

        The 64-bit version of x86 actually has twice as many general-purpose registers as the PDP-11.

        In any case, the number of CPU registers is irrelevant for any language that works at higher level than macro assembler. Don't be misled by the "register" keyword in C. Modern C compilers actually ignore it, except for enforcing the restriction that you cannot take the address of a "register" variable, and it never was more than a hint to the compiler that it should give some priority for optimizing this variable.

    3. Richard Plinston

      Re: C and then some

      > If you absolutely, positively need performance above all else, then there aren't really many useful alternatives to C

      Several years ago I wrote a program in C that merged data into a postscript template, eventually to produce PDFs for such things as invoices and statements. Later, I rewrote it in Python and it ran several times _faster_ than the original.

      The reason that the C code was slow was that it used the std library string functions. All the data merging was done using strfind and strcat and these keep scanning along the string to find the null terminator. Python keeps the string length in its object and thus has much faster string handling.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021