The quest for faster Python: Pyston returns to open source, Facebook releases Cinder, or should devs just use PyPy? Facebook has released Cinder, used internally in Instagram to improve Python performance, while another faster Python, called Pyston, has released version 2.2 and made the project open source (again). Python is the world's second most popular programming language (after JavaScript) according to some surveys; but it is by no …
Many years ago I thought so too, and wrote an exploratory project in Python (a compiler, of sorts). It was so excruciatingly slow that I re-wrote it in another cross-platform dynamic interpreted language that happened to have a good JIT (Racket scheme) and was completely happy ever after. Python is OK for what it is, and as a general purpose wrapper for fast C code it's quite excellent, but it has some dynamisms that are particularly egregious for attempts to go fast, after the fact.
As this article states, optimization after choosing Python is it's own realm of hell. Imagine all the insane architecture hacks thrown into the system before actually concluding that writing your own Python runtime is the next logical step.
I've seen companies build expansive nightmarish Python optimization architectures for back-end code that would be just a few pages of ordinary Java, C++, or even Go.
let's run an interpreted language on a virtual machine that runs on another virtual machine and shoehorn that onto another virtual machine that runs on an operating system... that may sit in another vm ...
Gimme back my 8k floating point basic interpreter that ran directly from ROM. Those were the days... i'm getting old. we need 5 billion transistors and 500 megabyte of code to print hello world.
Slowness is relative and poor programming can create exceptionally slow code. Multi ms hops to a DB on another server, after already spent multi ms searching a cache only to miss, then waiting for the DB to curn out a result and send it back to you is probably always going to be slower than the python code that you're executing.
"Python performance is not always a problem, since library developers have followed Van Rossum's advice and written performance-critical code in C. "
If you absolutely, positively need performance above all else, then there aren't really many useful alternatives to C. The accepted practice is to write the the application in Python and to then benchmark it. If performance meets your targets, then you're done.
If it doesn't meet your targets then use the benchmarks to guide your optimizations. Optimize in Python first. For those bits that still don't meet your target, then write them is C.
Python is designed to interface well to C. The reason why CPython persists in remaining the dominant implementation is because it interfaces so well to C. If you are doing something typical in Python, chances are there is a widely used library which does it, and if performance is critical it will be written in C.
There is also increasing use of Cython for these sorts of optimizations. Cython is an extended version of Python which translates Python to C which you then compile as a C extension. It's a good answer for those who feel allergic to C.
then there aren't really many useful alternatives to C.
Depending on the domain, you may find plenty of Fortran and Lisp fans out there… ;-)
PyPy does a fantastic job of optimising Python byte code and has over time achieved quite astonishing results. Unfortunately it doesn't play well with Cython, which is a great alternative for those occasional hotspots in libraries.
But the general drive to shoe-horning types in Python so that compilers can optimise should be resisted: code becomes bloated and unreadable and thus difficult to maintain. And then it turns out that YAGNI but it ticks a load of boxes, which is great for the project managers.
Where Python does need to improve is on multiple CPUs and, due to the way Python works, this is hard™ but there have been significant improvements and investments, due to the ecosystem built around Pandas and Numpy.
The main driving force behind adding optional type hints into Python has nothing to do with compilers. It's there strictly because people flogging IDEs want it so their static analysis algorithms in their IDEs can work better at code completion. The original "unofficial" version was ported in from an IDE by the maker (I can't remember which one), and the current "official" one was created to get rid of that one with something better. I was a case of "well if you're going to do it anyway we might as well make sure that it gets done properly".
Cython have their own type declaration system, but it's oriented just towards giving the C compiler more information. It isn't part of the Python language itself though, and Cython isn't backwards compatible with Python. This is why you add the type declarations as part of the final optimization process after you are satisfied your Cython program is working correctly.
Personally I do however think that the Cython approach is the one most likely to produce the greatest performance improvements, although I wouldn't do it exactly the way they did. The main issue with performance is actually related to how dynamic the language is. This requires constant run time type checking and all the baggage that goes with that. It lets you do things in far fewer lines of code than most other languages, but it comes at a price. If you could selectively sacrifice some dynamic features on a method by method basis to automatically generate C extensions then you could have the best of both worlds. There are some third party add-ins such as Numba which do this for numerical processing, but I think that much better could be done if it were part of the language.
i still cannot understand why C is such a popular language. C was designed for a register based processor (pdp-11 is a register based architecture) . x86 is NOT a register based architecture. The damn thing fits like pliers on a pig. C on ARM (or 68k or powerpc) produces much better and denser code than on x86.
The reason that C is such a popular language is because it does what it's intended to do well and it's ubiquitous. It's not what I would choose if I were designing a language, but it's there and I have reconciled myself to it. I wouldn't use it for everything, but then I don't believe there is any language which is best at everything, which is why I prefer to use Python for somethings and C for others, as well as other languages where they are appropriate.
As for x86, its assembly language is such a dog's breakfast I doubt that any reasonable language would fit it well. It's just layer upon layer of more "stuff" added in no logical or consistent fashion, SIMD operations being a good example.
The 64-bit version of x86 actually has twice as many general-purpose registers as the PDP-11.
In any case, the number of CPU registers is irrelevant for any language that works at higher level than macro assembler. Don't be misled by the "register" keyword in C. Modern C compilers actually ignore it, except for enforcing the restriction that you cannot take the address of a "register" variable, and it never was more than a hint to the compiler that it should give some priority for optimizing this variable.
> If you absolutely, positively need performance above all else, then there aren't really many useful alternatives to C
Several years ago I wrote a program in C that merged data into a postscript template, eventually to produce PDFs for such things as invoices and statements. Later, I rewrote it in Python and it ran several times _faster_ than the original.
The reason that the C code was slow was that it used the std library string functions. All the data merging was done using strfind and strcat and these keep scanning along the string to find the null terminator. Python keeps the string length in its object and thus has much faster string handling.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
