First Python feature release under new governance model is here, complete with walrus operator (
:=) Python's new feature release – version 3.8 – has landed, the first since June 2018. Python is the third most popular language in the Redmonk language rankings and fourth in the StackOverflow developer survey – second if you do not count HTML/CSS and SQL, behind JavaScript but ahead of Java. The download page identifies 17 " …
This, and many other queries and alternate suggestions, are dealt with in the PEP:
https://www.python.org/dev/peps/pep-0572/#why-not-just-turn-existing-assignment-into-an-expression
PEPs are often quite interesting to read.
It's not just that it "could" lead to mistakes, it's just that it makes them practically inevitable, especially when you consider the C idiom of writing something like "if (x = SomeFunc()) ..." for a test of non-zero return. In fact, some variation of "if (x = 0) ..." might be the single commonest error in C code.
In fairness, this occurs not just because of the use of '=' for assignment, but because of the intersection of three design choices in C: (1) the assignment operator; (2) that C is an expression-based language, so the assignment "x = 0" is also a valid expression; and (3) the lack of a formal Boolean type, instead having the convention of 0 as False, everything else is True. Change any one of those and the most pernicious version of the trap goes away -- although the example you provided still exists.
>but "if ((x = SomeFunc()) > 0) { /* use x */ }"
This works fine but I tend to avoid such constructs because people who try to cut down keystrokes by using such forms often are tempted to leave out additional -- and what they regard as redundant -- parentheses. Most compilers will generate the same code if the assignment is given its own line so I'd only use this type of construct if 'x' was something like a register variable that I was using to second guess the compiler's optimizer.
I'd be torn about this. (Though I don't really have a dog in the fight. I've programmed in C daily for decades, but do virtually nothing in Python.)
Most C and C++ compilers (that I've tried, at any rate) will give you a warning in almost all cases where you use = in a place where == ought to have been used. But you have to have compiler warnings turned on (I'm _amazed_ at how many people don't do this, but... they don't.)
Much of my code is open source, and may be read/used by people with a range of coding skills and C knowledge. I avoid assignments within conditionals in any code that other people will see, and rarely use it even in code that will never leave my machine. It's not that an assignment within a conditional is always terrible, but it _usually_ indicates that the bit of code in question should be rewritten and made clearer. Certainly in the instance you give, where I would argue that the following is easier to read:
x = SomeFunc( );
if( x > 0) { /* use x */}
> it _usually_ indicates that the bit of code in question should be rewritten and made clearer
Very strongly agree. Good code is written once and read many times. Making code quicker to write* is often not a time-saver in the long term.
* as in less keystrokes. Constructs/libraries/etc that let you reduce the complexity of code is good.
"...Good code is written once and read many times..."
That took me a while to figure out. I have quite a bit of code that I wrote starting in 1992, and I wasn't really thinking about the possibility that I might be looking at it 27 years later, wondering what the %&!# my younger self was trying to do. Eventually, I came around to this point of view (written concerning a clever, but impenetrable algorithm):
https://github.com/Bill-Gray/lunar/blob/master/date.cpp#L103
/* Personally, I like to be able to look at a chunk of code and see what it means. It should resemble the Reformation view of the Bible: anyone can read it and witness the truth thereof. */
C++17 does the best stab at this yet with its if-statement initialisers, I think:
if(int x = someFunc(); x > 0) { /* use x */ }
So, much the same as the way that 'for' has always worked. Note that x is scoped so that it isn't visible outside the if. Unlike if you had assigned x prior to evaluating the if. If you didn't want that then you could declare the x first and assign it only within the if, but that'd be a bit silly, even if still mildly less error-prone than an old if(x = ...).
> Algol 60's assignment operator.
No. In C the equal sign (=) is an _operator_ that causes assignment in an expression. 'x = 0' is an expression. This is why assignment can be done in an if statement evaluation.
In Algol and Pascal the ":=" is part of the syntax of an assignment statement. 'x := 0' is a statement, not an expression, because ':=' is not an expression operator.
This is a fundamental difference between Pascal like languages and C like. Assignment in an 'if' can lead to errors, not only because '=' can be mistakenly written instead of '==' but because of 'short circuiting' evaluations may fail to do the intended assignment.
Why TF would anyone want to *add* this? It's a well-recognised anti-pattern in C and C++, to the extent that coding standards such as MISRA explicitly ban its use. At least the "walrus operator" is relatively harder to use accidentally, which is its main reason for being excluded from C/C++ coding standards; but in practise it really doesn't bring you anything except less-clear code.
Not only is this construct an anti-pattern in C/C++, creating different ways to do the same thing is the most fundamental anti-pattern in Python.
Sometimes I feel that a lot of people have forgotten why some of the worst elements of C are in the language, namely to allow the programmer to express optimizations that a compiler from the early 1970s might miss, but which modern compilers / interpreters have been capable of implementing for decades. There's no reason to perpetuate them in modern languages.
Yes, the original code is ugly, but there will be be better ways to tidy up it up than using this operator.
Fundamentally, the problem with assignment within an expression is that it is a one-line violation of CQS (command / query separation), which frankly is one of the most important rules you can learn for writing reliable, good quality code.
Except it doesn't. For me, the "improved" version is much harder to follow at a quick scan. One of the key features for Python is that it *didn't* have bizarrely arcane language grammar, so it was very suitable as a training language. And now they've added some. Yay. Not. If that's the best example they can find, I'd say that my most charitable assessment would be "no proven benefit, some proven negative".
If it was already in the language (like C/C++) then fine, leave it be. When it isn't already in the language, it's a ton of extra work for no reason other than one person (GvR) thinks it'd be neat. And that's certainly a compelling reason for GvR to step down, if he's got bored with the Python language in the form which made it successful and decided to drop in changes which break its reason for being successful in the first place.
In previous discussions that were similar Guido tended to follow the argument that to do nothing would cause the least problem. This is exactly the kind of the syntax that, because it's occasionally useful, it gets used all over the place where not only it isn't useful but in fact downright confusing, aka dangerous.
Seeing as it's backwards incompatible
i'm no advocate for this operator. I haven't even used python for over a year, but I think you are mistaken in your belief that it is just there for fewer lines of code and therefore makes your code look neater.
In most programming languages you could remove 20%+ of the spec and still be able to achieve what you want to do another way. However simplification of certain functions or multiple error and sanity checks can be avoided making the code much easier to write, parse, debug and less prone to unintended bugs and security risks. To a casual observer this could be interpreted as just reducing lines and making it neater.
I can think of a number of areas that this operator could be used and I would expect this to and fro about the usefulness has happened regularly over the last few months of trying to get this sanctioned. However people intimately involved in the construction of this language are convinced it is useful, but you may be more qualified in your statement that "There are circumstances that make this good practice" (maybe you missed out a IMHO tag?)
For everyone else, either use it or don't it is up to you. If you have to parse some code that uses it even though you choose not to, then either appreciate the ease of parsing it or decide that your eyes would rather be scratched out than have to try to acknowledge it and skip over any such functions that use it.
It took me quite a while to parse the lambda (=>) and namespace alias qualifier (::) in C# when I started trying to debug it.
It is explicitly banned as an anti-pattern by MISRA, who are one of the leading organisations devoted to setting software development standards. The reason is twofold: combining assignment and test in one line makes it harder to follow (as assessed by MISRA, who set the coding standard based on code being easily reviewable and maintainable by other people); and it also makes it impossible to tell whether the assignment was *genuinely* supposed to be there or whether the coder simply made a typo.
So not just IMHO, but also in the less-humble opinion of all people involved in setting a leading safety-related software development standard. Incidentally, I've also met the same rule in in-house C coding standards for automotive and aerospace software, again set by the senior engineers of those organisations.
I'd be genuinely interested to find people who can advocate for it with any argument other than "it's less lines of code", an argument which we all know (or should know) is fallacious.
MISRA aren't really relevant when talking about Python or these use cases, I would suggest. Not really a safety critical embedded system or a vehicle system. I mean even a = and == for assignment and test is an easy to miss potential security flaw.
On a higher level, the Java string comparator vs a normal == comparator has caught me out a few times. Easy to introduce bugs with that one.
That's kind of missing the point of the coding standard. It doesn't matter whether the code is going into Flappy Birds or a nuclear power station. The point is that applying the coding standard gives you code which is less likely to be wrong in the first place, and thereafter can be easily reviewed and understood by most people at a glance.
Many languages have known holes which users will frequently fall into. A coding standard is about covering over those holes. Regardless of whether the bottom of the hole just contains dogshit or a claymore mine, not putting your foot in the hole in the first place is still a good thing.
So when the architect of Python *intentionally* creates a new hole, which we have been fully aware of for the last 30 years or more (the original Lint warned about this) and furthermore was intentionally absent from the language in the first place, and he says it's for the alleged benefit of novice coders who are unable to formulate an if-else statement, I can only be sceptical about his competence not just as an architect but also as a software engineer.
It's not uncommon that people get promoted out of day-to-day technical work, and that's fine. But then accepting that you're no longer as technically adept as you used to be is essential. I'm glad he's stepped down; it's just a shame he's tracked dogshit through the office before he left.
I find it easier to follow as long as it's kept simple.
The alternatives are either nesting which makes it harder to track, or assigning everything before an if/elseif block which in some situations is wasteful if the values are assigned the return value of computationally expensive functions.
If you look at the PEP, one of Guido's justifications was that reviewing existing Python code, he was finding plenty of examples of people duplicating work (eg [ f(x) for x in x_list if f(x)>0 ]) or doing redundant work to avoid nested ifs, and he saw this as a solution for real-world problems.
The other advantage is removing what the linguist part of me would call "long-range dependencies". The closer the assignment is to its use, the easier it is to reason through the code. Particularly, mathematicians and scientists are used to reasoning through things in semantically dense formulas, and less used to the step-by-step imperative style.
I'm not sure that the syntax change will stop people writing code to avoid nested if statements.
DSLs are probably the way to go for the small group of people (relatively) who need this.
Python has a history of introducing syntactic changes only to reverse them later (backticks, print >>, map, reduce, etc.). Some stuff stays because it is genuinely useful but this can take years of refinement before it becomes standard. My initial reaction to this change is that it is, unfortunately, yet more special use syntax forced on the rest of us. It certainly won't aid readability and the improvements are marginal. But let's see what the take up is in the next few years.
When you remove the possibility of =/== substitution bugs, the main danger of C-style assignment is gone.
The main reason I see the walrus as a good thing is in list comprehensions.
Consider :
newlist = [ f(x) for x in oldlist if f(x) > 0 ]
Now imagine the that f(x) is O(^n) or O(n!) -- you've either got to take the performance hit of running it twice or expand the code out. For a scientist or mathematician (and academia is a major target audience for Python), this one line, pseudo-mathematical, pseudo-functional approach is much more readable.
My personal preference would have been for local variable declarations, eg:
newlist = [ result for x in oldlist if result > 0 where result = f(x) ]
but that's not the way they chose to go. Instead we have:
newlist = [ result for x in oldlist if (result:= f(x)) > 0 ]
...which is less restrictive than my way, and Guido set out good reasons for it. But when you look back at the examples, I think it boils down to this: a great many users of Python aren't immersed in the imperative programming style the way most pro software devs are, and the reliance on branching blocks and lines is more confusing to them as it renders code logic implicit. I'm surprised it took Guido as long as it did (v2.4) to introduce conditional expressions, actually, as that's really useful for bringing a single mathematical function into a single line.
It will be a very useful feature in list comprehensions, where it offers the potential for performance improvements as you no longer have to repeat an operation in order to use the result several times in the same expression.
Here's an example directly from PEP 572.
results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]
Without it you would have to write
results = [(x, f(x), x/f(x)) for x in input_data if f(x) > 0]
The other alternative would be to unroll the comprehension into a for loop, but comprehensions (of various types) are generally faster as they have less loop overhead. This means they are a common optimisation method, and anything which makes them even faster is generally a good thing.
I run into this type of problem on a regular basis and look forward to using it once version 3.8 percolates out into the common Linux distros.
This means they are a common optimisation method, and anything which makes them even faster is generally a good thing.
Premature optimisation is the root of all evil and a great way to waste time. I always advise people to write clear code and profile if they need more speed. Comprehensions don't necessarily loop much faster, not that Python's loops are that slow, but they do allocate less memory. That said, running nested loops through a JIT like PyPY or Cython will generally lead to C-style speeds.
>It's a well-recognised anti-pattern in C and C++
Its an assignment operator in ALGOL -- the great-grandparent of all block structured languages. It differs from an equate because its an assignment to a variable reference rather than an identity -- that is, it has a rather nice property of being unambiguous, you don't need to know the context of its use to understand its meaning.
So, with regard to "less clear code", it actually does the opposite -- its utterly unambiguous. But then people have got so used to using '=' as assignment, as identity or as a comparison that they probably don't even realize there is a difference.
So we're going to end up converging on the features of Bliss decades later? Maybe not, but the feature by feature intertwining of languages is getting more than a little confusing. Do we do exceptions here like there or like that other language? Wait, what language am I staring at?
Also, I guess the article said GvR was a proponent of the expression assignment operator. Could the opposition have been described as reactionary, wanting to keep the language pure from borrowed features? I think another major religion is having this problem right now. I think the BDFL there is named Francis?
I imagine part of the debate raged around 'common sense' and what programmers might like versus 'Python Purity'. For example "a = 1" being okay but not "a := 1", "a = b = 2" being okay but not "a := b := 2" nor "a = b := 2", though "a = ( b := 2 )" is acceptable but frowned upon.
I would have just defined "=" for both purposes, or had "=" and ":=" as synonyms so people can use whatever they choose to.
Nothing in 3.8 stands out as a must have, or even more than mildly useful. Where progress is desperately needed is in threading, which sucks on Python due to the Global Interpreter Lock. Yes there is multiprocessing, which is just about usable on Linux, but completely hopeless on forkless Windows.
Not all Python interpreters have a GIL.
CPython is the reference implementation, so you often have to do without the latest features or port them yourself (usually not hard).
Stackless, or PyPy for example can provide enormous gains on web server work and massive-multithreading.
C extensions are trickier, though usually just a header-file-swap away.
There isn't enough information for your comment to be useful.
Compared to what? CPython? Do you have benchmarks?
Edit: looking over the link above, it seems that for multi-microthreaded applications (like a raytrace or game application) PyPy is indeed worse. For tiny threads, I imagine the JIT compiler is giving a performance hit.
I also see that some frameworks like Django performs worse with PyPy.
Not so hopeless - "spawning" a process is slower and the semantic is different from forking, but it does work and if the processes are long-lived it's not usually an issue. " Note that safely forking a multithreaded process is problematic." (from https://docs.python.org/3/library/multiprocessing.html).
BTW, in 3.8 "spawn" became the default method is macOS too.
But I agree the GIL in CPython is something they have to work on.
I think threads are generally Evil. Except in Erlang, which was designed from the start to handle threads correctly and efficiently. If not Erlang, Processes is the way to go, at least when one values ones sanity and cares about not getting support calls.
Threads too often are too low-level and kinda glommed-on 'because X has it' to be conveniently managed and instrumented and yet they are, in most cases, especially Java, still hideously bloated to obscene levels of resource usage, levels where one might just use a damn process and be done with it faster and easier!
FreeRTOS, being an embedded therefore 'has-to-work'-OS, uses full processes and a mailbox system. In FreeRTOS a process is a few hundred bytes, so it really doesn't make sense to use threads.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
