Google's Dart language soon won't take null for an answer When the third major release of the Dart programming language debuts in mid-2023, null values will no longer be allowed where they're not expected. Null in this context is an assignment value indicating the absence of a value or referenced object. Null references date back to around 1964 when British computer scientist Tony …
Its actually useful having a value that represents 'no value' provided you are prepared to use it as such. There's lots of reasons why you might want to have a variable holding a location in memory and there are going to be times when it won't have anything useful in it. If you assume you can't do this then you'll just have to create with every pointer the thing its pointing to; it won't necessarily have valid data, of course, so all you've done is move the problem one level. Having a concept like floating point's NaN is very useful in this situation.
The present situation is only undesirable because we insist on trying to give an empty pointer a value that resolves to something tangible like zero. This invites people to tinker with addresses as if they're integers. It also resolves to a location in memory which prevents the processor trapping out with a data segment access fault -- the code blithely collects what's in that location and makes use of it to cause a problem.
Back before virtual memory it might have made sense to start counting addresses at 0, so if NULL equals zero then it would reference an actual memory address. Such systems should have made NULL equal to -1 or something else outside of the possible memory range, though I guess back in the days of 16 bit systems you might not have any invalid pointer values.
The real shame is that anything supporting virtual memory ever both 1) mapped 0 into the address space and 2) had NULL equal to zero. The problems with that should have been obvious to the designers of the system (hey Sun architects, I'm looking at you!)
The zero page hasn't been mapped for decades.
It traps on all current OS that support virtual memory - which is everything you could run Dart on.
And yes, it's the whole page. Deref of a null pointer rarely actually accesses the exact value zero as only one struct field can be at zero.
HP-UX allowed you to select whether an executable would have no zero page so all dereferences of NULL result in a segv or whether you wanted a zero page of read only zeros.
The classic strcpy function required a dereferenced NULL ptr to return a zero byte.
OK you said "for decades" :-)
I've never thought to check on Linux - Duh better go take a look just for interests sake.
HP-UX did that because SunOS mapped a page of 0s at 0x0, so NULL pointers could be dereferenced freely. Since SunOS was so popular in the 80s and early 90s when Unix really began to spread, a ton of open source and commercial software was developed on it.
Hence HP-UX needed the option to map a page of 0s because people kept saying it was "broken" since ported software was randomly crashing when an unfixed NULL dereference bug would surface!
Interesting aside. SUNOS was the name of the lower level OS on top of which HP-UX was originally implemented. The original version was based of SYS III and neither SYS III, SYS V or 4.2BSD had support for multiple processors and the original HP9000 500 series was a multiprocessor platform. Their SUNOS also formed the lower levels of the BASIC environment which was available on the 520.
I guess when Sun Microsystems was formed and released their SunOS, HP dropped using the name since it was never a "product" but a lower level on top of which the product was built.
The real shame is that anything supporting virtual memory ever both 1) mapped 0 into the address space and 2) had NULL equal to zero.
Neither is true. The page at address zero has long been unmapped to cause a fault(*) and on Pr1me V mode machines the native null pointer was 7777/0(†) and much hackery was needed to implement C. (IX mode was invented to properly support C.)
(*) Back on PDP-11/70s address 0 was mapped which led to the joke that Unix error code E2ps meant dereferencing NULL, because that's what you got if you fed NULL to printf("%s").
(†) That means segment 7777 (octal), offset 0. Actually segment 7777 was completely unmapped so 7777/n faulted for any n.
The OS configures the CPU to trap on access to the zero page, which is why a badly-behaved application accessing a null pointer either throws an exception or is safely terminated by the OS, instead of corrupting memory. (Depending on language & implementation.)
So I don't actually understand what Dart is changing. What did it do before?
And addresses are integers, though there's certainly an argument for requiring an explicit cast to do pointer arithmetic. It is after all fundamentally necessary for serialisation and to implement containers, so cannot be prohibited.
I believe some languages already do this.
They may be convertible to integers, but they cannot always be manipulated as such. For example:
"The bottom two bits on an architecture that only supports 4 byte alignment may be used to store other pointer-related information"
That strikes me as a horrible habit born of bad practice even in the costly memory days. An address should be an address, so these two bits should for safety be set to zero. Masses of similar "space saving" kludges abound (e.g. the storing of small files in the MS MFT itself) and they are all indicative of sloppy engineering thinking -- trying to be "cunning" without anticipating the possible consequences (and quite possibly also due the curent level of abstraction from the actual machine inhibiting thought in sufficient detail).
Semantically, an address is an address as long as the format the code uses and the format the CPU uses are compatible. That means, if they agree to interpret it as a 30-bit (or 62-bit or 14-bit or n-2-bit) value which is to be shifted left by two to get the location, packed with a two-bit field which could be CPU-parsed or could be internal to the program, then that is an acceptable method to store an address. It's up to the user to understand and not mess with the values they're using, and making every pointer have two unused bits, though it will prevent one class of errors, will not automatically fix things. It will not, for example, fix a user who doesn't understand that they can't use nonaligned addresses and tries to do manual pointer arithmetic without taking that into account.
It's changing the type system. Instead of relying on documentation to know whether a value can be null or not, the type will be explicit about whether it can be null or not, and if it can be null then you'll have to handle the null vs non-null cases or get a compile-time error.
> And addresses are integers
I think that's a semantically, um, null statement. If you're talking about how addresses happen to be represented/stored digitally, well, then a floating-point number, a character, ... also "are integers" - if you choose to interpret the corresponding string of bits as such. Semantically, an address is a datum expressing the location of some other data in memory. The question is about how a given OS/language allow you to access and manipulate addresses.
In practice memory tends to be modelled (and possibly, but not necessarily, or even usually, implemented) as a contiguous serial array of locations, so that at the language level addresses map, not so much to "integers", as to integer offsets.
I've not looked into the Dart implementation, but null safety doesn't necessarily mean no nulls. There are different ways of handling things that can be optional or required.
1. Kotlin, for example with it's "?" on type declaration, allows null. It requires the code to say whether a variable can be null. "val name: String?" allows null to be assigned, whereas "val name: String" (no ?) doesn't allow null to be assigned. There are plenty of operators, pattern-matching, etc in the language to allow safe dereferencing. The point is that the code MUST explicitly handle null; NPE's are not possible (with caveats when interoperating with Java).
2. Functional languages have the explicit Option/Optional type: here there is no null at all; the code must unwrap the option values ("none" means no value, or its "some" + value. Pattern matching makes this elegant. Again the code must handle the "none" case explicitly (or pass it on. Again, very good language support to handle Option[al].
Specifying in code if a values is required or not is very nice (my experience is Kotlin).
"NULLs have their place in almost anything exept database tables."
... the reverse.
In reality most fields in database rows are nulls: No data. Non-nullable fields are typically table primary key and additional index fields, everything else can and often will be nulls.
They are OK, up until you use C/C++ libraries to interface with the DB.
Then you are faced with either:
1. Custom code paths to return the NULLs out of band with the actual data buffers
2. Return a struct for every data value containing a bool representing Null & the data buffer
3. Use an actual binary code that is normally mapped to data to map to Null (e.g. -1, "\0"), etc.
All three options stink, can introduce hard-to-find bugs, and waste a bunch of performance as you can no longer have simple, consistent, buffers of ints/floats/text info but have to search for Null-ness or holes, etc.
Note, even higher level languages like Python interface with DBs through C/C++ libraries so suffer in various ways from the 1-3 choice above.
That’s a problem with the language you are using, not the DB. If you don’t want nulls reaching your code it’s trivial to write your queries so they don’t return null.
And in real implementations there will always be data that is unknown, hence Null, how else would you distinguish between a real value and an unknown value? Using some “placeholder” value like -1 for integers? What if your integer can be negative? For strings?
You would have to make assumptions about what value will be stored and handle them individually for each field, and that, imho, is more error prone than a consistent handling of null values.
"what is wrong with nulls in db tables?"
In my experience, the problem is not directly technical, but in any system that says "The database will take a null, so I won't bother to make the user specify a valid value here". Someone comes back later and finds that the database has missing data replaced by a bunch of nulls and has to figure out what should be done with each bit of stuff that, in many cases, was data they expected to have at one point.
Nothing, but they're not the same thing as NULLs in languages. NULL in DB speak means 'unknown', it's not a value.
So in a DB anything compared to NULL will be false, even NULL. (Hence, 'IS NULL', meanling 'is Unknown').
If that's apropriate for your data model, then happy days, if not, then make it non nullable.
An example might be an HR system, you may want to enter details of a new starter that you don't have full details for. In general you'll normalise and group the data into logical tables so there simply won't be an entry in most cases, however there will become a point where normalising any further becomes silly, and you just have a nullable field to be filled in later. (e.g. a postcode on an address).
> They're indicative that the model has not been properly normalised
In practice, very few databases are fully normalised, usually for well thought out reasons.
> and that consquently many projections will suffer the problem of three-valued logic.
Three valued logic is not a problem, as long as you're expecting it.
JS (and therefore TypeScript which is just JavaScript with a type checking front end) has both null and undefined. undefined is the value of a variable is if it hasn't been assigned to anything. However, a variable can also be explicitly assigned a value of undefined. For that reason null and undefined have overlapping behavior.
A better functionality undefined might have been a runtime error whenever the r.h.s. value of an assignment was undefined, e.g. a variable with undefined as it's value. (Typescript would detect that during static checking, which it does reasonable well now to the extent possible).
(different AC… or Alzheimer is catching up with me)
> JS has both null and undefined.
For some reason I really like JavaScript having both null and undefined (they're not the same thing). It's not like it's terribly useful, but it does appeal to my sense of aesthetics.
The problem you have with having 'undefined' and 'non-assigned' values is that you need an efficient way of managing them at run time. You need essentially a two dimensional variable, one that has both a value and a property. Its difficult to efficiently implement this; memory accesses tend to be either Intel style, offset from a segment base register or RISC style where they're offset from a general purpose register. No amount of fiddling with the source could cope with overflow (for example), there's always going to be ways to devise illegal pointers, especially for the maliciously inclined, so we really need a built in trap mechanism for illegal addresses. This could be done by taking over the unaligned access trap (in a RISC) but this would probably cause a problem because somewhere in the volumes of legacy code someone's bound to be using the trap to manage a packed variable.
The answer's definitely in the hardware. The memory manager has to detect and trap on access to defined values. (You've then just got to hope that the programmers actually use the trap!)
"Which does make one wonder why Dart?"
That's an easy question: Vendor lock-in. Google wants people who know how to operate only in Google playground and vendor spesific language is a major tool for that.
See: MSFT, Azure and C#.
No other reasons exist.
That's what it comes down to. A snobbish attitude to languages is the reason we're blighted with multiple memory overflow vulnerabilities. We were offered "safer" languages but we simply have to be big boys and use the dangerous stuff because it makes us feel clever.
And ironically the smart people are the ones who don't paint themselves into a corner trying to defend bad languages. I've written C and C++ for close to 30 years and know its flaws but it's amazing to see the C/C++ defense force leap out of the woodwork to defend it if I point them out.
"We were offered "safer" languages but we simply have to be big boys and use the dangerous stuff "
Logic is correct so far, but reasoning goes waywire after that: Safe tools are typically useless tools.
Imagine a safe knife. Or even better, a safe axe. Both would be totally useless for their intented purpose.
Reminds me of a joke I heard a long time ago about a lumberjack buying a chainsaw and then complaining to the store manager that it takes longer to cut a cord a wood than when he used his hand saw. That is until the manager fired it up and the lumber jack was surprised at the noise.
A reference in C++ is just syntactic sugar for a pointer and has its own limitations and booby traps. My favorite is seeing functions that return a reference e.g. "std::string &getName()" and inside the function there is a branch that returns a reference to a temporary value. The auto keyword is also horribly designed for references leading to all sorts of fun where code thinks they're changing values on a reference but in fact they're changing values on a copy. Not to mention stale references where the original object has been destroyed and since it's just pointer sugar, now you're pointing at garbage.
“ True, but it can accidentally….”
True. But what you describe is undefined behaviour And therefore outside the scope of the language.
I’m not saying what you point out doesn’t matter but we’re starting to move further away from the argument for and against a null pointer. What you describe is programmer error/incompetence.
It probably does represent a language inadequacy but allowing that inadequacy to manifest itself is a programmer error.
I’m not defending C++ by the way - I use it but like many others, I’m well aware of its shortcomings including the dreaded “undefined behaviour” thing
Just doing something like this will crash -
const std::string &foo() { return std::string(); }
auto x = foo();
Some function like a getter returns a reference to a temporary object and of course it's gone by the time the caller gets the reference. And since "auto x = foo()" makes a copy of the reference it blows up.
And the amount of times I've seen code errors on references is unreal. Some compilers might warn on this simple example. It's not hard to make examples where they won't complain.
References in C++ are a poor man's pointer. You can't use them to indicate null node pointers in a tree structure, for example.
References also make the meaning of C++ source code obscure. In C, the syntax makes it clear whether a passed parameter is by value or by pointer. So when debugging you know when an argument can't be overwritten by the called function. Not so in C++, where you've got to look at the called function's signature to know this. As if debugging isn't difficult enough anyway.
I think in modern C++ the best solution would be to use optional type to wrap things like tree nodes, e.g. getParent() returns std::optional<TreeNode &>, but it doesn't do anything for the vast swathe of code out there which returns a pointer with NULL or nullptr to signify nothing.
Also references have no lifetime guarantees so code can hold a reference to a node in the tree even after it was deleted. In a garbage collected language this wouldn't be a worry. Nor would it be a worry in a language like Rust which would force lifetimes to be defined before it even compiles.
There is nothing poor about references, by using references you can skip the extra code like testing for null that you need for dealing with pointers.
References don't have nulls, multiple levels of indirection, reassigning, or arithmetic - you are really dealing with the referenced variable itself. If the variable is constant, so is the reference.
If you don't need full-fat pointers, use a reference. Obviously in a linked list with elements which could be null, you wouldn't use references.
TypeScript supports non-nullable and non-undefined types with the --strictNullChecks option. You can still break it by assigning a null of "any" type; but you can also prevent that with the ESLint rule "no-explicit-any". In theory this means TypeScript is perfectly type-safe; but in practice many developers don't use these checks; or don't enforce them on CI builds.
"He considered them his "billion dollar mistake" for the amount of time and money they've cost in error repairs – an issue to this day."
... and he's an idiot. Null specifially means "no value" and *has to exist* as a concept. Specifially a database would be useless if every record and every field must have value, even if it's meaningless.
Using null as a pointer value is the only way to indicate uninitiated pointer, but the fail is the pointer itself, not the null. It would be trivial to zero (or some other fixed value) all pointers but it would even more wrong than a pointer with no value.
So: It's the pointer, not the null: What kind of language allows null references anyway?
Null specifially means "no value" and *has to exist* as a concept.
Modern (hell, 90s) type theory would subtly disagree with you. Safe languages use sum (union) types instead, one or more arms of the type indicating various forms of missing/invalid data.
Using null as a pointer value is the only way to indicate uninitiated pointer
Uninitialised (presuming that's what you mean by uninitiated) pointers could have any old random pattern of bits in them (and do for C stack storage). NULL as a valid(ish) pointer value is a kludge that was invented before sum types were properly understood.
"Specifially a database would be useless if every record and every field must have value, even if it's meaningless."
I disagree. Databases with lots of nulls often have a lot less meaning than ones where the designer thought of the problem and fixed the representation of the data so they would not need null. When I see a null in a row, what does that mean? Does it mean "answer unknown", "answer not written down", "there is no answer", "answer incompatible with the format", "not applicable to this data", or something else? These are different cases. If I request all records with a value greater than 100, then results that lack a value because they're not applicable probably shouldn't be included, but unknown values could be desired because they're possibly over 100. Differing values allows the query to specify exactly how the different kinds of exceptions should be handled. Throwing everything in as null fails to provide that power. If you're willing to accept data not existing, you should probably have a way to handle the reasons it doesn't exist.
> When I see a null in a row, what does that mean? Does it mean "answer unknown", "answer not written down", "there is no answer", "answer incompatible with the format", "not applicable to this data", or something else?
To be fair, if the answer to those questions is important to you, then you should probably add that information to your model. If it's not important, a null will do just fine.
If I'm designing the database, a lot of my columns are restricted to being not null for exactly this reason. The problem is when people design a system without considering these things and whether nulls will be a problem, and the dataset arrives with a lot of missing data and no information about why it's missing. Someone assigned to do something with the system has to go find someone who can answer a question like "What do we do with user accounts that exist in the database as verified but have a null email address", when a proper system could have made such a situation impossible if that was desired. Given how many of the questions are answered with "I don't know" and that some of them have important consequences, there are reasons to want that fixed.
I appreciate your problem, but at the end of the day that is a database design / requirements competence issue. It's not going to stop anybody from setting your non nullable field to a blank string or to the word "null" or to "xxxx" or stuff like that. I've seen that happen. Address the root cause.
I do try to get things fixed at all levels, such as having a form that enforces restrictions on provided values or having larger constraints on database types than just not being null. The problem is not solved by blocking nulls, but it is enabled by allowing them unless there is a plan (and in most cases, there is not). The database-specific benefit of not allowing nulls is usually to deal with automatically-imported data which was imported incorrectly. When someone tries it and doesn't prepare their data, I'd much prefer that their database transaction is rejected and they have to fix it or ask for my help than that it gets accepted and someone has to clean up after the mess when it's discovered six months later.
Databases with lots of nulls often have a lot less meaning than ones where the designer thought of the problem and fixed the representation of the data so they would not need null. When I see a null in a row, what does that mean?
You describe a perfect excample for boyce codd normalization. Perfect normalization results in a missed table row and no nulls. In my previous job I had this normalization wizard that fixed our 17Tb 120 tables 25 year legacy into a 12Tb 320 tables a with 300% query performance. Developers over the years had just added collumns to existing tables in stead of making new tables with foreign keys. Joining tales with loose connections and lots of ORs kills performance and waste memory.
https://en.m.wikipedia.org/wiki/Boyce%E2%80%93Codd_normal_form
Writing a load of C++ and running under a DOS 386 extender to enable protected mode. Sometimes during testing it would just jump to the start of the program again. Figured out that calling a virtual function on a NULL pointer would send EIP to 0x00000000 which was the top of main.
Then found the switch on the DOS extender to not map the zero page. Oh how we laughed.
Fast forward a few years. On Win32 and a colleague had written: catch (...) { }
Turns out that this catches access violations too. That guy's speciality was creating bugs that took days to track down. That and farting. And sexual harassment. I don't miss him.
All popular operating systems will stop a program that tries to access a NULL pointer. Completely deterministic, debuggable and secure behavior (an attacker can't subvert a stopped program.
Much more dangerous are random values inside an invalid pointer(because the pointer itself is on the stack), as they cannot be detected (efficiently in time and space) and open the castle to all sorts of attack opportunities. Same is true for a pointer which points to freed/deleted memory.
So, a memory safe language should ensure (smart) pointers are either valid or null. Just like Java, Go, Swift, Sappeur already do.
Memory Safety needs several other properties, such as
+ Array Index Check
+ Stack Overflow Check
+ Multithreading Safety of heap, (arc or GC) pointers
+ Strict type conversion rules (e.g. pointer to car can never be converted to pointer to aircraft)
"Commercial users of the Sappeur Compiler need a written license per developer using the compiler. Cost of license per user is 300 Euro per developer-user." ... plus contacting a server for licence checks, so can it work offline? [note: it's free for non-commercial projects (it isn't defined what this actually means)]
You know, this isn't 1990 any more. Go is an open source language, C++ has freely available open source complete toolchains, and Dart is on GitHub. There are numerous Java compilers available too.
Sappeur - no source, no ability to audit the code, and it's €300 per developer compared to those languages you're claiming to be better than. [note: it's free for non commercial projects]
I wish you luck, but can't see this being embraced in a hurry in a world that's already saturated with programming languages...
"I do not believe in communism."
Cool. That's related to the thing you replied to how, exactly? If you're equating free open source languages to communism (either the concept or the execution), you are demonstrating a lack of understanding of both concepts. Their complaint wasn't just that it costs money, although that's been a problem for language creators repeatedly, but also the lack of the other benefits of open source languages such as the auditing of the source for security vulnerabilities or the decreased need to deal with legal factors such as defining who needs a license in a company of multiple developers and multiple non-developer people with different jobs, using multiple systems for build and test. These are not intractable problems, but preferring open source code is in no way communist.
Derefing null is a problem but one Java doesn't face. They have null and throw NPE if you try to dereference it.
optional I find annoying, it's just a pointer to a null and I have seen
opt.get().doThing() cause npes just as often as codes that allows returning null to mean "nothing here yet" .
I don't see why null matterers if you have Java's level of control over pointers, I.É. You can't find the value, you can just compare it to null. If your language can convert null to zero and play with it. Fix that first: then null is a better optimization that having one more level of Indirection for every pointer.
In the IT sphere, you are more or less right. A NULL pointer causes a deterministic, debuggable stop of program. Service will be stopped for an hour or so. Customer uses cash to buy a cup of coffee and comes again later to run the service again.
In other spheres such as auto, rail, aerospace and medical, you cannot just "stop the process and debug it". Imagine getting a NULL pointer inside an ABS brake algorithm, while the driver hits the brake pedal. How would you handle this as a software engineer ? Restart the process and maybe get another NULL pointer a few milliseconds later ? No, you must prove there will never be a NULL. One proper way to do this is to have the type system+compiler assuring this.
There are cases where a NULL simply cannot be handled. Certain pointers MUST be non NULL, or the algorithm will fail. But it MUST be active at a very high availability rate (one instance in 1 million years or similar). Because it is a flight control system and cannot simply be turned off or restarted. If the compiler can guarantee that for the software, the organization can put more effort on the hardware side (which has physics- and chemistry-related failure rates).
Some high performance aircraft are unstable By Design and you cannot "use direct law" in case the software "has issues".
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
