Talk about a calculated RISC: If you think you can do a better job than Arm at designing CPUs, now's your chance Microprocessor designer Arm will allow chipmakers licensing its blueprints to, in certain circumstances, alter the holiest of holy scriptures: its CPU instruction set. Specifically, the Softbank-owned biz will allow customers to add custom software instructions to Armv8-M CPU cores, free of charge, starting with the Cortex-M33 …
I did not know that ARM actually prohibited adding instructions. I just assumed the vast majority of ARM licensees simply didn't want to trouble themselves. I mean, I wouldn't.
"Basically, chip makers will be encouraged to come up with libraries and APIs that access their special instructions in a standardized way, and provide these frameworks to developers who then purchase and use the system-on-chips."
Very smart. This is like the "C intrinsics" for Intel CPUs; these are a special set of headers that let you use MMX, SSE (/SSE2/SSE3/etc.) while still having portable code. You want to use "MMXdofoo(a,b,c)" in your code? OK, use the header. If you're building for Intel (and not something like a 486...), inline assembly for MMXdoofoo is put into your code (so there's no overhead from using this type of header compared to putting inline assembly in yourself.) If you're building for something else, a C loop that implements MMXdoofoo is put in. Interestingly, I saw a few years back an ARM port of the Intel intrinsics... so code using Intel intrinsics to use MMX, SSE, SSE2, SSE3 instructions would use the ARM NEON equivalents; instead of merely being able to run some MMX-using code, it could actually run it with the equivalent speedups.
As you alter the ISA, you need to be able to ensure software works across different ISA's via CPUID flags.
This will likely mean a change from the relatively simple ARM CPUID register to something resembling the x86 CPUID instructions that are pretty ugly (https://en.wikipedia.org/wiki/CPUID)... Maybe ARM has a better plan for managing variations?
> As you alter the ISA, you need to be able to ensure software works across different ISA's via CPUID flags.
No. That is not how it works at all.
Instructions, their registers, and any additional / ancillary flags are encoded into a 32-bit integer for ARM, ARM64, X86_64, SPARC, SPARC64, MIPS, PPC, PPC64, etc. For X86, instructions are at most 15-bits wide.
In other words, an instruction is a 32-bit (or up to 15-bit) integer that, when read and subsequently loaded into the fetch-decode-dispatch-execute queue, instructs the CPU to do certain pre-determined things. These instruction encoding numbers are also known as opcodes.
If you add custom instructions to the ISA, you are effectively adding new and unknown numbers (opcodes). If the CPU runs into an unknown opcode, it will raise an exception and set one of the exception registers. The kernel will read the exception register, trap, and will send SIGILL to the offending process, and the process will die and dump core.
Some CPU's do not raise an exception when attempting to decode an unknown opcode. They either ignore it, or in some cases wonder into undefined behavior land.
Most CPU's expect that the instructions that are fed to them are legal. That is, the CPU knows how to decode them.
If the CPU ignores unknown/illegal opcodes, and arbitrarily skips over them without raising an exception, your program will misbehave, often with catastrophic results.
There are CPU architectures that can fetch-decode-execute instructions for multiple ISA's, but these are very specialized, sophisticated and rare breeds of CPU's. This domain is reserved to VLIW ISA's and the CPU's that support them. And even for this case, you can't feed some arbitrary instruction opcode and expect it to decode-dispatch-execute correctly. There has to be a pre-defined and known set of encodings that the CPU knows how to handle.
An executing program can't keep calling CPUID - or its equivalent - repeatedly to determine whether the next instruction in the queue can be decoded and executed or not. The performance hit would be unacceptable. It's not just the cost of calling CPUID. You need to factor in the cost of the conditional branch associated with the decision on whether to decode-execute the next instruction, or not.
It really depends what you're doing. In many of the cases mentioned in the article, the unknown instructions are probably very manufacturer-specific, and therefore little care needs to be taken because the code will only run on chips made by that manufacturer. But there is code to check CPU IDs and change what instructions are run. Perhaps the simplest example of that is CPUs with an AES acceleration capacity. It's frequent to have a check performed at the beginning of the code to determine if the processor executing the code has acceleration instructions for AES. If it does, a branch using those instructions runs. If it doesn't, a branch that has the functionality implemented in software and compiled into traditional instructions is run instead. It's not checked immediately before running the encryption; instead it's checked at the beginning and the result determines what code is run for minimal overhead. The same could be a factor depending on what manufacturers choose to do with the ability to create new instructions.
The point of a CPU ID check is to ensure that you never get into the SIGILL situation in the first place. The executing code looks to see if the instructions are supported and if so, it uses them, otherwise it will have to use a software emulation library, using standard ARM instructions, or exit gracefully.
> The executing code looks to see if the instructions are supported and if so, it uses them, otherwise it will have to use a software emulation library [ ... ]
There is a cost associated with a conditional branch, plus a kernel trap plus a mode switch sequence - i.e. kernel-land <-> user-land <-> kernel-land.
Conditional branches themselves are very expensive. Usually between 12 and 16 cycles. Plus the side-effects of a conditional branch, namely flushing the dispatch queue. Add the cost of the trap plus the cost of two mode switches. How many clock cycles are we talking about here? Somewhere in the vicinity of ~90 cycles? Just to determine whether or not the next opcode is legal, and if it's not, punt it to the emulation layer, and then come back.
Assume an ADD %rs1, %rs2, %rd instruction with a cost of 2 cycles. You are adding an overhead of ~90 cycles just to test if a 2-cycle instruction is legal, or must be emulated in software.
There is no free lunch here. Any which way you want to implement your trap to the emulation library for the unknown opcodes, it comes with an unacceptably high cost.
Not to mention the side-effects on any possible optimizations that the compiler might try to do. That is, it becomes impossible to do them.
Isn't every instruction conditional on ARM? So you could set a flag if the hardware supports the fancy instruction, have the real instruction predicated on this flag so it will be ignored if hardware support is absent, and follow this with a BL predicated on the flag not set to do the software emulated version which will be ignored if hardware support is present.
"An executing program can't keep calling CPUID - or its equivalent - repeatedly to determine whether the next instruction in the queue can be decoded and executed or not."
I never suggested it did. I was saying that the current tidy ARM CPUID register would become more like the x86. To clarify I was comparing how this change would be handled on the ARM ISA to how this functionality was implemented within the x86 ISA - I was not considering other ISA's.
In terms of functionality, CPUID usually determines what functions will be used at runtime - i.e. either via the OS checking and loading appropriate binaries/libraries to support the optimised/general purpose functionality or the checks are done at application level if you need the improved functionality but to retain compatibility with older OS's. As long as you default to the general purpose instructions if a CPUID check fails, you should be able to support all of the different ISA variations.
The mess I was referring to was how ugly the x86 CPUID flags have become with multiple vendors and similar functions implemented in different ways, occasionally resulting in sub-optimal code being used for some functions such as SIMD extensions. I hope ARM has a more structured approach.
> [ ... ] CPUID usually determines what functions will be used at runtime [ ... ]
No, that is not what CPUID does. CPUID has no effect on the fetch-decode-dispatch-execute queue.
On Intel, CPUID returns information about the manufacturer, model, micro-architecture and stepping that uniquely identify a particular CPU model.
If you want to use the information returned by CPUID to affect program execution, you need to - at a minimum: parse the information provided by CPUID and then jump to a different place in the opcode stream to continue execution.
Parsing the return of CPUID is not free, and neither is the conditional branch associated with the jump.
"No, that is not what CPUID does. CPUID has no effect on the fetch-decode-dispatch-execute queue."
Again - I agree, its not what I am saying. I'm basing my statements on how operating systems present CPU functionality to applications - for this discussion, we don't have to consider exactly how this will be executed on the underlying CPU as the OS or compiler will take care of this for us.
"If you want to use the information returned by CPUID to affect program execution, you need to - at a minimum: parse the information provided by CPUID and then jump to a different place in the opcode stream to continue execution."
This is exactly what I am saying - your OS (typically) or application determine which path is taken at runtime based on CPUID checks. From a binary perspective, multiple code paths exist and are selected based on CPU support. You use more memory/storage for a larger binary image with this approach, but that is generally trivial compared to the speed increase you get from an optimised code path.
Regarding the cost of the CPUID, it is likely a one off evaluation to set variables. That variable is then used for your subsequent calls (i.e. using SIMD vs general purpose maths for vectors) but the cost is hidden in cache and branch prediction. There is likely to be no real performance cost for a modern CPU.
In most OS's, all of this is hidden by the OS and ignored in higher layers unless you require specific functionality not supported by the OS. From a programming perspective, this optimisation is likely handled by your compiler.
> your OS (typically) or application determine which path is taken at runtime based on CPUID checks
No, that is not true at all. The application programmer and the compiler determine the code execution path of a program. Your application - written in a high-level language - is compiled, assembled and linked to an executable binary. The compiler + assembler generate the object files, and then the link editor glues them together into an executable. The code execution path is determined by (1) the code written and (2) the compiler. Nothing else.
I know of no compiler that arbitrarily inserts CPUID instructions at random points just to figure out where the code goes next. That's just bonkers.
CPUID will fill your registers with all kinds of values. You then have to store these values someplace. That implies stores. Where do you store them, so you can access them later? You can't keep them in registers, because these registers will be clobbered - used later on during program execution. The compiler can't invent imaginary variables to arbitrarily store the return of random CPUID instructions.
If the high-level code did not call CPUID explicitly with an assembler directive, the compiler will not hallucinate that call for you, nor will it store its results anywhere. It just doesn't happen.
> Regarding the cost of the CPUID, it is likely a one off evaluation to set variables.
No, it's not. Wikipedia for CPUID.
"Likely" does not compute. There is a known cost for the CPUID instruction - just like for every other instruction - and that cost is always the same.
OK, one last try as it feels like you are deliberately either reading too much or too little into my comments, particularly given where this started as a critique of the results returned by the x86 CPUID instruction.
The CPUID instruction allows you to read information from the CPU that typically contains details about features the CPU supports. To use those features, a program (typically the OS i.e. https://github.com/torvalds/linux/blob/master/arch/x86/kernel/cpuid.c) will examine the CPU features and either enable support for that task via specific instructions or default to a general purpose function that performs the same task with the instructions found in the base ISA (albeit slower).
As an example, the FPU detection and setup code in Linux under linux/arch/x86/kernel/fpu/
Did you even read anything that I wrote?
That's an example from the kernel. It sets up the /dev/cpu/<cpu-number>/cpuid char device at boot time. What does this have to do with userland code execution path?
Do you have an example of a compiler - any compiler - arbitrarily inserting CPUID instructions in software? No you don't. So WTF are you talking about?
Compilers don't arbitrarily insert CPUID instructions in code. And neither the OS, nor the kernel change execution paths depending on CPUID. /dev/cpu/<cpu-number>/cpuid is not world-readable. So userland software can't even read it.
I don't think you understand how computers work.
I don't think you understand what is being said. You are telling us that, if you request and parse the CPU ID before every questionable instruction, it would add a lot of overhead. When you say this, you are right. When you say this, you are missing the point. The ID is not retrieved and parsed before every possible instruction. It is retrieved and parsed (only if applicable), at the beginning of execution. Then, things are updated to use the proper instructions. The overhead is only incurred once, and that is a negligible time cost. You ask for examples of things doing this. I would direct you to nearly every program that uses one of the instruction sets that are widely supported but not universally so. Using my example of AES, look at disk encryption programs. They will do exactly this. Other extra instructions are frequently used conditionally by programs such as VM hosts or anything where the marketing includes the phrase "hardware acceleration".
As an example on how this is done, consider this pseudo-assembly, with AES acceleration as the example:
#function that does encryption in hardware:
run_cpu_aes_instruction parameters
return
#function that does encryption in software:
bunch_of_normal_math_instructions parameters
return
#main function:
encryption_function = do_it_in_hardware
//Note: I've written this like a variable. I do know about computers, so I know this would be implemented by storing a number in a memory location or register. I wrote it like this for simplicity of reading
Retrieve CPUID
Parse CPUID
if (CPU can't perform hardware AES):
encryption_function = do_it_in_software
#rest of program
In this simple case, the only thing that's changed is the value of the pointer encryption_function. The rest of the code merely jumps to it. In the real world, there would be more complexity because they'd probably write the code to avoid the function-calling overhead too. But I hope you get what we're trying to explain.
One last try...
What is the purpose of the CPUID instruction? To allow an OS or application to determine the capabilities of a CPU where the base ISA has been extended through additional instructions/functionality that is not universally supported across the CPU family.
Where is this used?
Linux Kernel example for enabling some FPU functions: https://github.com/torvalds/linux/blob/master/arch/x86/kernel/cpuid.c
Application level where enabling SIMD for vector math:
http://softpixel.com/~cwright/programming/simd/cpuid.php
Do programmers need to be aware of the underlying use of CPUID? In most cases no. At compile time, the compiler will produce binaries that support both general purpose or optimised code or rely on system libraries that contain similar functions.
For many of the other points you raised, I don't understand why you are raising them as I have either repeatedly said that they are not what CPUID is used for (i.e. Compilers arbitrarily inserting CPUID instructions, executing CPUID instructions repeatedly for conditional code or that "likely" is not how code works when I am trying to summarise general cases - while the OS is typically used to identify CPU functionality, there are cases where it is done at application level for vector math etc). With the rest of your descriptions of how a computer works, they largely depend on your assumptions that I have repeatedly rejected.
Maybe a better question would be how do you see ARM chips with differing instruction sets (as per the article) being handled in software? How will the software stack know what instructions can be run on which manufacturers ARM ISA's?
The 68000 series provided a trap (interrupt) for illegal instructions (i.e. the decoder did not recognise it) and was quite commonly used to provide customised instructions. The trap, IIRC, actually passed the offending instruction to the handler so it was possible to implement numerous interesting instructions in software.
This was not only possible, but positively marketed by (then) Motorola SPS (since rebranded to Freeescale and thence NXP).
This was,of course, not used in that many instances compared to all possible implementations using 68k series parts, but I have seen a single illegal instruction used to make a multiple instruction sequence atomic on the 68k series.
Apple used illegal instructions to call system subroutines in the original MacOS, including many simple library things such as NewPtr() or opening and closing and reading disk files. Every M68000 instruction of the form 0xAnnn is an illegal instruction and Apple eventually used most of them.
So what about the AMX instructions in Apple's A13? Are those not custom, but rather Apple just implemented them before ARM even announced them?
I thought the deal was that if you licensed the cores from ARM you could only tweak them not add to them. If you had an architectural license like Apple and design your own cores, you had more freedom - you still had to implement the ARM ISA but I didn't think it stopped you from 'enhancing' the ISA with your own instructions.
My understanding is that ARM ISA's have coprocessor instructions:
- COPROCESSOR DATA XFER (LDC/SDC/LDC2/STC2)
- COPROCESSOR DATA OP (CDP/CDP2)
- COPROCESSOR REG XFER (MRC/MCR/MRC2/MCR2/MCRR/MRCC/MCRR2/MRCC2/FMDRR))
These allow you control multiple coprocessors but you don't get to modify the ARM ISA itself. There maybe additional instructions that I haven't included, it's been a few years since I looked into this...
The management of the coprocessors leads to a small amount of overhead that is likely becoming an issue as clock speeds go past 2GHz (likely lower) as the interface CPU and coprocessor have to check availability via a common interface to see if the coprocessor is available to take more instructions. Moving that to the ISA would allow you to avoid or reduce the check and incorporate sufficient resources to handle synchronisation and concurrency without impacting other coprocessors or the CPU.
Until now ARM absolutely did forbid licensees from altering the ISA in any way whatsoever. I believe the tagged pointer instructions were indeed added at the request of Apple, but you’ll find them documented in the ARMv8.4-A (I think) manual for all ARM licensees to use. ARM themselves haven’t shipped any CPUs implementing them, but then they also haven’t yet shipped any cores implementing SVE and that was announced and documented three years ago.
My first thought was How are they going to know I've added any instructions?
They've only tested the samples I submitted to them to make sure unrecognised opcodes raise exceptions as expected. What stops me from changing the photomasks for the actual production run?
If there's some benefit to be had from adding unapproved enhancements (and if there wasn't, why would they be so keen for you not to?) then it can be weighed against the cost of the scam.
This is like the "C intrinsics" for Intel CPUs; these are a special set of headers that let you use MMX, SSE (/SSE2/SSE3/etc.) while still having portable code.
Motorola did this earlier with AltiVec on PowerPC. The difference was that, whilst Intel screwed around with MMX, SSE, SSE2, etc, Motorola got AltiVec right first time. Thus a ton of software got written to use AltiVec, whilst it took ages for SSE to settle down. Even today it’s far easier to use Intel’s MKL/IPP libraries, which provide routine for useful functions optimised for every variation of Intel chip.
If required, they already can - the CPUID function has an implementor code assigned. 0x61 is Apple.
However, adding additional instructions and not compiling a general purpose code path applicable for the base architecture would seem to be a path strewn with regrets.
You have always been able to a secondary processor alongside the ARM core.
So the 'prohibition' consisted of two parts: you couldn't modify the ARM core, and the ARM core provided no means of adding instructions.
Also, FWIW, the quotation in the article is that you can use RTL -- register transfer level -- to add your instructions to the new interface. That means you'll not necessarily have to use "logic gates" as the author has suggested.
"Embrace, extend, and extinguish", also known as "embrace, extend, and exterminate"
Hmmm Microsoft, seen this one before
https://en.wikipedia.org/wiki/Embrace%2C_extend%2C_and_extinguish
Softbank has some interesting holdings and is changing the way Arm does stuff,
it's what happens when money takes over a successful tech company.
This might be really good for RISC-V
"This would turn China into the centre of development for RISC-V and could quickly lead to Intel and ARM becoming also rans."
I think you're a little optimistic - RISC-V and MIPS are not too dissimilar in terms of being freely available CPU designs with MIPS having the performance advantage and RISC-V having the tooling advantage.
If you integrated many of the performance enhancements found in the MIPS design into RISC-V, you would likely be able to compete with a current high-end ARM processor but it would likely take 5+ years to get the design validated and on the market by which point your competitors are another 5 years ahead. Plus you need to develop your tooling at the same time AND make some money.
That's not to say RISC-V will not be successful, I think it will - just not to the point where it can challenge ARM let alone Intel or AMD.
If you were instead to look at ARM/Intel/AMD making mistakes that cost them their positions in performance or market share, then there is a much greater opportunity there. Advanced CPU designs are complex and expensive, particularly when you make mistakes.
The M33 etc more likely to be in a washing machine, remote controller, clock plus weather station, hardly even in a TV or even a router/modem.
The Neural Networks thus seems like a bad example. More likely an instruction related to I/O, perhaps analogue via ADC or DAC or for driving basic dot matrix LCD or VFD connected directly.
I think they are looking at an M33-class CPU to manage communications/IO/management of a more specialised coprocessor.
Without wanting to win buzzword bingo, the assumption is that the brave new world of AI and IOT will use sensors that do some local data processing before passing the data back to a cloud based mother ship. The M33 has the IO/sensor side handled through existing applications so I guess its how they plan to manage the AI pre-processing and power management to keep it competitive.
This post has been deleted by its author
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
