Re: FPGA?
I'll chip in; many explanations don't ever cover the "realities" of such devices, which is often what is needed to understand them better.
The basics of an FPGA are that they're chips that contain a lot of logic gates, clock distribution circuitry, programmable look up tables for those logic gate, and the ability to join gates up to other gates in a reasonably arbitrary way. If you ever did any digital electronics using discrete logic chips like a 7400 (a chip that has 4 2-input NAND gates), then an FPGA allows you implement that logic circuit by programming its lookup tables (e.g. to give them the same truth table as a NAND) and defining the interconnect between gates that you want (which is like building the printed circuit board to hold your 74-series chips). The words are all there in the name: "Field Programmable" (the device is programmable and you don't need a lab to do it) "Gate Array" (a large number of logic gates that can be programmed).
This sort of chip emerged from earlier more primitive concepts, where the circuit was less arbitrary, couldn't be as complex. Programmable logic has long been a way for electronics engineers to build circuits out of fewer components with more flexibility than is achieved by soldering discrete 74-series fixed-function chips to a board that you can't readily change. FPGAs emerged at the larger end of this market segment.
In principle, you can recreate any logic circuit you like (within boundaries). And as a CPU is just a large amount of logic on a chip, if you have a big enough FPGA you can emulate that CPU, down to the last transistor (kinda).
Early Problems
A major consequences of this approach was 1) cost, 2) speed, 3) thermals, 4) programming difficulties.
If you bought a chip capable of emulating 1000 logic gates, but used only 10, you were throwing money down the plughole.
The designers of the chip had to account for every conceivable clock configuration, and so the logic circuit you can implement can't be clocked very quickly.
By having far more transistors than was strictly necessary, there's power being dissipated for no good purpose.
And, as they got more complex, programming became harder and harder. And they are not infinite; use too small an FPGA for the circuit you want, start again on the board design. That's really news, late in a project. In the bad old days, if you had a large project you could also run into the problem of the FPGA build tools failing to complete place and route; it was purely a matter of random chance as to whether the initial "guess" it would have to take would successfully fit everything in. Folk literally ended up with code that - one day would build, another day it wouldn't. And it'd take a whole day to tell you it'd failed too.
All in all, kind of OK during development (at best), hopeless for production, and not competitive with a specifically designed chip for speed. About the only thing they're good at is having lots of copies of a logic circuit all running in parallel.
In fact, a valid question is, "why did they ever succeed in the market?". The short answer to that is that - really - they didn't. Whilst the likes of Intel and Co saw massive markets build up around their products, the FPGA manufacturers always seem to have struggled. All their frailties pointed to "niche", and that's the market they've filled; the niche where people will overlook their problems, to some extent.
Improvements
To address some of these frailties, the FPGA manufacturers started to do things like add "hard cores"; bits of silicon dedicated to specific functions. These range from the kind of logic one needs for some signal processing maths operations, to whole CPUs (ARM or PowerPC), bus interfaces (PCI), memory (DDR). The idea being that - probably - your design may want some of this and - instead of having to implement it in look up tables - can just wire up to these existing functions.
Why Isn't This Software?
Good question! The answer really lies in the programming. On a CPU, one has the idea of being able to load up any software one wants. Whereas an FPGA - whilst it doesn't have to be programmed in a lab using special tools - there is the general expectation that the programming would not be changed once programmed. Some FPGA technologies this is even physical; the look up tables and routing are programmed by un-burning anti-fuses (yes, that's right) in a one-time operation.
As with everything else in FPGAs these days, the lines are now somewhat blurry; you can on some models partially program them, and have them load up the rest later on (much like an OS and application).
However, their most useful attribute today is that the firmware they're programmed with can be either physically permanent, or very hard to alter (lots of cryptographic protection and authentication of firmware content). This is ideal in a security environment where you want to separate something from something else with some sort of processing / channel in between. Built that processing and channel inside a modern FPGA and you can have a ton of assurance about that implementation that you simply cannot achieve with an OS + software.
Signal Processing and other Maths
The vendors like to big up their chips as being good for maths and signal processing. And indeed, a modern FPGA can be pretty pokey when it comes to ops / sec.
However, their frailties still bite; a big, expensive modern FPGA may have a DDR interface on it, but a big cheap modern CPU might have 4 or more. And the FPGA might clock at only 400MHz, whilst the CPU can probably reach 4 or 5 GHz. And whilst the FPGA might still have a lot of "parallel", CPUs (with things like MMX, SSE, Altivec, AVX) aren't exactly short of parallelism either; a modern CPU can have 192 full CPU cores, each with 512bit vector units; that's a lot of parallelism clocked 10 times quicker. CPUs most severely and comprehensively outgun most FPGAs, especially when it comes to floating point arithmetic.
If - as often happens in FPGA projects - there is some uncertainty as to the exact required algorithm, and as the project evolves so does the algorithm's memory requirements, it reaches a point where the FPGA's internal memory is insufficient and it's now reliant on that external DDR connected memory. At that point it's game over; a CPU would certainly have been the better bet.
Real time? FPGAs are often cited as being good for real time; except, a modern CPU properly used is perfectly capable of being "real time" too. With a proper BIOS even Intel CPUs can be coerced into staying in a fixed power mode and not switching clock rates and voltages all the time. And a decent RTOS (VxWorks, Linux-PREEMPT_RT, INTEGRITY, etc) do a good enough job for most purposes.
Summary
In short, you've got to have some sort of burning engineering reason to want an application-specific integrated circuit in one's project, but can't afford to pay an ASIC manufacturer to make your part and o opt for an FPGA instead. And, not many of us do.
Which is why Intel sold off Altera - which it had bought only a few years previously - with the company being lucky if it does $1billion a year. Xilinx - the other big player in FPGAs - got bought by AMD, and so their performance is a little bit opaque at the moment; I think it was doing about $1billion a year too before acquisition. I very much doubt AMD makes much money out of Xilinx. Meanwhile NVidia - GPU''s being the other major alternative "CPU-maths" - are clearing $100billions+ a year.
So, Why Didn't AI pick on FPGAs?
Some folk did. But at the end of the day, NVidia did a good job with CUDA, and a GPU is comparatively cheap.
Another aspect is build time. If you're compiling CUDA code, it'll take, what, a few minutes at most? Whereas doing the place / route for a large FPGA can take a seriously large workstation an entire day. Your developer isn't doing much development for about 50% of the time. Being quicker to market matters, and you get their quicker with GPUs.
Disclosure
Personally, I hate FPGA for all but a very few specific use cases. Throughout all the time they've been around they've been marketed as the solution to all your problems, and universally failed to deliver on any of them. I've been stung by the manufacturers making things up, lying in data sheets and concealing silicon faults until you yourself stumbled across them (here's a lovely errata sheet instead of a working part). In contrast, I've built some fairly major specialised signal processing system with CPUs and rarely hit a problem at all. The lifecycle on FPGA projects is so slow that they're always behind the curve, always struggling for relevance.
