So, you want a rebooted Series 5?
There are two questions you have to ask, what features could you not live without, and how much are you willing to pay. For consideration;
OS - As framed here, the choice is Android or Linux of some flavour. What are the pros and cons of these, and other potentials. Bare bones Linux is a good starting choice, if a little bloaty compared to E32, but the simple fact is that there exists a working kernel for nearly any HW you'd care to choose. The main issue would be UI, are you expecting a proper desktop, and all your apps 'just work' or are you looking for something more terminal focused? Getting a meaningful desktop experience on that screen form factor would require some effort, even more so if you don't actually update the screen. Android seems like a better fit, UX wise, but it's very resource intensive. You'd need a high end ARM SoC to run it at anything like responsive - just try out a cheapo Chinese Android with any of the 6.x and 7.x flavours.
Purists will say Epoc32/Symbian OS, but even if you had the source code, the fact is part of Symbian's downfall was down to the sheer amount of effort it took to bring it up on new HW platforms, also, the kernel wouldn't be readily compatible with ARMv7 anyway.
Fact is, unless you're willing to spend big bucks on development time, you're going to go with the path of least resistance, which is likely to be something Linux based. Someone mentioned Sailfish.
Hardware - Shell. The key selling point of the Series 5, and why we're all still here 20 years after it launched, bemoaning the fact that it has never been bettered (OK, the 3mx was better, but let's skip that), is it's form factor. You have two choices here really, do you try and get new housings/keyboards made, or scavenge eBay for spares. If you get new parts made, aside from the expense of tooling, you're opening yourself up to patent infringement for the design. I've no idea who currently holds the patents, it was Psion Industrial IIRC. I'm sure they'd crawl out of the woodwork if they smelled $$$. Assuming again the path of least resistance, and cost, your looking at essentially an upgrade kit for existing Series 5s. This would have implications.
Hardware - Main board. If you're making an upgrade kit, your choices are either have something fabricated, or find something of-the-shelf that's suitable. There's not a great deal of spare space inside the Series 5 case, and it's very low-profile. Assuming again you're going to skimp on cost and development effort you're down to looking at what can be made to fit. I still have an OpenMoko board stuck to the inside of a 5mx case from the last time I seriously looked into this, the good news these days is, I think a Pi Zero would be perfect.
Screen - Here you have to make a real choice between cost and 'must have'. Custom screens are not cheap, though a lot cheaper than they used to be. The last quote I had for this particular application was ~£10ks for prototype quantities of either LCD/E-Ink/OLED. A whole order of magnitude less than when I first looked into this, but you're still into having to have pre-orders, or money to burn territory. You could of course re-use the existing screen - for the things the Series 5 was good at, the screen was more than adequate, if a far cry from today's luxurious 6" 400dpi+ panels. Re-using the original is not without issue either, as you'll need to interface it to your main board, and then write the display driver, but that at least is getting near to 'hobbyist' level of effort/cost. The real question is, is the screen a deal breaker?
Personally, the only thing lacking from the Series 5mx as it stands today is connectivity, and even then I think just Bluetooth would be good enough.
So would you pay;
£100 for an upgrade kit that turned your existing Series 5 into a wireless, portable ssh client?
£1000 for a reconditioned Series 5mx with e-ink/oled display and usable GUI?
Or something in between?
If a few hundred of you pick the £1000 option, then let me know! ;-)