Concorde wasn't retired because BA's load factors were dropping or maintenance was getting expensive.
Concorde was retired because Air France's load factors had dropped, and Air France's maintenance costs were problematic.
Because of the age of the design and the very few aircraft that entered service, spare parts were in short supply from about 1990 or so. In order to keep BA's twice-daily London-New York flights, BA needed at least 3 operational aircraft (one to operate BA001 and BA002, one for BA003/4, one spare), although they usually had 4 in order to support the odd charter and the winter Barbados flights (when they were doing those), which they supported by moving parts around their 7 aircraft.
New York-Paris was never as busy a route, so Air France only operated one round trip a day (needing one aircraft plus a spare) and did a lot more charter work, which needed another 1+1, but not every day. This was fine until the crash, at which point AF only had 4 aircraft remaining, so they couldn't move parts around as BA was doing.
And then politics entered the mix: once Air France decided they could no longer support their Concorde operations, they could have sold the aircraft to someone else (Branson publicly offered to buy at least one, but that was probably just grandstanding), and certainly BA was interested in buying parts from the AF fleet if it was going to be grounded. However, grounding the fleet would have pushed costs from Airbus (former Aerospatiale former Sud Aviation) and BAE (former BAC) all onto BA instead of sharing between AF and BA, plus French national pride/ego wasn't keen on the Brits continuing to operate the thing when they weren't (even though... "The E shall stand for England" -- an obscure reference to the naming of the thing). So AF refused to sell their airframes, Airbus pulled the plug on the airworthiness certificate, and the first supersonic age came to a close.
(Some years later, the USAF bragged that the F-22 could "supercruise" -- fly supersonic without afterburners -- at Mach 1.8. Whoo-de-doo: I've "supercruised" at Mach 2.0 while enjoying a Johnny Walker Blue Label, on G-BOAG and G-BOAF).