ECJ as JCPC?
Some assorted former British colonies still have the (UK) Judicial Committee of the Privy Council as their highest court of appeal - 12 countries according to Wikipedia, out of 31 territories in total. They include Jamaica, which still has the death penalty. I don't know if it has been asked to rule on any crimes carrying a death sentence since the UK abolished them. If so, it may be the last remaining UK court to rule on them.
So by analogy, leaving the EU would not require the UK to leave the jurisdiction of the ECJ at all, unless it wants to.