Re: "Fix it"...?
I think it would be more appropriate to say you need to know enough to manage the risks. So if you're using a spreadsheet package, you need to understand where you use it in your business, what the dependencies are, what the impacts of outage are, and so on.
At that point you need to manage the risk: if it's a proprietary package, that means reviewing the supplier, including their financials, code-escrow, and so on; if it's open-source, then it could mean hiring a developer yourself, or it could mean getting a commercial support contract outside your firm: that might even be bought from the core dev-team.