I can highly recommend Dependency Track for your first steps into controlling this
As a CTO looking after a significant amount of Java and other languages I can highly recommend Dependency Track, dependencytrack.org, as a first step into controlling this issue and gaining visibility of your vulnerabilities.
I've gone from manually figuring out our exposure to CVEs to having a system email me and the architects when a new issue is raised that we should be aware of. I've been able to massively reduce the issues we have to worry about and set up process for handling new ones and deciding a course of action. Now we are much closer to handling one-off library updates little and often instead of putting off major library updates for years.
The other side is that we are benefitting from a multi-year hard investment in Katalon testing and our QA team to give us extensive coverage. This allows up to do platform upgrades with much less risk.