"working with small scrum teams was a much better way of working than waterfall"
That's because waterfall requires that you know what you're doing and how to do it. Here, you didn't have a clue, you were making it up as you go, so yes, agile was the solution.
Agile is not always the solution and, as we see here, neither is waterfall. As for a hybrid approach, I don't see what the problem is if that gets the project done.
A good project manager will choose the tool best suited to the job.