Agile Customers
Having been in quite a few "Agile" projects over the past decade or so I have found the MAIN problem is the customer/end user and their requirements.
Easy example: Finding the current state of a process to analyze it for the new/improved (mutually exclusive terms btw) process.
What they SAY they currently do, what they ACTUALLY do, and what they are SUPPOSED to be doing are 3 VASTLY different things. Add into the mix years of "I have been doing it this way for decades for reasons LONG forgotten" and of course the "I'm retiring so I'm only going to teach the new kid WHAT to do and not bother to explain WHY I'm doing it" and then the poor new kid is thrown in with the poor BA (who is usually an outside hire specifically for the project and who has NO clue about the actual business) .
You know you're in trouble when your code monkey (That would be ME) gets the "specification document" and has to go back to the user and BA and tell them "But what about XYZ scenario" which completely blows the current solution out of the water. Of course this only happens if your developer has been around a while and actually has a fairly good understanding of the business and its processes (usually much to his/her disgust). If you have a bunch of relatively new/young hot shot developers, they will dive headfirst into the code based on the specs, deliver what is asked for and then get my favorite quote of all time "Yes, this is what we asked for, but it is not what we want".
Personally I like the "horses for courses" approach. Regular feedback is important. Daily standups (no more than 15 minutes), while a bit of a pain do add value in the sense that everybody is now aware of where we are/what any holdups are BEFORE they become mission critical. MOST of the other "ceremonies" in agile are just a waste of time. But if you don't have a clear and DEFINED endpoint in mind then working rapidly towards that ever moving goal is just going to cause frustration.
My 2c (Given the current exchange rate, worth even less)