I run email systems for a living
A few suggestions if you want to DiY:
* You *must* have a static IP with A, MX and PTR records
* Exim for the MTA. You are welcome to try others eg Sendmail, Postfix or Qmail
* Greylisting is a great idea but it will get on your nerves after a while and is no more effective than blacklisting (sign ups to new services will be delayed by your greylist)
* Use Spamhaus, Hostkarma and co for blacklists - they are very good for an initial filter
* SPF, DKIM, DMARC - they will improve your "reputation" but be careful - they are complex beasts.
Test with this: http://www.mail-tester.com/
To really get to grips with it, from a standing start, allow at least 1 year. I'm not joking. You can get good results in a couple of hours but you will still be learning for years. I am.