A few months ago my Apple TV rentals used to take hours to get ready to watch. And I am not talking about the whole movie... that was just the loading time for the first few minutes.
When I decided to look deeper into this problem, I found out Google DNS service was the cause (It is not Google's fault though).
Then I switched to Open DNS and everything turned into heaven again. But it looks like more and more Apple TV owners were switching to Open DNS as well. A few weeks later things went pretty bad to me again.
Now I am switching back to my ISP DNS which I hope will fix the problem permanently (or at least as long as it is up and running).
It doesn't only helps Apple TV to go faster, but boosts iTunes as well.
For a good explanation on why it happens, please read here.
PS: The DNS switch also had the benefit of fixing my iOS 6 Passbook, which wouldn't work even after the clock ugly hack.