It depends almost entirely on you. If you're a morning person and do your best thinking then, take it in the morning. If you hit your stride in the afternoon, take it then. Don't try to force yourself into a time slot that isn't a good time for you personally. Doesn't matter what time of day is good for anyone else but you.
As far as day of the week, try to avoid Saturdays - the testing centers are busier then, and they could be noisier. I would guess that middle of the week would be less crowded. Call the testing center you'd be going to and ask them when it tends to be less crowded.