Do I (we) really need a bug tracking system

I got an interesting comment from my previous post which the commentor asked: "You really anticipate so much bug that you will need a bug tracking system?"
Of course he didn't ask this question for himself as he is a great developer who always strongly supports any good software development practices where bug tracking system is absolutely not excluded. In fact I believe he asked that to get attention from those who have the authority to implement bug tracking system but somehow have not yet done so.
I really want to borrow a good quote from Joel:

One of the biggest incorrect facts that programmers consistently seem to believe is that they can remember all their bugs or keep them on post-it notes

I find it true wholeheartedly because I can't remember more than two or three bugs at a time, and the next morning, or in the rush of shipping, they are forgotten.
A very simple question to those who think that bug tracking system is not always needed: "Would you like your manager come to chase you for fixes every morning?"