There's a very basic rule: Who and whom are the relative pronouns for people; that and which are the rel. pronouns for everything else.
For me, too, it's the man who was driving the bus, never the man that was driving the bus. (I wouldn't call the latter usage a mistake, although like Wallace I tend to view it as a class-marker.) Wallace qualified this advice with "As of 2003," and it does seem to me that using that to refer to people has become much more common over the past dozen years. I wonder if it will ever sound archaic to say the man who was driving the bus. (Probably it will, when buses drive themselves.)
My own problem with who/that arises when talking about animals. A certain sympathy inclines me to call them who. My cat? Definitely a who. And in the children's book Click Clack Moo: Cows That Type, the word that always makes me frown. Those cows can type, for crying out loud! Give them a who! (Some pedant copyeditor probably put a stop to it.)
A train that left the station a long time ago was the use of agenda as a plural noun. Nowadays, agenda is a singular noun that means, more or less, a list. Once upon a time, an individual item on such a list would have been called an agendum, but today it would be eccentric to use that word. Now it's an agenda item.
Criteria, on the other hand, is still plural. A person looking for an apartment might have ten, twenty or a hundred criteria, but if he only has one then it's a criterion. Some people seem unaware that criterion exists, so they say Now let's look at the first criteria or This criteria is shorter than that one. People who habitually use the word criterion initially hear these statements as utter nonsense. (This giraffes is shorter than that one?). Naturally the intended meaning comes through a half-beat later, but it's apt to be accompanied by an unkind mental note about the speaker's vocabulary.
Sometimes people try to defend the use of criteria as a singular noun by analogy to the way agenda and data have become singular nouns. The analogy doesn't work, though, because agendum and datum have both disappeared, whereas criterion has not. As long as singular and plural forms are both available, there's no choice but to match them to singular and plural verbs and pronouns. Hence the usage note in the American Heritage Fourth Edition:
Like the analogous etymological plurals agenda and data, criteria is widely used as a singular form. Unlike them, however, it is not yet acceptable in that use.
This is good advice, because when a listener does perceive this error, it is perceived as catastrophic. Someday criterion will be joining agendum in the obsolete category, but right now it doesn't pay to be out in front on this one.