Most of us know mean people–people who always seem grouchy, grumpy, snappish or who rarely smile. They may be sarcastic, biting or negative. Mean people are easy to dislike, and if we’re well acquainted with the topic of narcissism, it may be tempting to label people we don’t like narcissists.

But they may not be narcissists at all. Grumpy, negative people may just be…grumpy and negative.

They could be depressed (depression often manifests as irascibility or quickness to anger), going through a hard time in their lives, in constant pain, or just having a bad day. Some people are just more anger-prone than others. They could seem cold and unfriendly because of another disorder, such as a schizoid personality, Aspergers, or OCD.

They are not wearing a mask. They are not showing you something they are not. Some people just feel bad all the time and don’t bother to try to put on a nice front to impress you.

Most (though not all) narcissists come across (at first) as the nicest people you could ever hope to meet. They are friendly, welcoming, effusive, and greet you with their most winning smile. They are likeable, outgoing and charming.

Let’s say you work in an office with a woman who is sarcastic and negative all the time. She is quick to criticize and complain. No one really likes her. There’s a man there who is much more pleasant to deal with. He’s always smiling, joking, patting you on the back. Everyone at work likes him.

Quick: who’s the narcissist?

Answer: it’s probably not the grouchy woman. Narcissists work hard to impress you, which means pouring on the charm to get you to trust them. The angry woman isn’t wearing a mask. She is showing how she really feels. She doesn’t care what you think of her because she has stopped caring about much of anything.

You hear through the grapevine what her problem is. Her husband left her and took the kids. He’s dating a woman 20 years younger than she is, and left her with practically no money. She also suffers from constant migraines. She’s too embarrassed to talk about all this in the office, but she feels terrible every day and takes it out on everyone else. No, she isn’t a nice person but she isn’t a narcissist (in fact, her ex-husband is).

It may not be the nice, friendly man either, but if one of them is a narcissist, it’s more likely to be him. If so, he’s far more toxic than the woman, but he doesn’t want you to know this. He’s biding his time, reeling you in with his phoney charm.

So don’t assume someone is a narcissist just because they have an unpleasant personality. You may want to call that person a narcissist because you don’t like them, but sadly, it’s more likely to be a person you like. Be careful who you pin the N label on.