Attitude problem

There are many things that are not wrong, immoral or abnormal... but they just don't feel right.
And sometimes it belongs to attitude problem.
To say something offensive followed by "Oh I'm just joking, really" is considered a serious attitude problem to me.
My sense of humour is not so high threshold yet.
so what does "I really don't mind working this saturday for you if you are not free. just poor me lar... i'm just joking la..." mean..

And I feel stupid to be upset by other's attitude problem. because it's their problem and they actually do no harm to me. And I'm getting so emotional over all these careless unthoughtful words/actions...