Saying no can sometimes be seen as threatening to an ego who has an investment in how others see it. Could this have anything to do with your reluctance to say no? Again, the mistake is in thinking that the story is who we are. It's not. The essential “I” behind the eyes has no story. There's no investment in needing to impress or prove some identity. This leaves us free to come and go … to say yes and no, accordingly. Now that's true freedom.