I don't know if high society is different in other cities but in Hollywood important people can't stand to be invited someplace that isn't full of other important people. They don't mind a few unfamous people being present because they make good listeners.
You can become famous but you can't become unfamous. You can become infamous but not unfamous.
It takes as much courage to have tried and failed as it does to have tried and succeeded.