That's a good point... I was out one day with my partner (who was always told by his ex how ugly he was and that he was lucky she stayed with him; FWIW he's beautiful--internally and externally) and he's trying to figure out what makes a guy "good looking" so he'll often ask me about people we see... one day he saw a very pretty girl and a not as externally attractive man who were clearly together and he asked if that guy was good looking enough that he was somewhat of a match for the woman. I replied, "Not in the traditional sense..." and then he asked if the guy was rich. I said, "Perhaps. Or perhaps he's kind and funny and treats her well and not like an object and that's refreshing to her." He chewed on that for the rest of the afternoon.

It's true though, that we often see beautiful women with men who are less conventionally attractive and far less often see beautiful men with women who are less conventionally attractive. Though looks are definitely not everything (as I highlighted above).

I think very often when we are admonished to shift our standards, its from insecurity on the part of the admonisher.