How do you deal with the judgement?

I'm about to be 40, last year I met a guy online and we really hit it off. After taking for almost a month I made a trip out to see him. He looks older than he is and I didn't ask his age until a couple days before we met in person. he's only 23. I was really nervous but his parents love me and I'm not his first ( previous gf was 42). He proposed two weeks ago and I'm over the moon. But I have some people who have said things accusing me of basically grooming him and taking advantage of him. I've never dated someone younger than me. How do you deal with the judgemental remarks and looks?