Idk what to do.................?

Well, I don't believe in sex before marriage, but clearly you think differently, so... I would say to let things get farther first. 3 weeks actually isn't very much time, and seeing as though everyone you ask says he just wants sex, that's probably true. You should keep dating him without having sex and see if he still wants you after about a month or so. Then you will find out for yourself.