Whether they're innocent or malice-ridden moves, you should be upfront and tell him you're uncomfortable. This can be verbally addressed or through body language. If he touches you, just recoil. If he's human enough, he'll get that you're not interested. If he persists, keep distance. If that's impossible (ie. He's part of your team or you share certain work processes or functions, etc.), talk to your HR.