I'm afraid he's already given you the answer to your question. Once this kind of relationship has been established, they rarely progress to anything more. You've set a pattern of letting him have his cake and eat it too. Besides which, if he had wanted more in 3 and 1/2 years, he'd have let you know.
Seems to me you have 2 options here.
One, continue as you have been with no expectations of more. I can tell you from personal experience, that this can be very difficult to do. It's very easy for us women to read something into those intimate moments where nothing exists.
Or, two, cut off contact because the friendship will be very difficult to maintain if you take away the sex. It can also be difficult to resist if you have taken away the sex and he tries to go back to the way things were before.
I'm sorry I wasn't able to give you more positive advice here but the truth is that these things rarely, if ever, develop into more especially in a situation where he's told you he doesn't want more. You can't say or do anything to make him change his mind if he's not feeling it.