I think this is where your going to have to sit down and talk with him about it.
It's obviously coming from his side. I think you need to do things slow again, do foreplay to get him and you in the mood. It will feel like starting over at one again, by this I mean take things slow and be romantic don't lead straight into sex.
When you talk to him about it ask him why hes not having sex with you so much any more. Make sure you talk about everything on your mind with him, and make sure you sound concerned not upset about it.
Try to find a middle ground for both of you to fix this problem. It must be agreed upon by both people its not supposed to be one sided. Sex just like a relationship needs a lot of communication.