People don't naturally somehow change their ways. They make the choice to change or the choice to stay the same.
If a person doesn't believe that they didn't do anything wrong, like your boyfriend, then they aren't going to change and continue to do what they do. He thinks it's ok to buy his gifts and will continue to do it regardless of how you feel about the situation.
"I don't think he forgave me" Forgive you for what exactly? having self respect? or not letting someone treat you badly? Believe me, you did nothing wrong.
But it doesn't matter if I believe it, you need to believe it. If you don't believe it you will end up getting hurt again.
Not buying an ex-girlfriend a gift is relationship common sense. Everyone knows that doing things like that is not acceptable.
You kicking him out was a natural reaction to what he did. Since, it's common sense to know not to buy ex gfs gifts he knew what the consequences of his actions would be and choose to do it anyways.
Him being upset that you kicked him out is a sign of a victim mentality. Victims never take response-ability for their actions or their consequences. They pretend like things happen to them instead of understanding that those things happened as a result of the choices that they make.
My girlfriend understands that I would stop dating she if she started talking to and buying gifts for an ex. If she chose to do that she would ALSO be choosing to end the relationship.
Because she understands this dynamic she wouldn't be upset if she got dumped because she was the one who decided to end it.
I think what is going on here is an issue of low self esteem and confusion. If you loved or respected yourself a little bit more he wouldn't be in the picture at all. You would respect yourself too much to put up with crap like that.
Another aspect of it is sticking to the choice that you made a year ago. You decided a year ago that it was ok for him to talk with an ex-gf (you make it ok by staying with him)
When you make a choice that goes against your principles (honesty, not flirting with ex's) your self esteem and power will start to slid.
Combine that with a fear of losing him and a fear of finding someone else and you will rationalize staying with him.
When you want something that you know is unhealthy and appropriate your mind will rationalize that decision. Most times the person will makes excuses or ignore that bad behavior (him saying that there was nothing wrong with what he did).
Other times the person will try to distract themselves from the negative behaviors of their partner. You did this by feeling guilty and feeling like you were a little too hard on him. The guilt allows you to keep what you want (him) while ignoring all of the bad things.
Underneath everything you know that your relationships shouldn't be this way. There shouldn't be a struggle or having to keep a watchful eye on him.
At this point you need to make a choice: stay with someone who will talk to his girlfriend or not.
If it were me (and I have been in that position) I would let him go completely. In the long run it would hurt me much worse than if I ended it right now.