HELP baby daddy drama?

So basically when I was pregnant my baby's father split. It wasnt until My now son was 14 months and coincidentally when they started detaching his wages for CS that he came around. He wanted to meet the baby. Since we live in different cities I was the one driving 4 hours to see him once a month and paying for the hotel (since he still lived with his parents and they didnt know about him seeing me) after about 6 months of this seeing each other on a monthly basis he decides he thinks me and him should go our separate ways because all he does is ignore the child and he thinks he wants to move on with his life and just pay the support. He says i need someone that will care and not just want a good time.. When I probed him about it he pretty much told me that the child is the biggest mistake of his life and that everytime he sees us he is reminded of his mistake. So I asked him by not seeing me and his son if that was his way of moving on.. his response was "yes kinda" I was very confused because at first he was so great.. and he said that was because he was optimistic that it would work out and now he knows it wont but he didnt give me a reason. And since then every time i offered to come down and see him with the baby he firmly says no thanks! i havent seen him in 4 months and it hasnt gotten any easier.. and to top it off my mother died 2 months ago. I desperately want to see my childs dad so my son can be happy and have a guy to call daddy if only for that weekend he sees him. How can I convince him to see us? Please no judgements.. We're both 22 by the way