I want to be friends with my ex, but he's confused the situation.?

Basically, we decided we should give friendship a go after our break-up. There was a holiday booked for me, him, our friend and her boyfriend, about 7 months after we broke up and I'd thought things might get awkward. Things were fine. We were sharing a room and the second night, the inevitable happened. I don't know if it was just because we were on holiday.Well, I'm not sure because there have been other times when we've been out together with friends and things have happened between us. The next morning after one of these episodes, he admitted he didn't know what he wanted from me, and then went on to say he wanted a friendship. Strange friendship, aye? We get on well and talk most days without incident, but he won't accept my invitation to call around to see him. Do you think he's worried things will happen? He's okay if other friends initiate the contact between us but refuses if it's me doing the initiating. Tried talking to him , he clams up. Any other ideas? I want this to work