Yondu was happy that Quill turned out better than him, and had a better life too. But he also knows that 'better than him' doesn't mean 'good'. Yondu wasn't a great father. He's pretty sure Peter would want to kick his ass for the simple fact he's been considerably better to these kids than he was allowed to be with his son.

All that aside, he does remember their previous conversation, even if it had been cut short by emergency and he'd just never properly followed up.

"You said you raised a youngin'?"