I'm no authority on social norms, but I think it's more acceptable to smile and such at babies in comparison to toddlers/kids.
What you've mentioned is somewhat on the radar though, yeah. I've been considering getting a career involving children/teenagers. I'm worried that, as a man, it would be more difficult to get hired.