Yes, everyone you plan to bring with you will count towards the total people in the accommodation - even babies & toddlers.
Whatever size or style of accommodation you're booking you'll see both the maximum number of people and also the maximum number of adults that it can accommodate and this INCLUDES any babies, infants or toddlers:
So for example if your accommodation states that it "Sleeps 6 (max 6 adults)" this means that it can accommodate a maximum of 6 people including any babies.