User Level: 

Managing your Organic Group includes both managing your users as well as managing the content that they produce.

The person/people responsible for managing a group are those with the Group Manager role.  Please note that Group Managers can not add new people to the general site, this must be done by a Site Manager.  Once people are added to the site, though, the Group Manager can add them to their group(s) at will.