Current group Assignments are limited to a single Group Set. This means if a course has multiple Groups Sets, only one of them can be assigned to the Group Assignment. Read How do I Create a New Group Set? for more information on Group Sets.
Work Around 1:
Combining the groups into a single Group Set. This is only an option if different Groups Sets are not necessary AND none of the groups in one of the Group Sets has already submitted work any graded assignment. This option is only available the very first time the groups are submitting work to the course.
Work Around 2:
Create a new assignment for each of the Group Sets that are required to submit to it. This will also require the Assign To field to be changed to just those groups, if not than total grades will not be accurate.