As an elementary music student, and truthfully all the way up to about 10th grade, I felt that conducting patterns were pretty mysterious. No music teacher covered learning or practicing conducting patterns, and when I played flute in elementary and junior high school band, I always wondered what the director was waving around up there.
In my music classes, I introduce conducting patterns for 2/4, 3/4, and 4/4 time signatures beginning in fourth grade and continuing review until sixth grade. Students this age like taking the part of the leader of a group, and we practice conducting as a class to recorded music for each time signature.