Your child may be ready for more advanced skills or topics in the subjects in which he seems to be getting more easily distracted or completes quickly. Tim Hawkins, a famous Christian comedian, has a wonderful song about homeschoolers (check it out on YouTube) and speaks about this subject frequently. He says that he is asked what grade his kids are in and he doesn't know what to respond except to say, "We homeschool."
It's true that many parents assign a grade level to their students, but I would say that is just to give an age-related grade level for testing or evaluation purposes. If your child is excelling in Math and could be challenged (and not overwhelmed!) by being moved to the next level, I would encourage you to do that. Many curricula have assessment tests available, too, so you can ensure you are moving your child to the next grade or level when they are in fact academically ready and possess the knowledge and understanding needed to move on.
Likewise, if your student is struggling with a higher level math, assess their understanding and if the child isn't ready for that level, don't be weary of moving that student to a lower level for that subject. You aren't changing the child's grade level necessarily, just the level of that subject.
This is why I believe Tim Hawkins' comments on homeschooling and the grade level our children are in is so appropriate! At one point I had a child enrolled in first grade, doing 5th grade reading (and comprehending what was read!), 3rd grade math and first grade science and social studies. Additionally, I have had children who were behind in math compared to their grade level, but excelling in the other subjects.
Remember, homeschooling isn't always about filling up a child with knowledge but teaching them how to become learners who desire to learn more! I want to encourage you to not worry about standardized grade level expectations and instead to set goals for your children each year that will give them an excellent education and will help them identify and develop their special passions and talents.