I've known people who have done this and it seems to work best when the kids move to the separate-teachers-for-each-class format of middle school. In elementary school, the problem is that the otherwise homeschooled kid is not a part of the group at any time other than when he/she is there. But when they mix up the groups for each class, it's less obvious that the homeschooled kid is different.