Yesterday in browsing about math universities, I ran into a thread discussing how people getting into top math PhD programs were ones who were taking Linear Algebra their freshman year of college and beginning graduate math courses by their junior year. So, if that's a door to keep open wider; sooner seems like a good idea.