Make fun your primary objective. I agree with the keyboard idea. We have one - it's about four octaves - nothing fancy (might have been $50? Can't remember). My kids have been playing with that thing since they were little. Two of the black keys are now held on with electrician's tape, and we've lost the plug-in cord as well as the battery door cover. It still works!
Interestingly, my son ACED the pitch pattern recognition portion of his CAPD assessment (kids his age are supposed to score 40% and he scored 97%). I think early, fun and child-led exposure to music helps to build their brains
