I'm in a rush... but I now that Microsoft also has some good children-oriented programming resources:

http://msdn.microsoft.com/en-us/beginner/bb308754.aspx

is what I got very quickly. My children are too young to start, but I'm sure they'll get into programming with them time comes.

It depends what your child is interested in and wants to get out if. C# or Java are good places to start programming if they really want to learn the computer science aspects. (Although I wouldn't expect anyone to really learn object oriented programming without a lot of guidance.) I know that a lot of people tend to be down on Microsoft, but PCs are very popular and they do offer very good & free tools to the hobbyists.

VB, COBOL, scripting languages are all good for what they do, but I wouldn't recommend it to a hobbyist... Not the fun stuff IMO. StampBasic is something really consider too because you can build little robots and such and get a bit of exposure to hardware.

Also, the Alice program (http://www.alice.org/) is supposed to be good, but I haven't tried it. This is what Randy Pausch was working on.

JB