You can learn just fine on any digicam with a manual setting.
Small digital cameras with manual settings tend to be more expensive than normal point and shoot cameras, where it might make more sense to just buy a cheap SLR anyway (unless one wants to trade higher expense and lesser functionality for smaller size, of course). Smaller fixed-lens cameras with some manual settings typically don't offer as much photographic capability as even the most basic SLR, even aside from the lack of interchangeable lenses-- in terms of settings, image quality, depth of field control, major ergonomics differences including the viewfinder and grip, etc., differences which you may already have thought about.
But if you're looking for just a rugged little fixed-lens camera, you might like this
search engine at dpreview.com, where I found plenty of rugged cameras. You might also have luck finding a great deal by including discontinued models in your search, which I didn't do at DPReview. This
refurbished Olympus Stylus Tough 8010 seems to offer a lot of toughness for a decent price, though-- I found that one by googling for reviews of tough cameras. You might also consider buying a cheap digital camera on a really great sale (closeouts can go quite low these days if you troll the deal sites such as FatWallet) and seeing if you can find a
protective skin for it.
The spouse and I gave an older Canon DSLR, a 30D, to DS7 and he has greatly enjoyed using it. You can sometimes find like-new or even new DSLR bodies for under $200, which of course is still a lot of money for a camera that will need accessories, relative to cheaper point and shoot cameras anyway. His first lens was a cheapo 50mm prime, total value of everything well under $300. I was worried at first that he wouldn't take good care of it, but he's been very careful. He's also learned a lot more about photography than with his old point and shoot.
