I think that for the most part the idea that Indians are good at math is inaccurate. A small percentage of Indians do go abroad (usually these are the bright kids anyway) and I think this may be the reason why so many people think that Indians are generally good at math. I mean, I'm Indian and I think I'm about average. But there is a certain truth about the schooling system; it is very science focused (you won't find too many high schools with drama or jazz band!)
What do you consider average?
Lets put it this way, in the US there are many undergrad universities that offer workshops for pre-algebra(or math review). Students get through high school, and do so poorly on a math placement test that they prove they arent not even ready for algebra level math at the age of 18, BUT are still accepted to college...
Although this case is below average, its not that rare that an American will start college with very limited math skills, and graduate without a grasp on simple statistics and calc. Math education in the US is sub-par, which is why there are so many Indian engineers here. There's a gap in our education system that India has a more focused approach to.
I think you rightly point out that simply stating average in my case might be misleading. I guess when I said I was average I was comparing myself to my schoolmates in India and the US so even though I may be average in this set my math skills/level may be considered above average nation wide (both US & India), considering I went to a good high school (International Baccalaureate) and college. I spent 7 years in the US (4 years in college and 3 years working; I moved back two years ago). During my senior year in college, I volunteered at the local middle school (in WA) as a teaching assistant and student mentor and I did notice that the level of math was way low compared to what kids in India study at those grade levels.
But, what I was trying to point out is that if you compare the math level per capita (if there is such a parameter!) between the US and India I suspect that the US will win hands down because of the high literacy rates. However, the proportion of kids who do get a decent education in India is probably way higher than that of kids in the US which may mislead people into thinking that Indians OVERALL have better math skills than Americans. Also the typical Indian that you will come across in the US is in all likelihood will probably have a decent education and is unrepresentative of the total Indian population.