Why do Hindus and non-Hindus alike think that caste is hereditary?

In the Bhagavad-Gita 4:13, Lord Krishna pointedly declares that not by birth/hereditary reasons, but that the caste of an individual is based on each person's inherent nature and capability in doing work.