Most of the words we capitalize in English are what we call proper nouns. They're the names of specific, unique things like Mount Fuji, Idaho, or your friend David. Meanwhile, when you're talking about a common thing of which there are many - like mountains or states - you don't need a capital letter for those words.
Of course, most grammar rules are not this cut and dry. So, let's review some of the most important rules of capitalization. They'll provide a nice foundation for much of your future writing.