The notion of "precious" is linked to the hardness and scarcity of a stone.
One says that a material is harder than another if the former can scratch or score the latter. After numerous tests, Friedrich Mohs, a German mineralogist, created a 'hardness' scale at the end of the 19th century. The Mohs scale goes from 1 to 10, from the softest material (talc, 1) to the hardest (diamond, 10).
The world we live in is primarily composed of quartz derivatives, which hardness is 7. Any material whose hardness is lower than or equal to 7 will have a tendency to be attacked, and therefore corroded, by our environment. But any material whose hardness is higher than 7 will be a lot less affected. Among the stones which hardness is superior to 7, one can find emeralds (7.5), rubies (9), sapphires (9) and diamonds (10).
As to scarcity, one can mention that 250 tons of minerals need to be displaced in order to find 1 carat (0.2 grams) of diamond. Scarcity takes on yet another dimension when one considers the size of the stones that are extracted: amethysts weighing several kilos can easily be found, whereas rough diamonds weighing 4 grams (20 carats) are a real rarity on the market.