Glaciers are permanent ice sheets. They form in layers. Naturally, the oldest layers are at the bottom, while the layers on top are from more recent years. As you said, glaciers do melt. They melt from the bottom under pressure which creates a slick surface for the glaciers to slide. The key thing about glaciers is that they slide due to gravity towards the sea just like rivers and when they lose mass, they lose it by calving off icebergs. Also, as you said, we can look at glaciers on major mountain ranges and see them shrink and expand. They can melt entirely.

If you look at the papers where they talk about using ice cores for paleoclimate studies, you'll see they focus on two sets of ice cores: ice cores from Vostok, Antarctica representing the eastern Antarctica ice sheet as it lies over a frozen lake there, and from Greenland near Thule Air Force Base. The reason scientists use the Greenland and Antarctic ice sheets is that they're so cold, they don't melt in the summer. They do flow and calve at the edges, but they don't melt top down. They're highly stable. The annual layers are preserved by the sheer cold of where they're at on the planet and thus can be used for this kind of study.

The age of the various layers is verified experimentally by looking at isotope ratios of the various gasses which have known rates of decay. It's not just a matter of counting layers and hoping we count accurately and that a hot season didn't melt off a bunch of layers. The Antarctic ice is thicker which gives us a longer record, about 800,000 years whereas we get about 400,000 out of the Greenland data. By contrast, if you used Alpine glacier from Italy, you might only get a few hundred years in some places.