Perhaps the most deeply hidden motive of the person who collects can be described in this way: he takes up the struggle against dispersion.
Walter Benjamin, "The Collector" in The Arcades Project
There are now many resources available that allow us to look at primary sources in mathematics - the seminal documents that first introduced a new concept, or that changed the course of thinking and research. For example, the MAA has its mathematical treasures site, and the Royal Society has recently publicicized many of its treasures through its Trailblazing exhibit.
But what about old 'secondary' sources, like old textbooks? Primary sources are often interesting in how they stand out from their time - how they provoke and anticipate future directions. Secondary sources are often interesting because they do the exact opposite by encapsulating and reflecting the milieu in which they are produced.
School mathematics textbooks, individually and collectively, sum up many aspects of the times that produced them. They tell us about school mathematics, but also pedagogy, curriculum, who was expected to attend school, and the kind of people who taught at them. They tell us about typesetting, bookbinding, who authored (and authorized) texts, how they were distributed, and much more. For math teachers, it is interesting to see how perpetual the notion of reform is in mathematics education, and how current debates about the place of technology in mathematics education have previously played out in discussions about calculators, slide rules, and trigonometry tables.
In his notes on collectors in The Arcades Project, Walter Benjamin explains that "for the collector, the world is present, and indeed ordered in each of his objects... and for the true collector, every single thing in this system becomes an encyclopedia of all knowledge of the epoch, the landscape, the industry, and the owner from which it comes."
It would be interesting, for example, to chart how textbooks have tried to justify and encourage the study of mathematics over the years - the image below is an example taken from a text published in the 1930s; today what image would we use?
Pat Ballew (of Pat's Blog) has put together an interesting survey of the history of high school math texts that highlights the emergence of some key concepts that we currently take for granted. His overview shows how interesting and useful looking at old text books can be. For your own collecting, if you don't like the mustiness of old books, Google Books has made many very old math texts available.