Today we celebrate the memory of all the baptized who have gone before us and whose lives were virtuous. We keep the memory of all these, not just those who have been formally canonized as saints.
The Feast of All Saints began in the early centuries as the “Feast of All Holy Martyrs,” a remembrance of martyrs whose names were not known and who therefore did not have their own festivities. It was introduced in the year 610 by Pope Boniface IV, when the Roman Emperor, Phocas, made a gift of the ancient pagan temple of the Pantheon to the Church. Initially celebrated on May 13, it was transferred to November 1 for the universal Church by Pope Gregory IV in 835, and by then it included all the saints.
The reason for the new date may have been a practical one. It seems that so many pilgrims came to Rome for the feast that it was moved to the fall when more food would be available after the harvest.