Wat Phra Kaeo, the state temple of the old walled city
The area around Kampaeng Phet has probably been settled for a few thousand years, at least. The location on the Ping River puts it on a strategic trading route between Lanna in the north and the ocean ports of the south. The first historical records date from the mid-fourteenth century and mention the settlement of Nakorn Chum on the west bank of the river. Kampaeng Phet seemed to rise in significance later on, in the fifteenth century, along with the shift in power from Sukhothai to Ayutthaya. The city seemed to have hit its peak in the fifteenth century, and this is when the famous walls were built, along with most of the temples of the city as well as the forest temples of the Aranyik. The walls were restored in the sixteenth century with Portuguese help, to aid in the defense of Ayutthaya against the Burmese. When the Burmese attacked and sacked Ayutthaya in 1767, Kampaeng Phet was one of the towns that was completely emptied and left deserted for many years.
The pagoda of Wat Phra Baromathat in Nakorn Chum
The city was re-settled over time, and in the twentieth century several restoration projects were undertaken, culminating in the listing of Kampaeng Phet's monuments as a World Heritage Site, as part of the inscription of Sukhothai and Si Satchanalai. Although considerably less popular than the other two sites, Kampaeng Phet is interesting in it's own right and well worth a visit, either as a day trip from Sukhothai - where the tourist infrastructure is far better - or as a stop on the way between Bangkok and Chiang Mai or other points north.
You can easily see the main sights of the old city in a day, so staying two nights in Kampaeng Phet will give you plenty of time to see everything. If staying in Kampaeng Phet, start with Aranyik and then work your way back to the walled city sights. If you have time, you can visit the museum, but I wouldn't make a special effort to see it. If you have more time, you can see the sights of Nakorn Chum on the other side of the river.