Both the pagoda and the temple were built in 706 in the Tang Dynasty. Shan Dao's disciple Huan Yun once wrote that the pagoda was built not as a tomb for Shan Dao, but as a Buddhist pagoda to commemorate him. When the pagoda was completed, Emperor Gaozong is reputed to have given it a thousand and so Buddhist relics, and Empress Wu Zetian visited it.

Shah Dao was a monk of great achievement. He wrote several Buddhist classics on the Pure Land Sect, expounding its doctrines, which were introduced into Japan in the eighth century. An eminent Japanese monk founded the sect in his country on the basis of Shan Dao’s doctrines, which spread wide. Even today disciples of this order consider Xiangji Temple their founding temple. On May 14, 1984, monks of the Pure Land Sect of both China and Japan held a ceremony at Xiangji Temple to commemorate the thirteen hundredth anniversary of Shan Dao's death, adding a new chapter to friendly exchanges between the two peoples.

The existing pagoda is the original structure erected thirteen hundred years ago. It was built on a square plan, with each side of the first storey measuring 9.5 meters long. The first storey on its square pedestal is huge; all eleven storeys reach a height of thirty-three meters. The steeple has been destroyed. At the entrance of the pagoda is an inscription made during Qianlong's reign in the Qing Dynasty. One characteristic of the pagoda is that the storeys are higher than those of ordinary multi-eave pagodas. Also, the pillars, beams, brackets and window lattices are painted red, a rare sight among multi-eave pagodas,