The pagoda is built of a kind of hard stone produced in the area. After more than a thousand years the pagoda shows no sign of decay. Except for some small sculptured stone pagodas and pagoda-shaped columns in grottoes, Simen Pagoda is the oldest extant stone pagoda in the country. Its structure is very simple. It is square, each side measuring 7.4 meters long. The entire pagoda is 10.4 meters high. In the middle of each side is an arched door. Over the main body are five tiers of stone eaves, progressively wider as they move upward. Many pagodas built later in the Tang Dynasty followed this same style. The roof is made of twenty-three tiers of slabstones, one overlapping the other, making it pyramid-shaped. It is crowned with a stone steeple that itself looks like a box-shaped pagoda, with Buddhist scriptures carved on it and a Sumeru pedestal with corners in the shape of banana leaves. The spire is formed by five discs. This is the common style for one-storey, pavilion-style pagodas.

Inside the pagoda is a huge square central pillar, surrounded by a winding corridor. On the ceiling a triangular beam links the pillar and the outer walls to provide support for the roof. On the four sides of the central pillar are stone statues of Buddha seated cross-legged. The appearance of Buddha is vivid and the lines of his clothes look smooth. On the base of the statues is an inscription dated 544 during the Eastern Wei period, but since the inscription on the stone tablet inside the ceiling, discovered in 1972, said, "Built in the seventh year of the Daye period of the Sui Dynasty," the statues may have been moved there later.