The centrepiece of the festival is at Pak Tai Temple where are the "Bun Mountains" or "Bun Towers"(包山), three giant 60-feet bamboo towers covered with buns. It is those bun-covered towers that give the festival its name. Historically, young men would race up the tower to get hold of the buns; the higher the bun, the better fortune it was supposed to bring to the holder's family; the race was known as "Bun-snatching" (搶包山). However, during a race in 1978 one of the towers collapsed, injuring more than 100 people. In subsequent years, three designated climbers (one climber to each tower) raced up their respective towers and having cleared the top buns proceeded to strip the towers of their buns as they descended.

The three "Bun Mountains" are still placed in the area in front of Pak Tai Temple, and are constructed using the traditional fixation method – bamboo scaffolding.
In 2005, a single tower climbing event in the adjacent sports ground was revived as a race -- with extra safety precautions including proper mountain-climbing tools as well as tutorials for participants (which now include women). A teamwork version of the event was added in 2006.The revised version of "Bun-snatching" as well as the traditional three "Bun Mountains" still have their buns removed from the towers at midnight of the Festival.
In February 2007, it was further announced that the buns on the single-tower construct will henceforth be made of plastic.