This paper investigates the influences of tall building shape and layout on the pedestrian-level wind (PLW) environment in the urban area by combing the wind tunnel test and Computational Fluids Dynamics (CFD) simulations. The maximum wind speed-up ratio and integrated normalized speed-up area ratio were used to quantify the effects of five different building shapes and four building layouts on the PLW environment. The favored building shape and layout with omnidirectional equal-probability distribution were determined, and the underlying mechanism of building shape and layout that influences the PLW environment in the urban area were elucidated according to the whole flow-domain information attained by CFD simulations. The PLW of groups of Y-shaped tall buildings in the staggered layout was most favored, while the most unfavorable scenario was H-shaped and enclosed layout; In addition, the most unfavorable wind directions of square, H-shaped and X-shaped tall buildings were in the oblique direction, while these of the cross- and Y-shaped buildings were in the normal wind direction. The wind speed-up phenomena of groups of tall buildings at the pedestrian-level were mainly attributed to the flow separation at building corners and channeling effects.