In order to satisfy the increasingly significant demand of the digital city for the detailed 3D model and solve the low efficiency problem of traditional modeling method, this paper summarizes a method of building the urban detailed model based on unmanned aerial vehicle (UAV) image sequences. Through the in-situ aerial photography test with 25 sorties, detailed models for 20 experiment area (300 m×300 m) and a library building were built. The accuracy of all detailed models were evaluated by field check points. The effect of 5 different flight heights and 4 different control points on the accuracy of the detailed models were discussed. The results show that the errors of the detailed models can be effectively decreased and the accuracy is improved by declining the flight height or increasing the number of control point. The highest accuracy of detailed models is ±3.4 cm in the plane and ±3.1 cm in the height, respectively. The accuracy of the detailed model for the library building is ±3.4 cm and ±1.5 cm, respectively. The method of building the urban detailed model using UAV image sequences can satisfy the accuracy requirement of the urban real scene model, which has great practical values in practical engineering.