Diabetic retinopathy （DR） is one of the major causes of blindness, and the appearance of retinal neovascularization （RN） is an important sign of DR deterioration. In order to detect RN more accurately, a method based on color fundus photograph for retinal neovascularization detection is proposed. First, an improved U-shaped convolutional neural network is used to segment the blood vessels. Then, a sliding window is used to extract the morphological characteristics of blood vessels in the specific area. A support vector machine （SVM） is used to classify the blood vessels into normal vessels and retinal neovascularization in the window. The experiments use color fundus photographs with retinal neovascularization from the MESSIDOR dataset and the Kaggle dataset for training and testing. The result shows that the accuracy of this method for the RN detection is 95.96%; This method has potential application prospects in the computer-aided diagnosis of diabetic retinopathy.