In view of the improvement requirements for accuracy and real-time performance of traffic sign recognition in natural scenes, an improved traffic sign recognition algorithm was proposed based on a multi-scale Convolutional Neural Network (CNN). At first, the comparison experiments on image enhancement methods was carried out, and contrast limited adaptive histogram equalization method was chosen as the preprocessing method to improve the image quality. Then, one kind of multi-scale CNN model was proposed to extract global and local features of the traffic sign images. Finally, the traffic signs were recognized after the combined multi-scale features were put into a fully connected SoftMax classifier. The effectiveness of the proposed algorithm was examined on the benchmark dataset---the German Traffic Sign Recognition Benchmark (GTSRB). The examination results show that the proposed algorithm can achieve 98.82% recognition accuracy and real-time processing with 0.1 ms per image on GTSRB dataset, which verified its superiority to some extent.