This paper presents a structural sub-pixel displacement measurement by using the digital image correlation technologies for structural displacement and deflection measurement. Pixel-based displacements of a monitored structural location were determined using an improved sub-pixel algorithm. To transform pixel-based displacements to engineering units, a camera calibration method was developed. The numerical simulation test was carried out based on digital image processing algorithm. The simulation results were in good agreement with theoretical value, which validated the reliability of systematic measurement accuracy. A machine vision-based displacement measurement system was developed. A steel frame model was established for conducting experiments to investigate the follows: (i) comparison study on the measurement results obtained by the vision-based system and the laser displacement sensor, (ii) the effect of the measurement distance on the accuracy of the vision-based system. The mid-span deflection of fire-fighting bridge was measured by the developed system. The experimental results show many advantages of vision-based method such as non-contact, low-cost and convenient to implementation in realizing the structural displacement measurement with the accuracy level of millimeter. The method has wide application prospects in the deformation detection of bridges and other structures.