An elevation measurement method via the fusion of vision and barometric sensors is proposed to achieve convenient and accurate measurement of the elevations of operators during aerial work. A YOLOX deep neural network is constructed to detect the aerial operators and their positions in the pictures of the aerial work site. The changes of real altitude in the relatively short period of climbing are measured via a barometric sensor. Using the barometric elevation measurement and vision detection results， the support vector regression （SVR） is applied to construct and update the regressive model between the image position and the actual elevation of the operator， and the high accuracy elevation measurement results are obtained through the regressive model. In real-world aerial work experiments， the proposed method is compared with the direction elevation measurement using a single barometric sensor and the differential elevation measurement by two barometric sensors. The measurement errors， in the mean absolute error as well as in the root-mean squares-error， of the proposed method are all lower than 0.2 m， and are superior to the two rival methods. The performances of the proposed methods are achieved by fusing the vision detection and the barometric elevation measurement， which overcomes the serious signal shifting and degradation of measurement accuracy of the barometric sensors， and also avoids the laborious calibration procedure of the vision detection system. The proposed method’s elevation measurement accuracy satisfies the requirement on the operator in aerial work， and it is easy to apply in work sites without putting extra burdens on the operators， making the method a practical choice for operator safeguarding in aerial work.