To address the issues of high model complexity and poor feature extraction performance in Convolutional neural network-based dehazing algorithms, this paper proposes an end-to-end image dehazing algorithm based on joint mapping of two-branch features. Firstly, the atmospheric scattering model is transformed to separate the mixed-parameter feature and the single-parameter feature model. Then two feature extraction networks, MPFEM and SPFEM are designed according to the two-branch features and the outputs are weighted by two attention mechanisms. Finally, the extracted two-branch features are sent to the restoration module to restore the clear image and perform color-enhancing to obtain the final restored effect. To avoid the loss of texture details caused by using a single loss function in the model training process, multi-scale structure similarity and mean absolute error weighting are used as the loss function. Experimental results show that the proposed algorithm has a simple network structure, obvious dehazing effect, accurate color brightness restoration, and strong edge preservation.