Classical Gaussian mixture model (GMM) can describe the multimodal state of the video pixels and GMM has certain robustness in dealing with complex scenes, such as slowly changing lighting. However, it still causes false detection because of the change of pixel values in the same position when the background of the scene is re-exposed after being covered. To solve the repetitive background problem, a Gaussian mixture model based on history background (HBGMM) was proposed in this paper. Compared with traditional Gaussian mixture model, this model can quickly adjust the learning rate by marking the historical background and counting the matched times. We also processed differently between the historical and non-historical model weights lower than threshold to update the model weights to reduce the false detection rate. Experiment results show that the proposed HBGMM can realize the function of remembering the scenes and adapt to the changes of scenes more quickly, thus decreasing the false detection rate.