An improved Hidden Markov Models (HMM) was proposed to recognize the user's behavior. In order to improve the learning efficiency of Baum-Welch algorithm in HMM, and to solve the problem of initial sensitivity, the improved GA s used to optimize the initial parameters of HMM, in which the Chaos operator is utilized to avoid the problem of stagnation and premature convergence of the traditional GA in the convergence process. Finally, the experiment results based on ADLs data in UCI show the algorithm's availability and reliability for user's behavior recognition.