The transfer of harmonic between the AC and the DC system was investigated on the basis of the Fourier analysis of the voltages at both converter sides. Unbalanced AC voltages at AC side were represented by their positive and negative sequence components. The above method was used in the inductive filtering based HVDC system to analyze the voltage on the DC side of an HVDC converter bridge under unbalanced conditions. A comparison of the inductive filtering based HVDC system with the traditional DC transmission system in the same AC system fault conditions has shown that the former is superior to the latter for DC side harmonic amplitude suppression. In the end, referring to the parameters of the inductive filtering based HVDC system in the lab, a dynamic simulation was executed. The results obtained by switching the 2, 11 and 13 harmonics LC filters on the valve side of inductive filtering transformer have shown that the contents of non-characteristic harmonic voltage in DC voltage are suppressed significantly and the harmonic instability is suppressed effectively in the inductive filtering based HVDC system.