The water damage of asphalt pavement shortens the service life of pavement, and it can be effectively improved by improving the cohesion of asphalt and the adhesion between asphalt and aggregate. In this paper, 3-aminopropyl trethoxysilane (KH550) was used to modify montmorillonite (NH2-MMT) so as to enhance the cohesion of styrene-butadiene-styrene block copolymer (SBS) modified asphalt. The granite (Gr) aggregate (NH2-GR) was modified by amino-silane KH550 to improve the adhesion between asphalt and aggregate. The montmorillonite and Gr aggregates were characterized by X-ray diffraction (XRD), scanning electron microscopy (SEM) and X-ray photoelectron spectroscopy (XPS). The adhesion work of NH2-MMT/SBS modified asphalt to NH2-GR aggregate is increased by 19%, and the peeling work is decreased by 24% by simultaneously increasing the cohesion of asphalt and the adhesion between asphalt and aggregate. Compared with base asphalt, the cohesion work of SBS modified asphalt, NaMMT/SBS modified asphalt and NH2-MMT/SBS modified asphalt is increased by 15%, 22% and 28%, respectively. Compared with base asphalt and Gr aggregate, the cohesion work of NaMMT/SBS modified asphalt and NH2-MMT/SBS modified asphalt is increased by 6% and 12%, respectively. Photoelectric colorimetry was used to test the adhesion between asphalt and aggregate. Compared with base asphalt/Gr aggregate, the UV absorption peak value of NH2-MMT-SBS modified asphalt /Gr aggregate is increased from 0.483 to 0.499. The UV absorption peak value of NH2-MMT-SBS modified asphalt/NH2-GR aggregate increases from 0.474 to 0.557. The results show that the simultaneous modification of cohesive force and adhesive force can effectively improve the water damage resistance of asphalt, and the effect of modified aggregate is more obvious.