Small populations with low genetic diversity are prone to extinction. Knowledge on the genetic diversity and structure of small populations and their genetic response to anthropogenic effects are of critical importance for conservation management. In this study, samples of Ancherythroculter nigrocauda, an endemic cyprinid fish from the upper reaches of Yangtze River, were collected from five sites to analyze their genetic diversity and population structure using mitochondrial cytochrome b gene and 14 microsatellite loci. Haplotype diversity, nucleotide diversity, and expected heterozygosity indicated that the A. nigrocauda populations had low genetic diversity, and decreased heavily from 2001 to 2016. Significant genetic differentiation was found among different populations in the cyt b gene and SSR markers based on the genetic differentiation index (F ST), whereas no differentiation was found in 2001. Haplotype genealogy showed that eight out of 15 haplotypes were private to one population. The SSR STRUCTURE analysis showed that there were four genetic clusters in the A. nigrocauda samples, with each population forming a single cluster, except for the Chishui River (CSR) and Mudong River (MDR) populations, which formed a common cluster. Therefore, loss of genetic diversity and increased genetic differentiation were found in the A. nigrocauda populations, which could be attributed to dam construction, overfishing, and water pollution in the upper Yangtze River. It is therefore recommended that the government should ban fishing, control water pollution, increase river connectivity, and establish artificial breeding and stocking.