The hitting and mixing times are two fundamental quantities associated with Markov chains. In the literature, it is shown that the mixing times and maximum hitting times of reversible Markov chains on finite state spaces are equal up to some universal multiplicative constant. We use tools from nonstandard analysis to extend this result to reversible Markov chains on general state spaces that satisfy the strong Feller property. Finally, we show that the asymptotic equivalence of mixing and hitting times holds for a large class of Markov chains used in MCMC, such as typical Gibbs samplers and Metroplis-Hastings chains, even though they usually do not satisfy the strong Feller property.