Hello,

The doping alters the band structure of the semiconductor so that there are "missing" electrons (holes) in the valence band. This allows other electrons to "move" from an atom to a nearby one without jumping into the conduction band: they fill a hole "near to them", leaving a hole "behind them". This mechanism is modeled by virtual charges (the holes) moving in the opposite direction. All this happens in the valence band, and this is (intuitively) the reason why the mobility of holes is less than that of electrons. Actual conduction is always due to moving electrons, but when conduction happens in the valence band all is more "difficult" (the energy of the moving electrons filling hole after hole is less than the energy they would have if they were in the conduction band). 

Hope this helps.....