Controller X button is hard to press

Me too, it's the same with one of my Xbox controllers; either the button is broken/rusted or it has taken a hit and gets stuck and then unsticks at a certain point.