The tread compound and tread pattern have the most influence. The width is a very terrain specific factor and a wider tyre will be faster in some conditions.
Road: Given the same inflation pressure, a widertyre is faster. If you have 120psi in both a 20mm tyre and a 23mm tyre, the wider tyre will be faster. The contact patch of the wider tyre will have rounder shape while the narrow tyre will have an elliptical shape. The elliptical profile means that the tyre casing is distorted along more of its length and this distortion of the casing is what creates rolling resistance. A wider tyre is a heavier tyre though.
Mountain: In smooth hardpack with a gravel surface a narrow tyre inflated to a higher pressure will be faster but in rocky terrain a wider tyre can be faster because it absorbs more of the bumps that would otherwise slow you down.