If your pivots are 1/8'', then drill a 1/8'' hole. Countersink from the outside only and not too deep, Cadet scales are very thin. Make sure the edge of the pivot hole on the inside is sharp to elimiate any play.
But the most important thing is to have the pivots ground long enough so they stink inside the scales. The scales are the things that define the position of the pivots and therefore the overall tension. The more layers your mod has, the harder it is (and should be) to snap the final scale on.