I'd probably use a T handle stubby driver on those, it's a little less brutal. Then chuck the screws and put some allen key cap screws in there 
Naw, it's just a matter of using the correct screwdriver... Pozi and Philips heads are definitely not the same thing.The advantage of the impact driver in this application is that the bits usually fit the screws better than than screwdriver tips do.