We introduce a new method for improving the coverage accuracy of confidence intervals for means of lattice distributions. The technique can be applied very generally to enhance existing approaches, although we consider it in greatest detail in the context of estimating a binomial proportion or a Poisson mean, where it is particularly effective. The method is motivated by a simple theoretical result, which shows that, by splitting the original sample of size n into two parts, of sizes n_1 and n_2=n-n_1, and basing the confidence procedure on the average of the means of these two subsamples, the highly oscillatory behaviour of coverage error, as a function of n, is largely removed. Perhaps surprisingly, this approach does not increase confidence interval width; usually the width is slightly reduced. Contrary to what might be expected, our new method performs well when it is used to modify confidence intervals based on existing techniques that already perform very well--it typically improves significantly their coverage accuracy. Each application of the split sample method to an existing confidence interval procedure results in a new technique.