Barbour introduced a probabilistic view of Stein's method for estimating the error in probability approximations. However, in the case of approximations by general distributions on the integers, there have been no purely probabilistic proofs of Stein's bounds till this paper. Furthermore, the methods introduced here apply to a very large class of approximating distributions on the non-negative integers, among which there is a natural class for higher-order approximations by probability distributions rather than signed measures (as previously). The methods also produce Stein magic factors for process approximations which do not increase with the window of observation and which are simpler to apply than those in Brown, Weinberg and Xia.