Allen and Berkley's image source method (ISM) is proven to be a very useful and popular technique for simulating the acoustic room transfer function (RTF) in reverberant rooms. It is based on the assumption that the source and receiver of interest are both omnidirectional. With the inherent directional nature of practical loudspeakers and the increasing use of directional microphones, the above assumption is often invalid. The main objective of this paper is to generalize the frequency domain ISM in the spherical harmonics domain such that it could simulate the RTF between practical transducers with higher-order directivity. This is achieved by decomposing transducer directivity patterns in terms of spherical harmonics and by applying the concept of image sources in spherical harmonics based propagation patterns. Therefore, from now on, any transducer can be modeled in the spherical harmonics domain with a realistic directivity pattern and incorporated with the proposed method to simulate room acoustics more accurately. We show that the proposed generalization also has an alternate use in terms of enabling RTF simulations for moving point-transducers inside pre-defined source and receiver regions.