The resemblance of crystallographic and magnetic structures of the quasi-two-dimensional iridates Ba2IrO4 and Sr2IrO4 to La2CuO4 points at an analogy to cuprate high-Tc superconductors, even if spin-orbit coupling is very strong in iridates. Here we examine this analogy for the motion of a charge (hole or electron) added to the antiferromagnetic ground state. We show that correlation effects render the hole and electron case in iridates very different. An added electron forms a spin polaron, similar to the cuprates, but the situation of a removed electron is far more complex. Many-body 5d 4 configurations form which can be singlet and triplet states of total angular momentum that strongly affect the hole motion. This not only has ramifications for the interpretation of (inverse-)photoemission experiments but also demonstrates that correlation physics renders electron- and hole-doped iridates fundamentally different.Some iridate compounds such as Sr2IrO4 have electronic and atomic structures similar to quasi-2D copper oxides, raising the prospect of high temperature superconductivity. Here, the authors show that there is significant electron-hole asymmetry in iridates, contrary to expectations from the cuprates.