Flow over a parallel cantilevered flag in the vicinity of a rigid wall is numerically studied using an immersed boundary-lattice Boltzmann method (IB-LBM) in two-dimensional domain, where the dynamics of the fluid and structure are, respectively, solved by the LBM and a finite-element method (FEM), with a penalty IB to handle the fluid-structure interaction (FSI). Specifically, a benchmark case considering a plate attached to the downstream of a stationary cylinder is first conducted to validate the current solver. Then, the wall effects on the flag are systemically studied, considering the effects of off-wall distance, structure-to-fluid mass ratio, bending rigidity, and Reynolds number. Three flapping modes, including symmetrical flapping, asymmetrical flapping, and chaotic flapping, along with a steady state are observed in the simulations. It is found that the flag is vibrating or stable with a mean angle inclined in the fluid when it is mounted in the vicinity of a rigid wall. The mean inclined angle first increases in the steady state and then decreases in the unsteady state with the off-wall distance. In the unsteady regime, the dependency of the inclined angle on the off-wall distance is similar to that of the gradient of the fluid velocity. In addition, the rigid wall near the flag decreases the lift and drag generation and further stabilizes the flag-fluid system. Contrarily, the flag inertia destabilizes the flag, and large flag inertia induces chaotic vibrating modes.