With the recent advances in ultrahigh intensity lasers, exotic astrophysical phenomena can be investigated in laboratory environments. Collisionless shock in a plasma, prevalent in astrophysical events, is produced when a strong electric or electromagnetic force induces a shock structure in a time scale shorter than the collision time of charged particles. A near-critical-density (NCD) plasma, generated with an intense femtosecond laser, can be utilized to excite a collisionless shock due to its efficient and rapid energy absorption. We present electrostatic shock acceleration (ESA) in experiments performed with a high-density helium gas jet, containing a small fraction of hydrogen, irradiated with a 30 fs, petawatt laser. The onset of ESA exhibited a strong dependence on plasma density, consistent with the result of particle-in-cell simulations on relativistic plasma dynamics. The mass-dependent ESA in the NCD plasma, confirmed by the preferential reflection of only protons with two times the shock velocity, opens a new possibility of selective acceleration of ions by electrostatic shock.