Two-particle correlations in relative azimuthal angle (Δø) and pseudorapidity (Δη) are measured in sqrt[s(NN)] = 5.02 TeV p+Pb collisions using the ATLAS detector at the LHC. The measurements are performed using approximately 1 μb(-1) of data as a function of transverse momentum (p(T)) and the transverse energy (ΣE(T)(Pb)) summed over 3.1 < η < 4.9 in the direction of the Pb beam. The correlation function, constructed from charged particles, exhibits a long-range (2 < |Δ η | < 5) "near-side" (Δø ~ 0) correlation that grows rapidly with increasing ΣE(T)(Pb). A long-range "away-side" (Δø ~ π) correlation, obtained by subtracting the expected contributions from recoiling dijets and other sources estimated using events with small ΣE(T)(Pb), is found to match the near-side correlation in magnitude, shape (in Δη and Δø) and ΣE(T)(Pb) dependence. The resultant Δø correlation is approximately symmetric about π/2, and is consistent with a dominant cos2Δø modulation for all ΣE(T)(Pb) ranges and particle p(T).