Aside from ambient light noise, shot noise, and linear/nonlinear effects, strong low-frequency noise (LFN) severely affects the signal quality in LED-based visible light communication (VLC) systems, which hinders the implementation of data-driven end-to-end (E2E) deep learning approaches in real LED-VLC systems. We present a deep learning-based autoencoder to deal with this challenge. A novel modeling strategy is proposed to bypass the influence of the LFN and other low signal-to-noise ratio data when training the channel model of our E2E framework. The deep learning-based autoencoder then embeds the differentiable channel model and learns to combat the majority of channel impairments. In the E2E LED-VLC experiment, 1.875 Gbps transmission is achieved under the 7% HD-FEC threshold, 0.325 Gbps faster than the baseline. The E2E framework is robust to signal bias and amplitude variations, implying dimming support in the indoor environment.