The effects of turbulence in the very local interstellar medium (VLISM) have been proposed by Giacalone & Jokipii (2015) to be important in determining the structure of the Interstellar Boundary Explorer (IBEX) ribbon via particle trapping by magnetic mirroring. We further explore this effect by simulating the motion of charged particles in a turbulent magnetic field superposed on a large-scale mean field, which we consider to be either spatially-uniform or a draped field derived from a 3D MHD simulation. We find that the ribbon is not double-peaked, in contrast to Giacalone & Jokipii (2015). However, the magnetic mirror force still plays an important role in trapping particles. Furthermore, the ribbon's thickness is considerably larger if the large-scale mean field is draped around the heliosphere. Voyager 1 observations in the VLISM show a turbulent field component that is stronger than previously thought, which we test in our simulation. We find that the inclusion of turbulent fluctuations at scales ≳100 au and power consistent with Voyager 1 observations produces a ribbon whose large-scale structure is inconsistent with IBEX observations. However, restricting fluctuations to <100 au produces a smoother ribbon structure similar to IBEX observations. Different turbulence realizations produce different small-scale features (≲10°) in the ribbon, but its large-scale structure is robust if the maximum fluctuation size is ≲50 au. This suggests that the magnetic field structure at scales ≲50 au is determined by the heliosphere-VLISM interaction and cannot entirely be represented by pristine interstellar turbulence.