Fractal structures are very common in the phase space of nonlinear dynamical systems, both dissipative and conservative, and can be related to the final state uncertainty with respect to small perturbations on initial conditions. Fractal structures may also appear in the parameter space, since parameter values are always known up to some uncertainty. This problem, however, has received less attention, and only for dissipative systems. In this work we investigate fractal structures in the parameter space of two conservative dynamical systems: the standard nontwist map and the quartic nontwist map. For both maps there is a shearless invariant curve in the phase space that acts as a transport barrier separating chaotic orbits. Depending on the values of the system parameters this barrier can break up. In the corresponding parameter space the set of parameter values leading to barrier breakup is separated from the set not leading to breakup by a curve whose properties are investigated in this work, using tools as the uncertainty exponent and basin entropies. We conclude that this frontier in parameter space is a complicated curve exhibiting both smooth and fractal properties, that are characterized using the uncertainty dimension and basin and basin boundary entropies.