We demonstrate magnetic confinement of an ultracold neutral plasma (UCNP) created at the null of a biconic cusp, or quadrupole magnetic field. Initially, the UCNP expands due to electron thermal pressure. As the plasma encounters stronger fields, expansion slows and the density distribution molds to the field. UCNP electrons are strongly magnetized over most of the plasma, while ion magnetization is only significant at the boundaries. Observations suggest that electrons and ions are predominantly trapped by magnetic mirroring and ambipolar electric fields, respectively. Confinement times approach 0.5 ms, while unmagnetized plasmas dissipate on a timescale of a few tens of microseconds.