The porous Ba12(BO3)66+ framework of the so-called "antizeolite" borates with channels along the c axis is capable of accommodating various guest anionic groups, e.g. [BO3]3-, [F2]2-, [F4]4-, and [(Li,Na)F4]3-. Taking as an example the Ba12(BO3)6[BO3][LiF4] crystal, we put forward the argument that the optical properties of "antizeolite" borates are strongly influenced by the degree of channel packing with anionic groups and, correspondingly, by the conjugated intrinsic defects. With the use of optical, electron-spin resonance, Raman spectroscopy, and ab initio calculations, it was shown that intrinsic defects largely impact the absorption of light in the visible and UV regions (the color of the bulk crystals can change from colorless to dark brown), absorption-edge position, dichroism, and other optical properties. The change in the optical absorption in the visible range is caused by the appearance of new states in the electronic structure inside the band gap, which are associated mainly with the presence of single and double F centers-fluorine vacancies that capture electrons-in [□F4]4-, [F2]2-, and [LiF4]3- groups. The formation of F centers in the [F2]2- group is the most energetically favorable. It has been found that Ba12(BO3)6[BO3][LiF4] crystals are optically active gyrotropic with an isotropic point at 499 nm at 300 K and are of interest for practical application as narrow-band light filters.