The relationship between the structure and superconductivity of Bi4O4S3 powders synthesized by heating under ambient and high pressures was investigated using synchrotron X-ray diffraction, Raman spectroscopy, and transmission electron microscopy (TEM) observation. The Bi4O4S3 powders synthesized under ambient pressure exhibited a strong superconductivity (diamagnetic) signal and zero resistivity below ∼4.5 K, while the Bi4O4S3 powder synthesized by the high-pressure method exhibited a low-intensity signal down to 2 K. Further annealing of the latter Bi4O4S3 powder under ambient pressure led to the development of a strong signal and zero resistivity. The crystal structures of all Bi4O4S3 phases consisted of Bi4O4Bi2S4 blocks including a Bi-S layer and anion(s) sandwiched between Bi4O4Bi2S4 blocks, but minor structural differences were detected. A comparison of the structures of the superconductive and nonsuperconductive Bi4O4S3 samples suggested that the superconductive Bi4O4S3 phases had slightly smaller lattice parameters. The average structures of the superconductive Bi4O4S3 phases were characterized by a slightly shorter and less bent Bi-S plane. Raman spectroscopy detected vibration of the S-O bonds, which can be attributed to sandwiched anion(s) such as SO4(2-). TEM observation showed stacking faults in the superconductive Bi4O4S3 phases, which indicated local fluctuation of the average structures. The observed superconductivity of Bi4O4S3 was discussed based on impurity phases, enhanced hybridization of the px and py orbitals of the Bi-S plane within Bi4O4Bi2S4 blocks, local fluctuation of the average structures, compositional deviation related to suspicious anion(s) sandwiched between Bi4O4Bi2S4 blocks, and the possibility of suppression of the charge-density-wave state by enriched carrier concentrations.