Two zinc-diphosphonates driven by 4,4'-bipyridine (4,4'-bipy), [Zn4(HEDP)2(4,4'-bipy)2]·4.5(H2O) (1) and (H2-4,4'-bipy)0.5·[Zn3(HEDP)(H2EDP)(H2O)]·1.5(H2O) (2), were solvothermally prepared (HEDP = CH3C(OH)(PO3)2, 1-hydroxyethylidenediphosphonate). Compound 1 features a layered structure with the Zn-HEDP chains as supramolecular units and 4,4'-bipy as interchain linkers. Compound 2 possesses a Zn-HEDP layer with protonated 4,4'-bipy as template. The structure differences between the two compounds are mainly ascribed to the distinct role of 4,4'-bipy in the process of in situ assembly. In compound 1, the 4,4'-bipy moiety acts as coordinative unit to participate in the formation of the resulting architecture. However, in compound 2, the protonated 4,4'-bipy serves as template to direct the formation of the final structure. The luminescence and proton conduction of the two compounds were investigated. Compounds 1 and 2 exhibit intense blue fluorescence; compound 2 has a proton conductivity of 2.61 × 10-4 S cm-1 at 100% relative humidity and 70 °C.