The Ising model in two dimensions with special toroidal boundary conditions is analyzed. These boundary conditions, which we call duality-twisted boundary conditions, may be interpreted as inserting a specific defect line ("seam") in the system, along noncontractible circles of the cylinder, before closing it into a torus. We derive exact expressions for the eigenvalues of a transfer matrix for the critical ferromagnetic Ising model on the M×N square lattice wrapped on the torus with a specific defect line. As a result we have obtained analytically the partition function for the Ising model with such boundary conditions. In the case of infinitely long cylinders of circumference L with duality-twisted boundary conditions we obtain the asymptotic expansion of the free energy and the inverse correlation lengths. We find that the ratio of subdominant finite-size correction terms in the asymptotic expansion of the free energy and the inverse correlation lengths should be universal. We verify such universal behavior in the framework of a perturbating conformal approach by calculating the universal structure constant C_{n1n} for descendent states generated by the operator product expansion of the primary fields. For such states the calculations of an universal structure constants is a difficult task, since it involves knowledge of the four-point correlation function, which in general is not fixed by conformal invariance except for some particular cases, including the Ising model.