The continuous-variable quantum key distribution with entanglement in the middle, a semi-device-independent protocol, places the source at the untrusted third party between Alice and Bob, and thus has the advantage of high levels of security with the purpose of eliminating the assumptions about the source device. However, previous works considered the collective-attack analysis, which inevitably assumes that the states of the source has an identical and independently distributed (i.i.d) structure, and limits the application of the protocol. To solve this problem, we modify the original protocol by exploiting an energy test to monitor the potential high energy attacks an adversary may use. Our analysis removes the assumptions of the light source and the modified protocol can therefore be called source-device-independent protocol. Moreover, we analyze the security of the continuous-variable source-device-independent quantum key distribution protocol with a homodyne-homodyne structure against general coherent attacks by adapting a state-independent entropic uncertainty relation. The simulation results indicate that, in the universal composable security framework, the protocol can still achieve high key rates against coherent attacks under the condition of achievable block lengths.