The question is still open in the comments section and I haven’t been able to pin down anything in the World War II era, though there is plenty of material to be sifted through. In any case, as I mentioned in the comments yesterday, Hans Bethe was deep into fusion studies in the late 1930s, and I would bet somewhere in the immediate postwar issues of John Campbell’s Astounding we’ll track down the first mention of fusion driving a spacecraft.
My gut feeling on this is that it's going to turn out be later than the 1940s. Definitely by 1960 but it wouldn't surprise me too much if SF awareness of fusion reactions as a power source didn't gel until after Ivy-Mike in 1952.
What I recall from before that is fission or total conversion, without fusion in between them.