Are you still here on the forums?
The nurse in me got curious and did some reading. Apparently his diabetes can affect swimmer quality. I would think if its pretty well controlled it would minimize that risk. So his diabetes might make it more difficult, but not impossible.
If you've been trying for a long time (especially if >1 year), you can talk to your doc. They can do an analysis of his swimmers, and some testing on you too. There are some simple things that can cause infertility that are easily fixed. I had a friend ready to do IVF when they realized it was her thyroid that was off. They put her on thyroid medicine and she conceived the next cycle.
Don't feel bad about seeing docs for this stuff. That's what they're there for.