It does seem scientifically improbable that we're the only planet in the entire universe that has developed life, even intelligent life. It's not a conspiracy theory to believe in aliens in general.

Whether aliens have been here or Roswell was what the government says can be debated, but I don't think anyone who understands how truly enormous the universe (or even this one tiny galaxy) actually are would argue against the sheer mathematical probability that multiple other bodies are likely to have met the conditions under which life can flourish.