Unfortunately, not buying in to the GW thing puts you in the same camp as the people that reject it for entirely irrational reasons and, I think, this is why many people are timid to admit that they doubt it's reality.
I suppose that's what it comes down to: just because one side is dominated by denialists, doesn't mean the other side isn't. In this particular case, you've got cranks on both sides obscuring the truth.
Personally, I lean towards the GW is primarily man-made position (this may be influenced by wanting to lean in that direction), but I think it should be left to climatologists and engineers to decide what actions to take and how to allocate funds, not polititions and lobby groups.
I think we can all agree, though, that climate change is happening. Australia experienced the effects first-hand: longest drought in recorded history. We're still on water restrictions.