Yeah, but believe me it's still pretty disconcerting and scary under that mindset. Hard to find meaning or care for anyone in a world that you aren't sure is real.
I've suffered from severe and prolonged derealization after heavy mescaline abuse and subsequent psychotic mania, and while I agree it is disconcerting, I did not find it scary in the least. If anything it was liberating, in the sense that I didn't fear illness, injury or death, not that I sought them out or anything, the concept just didn't deter me from behaving how I wanted to. Even though today the derealization has faded, I still retain that lack of fear, and it's actually something I'm grateful for.
None of us are sure that the reality we perceive is the absolute nature of reality as a whole. If anything quantum physics has taught us that human perception is a huge limiting factor on our understanding of higher dimensional mechanics, super-symmetry and Planck relations. Such that true reality exists wholly within transcendental divisions of perception, ours being but one limited aspect of such.
Anyway, the point I'm making is that our only choice is to react to what we experience. Does it matter if it's real? Not in a pragmatic sense. Real is such a relative term anyway.