Of course no one in their right mind expects wizards, magic or any other fantasy stuff. But really, was it magic that endeared Harry Potter to me? Definitely not. The two most striking features of Harry Potter were, in order named: 1) superb psychology, 2) lousy logic.
And considering that, how come Rowling went into detective genre? Yeah, you need the first feature there, but the second will come out anyway and destroy the whole.