It's not misleading for houses to be slightly off from what they are, as long as positioning is right. It's not perfect, but it's still accurate enough to remain in OSM.

If you're not happy with the accuracy of these buildings, then engage with the people adding them in, but once they've been added in, the right thing to do is to fix them, not delete them.