I've gone ahead and reverted the address based name removals for now:
https://www.openstreetmap.org/changeset/147432472
https://www.openstreetmap.org/changeset/147432411

I think the question of whether these addresses are truly building names is an interesting one, and I may start a forum topic seeking other viewpoints.  It seems to me that for the most notable and well known of these buildings it probably does make sense that the name is the address.  For less notable ones I'm not so sure.