I don't have any local knowledge I was just going by the fact that you named it as "Woolwich baths" in your changeset comment but the name was missing.

Though I checked Bing Streetside (which we can use) and it does show the name signposted https://www.bing.com/maps?cp=-33.840041~151.1689&lvl=17&dir=98.562&style=x&v=2&sV=1 so I've gone ahead and updated the name.