You are quite correct about the garden polygon which is an enclosed area feature. But adding the tag barrier=fence to the area feature, then joining another area similarly tagged feature to it makes for two fences on the join which is incorrect. In cases like this, the fence ought to be added as a separate feature thus avoiding duplication.

There are actually more problems, one is that there are many gaps in the barrier lines that provide highway access into the garden areas, The barrier doesn't enclose the entire property. Another is that these are not all fences, it can be seen that some barriers are walls or hedges. Also I very much doubt if the barriers extend through the middle of the semi-detached houses. 

You've also mapped the gardens as extending over the buildings.

Regards Bernard.