You're right about the fencing, I see the very deliberate gap on the map, but see the street view (https://www.google.co.uk/maps/@56.0238204,-3.4318502,3a,71.2y,141.45h,88.09t/data=!3m7!1e1!3m5!1sSW_axill3VLqvawHNaONzg!2e0!6shttps:%2F%2Fstreetviewpixels-pa.googleapis.com%2Fv1%2Fthumbnail%3Fcb_client%3Dmaps_sv.tactile%26w%3D900%26h%3D600%26pitch%3D1.913866242945403%26panoid%3DSW_axill3VLqvawHNaONzg%26yaw%3D141.4549934927017!7i16384!8i8192?entry=ttu&g_ep=EgoyMDI1MDQwOC4wIKXMDSoJLDEwMjExNDU1SAFQAw%3D%3D) (looked exactly like this when I visited today).

The green sign indicates private property, no trespassing, and there's a small fence gently 'discouraging' pedestrians from continuing to the castle. The one page I could find talking about the castle suggests getting permission from the security first.

Suppose this is a case of 'technically private' vs 'no-one actually cares'.