It exists on the aerial imagery, so perhaps it would be better to retag the track rather than deleting it. Otherwise a well-meaning mapper (who has not surveyed on the ground) may re-add it from aerial imagery in future.

I suggest tagging as access=private or disused:highway=track depending on whether the track completely doesn’t exist on the ground (even if you had a key for the gates, say).