The Planning Inspectorate has dismissed an appeal challenging South Hams District Council’s decision over the future of a community cafe site.