A lack of investment, employment and "appalling" transport links are said to be putting the UK's coastal towns at risk.