First Minister Mark Drakeford has said there are "too many second homes" in some parts of Wales that are affecting communities and causing housing prices to rise sharply.