Majestic Wine boss Rowan Gormley has warned a Brexit would push up the price of wine as the cost of imported goods will inevitably rise.