Amateurs are handing victory to the Remain camp