What on earth makes you think that the Tories are anti-immigrant/populist? Establishment, neo-liberal, globalist (the 'modernising' wing of the party at least) perhaps. You don't honestly think that most Tories want to cut the level of cheap labour from overseas? Complete fantasy.
It may have escaped your notice but that the EU Referendum would not have been won by Leave without the votes of a significant number of voters who would be classified as Labour's core demographic. But the standard response is to condescendingly claim to 'understand their concerns' while simultaneously dismissing them as ill informed, stupid and racist. Yep, that's going to win voters back to the left.
That's why Corbyn and co are failing miserably in the polls and why they will probably be obliterated at the next election.