I agree, calling people scum won't win back the red wall. People who voted brexit weren't scum.
They were misled, fooled, ignorant, ill informed, lazy in their research, exhibited poor judgement, unwise, were compromised in their usual reasoning process etc etc. The best single word I can think of that applies to each of these synonyms is 'stupid', but the problem with that is while it is accurate, people don't like to confront the idea that they themselves have been stupid, even though we all are. Also, because there are all sorts of definitions of stupidity, when our opponent gives us a label, we tend to leap to a more negative one than they intend.