''Im thinking of pretending i have been invited elsewhere for christmas, and therefore stopping him visiting.''

That is avoiding the problem. ( plus you can not tell your son a lie!! Don't start that!)