I looked at passporting in more detail, and it means exactly what I had understood it to mean in the event of us not coming to some sort of agreement with the EU.
"The impact of lost passporting rights on the UK financial services sector and its economy could be huge. About 5,500 British financial services firms have passporting rights pre-Brexit. The loss of EEA passporting rights could mean the disruption of as much as 20 percent of the UK’s investment and capital markets revenue. In just a few years after losing the passport, the UK could lose 10,000 finance jobs, which could have a serious impact on the economy, especially since those jobs tend to be higher-paying. Without regulatory equivalence post-Brexit, the UK financial services industry could lose as many as 35,000 jobs. That could mean a loss of £5 billion of tax revenue seven percent of the UK’s total economic output."
https://www.investopedia.com/terms/p/passporting.asp
You'll have to tell me what you think I'm missing.