I had around 130 terms in a vocab and despite the first suggestion in this comment went ahead importing without increasing the phptimeout settings. Sure, enough, only 8-10 terms were imported.
Luckily, I had enabled the backup of taxonomy term data, but unfortunately, I noticed this problem only after doing plenty of work to rearrange the imported containers and categories into the new site structure.
Is there any way I can import just that vocab again without redoing all the work I have done?