panel pages using TID works in preview, but otherwise gives empty pages

I want to create taxonomy term pages with a panel displaying only nodes of one specific content type.

I created a view, using TID as argument. The page for that works fine.
However when use this view in a panel page, it shows up with a blank column. Other content (like a node) shows up where it's supposed to be.
I tried using the views-pane, as well as both the page and pane views. As well as adding a new taskhandler taxonomy/term/%.

However, when I used either the page or the panel-page as view, and add the TID the live preview works fine.