View yielding multiple blocks with pagers shows unexpected behavior

I have a view that call all nodes of a certain type. I have added several blocks to create categories within that node and show them nicely inside a panel page as lists of categories. I just noticed, however, that when I use the pager for any of the blocks (some of them have several pages of nodes that I have enabled pager or mini pagers for), all the pagers on the panel page layout change pages as well. Is that by design? Can I turn it off somehow? I just want the particular block owner of the pager in use to change, not everything else in the page.