2 different taxonomy_term views with same path for 2 different vocabularies

Hi guys,
I currently have two different vocabularies (vA and vB).
Vocabulary_A is associated with content type, Content_A; Vocabulary_B is associated with content type, Content_B.

I created two view pages under taxonomy_term views, one for Vocabulary_A, and one for Vocabulary_B. Both have the same page view path: taxonomy/term/%, but each view page has different "Taxonomy: Term ID (with depth)" argument with "taxonomy term" validator. Like the Vocabulary_A page view has "taxonomy term" validator = Vocabulary_A box checked, and Vocabulary_B page view has "taxonomy term" validator = Vocabulary_B box checked. Rest argument settings remain default.

Now the view page only works for Vocabulary_A, but it doesn't work Vocabulary_B. Every time when I go to a term under Vocabulary_B, it always shows "Page Not Found".

Does anyone know how to fix this problem? Do I need to set any other argument to make this work? It seems like there's some argument not passing through the url. I hope I have explained it.

Many Thanks,
Jerry