EntRef by Views doesn't search profile field (returns empty)

I have 2 profile fields (First Name & Last Name) in my user entity. So I created a view adding a EntRef display with the First Name & Last Name fields. In the 'Search fields' settings in the view I have checked First Name & Last Name. Finally, I associated the view to my EntRef field.

I proceed to try to so a search in the field, but it returns empty result. However if I search by uid (which I did not assign in the view), it returns the First & Last Name.

What did I miss out to make it work?