Filters aren't optional

I do not understand your problem. Could you give me more details? Filters are not exposed by default in a view, so please explain clearly your issue.

The patch you posted above is supposed to used custom filters, but none are present in the patch.

Moreover, the filter handlers are already defined in the views_data hook by each field widget, so I do not understand why you want to override them in the hook_views_data() function, because it is not the expected place.

And last thing, there are 7 field widgets (choice, color, date, file, list, numeric and text). Your patch take care of 3 of them, but ignore the others, defining for those remaining widgets a text filter handler that will not work with "file" widget for example.