How do I create a voting for Taxonomy terms?

I have been searching and found nothing that will do what I need. But it could be I just missed it.

In my website, I want users to have the current experience:

A user wishes to read a node that is about a certain subject (read: Taxonomy tag, as nodes can have multiple subjects)
When searching the directory, the nodes will be ordered by how relevant they are to that subject.
After selecting a node, and while reading it, a certain block will have a list of all the subjects this node is related to, with a value (between 1 to 100) of relevance.
Now, if the user thinks that the list is inacurate, he should have the ability to edit it, by "deleting" wrong tags, and "adding" correct tags, and changing the relevance value to whatever value the user chooses.
Now, this list is his personal list of tags for that node. And all searches will use this list of tags whenever possible (otherwise they will use the "public" list of tags).
The "public" list of tags will hold all tags users put on nodes, with the relevance as avrage (deleted nodes count as 0 relevance).
Finally, the list of existing tags and public list of tags should both be editable by permissions.

Can I do this thing with existing Drupal modules? If so, which ones, and in what way?
Any help would be greatfully appreciated.

With thanks in advance
Shany Topper