How to remove >> in front of menu?

The name of that image is raquo.png

On or around line number 155 of style.css you'll find:

ul li,
ul.menu li,
.item-list ul li,
li.leaf {
  background: url('images/raquo.png') no-repeat left 0.833em;
  list-style-image: none;
  list-style-type: none;
}

You'll need to change the background to transparent:

background: transparent;

The above will remove the image for all list items. If you wish to just remove the image for the sidebar menu items you'll need to target those specifically:

#sidebar-first ul.menu li,
#sidebar-last ul.menu li {
  background: transparent;
}