Can't delete

If you think there's a bug here, you need to explain in much more detail what it is...

You can remove images from a gallery via the "Remove" link you get when hovering over a thumbnail (if you're using the core Contextual Links module), or by clicking through to view the full-screen version of the image and then clicking on the "Remove" tab (next to the "Edit" tab at the top). Once #1055986: Add a preview sidebar to the media edit screen gets fixed, there will also be a checkbox on the edit screen that lets you remove items from there.

If you want to actually delete the image from your site (rather than just remove it from the gallery), then the Media module (not Media Gallery) lets you do that provided you've removed it from all places on the site that are using it; e.g., only if you first follow the steps above to remove it from the gallery. Then you should be able to delete it from your site entirely via the screen at admin/content/media.

Not sure what you mean by full view, full view brings the image up in lightbox - no remove option there.

Once you're in the lightbox, if you click on the image again I believe it takes you to a regular Drupal page showing the image, and that's where the Remove tab is.

(I think there's also an option for the module that gets rid of the lightbox entirely and makes it so clicking on the thumbnail images in the gallery takes you directly to the regular page where the Remove tab is, although that's probably not too common of a use case.)

i do have an edit media tab on my gallery page, going there though provides no option to remove or delete.

Right, once #1055986: Add a preview sidebar to the media edit screen gets committed, a series of remove checkboxes will appear on that page also (it will let you remove multiple images from the gallery at once), in addition to the checkboxes on the single-media-item edit screen.