Blocks are stored as configuration entities

Blocks in Drupal 8 use the configuration entity system and are therefore easy to ship for your module and easy to work with once you are familiar with the configuration entity system. For example the standard profile ships with core/profiles/standard/config/install/block.block.bartik_powered.yml for the Powered by block:

id: bartik_powered
theme: bartik
weight: 10
status: true
langcode: en
region: footer
plugin: system_powered_by_block
settings:
  label: 'Powered by Drupal'
  module: system
  label_display: '0'
  cache: -1
visibility:
  path:
    visibility: 0
    pages: ''
  role:
    roles: {  }
  node_type:
    types:
      article: '0'
      page: '0'
  visibility__active_tab: edit-visibility-path

Blocks can be loaded based on their unique entity IDs (in the above case bartik_powered) with the configuration entity API. The configuration entity can be further manipulated, values changed, or the whole entity deleted as appropriate:

use \Drupal\block\Entity\Block;
$block = Block::load('bartik_powered');
$block->set('settings.label', 'Powered by the awesome Drupal');
$block->save();