I *think* this is what I'm seeing...

Here's the flow:

        if (empty($this->definition['is revision'])) {
          $entities = entity_load($entity_type, $entity_ids);
          $keys = $entity_ids;
        }

So $keys is now an array of entity IDs.

Then:

        foreach ($keys as $key => $entity_id) {
          // If this is a revision, load the revision instead.
          if (isset($entities[$entity_id])) {
            $values[$key]->_field_data[$this->field_alias] = array(
              'entity_type' => $entity_type,
              'entity' => $entities[$entity_id],
            );
          }
        }
      }

We iterate over $keys, and use the key of $keys (!!!) as a key in $values. (!!!!!)

      // Now, transfer the data back into the resultset so it can be easily used.
      foreach ($values as $row_id => &$value) {
        $value->{'field_' . $this->options['id']} = $this->set_items($value, $row_id);
      }

We're iterating over $values and... oh wait! We're calling its key $row_id. But a moment ago it was an entity ID!