Problem/Motivation

Currently, our git sensor checks for dirty working tree. That's a good start.
However, if production is switched to a non-conform branch (or not a branch at all), the code is still broken.
With a release managed workflow, even a specific tag should be fixed and everything that is different to it (diff, untracked files) is just wrong.

Proposed resolution

Change the gitDirtyWorkingTree sensor to a generic git sensor.

Execute multiple commands to check for untracked or modified files, and additionally to check if there is any diff to a configured branch / tag.

Remaining tasks

-

User interface changes

New setting "Reference", that is either a hash, branch or tag
The setting could offer a list of available branches or tags to pick in admin settings.

API changes

class rename: SensorGitDirtyTree => SensorGit
New setting "reference"

Comments

miro_dietiker’s picture

Component: Code » Sensors
miro_dietiker’s picture

Title: New sensor: Git branch / tag » Extend sensor: Git branch / tag
blueminds’s picture

Status: Active » Needs review
StatusFileSize
new11.02 KB

Provided textarea where whatever shell command can be provided to check.

miro_dietiker’s picture

Thank you!

From the description above and discussions we did, we learned that one command execution is not always enough. (especially if we would want to check if submodules are initialized properly and more...)

Now you're passing the problem to find the right command to the user. That was not the intention...
I would have expected an optional dropdown to pick a branch or a tag.

Let's see what Berdir wants for the initial release. We should remain simple.

miro_dietiker’s picture

Assigned: Unassigned » berdir
berdir’s picture

Hm. It's also a security problem. This basically opens up the possibility to do any kind of script execution through the UI, fun :)

blueminds’s picture

Okay, did something fancier. What will be a real challenge is test coverage :)

First what it does:
- You need to specify the repository path, which is validate, then it loads branches and tags
- You may select either expected branch or tag - currently both at once is possible which does not make sense, I guess simple JS action that will preselect empty value of the other would be just fine
- It checks for expected tag/branch and then it runs git status --porcelain

See the screenshots - first three are configuration, the last is the sensor detail.

For the tests I suggest to provide a test class that will override the "check" methods and will fail/pass as requested by tests by setting a variable or so. What do you think?

Status: Needs review » Needs work

The last submitted patch, 7: 2161767-git_sensor-2.patch, failed testing.

blueminds’s picture

Status: Needs work » Needs review
StatusFileSize
new28.61 KB
new16.05 KB

Here are tests as well + some minor fixes.

3 Tests will fail - will fix them. But the code is ready for review

Status: Needs review » Needs work

The last submitted patch, 9: 2161767-git_sensor-3.patch, failed testing.

The last submitted patch, 9: 2161767-git_sensor-3.patch, failed testing.

blueminds’s picture

Status: Needs work » Needs review
StatusFileSize
new26.02 KB
new3.67 KB

Lets try, merged manually number of conflicts...

Status: Needs review » Needs work

The last submitted patch, 12: 2161767-git_sensor-4.patch, failed testing.

blueminds’s picture

Status: Needs work » Needs review
StatusFileSize
new27.35 KB

yup... unexpected refactoring ;)

blueminds’s picture

StatusFileSize
new28.53 KB

Somehow the old sensor git dirty tree got in.

miro_dietiker’s picture

Status: Needs review » Needs work

A bit feedback here.

  1. +++ b/lib/Drupal/monitoring/Sensor/Sensors/SensorGit.php
    @@ -0,0 +1,314 @@
    +      if (strpos($value, '*') !== FALSE) {
    ...
    +      if (strpos($value, '*') !== FALSE) {
    ...
    +      if (strpos($branch, '*') !== FALSE) {
    ...
    +      if (strpos($tag, '*') !== FALSE) {
    

    The * should be the first posision. Not an unspecified one.

  2. +++ b/lib/Drupal/monitoring/Sensor/Sensors/SensorGit.php
    @@ -0,0 +1,314 @@
    +  protected function checkTag($tag) {
    ...
    +      throw new GitRepositorySensorException(format_string('Unexpected tag @current expecting @expected', array('@expected' => $tag, '@current' => $current_tag)));
    ...
    +  protected function checkBranch($branch) {
    ...
    +      throw new GitRepositorySensorException(format_string('Unexpected branch @current expecting @expected', array('@expected' => $branch, '@current' => $current_branch)));
    

    This is bad design. With a check function, you expect that the check might not be successful. It should be true or false and could have an argument by ref.

    Exceptions are for unexpected function / API behavior.

  3. +++ b/lib/Drupal/monitoring/Sensor/Sensors/SensorGit.php
    @@ -0,0 +1,314 @@
    +      '#options' => $this->getBranches(),
    ...
    +    foreach ($this->getBranches() as $key => $value) {
    ...
    +  protected function getBranches() {
    

    Agree, this "*" is the output of the shell command. But it's totally strange to have an API like that and callers need to parse it... We should return something better. Unsure: Is using local variables an option?

  4. +++ b/lib/Drupal/monitoring/Sensor/Sensors/SensorGit.php
    @@ -0,0 +1,314 @@
    +    if ($this->getRepositoryPath($values['repo_path']) == NULL) {
    +      form_set_error('repo_path', t('The provided path %path does not exist or is not a git repository.', array('%path' => $values['repo_path'])));
    

    Before it was optional. I was talking about possibly removing it from UI. Now you make it even mandatory? Now it's clearly wrong.

  5. +++ b/lib/Drupal/monitoring/Sensor/Sensors/SensorGit.php
    @@ -0,0 +1,314 @@
    +  protected function getRepositoryPath($path) {
    ...
    +    return $this->repositoryPath;
    ...
    +        $this->repositoryPath = $repo_path;
    ...
    +  protected $repositoryPath;
    ...
    +    if ($this->getRepositoryPath($this->info->getSetting('repo_path')) == NULL) {
    ...
    +    if ($this->getRepositoryPath($repo_path) == NULL) {
    ...
    +      $repo_path = $form_state['values'][monitoring_sensor_settings_key($this->getSensorName())]['repo_path'];
    ...
    +      $repo_path = $this->info->getSetting('repo_path');
    ...
    +    $repo_path = escapeshellarg($this->getRepositoryPath($this->info->getSetting('repo_path')));
    ...
    +  protected function execAtGitRoot($cmd) {
    ...
    +    return $this->exec("cd $repo_path\n$cmd 2>&1");
    

    Way too complex. You store it as local variable but never use it. And pathes are passed around and checked with git status like crazy.

  6. +++ b/test/tests/monitoring.core.test
    @@ -687,6 +663,104 @@ class MonitoringCoreTest extends MonitoringTestBase {
    +      $dir_cmd . "git status 2>&1" => array(
    ...
    +  function testGitSensor() {
    

    We cannot be sure git is just there. This needs a check first.

    The sensor needs to deal with missing git application in general and show an error message - for instance in the settings form.

blueminds’s picture

Status: Needs work » Needs review
StatusFileSize
new6.18 KB
new27.36 KB

1 - 3 fixed
4. it is either the drupal root, or a subdir within the drupal root. In the latter case you need to specify the path. So if drupal root = git root you do not need to do anything
5. do not understand the problem there. The repository path is checked and if found it is stored in local variable. As there are cases when we need to use other than setting value it accepts the argument path.
6. there is no git in tests - we simulate the console output. See exec() method where we switch the test console.

... and yes, i do not like the d.o tracker UX, if anything, i would like to see the previous comments ;)

miro_dietiker’s picture

Status: Needs review » Needs work

Some moore feedback. :-)

  1. +++ b/lib/Drupal/monitoring/Sensor/Sensors/SensorGit.php
    @@ -0,0 +1,298 @@
    +    return trim(shell_exec($cmd));
    ...
    +    $cmd_output = $this->execAtGitRoot('git branch');
    +    if (strpos($cmd_output, 'fatal') !== FALSE) {
    ...
    +    $cmd_output = $this->execAtGitRoot('git tag');
    +    if (strpos($cmd_output, 'fatal') !== FALSE) {
    

    That's not a reliable check. Use exec and check the third $return_val argument. It should be 0 in case of success. Otherwise, it's an error and i guess throwing an exception would make sense here.

  2. +++ b/monitoring.drush.inc
    @@ -44,11 +44,11 @@ function monitoring_drush_command() {
    +      'show-execAtGitRoot-time' => 'Relevant for the table output listing all results. Will expand the table with execution time info.'
    

    Bad replace. :-)

miro_dietiker’s picture

We're in beta and the sensor is not ready yet.

Thus i committed the escape fix in #2176897: Escape shell commands
Make sure to rebase / reroll the patch.