Authentication plugins system to have different authentication system available on remote entity.

In a first step and to prove the concept, we want to have two plugins (from core's authentication method):

  • basic HTTP
  • Cookie

Quickly an Oauth plugin could come.

The UI to configure the plugin will probably be in a first such as in remotev2.png

The plugins should have the following method, this list can be modified during the implementation depending on the problems encountered during the implementation:

  • checkIfAvailable: to test on the remote server if this authentication method is supported. Will be triggered during the validation of the website entity form
  • getJsonApiClient: to be called in the prepareJsonApiClient method of the RemoteManager service.
  • getClient: to be called in the prepareClient method of the RemoteManager service.

In addition, basic authentication (nothing to do with the plugin) can be added separately to the plugin list to be able to handle websites behind an HTTP authentication.

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

Grimreaper created an issue. See original summary.

grimreaper’s picture

StatusFileSize
new25.93 KB
new19.3 KB

Add UI sketches.

grimreaper’s picture

Assigned: Unassigned » grimreaper

Before having this plugin system. A dependency on the basic_auth module should be added.

  • Grimreaper authored 39def54 on 8.x-1.x
    Issue #2856713 by Grimreaper: Update dependencies
    
grimreaper’s picture

Assigned: grimreaper » Unassigned
grimreaper’s picture

Title: [D8] Authentication plugins » Authentication plugins
Version: » 8.x-1.x-dev
dakwamine’s picture

Hello,

Following the 2930801 issue about http auth, is there anything I can do to get the authentication plugin system completed?

grimreaper’s picture

Issue summary: View changes

Hello,

Thanks for your help.

I have updated the issue summary with the TODO list.

So if you want you can provide a patch for this issue.

grimreaper’s picture

Title: Authentication plugins » Authentication plugins and HTTP authentication
Version: 8.x-1.x-dev » 8.x-2.x-dev
grimreaper’s picture

Priority: Minor » Normal
grimreaper’s picture

Category: Task » Feature request
fathershawn’s picture

Are you open to integration in this system with Key module for credential storage? Self storage in config can be the fallback if Key module is not installed, but since config often gets exported and committed to version control, it's a problematic storage system for sensitive information.

grimreaper’s picture

Hello,

@FatherShawn, I am ok with that if a fallback is provided to not have hard dependency.

Thanks for the suggestion, nice idea :)!

fathershawn’s picture

checkIfAvailable: to test on the remote server if this authentication method is supported. Will be triggered during the validation of the website entity form

My first thought is pinging the server to ask what it supports. Are you thinking of a simple config schema for the server to declare supported auth types?

grimreaper’s picture

I think with core/lib/Drupal/Core/EventSubscriber/AuthenticationSubscriber.php::onKernelRequestFilterProvider(), it raises exception if the authentication method is not supported.

I think only with that it will be ok.

fathershawn’s picture

I'm still a bit confused between your comment in #15 and the UX proposal in remotev2.png. The wire frame looks like it is for the client interface, when adding a site at admin/config/services/entity_share/remote/%/edit. As I read the code in AuthenticationSubscriber you are proposing that an authentication method is chosen on the clientside and users discover that it is not supported via failure?

Or is your perspective that since a prerequisite for Entity Share is that both sites entities have the same structure then it is extremely likely that the admin on the client side already knows what auth methods are supported, so attempting an unsupported auth method is an edge case?

grimreaper’s picture

Ok, no problem for the confusion.

The plugin system will be in Entity share, for the client interface, yes the form you wrote about in comment 16.

This plugin system will have plugins like:

  • basic http
  • cookie
  • oauth

Similar to the authentication providers but a little bit different, for Entity share usage. And they will be in entity_share_client to allow the RemoteManager to work with.

fathershawn’s picture

I think I understand your plan. So I think in the scope of this issue, we create the plugin architecture, and move basic auth to a plugin. Then we tackle Oauth2 in it's own issue.

id.aleks’s picture

Issue tags: +LutskGCW20

Tagging for Drupal Global Contribution Weekend

fathershawn’s picture

I have a solution partially created and then the crush of immediate work came. I'll see if I can make progress on the weekend!

id.aleks’s picture

Thanks, @FatherShawn. It would be great if you can handle it.

id.aleks’s picture

Issue tags: -LutskGCW20
fathershawn’s picture

Assigned: Unassigned » fathershawn
fathershawn’s picture

StatusFileSize
new37.67 KB

Here's an patch to get your feedback. I've successfully used this to reach a server channel. Not sure on what you want tests to cover.

grimreaper’s picture

Status: Active » Needs work

Hi FatherShawn!

Huge thanks for this patch!

I have not tested it yet, but it looks promising. Below is my quick review. Please do not rework your patch yet, I making a lot of changes on 8.x-3.x with #3009221: Add an entity share client settings form.

  1. +++ b/modules/entity_share_client/config/schema/entity_share_client.schema.yml
    @@ -0,0 +1,38 @@
    +entity_share_client.remote.*:
    

    Why changing the file name?

    I prefer the old one.

    I use module_name.schema.yml for simple config.

  2. +++ b/modules/entity_share_client/entity_share_client.services.yml
    @@ -60,3 +59,13 @@ services:
    +      - [setKeyRepository, ['@?key.repository']]
    

    Nice! To avoid the dependency on the key module.

  3. +++ b/modules/entity_share_client/src/Entity/Remote.php
    @@ -97,4 +90,41 @@ class Remote extends ConfigEntityBase implements RemoteInterface {
    +      // DI not available in entities:
    

    Yes. That's why we can create a dedicated service for data manipulation. Or putting it in the remote manager service later.

  4. +++ b/modules/entity_share_client/src/Entity/RemoteInterface.php
    @@ -5,11 +5,40 @@ declare(strict_types = 1);
    +   *   Is this client for JSON operations?
    

    Thanks this gave me the idea on how I refactored the Remote Manager service on #3009221-13: Add an entity share client settings form

  5. +++ b/modules/entity_share_client/src/Form/RemoteForm.php
    @@ -79,6 +90,22 @@ class RemoteForm extends EntityForm {
    +    $selectedPlugin->validateConfigurationForm($form['auth']['data'], $subformState);
    

    Add a check if the plugin instanceof PluginFormInterface. As done on #3009221-13: Add an entity share client settings form.

fathershawn’s picture

My pleasure Grimreaper!

1. I needed to add key.type.entity_share_basic_auth: to the schema, so it wasn't just remote config anymore, which is why I changed the name. Drupal doesn't seem to care what the file is named.

5. Added in my branch - I'll put up a new patch when I hear back about #1.

grimreaper’s picture

Version: 8.x-2.x-dev » 8.x-3.x-dev
Parent issue: » #3082611: Entity share 3.x

Hi FatherShawn!

Now that #3009221-17: Add an entity share client settings form had been merged, if you have time, you can take a look at the new code organization.

I also have updated #3082611: Entity share 3.x to order the issues.

This issue can be worked on without needing the alpha blockers.

If you don't have time, I will take it when alpha blockers will be done and if I will still have the time.

Regards, :)

grimreaper’s picture

Assigned: fathershawn » grimreaper
fathershawn’s picture

Hi Grimreaper!

I'm continuing to refine our approach as we finalize our work for a client. I'll get an updated patch to you soon

grimreaper’s picture

Assigned: grimreaper » Unassigned
StatusFileSize
new46.78 KB

Hi @FatherShawn!

Glad to see your comment and the reaction time! :)

Patch from comment 24 does not apply on 8.x-3.x. I tried to update it, but the remoteManager changes bugged me. And I didn't understand any more all the plugins.

So I moved back to 8.x-2.x where it applied and tested it. It works very well. Thanks!

(Except if the authentication is wrong, you have a fatal error on the pull form instead of just no data)

I attached a screenshot. Because when seing the UI changes, it was now clear why there is the complexity of the credential_provider on top of the complexity of the new plugin type + optional integration with the Key module...

Wow... Such a complex patch. I feel like 2 or 3 issues are handled at the same time...

I see that in case the Key module is not available, now you store username + password in a state and no more in config.

I understand that config has drawbacks:

  • password is displayed in clear in the config
  • config is expected to be the same when deploying a project accross environments, and a login/password is not expected to be the same accross environments

But also config has some benefits:

  • It is overridable in settings.php, with environment variables for example
  • It is overridable using additional modules like config split

And also to have tested the separately the Key module, I saw that it has a feature of config override in the UI. But once done it is no more editable... And it is also very nice to see that the Key module supports environment variables.

So I am very confused because I can't merge patch from comment 24 (outside technical reason of it no more applying on 8.x-3.x). And I don't want to reject a such big amount of work.

I hesitate a lot, because, I would like to reduce the complexity of the patch, aka avoiding to have a double storage system (local or key) to implement for each authentication plugins, by forcing the usage of the Key module.

If people use the Key module using keys in config storage it will be the same level of security as storing login/password in the remote config. Only 1 or 2 additional steps of configuration. With the appropriate documentation I don't find it very difficult.

And at the same time, the decision to force people using the Key module is hard. But it would also be the best moment because there are still no tagged version for the 8.x-3.x branch.

The problem I also have is: Will every authentication plugin store sensitive info that will require the Key module?

As you can see, I have a lot of questions and hesitation in my mind right now.

Regards,

fathershawn’s picture

The response time is from the fact that I'm working on this plugin system today :) We are close to moving this to the client, so I'd like to keep it against 2.x but I'm happy to refactor it for 3.x going forward.

  1. (Except if the authentication is wrong, you have a fatal error on the pull form instead of just no data) I'll double check but I think I improved this since #24
  2. State versus Config In the first place, I understand it's your module to maintain, and defer to your decision. My perspective is that confidential information should never be committed to version control. Even via configuration override on many hosting environments that would require committing the credentials to a settings.*.php. Since for basic auth this data is a username/password pair using State closely mirrors the usual storage of such data.
    It is true that Key module has a configuration override, but the module README.md also notes this is not to be used for production. When Key module is used, in addition to an external key service, the simplest approach for easy management is to use a file based key. I include a Key plugin for this here as well. The credentials are placed in a json file outside the web root:
    {
    "username": "username value",
    "password": "password value"
    }

    and all that is stored in config is the path to the file, which can safely be overridden in a settings.*.php. I don't think Key module support on a particular plugin is forced here. the base plugin class has the provider set to the module and hidden:

    return $form + [
          'credential_provider' => [
            '#type' => 'hidden',
            '#value' => 'entity_share',
          ],
          'entity_share' => [
            '#type' => 'fieldset',
            '#title' => $this->t('Stored in local State'),
          ],
        ];
    

    so if an implementing plugin does not wish to support Key, the credentials just need to be in $form['entity_share'] as in

    $form['entity_share']['username'] = [
          '#type' => 'textfield',
          '#required' => FALSE,
          '#title' => $this->t('Username'),
          '#default_value' => $credentials['username'] ?? '',
        ];
    
        $form['entity_share']['password'] = [
          '#type' => 'textfield',
          '#required' => FALSE,
          '#title' => $this->t('Password'),
          '#default_value' => $credentials['password'] ?? '',
        ];
    
  3. I understand the concern about the size of the patch if you want me to move Key module support into a related issue, that could layer on, but it all pertains to authentication security.
grimreaper’s picture

Hi,

Thanks a lot @FatherShawn for your reply.

so I'd like to keep it against 2.x

No problem. And even if not merged in the 8.x-2.x, I would like to give you credit like I did for another contributor in #3009258-18: Allow skip synchronization for already synced entities.

but I'm happy to refactor it for 3.x going forward.

That would be amazing! Thanks.

My perspective is that confidential information should never be committed to version control.

I share your opinion. The paradox is that I have never been involved on a project using Entity Share... Only preparing features and stuff for teams using it. So I may miss some practical problems.

That's why I am more and more in favor of having a hard dependency on the Key module and simplify the patch.

I will try or ask one of my colleagues to write a patch with a second authentication method like Oauth to have a better idea of what will be required. And a better diff/view of the Key module support implication.

Not sure when I will have time for that.

We will see.

Thanks again!

fathershawn’s picture

I will try or ask one of my colleagues to write a patch with a second authentication method like Oauth to have a better idea of what will be required.

I have written such a plugin for the project that we have in progress. I haven't posted anything new while you were on holiday but we're happy to share.

grimreaper’s picture

Hi @FatherShawn,

If you want/can post it. Sure no problem, even if it needs to be reworked or not. That will still help!

yarik.lutsiuk’s picture

Assigned: Unassigned » yarik.lutsiuk
yarik.lutsiuk’s picture

StatusFileSize
new38.08 KB

Hello,

add rerolled patch from #24

Later will provide patch which will include #25 points.

Cheers

yarik.lutsiuk’s picture

StatusFileSize
new38.04 KB

my bad,
prev patch won't apply to latest version of 3.x, here is new one

fathershawn’s picture

We have continued to refine this work your team was on holiday. How do you want our improved code? An updated to 2.x patch?

grimreaper’s picture

Hello @FatherShawn,

your team was on holiday

???

As you want, @yarik.lusiuk is currently trying to implement an authentication plugin on Oauth on 3.x.

If you can provide a patch for 3.x great. If only for 2.x because it is the version you are using, no problem about that.

Regards,

fathershawn’s picture

OK - I have a working 2.x Oauth that I'll post today.

fathershawn’s picture

StatusFileSize
new10.7 KB

Here's what I'm doing for Oauth at the moment, but I would like to see a good part of this plugin moved to plugins on the Oauth2Client module. https://www.drupal.org/project/oauth2_client

fathershawn’s picture

StatusFileSize
new51.44 KB

And I am fairly sure I've teased out the auth plugin work as it currently stands in our combined branch with all improvements since the original patch.

fathershawn’s picture

for Oauth we could also drop the proposed dependency on Oauth2Client module and directly declare a dependency on league/oauth2-client via composer

grimreaper’s picture

Hello,

@FatherShawn, I have not tested Oauth modules yet, and planned to do at least when testing patches for this issue.

If we can avoid a dependency on a module, I would prefer it.

About the Key module, I think we will enforce this dependency for security reasons, do not rewrite your patches to enforce it. We will see that later.

Thanks all for the efforts on this issue.

Regards,

fathershawn’s picture

Thanks for the guidance, I'll proceed with the perspective that we would have a dependency on league/oauth2-client via composer.

yarik.lutsiuk’s picture

StatusFileSize
new51.45 KB

Hello,

patch for 8.x-3.x from #42

yarik.lutsiuk’s picture

StatusFileSize
new51.48 KB
new3.02 KB

small changes from #46

yarik.lutsiuk’s picture

StatusFileSize
new54.03 KB
new3.14 KB

Hello,

updated tests and i think we need updated tests more according to changes we made for remote.

Cheers

grimreaper’s picture

Hello everyone!

As the patch is progressing, maybe it is too late.

But so you think that creating sub issues to merge easily and progressively will be usefull? Also this would require more work...

I am thinking of the following steps:

  1. Create the authentication plugin with two plugins, basic auth and anonymous (#3096716: Allow Clients to connect to Server anonymously is merged now)
  2. Create an oauth authentication plugin
  3. Have configuration options, outside of the plugins (but somehow linked with the basic auth as it overlaps when making request), to be able to request a server behind an HTTP Password protection (maybe the shield module will be helpful for tests on that)
  4. Adapt authentication plugins to use the Key module and force its usage

Can you all please share your opinions?

And thanks all again for your work. This will be a huge milestone.

grimreaper’s picture

Hello,

Also, do you think this will help that I ask pull request to be enabled on this project?

yarik.lutsiuk’s picture

StatusFileSize
new54.19 KB

Hello,

rerolled patch according to new changes in 8.x-3.x

fathershawn’s picture

I think having Pull Request would be super helpful. I just met with my dev team and we are preparing all of our changes from while your office was unavailable to offer back for discussion.

It is clearly easier to review smaller changes, so making related issues and breaking up this work is understandable. Most of the numbered items are then improvements for 3.x? Especially #4?

grimreaper’s picture

Thanks @FatherShawn for the feedback.

I asked to enable the feature in #3152637-100: Opt-in to the Drupal.org Issue Forks and Merge Requests beta

I need to dive deep again into the patch to see what is going on.

Maybe a call with @FatherShawn, @yarik.lutsiuk, @ivan.vujovic and myself will help to clarify the situation and to easily organize us?

grimreaper’s picture

Hello,

The fork and pull request feature is now enabled.

grimreaper’s picture

Hello,

"Playing"/testing the new fork/MR feature.

Now I am going to manually test the patch.

grimreaper’s picture

Sorry for the delay.

I saw that the Key module integration is still optional. I still hesitate to make it a hard requirement seing the amount of work already done in the issue...

I haven't tested the Oauth plugin yet, because of the setup I am not used to.

And still nice work :)

grimreaper’s picture

Hello,

Thanks @FatherShawn for the comments.

Gitlab comments are super cool. Easier to mention someone on it.

We will test with @yarik.lutsiuk, how updating the MR will be.

When it will be ready, I will do commit the old fashion way to be sure to control comment and credit attribution.

We will test merging on smaller issues.

fathershawn’s picture

OK - I have some updates based on your comments - You want them as a patch then? I cloned the MR branch, but I'm not clear if I can push another branch. Happy for the MR feature and we definitely have stuff to figure out

yarik.lutsiuk’s picture

StatusFileSize
new59.31 KB
new8.86 KB

Hello,

here is new patch with fixed tests + Anonymous plugin + some changes mentioned in MR
later will commit into branch

Cheers

grimreaper’s picture

Status: Needs work » Needs review

Hello,

@FatherShawn, thanks for the proposition, but @yarik.lutsiuk is also working on it. Maybe we should avoid telescoping each others. I don't know exactly how drupal.org MR works, but if I can give you write access on the fork for this issue no problem.

@yarik.lutsiuk, thanks for the updated patch. I will review it.

Changing to needs review to trigger automated tests.

The last submitted patch, 24: entity_share-auth-plugins-2856713-24.patch, failed testing. View results

The last submitted patch, 42: issue-2856713-auth-plugin-42.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

fathershawn’s picture

we should avoid telescoping each others.

I'll keep following the discussion then and offer insight into my code as needed. I'm happy to jump back in if needed.

grimreaper’s picture

Status: Needs review » Needs work

Here is my review. I have not tested it yet.

One part comes from comment on the interdiff. The other one on the patch itself.

  1. +++ b/modules/entity_share_client/src/Annotation/ClientAuthorization.php
    @@ -1,5 +1,7 @@
    +declare(strict_types = 1);
    
    +++ b/modules/entity_share_client/src/Plugin/ClientAuthorization/Anonymous.php
    @@ -0,0 +1,63 @@
    +
    

    declare(strict_types = 1);

    I will make a check before merging. No more lose time with that.

  2. +++ b/modules/entity_share_client/src/Plugin/ClientAuthorization/Anonymous.php
    @@ -0,0 +1,63 @@
    +  public function validateConfigurationForm(array &$form, FormStateInterface $form_state) {}
    

    We can remove it if it is empty.

  3. +++ b/modules/entity_share_client/src/Plugin/ClientAuthorization/Oauth.php
    @@ -110,13 +110,13 @@ class Oauth extends ClientAuthorizationBase {
    +    return '';
    

    Need to update the PHPDoc because the method no more throws exception.

  1. +++ b/modules/entity_share_client/config/schema/remote.schema.yml
    @@ -11,9 +11,25 @@ entity_share_client.remote.*:
    +  type: sequence
    

    This will need to be completed I think. And need one for anonymous and oauth.

  2. +++ b/modules/entity_share_client/entity_share_client.install
    @@ -40,3 +41,34 @@ function entity_share_client_update_8302() {
    +    $plugin = $manager->createInstance('basic_auth');
    

    I think this needs to be updated. As now it can be an anonymous connection.

  3. +++ b/modules/entity_share_client/src/Plugin/KeyType/EntityShareOauth.php
    @@ -0,0 +1,31 @@
    + *   id = "entity_share_oauth",
    

    Maybe this also needs an entry in .schema.yml file.

    I don't know.

  4. +++ b/modules/entity_share_client/src/Service/KeyProvider.php
    @@ -0,0 +1,94 @@
    +   * @see maw_luminate.services.yml
    

    ?

The only important part is to update the hook_update_N for anonymous connections. The rest of the stuff I can do it or it can be done in sub-issues.

@yarik.lutsiuk, almost done!!! :)

PS: Ok @FatherShawn, thanks :)

fathershawn’s picture

Maybe this also needs an entry in .schema.yml file.

I don't know.

Good catch. Every plugin that stores configuration needs a schema entry. I've found https://www.drupal.org/project/config_inspector to be so helpful in checking for missing schema. We should be able to load any of our plugins and have admin/reports/config-inspectorbe error free.

grimreaper’s picture

Thanks for the confirmation @FatherShawn!

Yep I know Config inspector, it is a development module on all my projects :). So helpful :)

yarik.lutsiuk’s picture

Status: Needs work » Needs review

Pushed changes related to #65

Also,
2.

We can remove it if it is empty.

Don't think so, if we remove it, it will trigger validation from parent class and it cause notices on page(it will try to validate credentials which we don't have for anon connection).

1.

This will need to be completed I think. And need one for anonymous and oauth.

i checked scheme for other plugins(in Key module) and it looks same, there is no fields, so there is nothing to add into scheme.
2.

I think this needs to be updated. As now it can be an anonymous connection.

as current websites have only basic auth method, we have update them accordingly, to use basic auth plugin.

Cheers,

grimreaper’s picture

Thanks @yarik.lutsiuk for your commits/patches!

And for checking all the points.

2: in this case a comment should be added to ensure it is done awarely.
1bis: Ok. I opened some sub-issues in the meantime with one dedicated on checking that. I will take it when this issue will be merged. Maybe nothing has to be done. Just checking.
2: As #3096716: Allow Clients to connect to Server anonymously had been merged, I agree that 99% websites have basic auth connections, but some may have started using anonymous connection.

grimreaper’s picture

Hello,

Thanks @yarik.lutsiuk for the updated commit.

It is ok for me.

I have done a last review locally.

This is the last things I changed:

  • Fixing some typos and coding standards
  • Move Authorization plugins base/manager/interface into a dedicated folder to uniformize like the import processors
  • Renamed the authorization plugin manager service machine name

Triggering the automated tests.

If its is green. Then it is merging time!!!

  • Grimreaper authored 0d85a74 on 8.x-3.x
    Issue #2856713 by yarik.lutsiuk, FatherShawn, Grimreaper: Authentication...
  • FatherShawn authored 51dac79 on 8.x-3.x
    Issue #2856713 by yarik.lutsiuk, FatherShawn, Grimreaper: Authentication...
  • yarik.lutsiuk authored f417f57 on 8.x-3.x
    Issue #2856713 by yarik.lutsiuk, FatherShawn, Grimreaper: Authentication...

grimreaper’s picture

Assigned: grimreaper » Unassigned
Status: Needs review » Fixed

And this is merged now!!! \o/

Thanks everyone!!! Now the sub-issues!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.