The Drupal.org Security Team and Infrastructure Team has discovered unauthorized access to account information on Drupal.org and groups.drupal.org.

This access was accomplished via third-party software installed on the Drupal.org server infrastructure, and was not the result of a vulnerability within Drupal itself. This notice applies specifically to user account data stored on Drupal.org and groups.drupal.org, and not to sites running Drupal generally.

Information exposed includes usernames, email addresses, and country information, as well as hashed passwords. However, we are still investigating the incident and may learn about other types of information compromised, in which case we will notify you accordingly. As a precautionary measure, we've reset all Drupal.org account holder passwords and are requiring users to reset their passwords at their next login attempt. A user password can be changed at any time by taking the following steps.

  1. Go to https://drupal.org/user/password
  2. Enter your username or email address.
  3. Check your email and follow the link to enter a new password.
    • It can take up to 15 minutes for the password reset email to arrive. If you do not receive the e-mail within 15 minutes, make sure to check your spam folder as well.

All Drupal.org passwords are both hashed and salted, although some older passwords on some subsites were not salted.

See below recommendations on additional measure that you can take to protect your personal information.

What happened?

Unauthorized access was made via third-party software installed on the Drupal.org server infrastructure, and was not the result of a vulnerability within Drupal itself. We have worked with the vendor to confirm it is a known vulnerability and has been publicly disclosed. We are still investigating and will share more detail when it is appropriate. Upon discovering the files during a security audit, we shut down the association.drupal.org website to mitigate any possible ongoing security issues related to the files. The Drupal Security Team then began forensic evaluations and discovered that user account information had been accessed via this vulnerability.

The suspicious files may have exposed profile information like username, email address, hashed password, and country. In addition to resetting your password on Drupal.org, we are also recommending a number of measures (below) for further protection of your information, including, among others, changing or resetting passwords on other sites where you may use similar passwords.

What are we doing about it?

We take security very seriously on Drupal.org. As attacks on high-profile sites (regardless of the software they are running) are common, we strive to continuously improve the security of all Drupal.org sites.

To that end, we have taken the following steps to secure the Drupal.org infrastructure:

  • Staff at the OSU Open Source Lab (where Drupal.org is hosted) and the Drupal.org infrastructure teams rebuilt production, staging, and development webheads and GRSEC secure kernels were added to most servers
  • We are scanning and have not found any additional malicious or dangerous files and we are making scanning a routine job in our process
  • There are many subsites on Drupal.org including older sites for specific events. We created static archives of those sites.

We would also like to acknowledge that we are conducting an investigation into the incident, and we may not be able to immediately answer all of the questions you may have. However, we are committed to transparency and will report to the community once we have an investigation report.

If you find that any reason to believe that your information has been accessed by someone other than yourself, please contact the Drupal Association immediately by sending an email to password@association.drupal.org. We regret this occurred and want to assure you we are working hard to improve security.

Thank you,
Holly Ross
Drupal Association Executive Director

FAQ

What happened?

The Drupal.org Security Team and Infrastructure Team has identified unauthorized access to user information on Drupal.org and groups.drupal.org, which occured via third-party software installed on the Drupal.org server infrastructure.

What information of mine was exposed?

The information includes username, email address, hashed passwords, and country for some users. However, we are still investigating the incident and may learn about other types of information compromised, in which case we will notify you accordingly.

Was my credit card information exposed?

We do not store credit card information on our site and have uncovered no evidence that card numbers may have been intercepted. However, we are still investigating the incident and may learn about other types of information compromised, in which case we will notify you accordingly.

Were projects or hosted drupal.org code altered?

We have no evidence to suggest that an unauthorized user modified Drupal core or any contributed projects or packages on Drupal.org. Software distributed on Drupal.org is open source and bundled from publicly accessible repositories with log histories and access controls.

Does this affect my own Drupal site?

This notice applies specifically to user account data stored on Drupal.org and groups.drupal.org, and not to sites running Drupal generally. However, we recommend that you follow best practices and follow any security notices from Drupal.org or third party integrations to keep your site safe. Resources include the following sites:

How did the access happen?

Unauthorized access was made via third-party software installed on the Drupal.org server infrastructure, and was not the result of a vulnerability within Drupal itself. We have worked with the vendor to confirm it is a known vulnerability and has been publicly disclosed. We are still investigating and will share more detail when it is appropriate.

What has been done to prevent this type of unauthorized access in the future?

There have been several infrastructure and application changes including:

  • Open Source Lab, the group that hosts the servers for Drupal and infrastructure teams rebuilt production, staging, and development webheads
  • GRSEC secure kernels were added to most servers
  • An anti-virus scanner was run over file servers, and run routinely to detect malicious files being uploaded to the Drupal.org servers.
  • We hardened our Apache web server configurations
  • We made static archives of any site that has been end-of-lifed and will not be updated in the future
  • Sites that were no longer going to receive feature or content updates were converted to static copies to minimize maintenance.
  • We removed old passwords on sub-sites and non-production installations

Do you have any information about the identity of the person or group who did this?

At this point there is no information to share.

What is the security team doing to investigate the unauthorized access?

We have a forensics team made up of both Drupal Association staff and trusted community volunteers who are security experts investigating.

How is my Drupal.org password protected?

Passwords on Drupal.org are stored in a hashed format. Currently, passwords are both hashed and salted using multiple rounds of hashing (based on PHPass). Passwords on some subsites were not salted.

Who maintains the Drupal.org site?

The Drupal Association is responsible for maintaining the site, with the assistance of many trusted Drupal community volunteers.

How can I delete my profile rather than create a new password?

Please email password@association.drupal.org with the request.

What else can I do to protect myself?

First, we recommend as a precaution that you change or reset passwords on other sites where you may use similar passwords, even though all passwords on Drupal.org are salted and hashed. Some older passwords on some subsites were not salted. To make your password more secure:

  • Do not use passwords that are simple words or phrases
  • Never use the same password on multiple sites or services
  • Use different types of characters in your password (uppercase letters, lowercase letters, numbers, and symbols).

Second, be cautious if you receive e-mails asking for your personal information and be on the lookout for unwanted spam. It is not our practice to request personal information by e-mail. Also, beware of emails that threaten to close your account if you do not take the "immediate action" of providing personal information.

Although we do not store credit card information, as a precaution we recommend you closely monitor your financial accounts if you made a transaction on association.drupal.org or if you use a password with your fianancial institution that is similar to your Drupal.org password. If you see unauthorized activity (in the U.S.), we also suggest that you submit a complaint with the Federal Trade Commission ("FTC") by calling 1-877-ID-THEFT (1-877-438-4338).

Based on the results of the investigation into this incident, we may update the FAQs and may recommend additional measures for protecting your personal information.

Comments

christefano’s picture

For anyone who's interested, Ars Technica just ran an article about the current state of password cracking:

   http://arstechnica.com/security/2013/05/how-crackers-make-minced-meat-ou...

If the largest corpus of Drupal developer passwords is really now out there in the wild, this can affect the security of all sites (Drupal or not) where the same Drupal developers have accounts and utilize the same (or similar) passwords.

It's not all bad news, though. Now there are more Drupal.org sites available via SSL!



Christefano  
Founder, CEO
Large Robot
954-247-4786
http://www.largerobot.com
greenwork’s picture

Nice article. Time to change all of our passwords on every site.

====================================
Website:LEED Exam V4

abasso’s picture

Nice article, however Drupal 7 password should not be hashed and salted just once, but 16385 times with stretching (if I read the notes correctly), it also uses SHA-512 instead of MD5 for the hashing. See https://api.drupal.org/api/drupal/includes%21password.inc/function/_pass...

greenwork’s picture

according to the notes it was not all salted but mostly salted

although some older passwords on some subsites were not salted.

I would still suggest people change their passwords for sure.

====================================
Website:LEED Exam V4

walterbyrd’s picture

I am logged in as myself.

But, when I go to: https://drupal.org/user/password

I get the message: "Access denied . . . You are not authorized to access this page."

Anonymous’s picture

How can I delete my drupal.org account? I can't seem to find this option.

holly.ross.drupal’s picture

Send a request to password@association.drupal.org. We would normally send you to an issue queue to do that, but we are trying to expedite the process for folks today. Thanks so much, and let me know if you have other questions.

Anonymous’s picture

I was searching for the same info on my profile, glad I've come across this comment.

Wolf_22’s picture

Have we identified the specific software responsible for all this? Something tells me this is a piece of information that everyone should be entitled to know--if not for the sake of curiosity, then definitely so that other server admins can avoid using it to help the internet rid itself of yet one more POS application.

However, if it HAS been identified but not released to the general public, that makes me likely to assume that it's not software tailored to managing servers but rather just someone's personal stash of torrents, etc., in which case, the person responsible should be removed from the Circle of Trust.

Also, have the authorities been contacted about this? Seems serious enough to merit that type of reaction... I mean, we're only talking about what, a million or more users? How many users are we talking about here?

All that aside, it's nice to know that the Drupal system itself isn't responsible for this. That would really take some wind out of peoples' sails, not to mention the entire Drupal initiative...

TolstoyDotCom’s picture

It'd be nice to know the vulnerable software, and it would also be nice to know what the malicious code consisted of. Was this just a drive-by, or was Drupal specifically targeted? Did the commands they issued indicate knowledge of Drupal's db tables? Was this leading up to redirecting search results or other commercial activity, or was obtaining the user list the primary goal?

pandaski’s picture

Second the above, please release more details.

Heine’s picture

"We would also like to acknowledge that we are conducting an investigation into the incident, and we may not be able to immediately answer all of the questions you may have. However, we are committed to transparency and will report to the community once we have an investigation report."

pandaski’s picture

Many thanks for the hard work

RobinJ’s picture

I second this. I want to make sure that this software does not exist on any of the e-commerce sites that I maintain until it is patched.

Owen Barton’s picture

The security and infrastructure teams have been doing lots investigation into the issue and are trying to get all the details right before we make a detailed technical post. There are already some issues in the queues of some modules and patches committed to others that would help prevent this kind of problem in the future on drupal.org and other sites. We basically got to a point where we wanted to let users know their information had been compromised before we got to the point where we're satisfied with the details of the attack to be able to fully share what happened in the attack. There will be at least one more technical followup in the future and possibly more. Those followups will get into more details of the class of problem we faced and what we think can be learned from the event.

[This post is based on security team lead greggles reply to a comment on HN]

charizard’s picture

A Technical write-up with the mentioned third party software and details would be great for those Drupalers who also maintain their production servers, look forward to it!

holly.ross.drupal’s picture

Just wanted to provide a little bit of clarity we've confirmed since our initial announcement. Understandably there are questions today about whether the vulnerability we experienced is something that may affect others. We want to let you know that we have worked with the software vendor to confirm it is a known vulnerability and has been publicly disclosed. We are still investigating and appreciate your patience. We have updated the FAQ to include this information.

Bevan’s picture

It would be useful to know exactly what date/time(s) passwords were captured from Drupal.org servers. For password security strategies that involve changing passwords at intervals, the time(s) the passwords were compromised helps understand the severity of the issue. Thank you.

Owen Barton’s picture

Right now our focus is on making sure users are secure, so to be on the safe side, we recommend changing your password anyway (as well as the passwords on any sites that shared that password).

Mesomet’s picture

Unfortunate incident to happen but since discovering Drupal, I have loved the functionality, assistance and wide support available I have required. Naturally I hope this sad moment leads to Drupal learning whatever lessons are necessary to ensure user safety but I still will continue to use Drupal and be registered here and continue to show my support Drupal.

Thank you for the information Holly, and thank you everyone at Drupal for the hard work you do, even in times such as this. I appreciate your continued work.

Meso

danjro’s picture

Should we all generate new SSH keys?

edit: Never mind, I just remembered that drupal.org would only have public keys.

danjro’s picture

This is why I love password managers. My password was a long random string unique to drupal.org, and now it is a different long random string.

charizard’s picture

Seconded, got lastpass to generate a new random password and now I can access Drupal on all my dev machines.

Demoshane’s picture

For an excellent communication and openness. Most companies and associations would have tried to hide this kind of issue and prevent it from seeing the daylight, you did not. This is remarkable feat and mark of excellent communication, openness and security. Yes, security because we can now act on the issue.

Raised my level of trust at least even though the issue is nasty (But as I can see it, it was not related to Drupal as software?)

You get my points for handling the issue.

holly.ross.drupal’s picture

Great to hear on a tough day.And yes, I did want to reiterate that at this time, we have no reason to believe that sites running Drupal software should be worried. Though if you use the same password to manage your site access, you should update those accounts as well.

Best,
Holly

kannan@kiluvai.com’s picture

I have same thing in my mind to share with Drupal Association, the way drupal association acted is appreciable. Every day we learn from things happening to us. :)

Bairnsfather’s picture

I too appreciate this effort and openness.

I am glad (or at least assuming) that Drupal will run in https mode full time now.

It would be nice if, in time, the Enabling HTTP Secure (HTTPS) page was updated to reflect how Drupal.org was updated.

Currently I run the patched version of Secure Pages.

topham’s picture

I'm pleased to see how the situation is currently being handled. I hope further details will be forthcoming soon as I may have to explain this situation to others. (Would be nice to identify the 3rd party software that is the risk-factor in question).

I also think the process regarding previously existing passwords needs to be updated; when a new password scheme is applied -ALL- old accounts should be marked to force a new password be generated using the new scheme. We shouldn't have some accounts salted, and some not, or some using a better hash than others, etc. An incident like this highlights exactly why such accounts are an issue. Also it means that in an account is sitting in the 'Set new password on next login' for a prolonged period of time should be forced into a Password recovery process.

The idea here being that a long dormant account passwords should be cleared/reset or set to an impossible value. This would reduce the amount of damage if an account is dormant on one site, but the same user/password is used on an active site elsewhere.
(We can't keep users from re-using passwords, but we certainly can reduce the risks to the bare minimum)

Grum’s picture

Password reuse can be prevented by checking against a password history file.

References:

phil.neale’s picture

Hello

A little guidance please for those of us just users and still climbing the way to a drupal website

I am only using local host on my machine for learning. Never published what I have to any other server.
Am I at risk of this breach ?

Have changed my drupal.org password that I use for forums. Do I need to change any of the the rather obvious one on my home local host only machine ?

thanks

phil

Owen Barton’s picture

If you have the same password you use on your drupal.org phil.neale account on your localhost sites (or any other sites, Drupal or otherwise) then changing it would be advised.

If you are using a different password on localhost (or any other sites, Drupal or otherwise) then you do not need to change those.

Of course, using a strong password everywhere (different for each site) is a good idea - even localhost sites may eventually get pushed to a live host, and it's easy to forget to change the passwords.

georgir’s picture

A crucial bit of information missing is if the password hashes used a salt or not.

reglogge’s picture

5th paragraph in the post:

All Drupal.org passwords are both hashed and salted, although some older passwords on some subsites were not salted.

vwilding’s picture

Any word on which third party app caused the 5/29 incident? We just want to make sure that our sites aren't using the same app.

greenwork’s picture

I agree the original statement is confusing people. APP on a server vs module in a drupal install. IS this a drupal related APP or is this a server app unrelated to drupal at all in any way. It would be very nice if you clarify what APP caused the exploit

====================================
Website:LEED Exam V4

StevenPatz’s picture

greenwork’s picture

so not a drupal module?

====================================
Website:LEED Exam V4

shekhtman’s picture

I am using OpenPublish 2.0. There are a lot of manual security updates to install. First, why not automatic?
Second, how do I do this? I tried update.php but it gave me errors and told me to view the error log.
After I download the suggested updates, where do I place them? I am running my own vps on aws.
Please provide detailed directions. Thank you.

--Alex
alexander@shekhtman.com

Jaypan’s picture

What exactly does that have to do with resetting your drupal.org password, which is the topic of the thread you have posted in.