API update to 6.x

linuxlover101 - November 30, 2007 - 22:57
Project:API
Version:HEAD
Component:Code
Category:task
Priority:critical
Assigned:Unassigned
Status:patch (code needs work)
Description

This is a work in progress to port the API module to Drupal 6.x.

I have:

  • Converted the menu system.
  • Converted the API Schema.
  • Switch to database functions instead of the switch-case method.
AttachmentSize
api.info_.patch385 bytes
api.install.patch11.04 KB
api.module.patch27.8 KB

#1

linuxlover101 - November 30, 2007 - 23:38

I'm having trouble instituting the menu system. I'm getting hook_menu(), hook_init() and hook_boot() mixed up.

#2

linuxlover101 - November 30, 2007 - 23:56

Fixed drupal_uninstall_schema() in api.install

AttachmentSize
api.install.patch11.03 KB

#3

linuxlover101 - December 1, 2007 - 00:30

Fixed no menu display problem. (Void comment 1) in api.module.

AttachmentSize
api.module.patch28.02 KB

#4

linuxlover101 - December 1, 2007 - 00:39

Updated to new FormAPI in api.module.

Added $radios = array(); to api_page_admin_form() in api.module to fix undeclared var.

I think this is now ready for review. Awaiting testing.

AttachmentSize
api.module.patch28.26 KB

#5

webchick - December 3, 2007 - 08:42
Title:API Port to 6.x» GHOP #42: API Port to 6.x
Status:patch (code needs work)» patch (code needs review)

Marking code needs review.

#6

agentrickard - December 4, 2007 - 17:09

The info patch fails for some reason, but that doesn't really matter.

Other patches apply. Testing functionality.

#7

agentrickard - December 4, 2007 - 17:16
Status:patch (code needs review)» patch (code needs work)

API settings menu item does not appear for user 1 or for users with 'administer API reference' permission.

#8

linuxlover101 - December 4, 2007 - 22:53

Hmm interesting. The menu setting shows up fine on the original admin accout when tested against HEAD.
I haven't tried other users yet. I plan to do so.

#9

agentrickard - December 5, 2007 - 01:05

Hm. I'm running CVS HEAD (about three days old) as user 1. Nothing in the menu_* tables, either. Will check again after updating Drupal CVS.

#10

agentrickard - December 5, 2007 - 01:26

FWIW: When I move this menu item from hook_init() to hook_menu(), it works.

    $items['admin/settings/api'] = array(
      'title' => 'API reference',
      'description' => 'Configure Drupal branches for documentation.',
      'access callback' => 'user_access',
      'access arguments' => array('administer API reference'),
      'page callback' => 'api_page_admin',
    );

But I haven't studied the new D6 menu system enough to know why.

#11

linuxlover101 - December 5, 2007 - 01:50

Hmm Ok. Thanks. As noted earlier...I'm confused about hook_menu(), hook_init(), and hook_boot(). Specifically how to change what the original code was before to initiate correctly in the new menu system. Any enlightenment appreciated. :)

#12

agentrickard - December 5, 2007 - 02:05

I'm having the same issue with all the menu items, actually, but don't have time to check through the code.

Take a look at http://drupal.org/node/103114 for documentation.

#13

linuxlover101 - December 5, 2007 - 02:10

I removed any instance of hook_init() and moved all menu options to hook_menu().

Then in the admin settings Go ahead and try to index some documentation (drupal Docs: http://cvs.drupal.org/viewvc.py/drupal/contributions/docs/developer/).

Then you might receive this error:

"notice: Undefined variable: first_key in C:\wamp\www\drupal\modules\api\api.module on line 989."

Could anyone help me debug this?

AttachmentSize
api.module.patch28.05 KB

#14

agentrickard - December 5, 2007 - 02:33

Those are -- and my term may be off here -- E_ALL strict errors.

They simply mean that you must define a variable (in this case $first_key) before you evaluate it.

#15

linuxlover101 - December 5, 2007 - 02:44

Well the line where I'm getting the error is:

  if ($radios) {
    if (!variable_get('api_default_branch', FALSE)) {
      list($first_key) = array_keys($radios);
      variable_set('api_default_branch', $first_key);
    }

And $first_key is not pre-defined. I'm guessing I should change this. Just declare it without a value?

#16

linuxlover101 - December 5, 2007 - 03:23

Fixed it (comment 15) with this code.

  $first_key = '';
  if ($radios) {
    if (!variable_get('api_default_branch', FALSE)) {

      $first_key = array_keys($radios);
      list($first_key) = $first_key;
      variable_set('api_default_branch', $first_key);
    }

AttachmentSize
api.module.patch28.41 KB

#17

webchick - December 5, 2007 - 03:42
Status:patch (code needs work)» patch (code needs review)

Bumping back to review. Great job on this, linuxlover!! :D

#18

agentrickard - December 7, 2007 - 15:12

These types of errors are going to be found throughout the code because of the new stricter error reporting.

#19

webchick - December 8, 2007 - 04:36
Priority:normal» critical

Bump. :( Student is waiting on us for a review since Dec. 4. :(

#20

drumm - December 8, 2007 - 04:46
Title:GHOP #42: API Port to 6.x» GHOP #42: API update to 6.x

Oh, I wasn't informed that this was a GHOP task. Thanks!

I was planning on updating the module myself this weekend, but I will review instead.

#21

agentrickard - December 10, 2007 - 00:29

My review would be: GHOP done and done well. Let the maintainer(s) fix E_ALL errors.

#22

webchick - December 10, 2007 - 01:20

@agentrickard: Does that mean you tested this and it works properly?

#23

linuxlover101 - December 10, 2007 - 10:56

>.< The due date is today (12/10/07). Please help review so this task on GHOP can be finished.

#24

agentrickard - December 10, 2007 - 15:35

@webchick: Apologies. I still have to test the new menu fixes from #13. I just think E_ALL is minor and could be considered out of scope for GHOP.

#25

agentrickard - December 10, 2007 - 16:28
Status:patch (code needs review)» patch (code needs work)

Menu fixes worked.

I get a white screen after submitting "Index PHP manual pages" -- but I suspect that behavior is in the D5 version as well.

After 6 tries, I can't get the module to index any content.

#26

linuxlover101 - December 10, 2007 - 21:08

Make sure your allowed script execution time is 0 (unlimited). This is because there is a VAST amount of info to index in the database. There are many functions. So the script may need more time to execute than usual.

Though, when I put in a server absolute path to a folder with a correct file with documentation (including code) it adds the info in the database. But I can't seem to get it to display the new (index) page. :\

#27

agentrickard - December 10, 2007 - 22:48

I don't think script execution time is the issue -- I never had that problem in D5, even on a shared host.

See http://therickards.com/api

#28

linuxlover101 - December 10, 2007 - 23:48

I feel I may end up redoing everything except for the Menu System and the database Schema.

#29

agentrickard - December 11, 2007 - 01:10

I don't think so. I didn't dig into my error logs very deeply, but I was seeing more 'undefined index' errors.

If the module is working in your test environment, then it means we need more testers.

#30

linuxlover101 - December 11, 2007 - 02:22

Wow I received a huge error when trying to index the PHP manual.

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:12:\"user warning\";s:8:\"%message\";s:1804403:\"MySQL server has gone away\nquery: UPDATE sessions SET uid = 1, cache = 0, hostname = '127.0.0.1', session = 'messages|a:2:{s:5:\\&quot;error\\&quot;;a:14274:{i:0;s:93:\\&quot;notice: Undefined index: signature in C:\\\\wamp\\\\www\\\\drupal\\\\modules\\\\api\\\\parser.inc on line 435.\\&quot;;i:1;s:94:\\&quot;notice: Undefined index: start_line in C:\\\\wamp\\\\www\\\\drupal\\\\modules\\\\api\\\\parser.inc on line 435.\\&quot;;i:2;s:94:\\&quot;notice: Undefined index: parameters in C:\\\\wamp\\\\www\\\\drupal\\\\modules\\\\api\\\\parser.inc on line 435.\\&quot;;i:3;s:90:\\&quot;notice: Undefined index: retur in C:\wamp\www\drupal\includes\database.mysqli.inc on line 167

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:12:\"user warning\";s:8:\"%message\";s:84:\"MySQL server has gone away\nquery: UPDATE users SET access = 1197331594 WHERE uid = 1\";s:5:\"%file\";s:39:\"C:\\wamp\\www\\drupal\\includes\\session.inc\";s:5:\"%line\";i:81;}', 3, '', 'http://localhost/drupal/admin/settings/api', 'http://localhost/drupal/admin/settings/api', '127.0.0.1', 1197331594) in C:\wamp\www\drupal\includes\database.mysqli.inc on line 167

Warning: MySQL server has gone away query: INSERT INTO watchdog (uid, type, message, variables, severity, link, location, referer, hostname, timestamp) VALUES (1, 'php', '%message in %file on line %line.', 'a:4:{s:6:\"%error\";s:7:\"warning\";s:8:\"%message\";s:130:\"Cannot modify header information - headers already sent by (output started at C:\\wamp\\www\\drupal\\includes\\database.mysqli.inc:167)\";s:5:\"%file\";s:38:\"C:\\wamp\\www\\drupal\\includes\\common.inc\";s:5:\"%line\";i:320;}', 3, '', 'http://localhost/drupal/admin/settings/api', 'http://localhost/drupal/admin/settings/api', '127.0.0.1', 1197331594) in C:\wamp\www\drupal\includes\database.mysqli.inc on line 167

EDIT: "The most common reason for the MySQL server has gone away error is that the server timed out and closed the connection." Hmm.

#31

agentrickard - December 11, 2007 - 16:28

Let's start with just indexing local files. I wasn't able to get that working, either.

#32

linuxlover101 - December 14, 2007 - 02:42

Well I know the process is. You put the files in a directory. You input which directory the file is in. Then you have to run cron.php. But still I'm having some troubles.

#33

agentrickard - December 14, 2007 - 03:07

What kind of troubles? Perhaps we can schedule some IRC time for this weekend.

#34

webchick - December 14, 2007 - 04:47

k. This task has been going on for about 2 weeks now, so it's time to wrap it up.

From what I can gather, this for the most part is done, but there remain critical issues which prevent it from working. And worse, these issues are very hard to track down and vary from system to system. Sounds thorny and messy. :(

agentrickard, linuxlover, could we come to a consensus of what we believe to be the minimum amount of functionality in order for us to mark this "Closed" and give linuxlover credit for the task so he can move onto another one?

I've extended the task deadline to 2007-12-18, but let's get this figured out ASAP.

#35

agentrickard - December 14, 2007 - 15:27

To me, minimum functionality is the ability to index local files properly.

I just started poking at the code and found a major problem:

Call to undefined function db_num_rows() in .../modules/api/parser.inc on line 422

http://drupal.org/node/114774#db-num-rows

Which explains some of the issues we've run into. So the port needs to be checked against all conditions in the upgrade list.

#36

agentrickard - December 14, 2007 - 15:33

db_next_id has also been deprecated. http://drupal.org/node/114774#db_last_insert_id

I think it is worth extending this task if we can get a few more reviewers to look at the output, or if we can wait for me to catch up.

#37

agentrickard - December 14, 2007 - 16:00

Fixing those last two issues helps a lot.

One big gotcha: the ability to split page functions into separate files caused the old "is this a unique did" query to fail. It is now necessary to add "AND file_name = '%s' to that query.

The issue is that parser.inc has not been updated to Drupal 6. There are some minor changes that need checking.

AttachmentSize
parser.patch2.17 KB

#38

linuxlover101 - December 15, 2007 - 02:03

Grr. I forgot to upload my parser.inc patch in the first place. Thats probably why there was so much inconsistency with the errors in testing.

#39

agentrickard - December 15, 2007 - 16:58

It explains a ton, and I should have caught that sooner. Make sure you run each file through the whole list of changes in the updating from 5.x to 6.x list.

Also watch out for the gotcha noted in #37. The ability to split modules into multiple files may cause errors that did not exist in the D5 version.

Then, I think if we can parse and display API functions, this is done. Only worry about the E_ALL errors if they harm functionality.

#40

linuxlover101 - December 16, 2007 - 04:08

Ok, Sorry about the mess. I'll apply your patch and make the necessary changes. I hope to have this done by tonight or tomorrow because I want to move on to the next task. Hopefully, the next one won't be as much of a hassle.

#41

linuxlover101 - December 16, 2007 - 06:08

Alright. I applied your patch put in a path for the local documentation. And I got a message that api_parse_file() doesn't exist. This is due to api_cron() in api.module not being able to access 'modules/api/parser.inc'. (That path was generated with drupal_get_path('module', 'api') .'/parser.inc')).

I've made sure the file is named correctly...and that it points to the right place. But it can't seem to open the file and I don't know why.

In api_cron() I changed include_once to require_once and got this message.
Fatal error: require_once() [function.require]: Failed opening required 'modules/api/parser.inc' (include_path='.;C:\php5\pear') in C:\wamp\www\drupal\modules\api\api.module on line 1114

EDIT:
There was something wrong with my file permissions. For some reason there's a difference between "-rwx------" and "-rwx------+". The first permission was put on due to me applying the patch...curious. So now it no longer gives me that error.

#42

linuxlover101 - December 16, 2007 - 06:42

Ok, api_cron() adds the correct info into the database. But it won't correctly call that info for viewing. I also get:

    * notice: Undefined index: groups in C:\wamp\www\drupal\modules\api\parser.inc on line 448.

EDIT:
In parser.inc api_save_documentation() uses &$docblock as a parameter. For some reason with in that array, there isn't an index named groups. This is probably hindering the processing of the documentation.

#43

linuxlover101 - December 16, 2007 - 07:29

Hmm. Well, For some reason the module can not determine the correct ID from the database so that it can display the correct page. I'll see if I can figure this out.

EDIT:
Something is going wrong with

  $result = db_query("SELECT documentation FROM {api_documentation} WHERE object_name = '%s' AND branch_name = '%s' AND object_type = 'mainpage'", $branch_name, $branch_name);
within api_page_branch(). Maybe its not getting the correct $branch_name...

If thats true then something is wrong in api_get_active_branch(). Maybe its the dynamic arguments (i.e. arg(1)).

#44

webchick - December 16, 2007 - 16:07

linuxlover, could you attach a patch of what you're working on so far? that'd make it easier for other people to help test it and see if they can help you track down the errors.

#45

linuxlover101 - December 17, 2007 - 20:00

Here is what I am currently working with.

Also, I have a question. I keep patching by comparing the original file to the new one. Am I doing this correctly? Or should I compare to the previous change (version) I made?

AttachmentSize
parser.patch2.48 KB
api.info_.patch390 bytes
api.install.patch11.03 KB
api.module.patch28.68 KB

#46

agentrickard - December 17, 2007 - 20:05

Probably easiest for all involved if you roll new patches against the original file(s). That way, people new to the issue can jump in.

#47

linuxlover101 - December 17, 2007 - 22:31

Ok Good. I've been doing it correctly then.

#48

add1sun - December 21, 2007 - 04:31
Status:patch (code needs work)» patch (code needs review)

These latest additions need review.

#49

aclight - December 24, 2007 - 15:15
Status:patch (code needs review)» patch (code needs work)

@linuxlover: we generally like patches to include changes to all files for the issue, so that only one patch is needed in a case like this. See http://drupal.org/patch/create for more information. It also looks like you're not using the correct options with diff when you create your patches. Normally patches will include the actual diff command used to create the patch.

If you're not using cvs diff to create your patches, I suspect that you will find that doing so will save you a lot of time and frustration.

Also, I noticed that you have made edits to many of your comments on this issue. Generally we only make edits when there is something like a typo or when you forget to upload a patch with a comment. The reason is that many people subscribe to issues via email, and when you edit a comment a new email is not sent. So, if you're adding useful information to a comment when editing it, people who don't actually browse this page itself won't see your edits.

I'm going to extend your due date on the GHOP task tracker by 3 additional days. It would be best if you could fix the patch as I specified above and then try to get people to review the patch for you. Dropping into #drupal or #drupal-ghop on irc.freenode.net and asking for reviewers can be an effective way to get people to review your patch (and also to get to know other community members better).

#50

linuxlover101 - December 25, 2007 - 06:10

Ok used cvs diff. This patch should be up to the standards provided. If not, please notify me.

AttachmentSize
api_changes.patch72.26 KB

#51

agentrickard - December 25, 2007 - 17:41

Still getting duplicate key errors from api_save_documentation when running cron.php.

Path 'api/files' shows no values even when data exists in database.

Path 'api/functions' shows no values even when data exists in database.

#52

webchick - January 2, 2008 - 21:48

I went ahead and gave linuxlover credit on this task, based on the work done so far. It'd be awesome if the community could help carry it the rest of the way.

#53

agentrickard - January 3, 2008 - 03:46

Agreed. Sorry that I have been working on my own stuff.

#54

pwolanin - January 23, 2008 - 02:06
Title:GHOP #42: API update to 6.x» API update to 6.x
Assigned to:linuxlover101» Anonymous

#55

bdragon - January 23, 2008 - 02:37

Rerolled against DRUPAL-5 instead of HEAD that has had DRUPAL-5 tacked onto it. (more signal, less noise, branch merges should be done as branch merges, not patches.)

AttachmentSize
api_correct_branch.patch43.7 KB

#56

bdragon - January 23, 2008 - 04:45
Assigned to:Anonymous» bdragon

Claiming task temporarily.

#57

bdragon - January 23, 2008 - 08:50
Assigned to:bdragon» Anonymous

Well, I didn't get it quite figured out, but I did a lot of work tidying up the menu hook...

Still definately "needs work." I'm seeing a LOT of warnings coming from the parser as well... Then again, I run with E_ALL...

AttachmentSize
api_menu_work.patch47.73 KB

#58

Robin Monks - February 27, 2008 - 12:25

This code is producing a message on each drupal page that says:

"

I couldn't pin down the bug, but, I didn't have much time either. Also, files don't appear to be crawled.

Robin

#59

ax - February 29, 2008 - 08:46

58 comments and no real progress - i think we need to tackle this issue a a little more structured. i posted a suggestion for a 6.x Upgrade Roadmap to the API module developers group at http://groups.drupal.org. please chime in, discuss that roadmap (it's a wiki page) and the implementation, and let the (small) patches (that would be applied to HEAD step by step) flow in. thanks everyone!

#60

matt_paz - March 26, 2008 - 12:25

subscribing ...

#61

Freso - April 17, 2008 - 13:00

@Robin: The '' comes from drupal_set_message(var_export(implode('/', array_slice($map, $index + 3, $end - $index + 1)), true)); which seems to just be a debug statement.

Line 1149 checked for $form_state['values']['default_branch'] which called a notice if it wasn't defined, so I added a !empty() around it. Apart from this and removing some excessive whitespace (a newline and two trailing spaces), nothing else is changed in the patch, as the parser bails out in a pgsql environment. I'm going to try and use my MySQL and see if it works there, and then possibly work on cross-compatibility. Possibly.

AttachmentSize
api.d5_to_d6.patch48.76 KB
 
 

Drupal is a registered trademark of Dries Buytaert.