The XML parser seems to be dropping empty parameters on incoming XML-RPC requests. Here's what I did.
First, create this minimalistic module:
<?php
// xmlrpc test
function xtest_help($section) {
switch ($section) {
case 'admin/modules#description':
return t('Tests XML-RPC support in Drupal.');
}
}
function xtest_xmlrpc() {
//return array('xtest.threeparam' => 'xtest_threeparam');
return array(array('xtest.threeparam', 'xtest_threeparam', array('string','string','string'), t('Tests a function with three parameters.')));
}
function xtest_threeparam ($foo, $bar, $baz) {
return array($foo, $bar, $baz);
}This just sets up a very simply XML-RPC function that reflects back the three parameters that it gets. Next, we do an XML-RPC call:
POST /drupalhead/xmlrpc.php HTTP/1.0
User-Agent: Frontier/9.1b4 (MacOS)
Host: pardosa
Content-Type: text/xml
Content-length: 259
<?xml version="1.0"?>
<methodCall>
<methodName>xtest.threeparam</methodName>
<params>
<param>
<value>foo</value>
</param>
<param>
<value></value>
</param>
<param>
<value>baz</value>
</param>
</params>
</methodCall>
Note that we're basically saying, xtest.threeparam('foo', '', 'baz'). What we should get back is the array ('foo', '', 'baz'). But wait! What's this? We get back
<?xml version="1.0"?>
<methodResponse>
<params>
<param>
<value><array><data>
<value><string>foo</string></value>
<value><string>baz</string></value>
<value><string></string></value>
</data></array></value>
</param>
</params>
</methodResponse>
We're getting the array ('foo', 'baz', '')! Why?
It turns out that when xmlrpcs.inc (line 52) calls the parser (line 125 in xmlrpc.inc) the parser converts the <param>value</param> tags to a PHP array but drops "empty" params like <param></param> instead of converting them to empty string.
This means that our XML-RPC server is going to eliminate parameters so they won't match up with the expected parameters of the functions. In the example above, the server dispatches xtest_threeparam('foo', 'baz') instead of xtest_threeparam('foo', '', 'baz'). Note how the middle parameter has been zapped.
Please, someone tell me I'm doing something wrong. I've tried both supported xmlrpc hook implementations (one's commented out in the above code).
| Comment | File | Size | Author |
|---|---|---|---|
| #27 | MAINTAINERS.txt | 1.38 KB | killes@www.drop.org |
| #25 | empty_values_0.patch | 1.04 KB | chx |
| #24 | xml_captured_in_xmlrpc_message_parse.zip | 3.64 KB | drewish |
| #22 | rpc_result.txt | 3.29 KB | drewish |
| #5 | empty_values.patch | 1.05 KB | chx |
Comments
Comment #1
chx commentedI do not know why but there was a barrier which stopped empty values to be set. Why, I can't fathom.
Comment #2
chx commentedBy looking at some schemas (no official, alas), http://www.ibiblio.org/xml/slides/xmlone/london2002/advancedxml/240.html then string has a value of value="([ -~]|\n|\r|\t)*" which means 0 or more characters.
Comment #3
jvandyk commentedTested. Works.
Comment #4
dries commentedCommitted to CVS HEAD and DRUPAL-4-6. Thanks.
Comment #5
chx commentedReopening because <value/> was broken. Here is another fix -- I unrolled the committed patch and did something else. As it's been stated many times, there are many ways to skin a cat. Tested with this XMLRPC request and the one from http://drupal.org/node/59835
Comment #6
chx commentedI corrected thte title. It's not value/ that's fixed here but that struct spawns empty parameters because of the previous fix. jvandyk and I have not found this problem before because we were not typing structs.
Comment #7
halkeye commented+1
Tested with Zoundry Blog Client
Comment #8
jvandyk commentedThe current patch only partially works. It works with the following:
But fails with the case where implicit strings are used:
where it fails with "server error. wrong number of method parameters".
Note that it succeeds with implicit strings as long as there's no empty parameter value; it only fails on
<value</value>.Comment #9
jvandyk commentedYay! The bug was in my testing code; ignore my most recent comment.
Tested on Drupal-to-Drupal and Frontier-to-Drupal calls with string, empty and array parameters. Works like a charm.
Comment #10
drewish commentedHumm, while it fixes the incorrect parameter count bug, returned arrays now seem to consist of empty strings. I'm not sure if it's related to the patch...
Comment #11
chx commenteddrewish, please elaborate. The XML that is returned is not sent through the parser, so that's a different issue. If you mean the params array then it's a bug and please submit a request XML which is parsed bogus.
Ps. I love concise bug reports.
Comment #12
drewish commentedokay well XMLRPC is being very wierd for me. this patch in #5 does seem to fix the parameter bug but i'm running into other issues. i'll look back through the cvs history and see if i can figure out what was changed.
Comment #13
chx commentedNow I am losing my patience. Is it so damned hard to copypaste the request XML here and the method signature? With this mysterious shit you are holding up this patch AND disallow us from helping YOU. Congratulations.
Comment #14
jvandyk commentedAlso tested with full and empty structs from Frontier, and it works. drewish, please specify a way for us to repeat any problems you are having.
Comment #15
drewish commentedchx, if trying not to distract everyone with something that maybe unrelated to this issue is being secretive then I'm guilty as charged. If I'd pasted in a bunch of unrelated code you'd have accused me of hijacking the thread. But, since you asked for it, here's the long version.
The station module has two sub-modules designed to run on different machines and communicate via XMLRPC requests. Some time after updating to HEAD on Thursday the RPC calls stopped working with the "wrong number of method parameters" error. As I've already said the patch attached to #5 fixes this.
Here's the code I'm running (using the devel module's execute function):
The problem, and the reason I was reluctant to post is that on one machine everything works fine but on another, it doesn't. All machines are running PHP 5.1.2 and Drupal HEAD. The kpsu.org server is also running the #5 patch. The results should be a program node for the currently scheduled program. In the broken results you'll notice that all the keys are there but the values are missing.
My localhost results:
array(28) { ["nid"]=> string(3) "251" ["vid"]=> string(3) "251" ["type"]=> string(7) "program" ["status"]=> string(1) "1" ["created"]=> string(10) "1129573274" ["changed"]=> string(10) "1140573932" ["comment"]=> string(1) "2" ["promote"]=> string(1) "0" ["moderate"]=> string(1) "0" ["sticky"]=> string(1) "0" ["revision_timestamp"]=> string(10) "1140573932" ["title"]=> string(11) "Liquid Beat" ["body"]=> string(109) "This show is dedicated to the preservation and cultivation of real Hip-Hop music and culture. Enough said. " ["teaser"]=> string(109) "This show is dedicated to the preservation and cultivation of real Hip-Hop music and culture. Enough said. " ["log"]=> string(0) "" ["format"]=> string(1) "1" ["uid"]=> string(2) "74" ["name"]=> string(14) "Matthew Nelkin" ["picture"]=> string(0) "" ["data"]=> string(245) "a:7:{s:12:"profile_name";s:14:"Matthew Nelkin";s:7:"form_id";s:9:"user_edit";s:18:"profile_name_first";s:8:"Matthew ";s:17:"profile_name_last";s:6:"Nelkin";s:17:"profile_badge_num";s:5:"61058";s:7:"contact";s:1:"0";s:14:"picture_delete";s:0:"";}" ["genre"]=> string(7) "Hip-Hop" ["url"]=> string(0) "" ["djs"]=> array(1) { [74]=> string(14) "Matthew Nelkin" } ["times"]=> array(1) { [15]=> array(3) { ["day"]=> string(1) "0" ["hour"]=> string(2) "14" ["link"]=> string(79) "Sunday at 2pm" } } [0]=> bool(false) ["taxonomy"]=> array(1) { [3]=> array(5) { ["tid"]=> string(1) "3" ["vid"]=> string(1) "1" ["name"]=> string(7) "Hip-Hop" ["description"]=> string(0) "" ["weight"]=> string(1) "0" } } ["current_time"]=> string(2) "15" ["node_url"]=> string(35) "http://kpsu.org/program/liquid_beat" }archive.kpsu.org results:
array(28) { ["nid"]=> string(0) "" ["vid"]=> string(0) "" ["type"]=> string(0) "" ["status"]=> string(0) "" ["created"]=> string(0) "" ["changed"]=> string(0) "" ["comment"]=> string(0) "" ["promote"]=> string(0) "" ["moderate"]=> string(0) "" ["sticky"]=> string(0) "" ["revision_timestamp"]=> string(0) "" ["title"]=> string(0) "" ["body"]=> string(0) "" ["teaser"]=> string(0) "" ["log"]=> string(0) "" ["format"]=> string(0) "" ["uid"]=> string(0) "" ["name"]=> string(0) "" ["picture"]=> string(0) "" ["data"]=> string(0) "" ["genre"]=> string(0) "" ["url"]=> string(0) "" ["djs"]=> string(1) " " ["times"]=> string(1) " " [0]=> string(0) "" ["taxonomy"]=> string(1) " " ["current_time"]=> string(0) "" ["node_url"]=> string(0) "" }Comment #16
drewish commentedwith the preview broken i didn't notice that the status had changed.
Comment #17
chx commentedThat maybe kills. Do you want the really short and blunt version? Put up or shut up. As you have not choosen the latter, I forced you to choose the former. There is not try.
So, indeed we may an issue here because we were inspecting the XML-RPC server so far. But as the parser is common there may be some problem. To further help, please add a watchdog('xmlrpc', var_export($xmlrpc->params, TRUE)); to the end of xmlrpc_message_parse just before the return TRUE; and inspect what you get. Please do so on the server and the client end too and inspect the results. Outgoing XML can be catched at xmlrpc_server_output and incoming can be catched at $data = file_get_contents('php://input'); in xmlrpcs.inc. Inspect and if you can't find out, please include (attach in a zip or whatever) the XMLs.
Comment #18
chx commentedPriority pingpong.
Sorry for not being polite enough, I am trying to debug and I really try to fix promptly identified problems and anything blocking me from fixing a bug (or determining there is none) frustrates me.
Comment #19
drewish commentedstrangely, all the watchdog statements are just logging NULLs on all three machines. i'll try to captuer some of the XML and see how that looks.
no worries chx, i've got thick skin and i appreciate all the hard work you put in.
Comment #20
drewish commentedahh, typo in the addition to xmlrpc_message_parse():
watchdog('xmlrpc', var_export($xmlrpc_message->params, TRUE));output from working machine:
array ( 0 => array ( 'nid' => '214', 'vid' => '214', 'type' => 'program', 'status' => '1', 'created' => '1129573274', 'changed' => '1129573274', 'comment' => '2', 'promote' => '0', 'moderate' => '0', 'sticky' => '0', 'revision_timestamp' => '1129573274', 'title' => 'Nuthin\' But Football', 'body' => '', 'teaser' => '', 'log' => '', 'format' => '1', 'uid' => '1', 'name' => 'admin', 'picture' => '', 'data' => 'a:2:{s:5:"roles";a:1:{s:18:"authenticated user";s:1:"2";}s:7:"form_id";s:9:"user_edit";}', 'genre' => 'Sports Talk', 'url' => '', 'djs' => array ( 23 => 'Ive', ), 'times' => array ( 16 => array ( 'day' => '0', 'hour' => '15', 'link' => 'Sunday at 3pm', ), ), 0 => false, 'taxonomy' => array ( 25 => array ( 'tid' => '25', 'vid' => '1', 'name' => 'Sports', 'description' => '', 'weight' => '0', ), ), 'current_time' => '16', 'node_url' => 'http://kpsu.org/program/nuthin_but_football', ), )output from broke machine:
array ( 0 => array ( 'nid' => '', 'vid' => '', 'type' => '', 'status' => '', 'created' => '', 'changed' => '', 'comment' => '', 'promote' => '', 'moderate' => '', 'sticky' => '', 'revision_timestamp' => '', 'title' => '', 'body' => '', 'teaser' => '', 'log' => '', 'format' => '', 'uid' => '', 'name' => '', 'picture' => '', 'data' => '', 'genre' => '', 'url' => '', 'djs' => ' ', 'times' => ' ', 0 => '', 'taxonomy' => ' ', 'current_time' => '', 'node_url' => '', ), 1 => ' ', )i'll try to capture the outgoing XML from the server.
Comment #21
chx commentedYou could also catch the XMLs in the beginning of xmlrpc_message_parse. I predict it'll be different. you want xmlrpc_message->message.
Comment #22
drewish commentedfor reference, here's the output generated by the server. i ran it from both hosts and then diffed the results they were identical.
Comment #23
drewish commentedhere's a zip file with the values from $xmlrpc_message->message, at the top of xmlrpc_message_parse(), and $xmlrpc_message at the bottom from both the working and non-working machine.
Comment #24
drewish commentedi'll try actually attaching the file this time.
Comment #25
chx commentedNow, that's interesting because the top file are absolutely the same and yet the parsed outputs differ. One of them is correct and the other is not.
Now, please try deleting chunks from the 'top' XMLs and try to find where your parser chokes.
However, I found that if you do not apply the patch (reroll of #5) then you get what the bad indicates. Maybe that's not applied? (Yes, the clients also need the patch)
Comment #26
drewish commentedDeleteded all the files in the includes directory, cvs up'ed, applied that patch to all the machines and they're all working perfectly. I think, as you suggested, the problem was that the "bad" client didn't have the patch.
Comment #27
killes@www.drop.org commentedapplied
Comment #28
jvandyk commentedThe patch by chx in #25 also needs to be applied to 4.6. Note that Dries applied the previous patch to DRUPAL-4-6, so XML-RPC is broken in 4.6 until this is applied.
Comment #29
killes@www.drop.org commentedmoving to 4.6
Comment #30
dries commentedCommitted #25 to DRUPAL-4-6. Thanks!
Comment #31
(not verified) commented