Ability to filter with comparison parameters (smaller/larger than)

My first attempt to add more query operators, it should work with latest dev version, unstable, maybe buggy, need preview and improve.

Use this patch like this:
http://server/<entityname>?$filter[created]=~ge~1370175882&$filter[changed]=~le~1369657482

Supported operators: "=", ">=", ">", "<=", "<", "<>"

$opeators = array(
    'eq' => array(
      'name' => 'Equal',
      'support' => 'string, int, float, dateTime, boolean',
      'query_operator' => '='
    ),
    'ge' => array(
      'name' => 'Greater than or equal',
      'support' => 'int, float, dateTime',
      'query_operator' => '>='
    ),
    'gt' => array(
      'name' => 'Greater than',
      'support' => 'int, float, dateTime',
      'query_operator' => '>'
    ),
    'le' => array(
      'name' => 'Less than or equal',
      'support' => 'int, float, dateTime',
      'query_operator' => '<='
    ),
    'lt' => array(
      'name' => 'Less than',
      'support' => 'int, float, dateTime',
      'query_operator' => '<'
    ),
    'ne' => array(
      'name' => 'Not equal',
      'support' => 'string, int, float, dateTime, boolean',
      'query_operator' => '<>'
    )
)

I'm not sure we should pursue this path since we are basically replicating what Views and Views Datasourse are already doing. Implementing a query builder here just duplicates what is already done in other modules.

I have personally not experimented with those Views modules, so I can't say much where they differ in their JSON output for example. To me it makes much more sense to have a Views display plugin that renders the rows with the exposed form filters applied, like we do in Drupal 8's REST module Views integration.

@klausi I partly agree with you, the trouble is none of those modules works very well.

In both https://drupal.org/project/views_datasource and https://drupal.org/project/views_data_export you must meticulously add every field you want included, so more work to begin with, and makes maintenance is cumbersome (when adding/removing fields).

You could build an index-feed using one of the above modules, and than fetch the resources through restws, but if you want to expose a nice, consistent API to a third party you have a problem.

I really need the "greater than"-operator for dates since a third party will be polling a drupal site for content changes, and we don't want to send all the content in every poll (just content changed since !timestamp). One could place a pubsubhubbub-hub in between and solve the problem that way, but more query-operators would be so much more simple and convenient. I will try (and review) the patch, and see if I can find any issues.

I really understand if you want to avoid adding extra complexity that should/could be handled by other modules, but for me this (and more complete file-resources) are the only major things keeping restws from being the perfect REST-interface for Drupal.

I had a look at the patch in #1, and even if it works as advertised I'm not sure it's the ideal way of going about this. It feels a bit kludgy to introduce an new $filter argument.

As a user I would expect http://server/<entityname>?created=ge:1370175882 or similar. But such a syntax would mean that you have to escape ":" could break existing code. :(

Here is my take at it. It's not perfect, but perhaps good enough. I've implemented the syntax in my previous post. To avoid breaking existing code I added the setting "restws_advanced_fitlers" that must be set to TRUE to enable operators in filtering. I don't know if this is a very good idea since I know one of the design-goals of this module is consistent behavior, but since this patch could otherwise break things (":" now have to be escaped) I put it there just to be safe.

Accidentally removed some comments, here is a new patch.

(Hopefully) fixed bug causing tests to fail (made a mistake when replacing parameters for fieldConditions with one value).

Right now I am short of time, but will update the patch with operators-specific tests a soon as I can.

Added some tests, though I can't figure out why the last two of them fails ('CONTAINS' and 'STARTS_WITH'). The exact same queries works fine in a non-test environment. I get 404 because as a result of an empty result in the count-query. Uploading this anyway, will try to sort the failing tests out.

Lets try again (think I put my variable_set('restws_filter_operators', TRUE)) in the wrong place, though it worked locally for me (probably because it was in my settings file).

Updates patch to latest dev.

Hopefully this fixes simple test.

I think it's a really bad idea to expect the operator in the value, as values may actually start with these strings which could break the results in unexpected ways. A far safer approach would be to append the operator to the field, e.g.:

?field_name%3Agt=64000

This makes parsing easier as well, as a simple explode() will do the job:

list($field_name, $operator) = explode(':', $key, 2) + array(null, '=');

As far as I know field names will never contain ":". Oh, and we should probably consider using "-" or "." instead of ":", as it doesn't require encoding. With that in mind the example would look like:

?field_name.gt=12345
?field_name-gt=12345

Thinking this through, it might actually be a lot easier and safer to just provide an additional parameter (e.g. "o", as "op" is already used too often for "operation") similar to what was proposed in #1. With that scenario the example would become:

?field_name=12345&o[field_name]=gt

This could later be expanded to support different operators per value, if multiple values are passed:

?field_name[]=12345&o[field_name][]=gt&field_name[]=54321&o[field_name][]=lt
// More readable with named indices:
?field_name[gt]=12345&o[field_name][gt]=gt&field_name[lt]=54321&o[field_name][lt]=lt

The huge benefit of this approach is that it provides a safe fallback in case the operator is not recognized (less values, not more).

I agree with @ciss. Could anyone please do a new patch? I have tested patch on @22 but doesn't work for me.

All patches are too old and I think this is a basic feature to have and that's why I put this as major priority. In my case, I have events and I want to filter them between two dates.

@kenianbei With your patch, this doesn't work:

<URL>/node.json?type=MY_CONTENT_TYPE&field_data=ge:2017-01-01

It justs returns an empty list!

However, the following examples returns at least one result:

<URL>/node.json?type=MY_CONTENT_TYPE&field_data=2017-01-01
<URL>/node.json?type=MY_CONTENT_TYPE&field_data=2017-01-02

Thanks!

How would you query field > 100 AND field < 200 ?

@rudiedirkx Please see the last section of my proposed solution, starting with "This could later be expanded to support different operators per value."

I'm actually not quite sure why I still included the o parameter in my last example (the part described as "more readable"). The example should probably have read:

?field_name[gt]=12345&field_name[lt]=54321

There may be a field called "field_name" with a column called "gt" - RestWS currently handles this and allows you to filter based on a field's column.

field_something[le]=120
field_something[value]=120

So I think to fully support properties and fields, you have to force specifying the field's column in order to have operators since I don't think we can differentiate between a column call and an operator. Maybe if we check that the field only has 1 column, we can allow it.

property_whatever[gt]=5
property_whatever[lt]=10

field_something[value][gt]=31
field_something[value][lt]=100
field_something[value2][gt]=5
field_some_other[value]=200
field_another=200

I extended the patch so it handles existing use cases but also handles multiple conditions discussed in #23. Probably needs some cleanup and more tests (for fields), but you can query like this now:

http://localhost/course_object.xml?coid[ge]=1&coid[le]=3&field_time_to_complete[value][ge]=100&field_time_to_complete[value][le]=150

Fixed tests.

@djdevin Patch on #30 works for me if I filter integer fields. But If I use date fields, the filtered rows don't make sense. Have you tested it using date fields?

I've tested with date fields and it appears to work. field_course_date is an ISO format date field.

/node.xml?type=course&field_course_date[value][ge]=2025-05-31T18:20:00

/node.xml?type=course&field_course_date[value][le]=2025-05-31T18:20:00

Re-rolled.

I've done some manual testing, added some tests that handle the three types of date fields (I think what I did should work without the Date module being required) and added other filtering tests.

This project has switched to GitLab CI, so merge requests should be used instead of patch files.

@drumm it looks like @aangel did open a branch, but missed creating the MR. I've done that: https://git.drupalcode.org/project/restws/-/merge_requests/2

@drumm I see a bunch of phpcs errors/warnings, but I'd imagine much of that is preexisting tech debt and not from the changes in this issue. How would you recommend moving forward?

Yes, those phpcs warnings were there before and I didn't want to expand the MR with non-issue related changes. I might have fixed a very few things but not many.