Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to make .filter(field__in=[value1,value2] #50

Open
onegreyonewhite opened this issue Jul 7, 2016 · 11 comments
Open

How to make .filter(field__in=[value1,value2] #50

onegreyonewhite opened this issue Jul 7, 2016 · 11 comments
Labels

Comments

@onegreyonewhite
Copy link
Contributor

It`s look like possible:
Finding Multiple Exact Values

@lauxley
Copy link
Contributor

lauxley commented Jul 7, 2016

@lauxley lauxley closed this as completed Jul 7, 2016
@onegreyonewhite
Copy link
Contributor Author

it is quite different. I do not want to use the "in" operator. I want to get satisfying results. Your example only checks the condition of the occurrence of the results.
I asked about construction like sql "field in [...]".

@lauxley
Copy link
Contributor

lauxley commented Jul 8, 2016

Ok i might have closed this issue a bit hastily, did you try with the __contains lookup ? If it doesn't work I will need a complete use case to make sure it's not feasible before adding a new functionality.

@lauxley lauxley reopened this Jul 8, 2016
@onegreyonewhite
Copy link
Contributor Author

onegreyonewhite commented Jul 11, 2016

For example, I have Params model.
If I call Params.es.filter(obj__contains=["1","2"]), everything looks like normal.
But when I call Params.es.filter(obj__contains=["1","2"], time__gte="now-1m")

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/django_elasticsearch/query.py", line 76, in __repr__
    data = list(self[:REPR_OUTPUT_SIZE + 1])
  File "/usr/local/lib/python2.7/dist-packages/django_elasticsearch/query.py", line 97, in __getitem__
    self.do_search()
  File "/usr/local/lib/python2.7/dist-packages/django_elasticsearch/query.py", line 270, in do_search
    r = es_client.search(**search_params)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/client/__init__.py", line 548, in search
    doc_type, '_search'), params=params, body=body)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/transport.py", line 329, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 109, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/base.py", line 108, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
RequestError: TransportError(400, u'search_phase_execution_exception', u'No query registered for [must]')

@onegreyonewhite
Copy link
Contributor Author

My models classes:

class MainEsIndexable(EsIndexable):
    id = models.CharField(max_length=25, primary_key=True, default=random_string)
    time = models.DateTimeField(default=timezone.now)
    server = models.CharField(max_length=255)
    param = models.CharField(max_length=255)
    obj = models.CharField(max_length=255, default="None")


    class Meta:
        abstract = True

    def __init(self, *args, **kwargs):
        super(MainEsIndexable,self).__init__(*args, **kwargs)
        self._synced = False

    def save(self, *args, **kwargs):
        self.es.do_index()

    def delete(self, *args, **kwargs):
        self.es.delete()

class Params(MainEsIndexable):
    value = models.CharField(max_length=255, default="0")

    class Elasticsearch(EsIndexable.Elasticsearch):
        serializer_class = ParamEsSerializer
        fields = ['time','server','param','obj','value', 'id']
        index = 'monitd-params'
        doc_type = 'param'
        mappings = {
            "_id" : { "path" : "id" },
            "server": {"index": "not_analyzed"},
            "param": {"index": "not_analyzed"},
            "obj": {"index": "not_analyzed"},
            "time": { "type": "date" }
        }

    class Meta:
        managed = False

For query Params.es.filter(obj__contains=[str(j) for j in range(7)], time__gte="now-1m", server="192.168.0.251", param="lmTempSensorsValue") results returned only with last (6) obj.
Created query:

{
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "range": {
                "time": {
                  "gte": "now-1m"
                }
              }
            },
            {
              "term": {
                "param": "lmTempSensorsValue"
              }
            },
            {
              "term": {
                "server": "192.168.0.251"
              }
            }
          ]
        }
      },
      "query": {
        "match": {
          "obj": {
            "query": [
              "0",
              "1",
              "2",
              "3",
              "4",
              "5",
              "6"
            ]
          }
        }
      }
    }
  }
}

But results must be with obj 1-7 values.

@lauxley
Copy link
Contributor

lauxley commented Jul 25, 2016

range(7) returns a 0 indexed list so it looks normal to me !
But does the PR#51 fix the query sql syntax error ?

@onegreyonewhite
Copy link
Contributor Author

onegreyonewhite commented Jul 25, 2016

It is normal.
But results are not normal. I have data with obj values in range 1-7, but result is only with "6" obj value.
#51 request solve all last problems, but query results is not full.

@lauxley
Copy link
Contributor

lauxley commented Jul 25, 2016

I'm not sure I understand what you mean, if you want documents with "obj" being equal to "7", you can use range(1, 7) ?
But i doubt it is really the problem here, and i'm not even sure it works with a match query.

@onegreyonewhite
Copy link
Contributor Author

onegreyonewhite commented Jul 25, 2016

@lauxley this behavior is very strange.
When I do a query from Django, I'm getting results with the obj parameter just equal to 6. For the range (1,7) results from obj = 7. But if I make a request directly to the ElasticSearch, the results come very strange, ignore blocks "term".

Could we using finding_multiple_exact_values for this case?

@lauxley
Copy link
Contributor

lauxley commented Jul 26, 2016

As stated in #53 we may need another lookup for this or at the very least to investigate a little bit more.

@onegreyonewhite
Copy link
Contributor Author

I added some fixes to PR #53. Now may use MyModel.es.filter(field__in=['one', 'two']).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants