Make collections/filters support empty lists #929

Activity

1. tsmith023 commented on Mar 5, 2024

   

   tsmith023

   on Mar 5, 2024

   Member

   More actions

   Hi Michael, this should be fixed by the latest 4.5.1 release. LMK if it doesn't otherwise I will close this issue 😁
2. MichaelOwenDyer commented on Mar 5, 2024

   

   MichaelOwenDyer

   on Mar 5, 2024

   Author

   More actions

   Hi @tsmith023, thanks for your response. Unfortunately the issue persists in filters.py - lines 108-111, 127, 134, 141, 207-217 all access value[0] without a length check.
   I would open a PR to fix this myself, but lines 207-217 look problematic to me:

   @staticmethod
       def __parse_filter(value: FilterValues) -> Dict[str, Any]:
           if isinstance(value, str):
               return {"valueText": value}
           if isinstance(value, uuid_lib.UUID):
               return {"valueText": str(value)}
           if isinstance(value, TIME):
               return {"valueDate": _datetime_to_string(value)}
           if isinstance(value, bool):
               return {"valueBoolean": value}
           if isinstance(value, int):
               return {"valueInt": value}
           if isinstance(value, float):
               return {"valueNumber": value}
           if isinstance(value, list):  # ISSUE STARTS HERE
               if isinstance(value[0], str):
                   return {"valueTextArray": value}
               if isinstance(value[0], uuid_lib.UUID):
                   return {"valueTextArray": [str(val) for val in value]}
               if isinstance(value[0], TIME):
                   return {"valueDateArray": [_datetime_to_string(cast(TIME, val)) for val in value]}
               if isinstance(value[0], bool):
                   return {"valueBooleanArray": value}
               if isinstance(value[0], int):
                   return {"valueIntArray": value}
               if isinstance(value[0], float):
                   return {"valueNumberArray": value}
           raise ValueError(f"Unknown filter value type: {type(value)}")

   I am not familiar enough with the rest of this library (or with Python) to know what the best way to fix this would be. Does an empty list have any element type to speak of, and if not, how do we know what to return here?
3. tsmith023 commented on Mar 5, 2024

   

   tsmith023

   on Mar 5, 2024

   Last edited by tsmith023

   Member

   More actions

   Ahh, I see what you're driving at. Performing these types of filters actually isn't supported directly with your use-case by the core Weaviate server itself. Trying your same filtering syntax with the GraphQL API fails also.

   However, you can achieve the same outcome by using a different logic instead. You can filter based on the length of a property and therefore achieve the desired response:

   filter = wvc.query.Filter.by_property("degree_programs", length=True).equal(0) 

   LMK how that works for you!

   P.S. To achieve this, your collection needs to have been configured with index_property_length=True in its inverted index config:

   collection = client.collections.create(
       ...,
       inverted_index_config=wvc.config.Configure.inverted_index(index_property_length=True),
   )

   👍React with 👍1Reacted by Michael Dyer
4. MichaelOwenDyer commented on Mar 5, 2024

   

   MichaelOwenDyer

   on Mar 5, 2024

   Author

   More actions

   Oh I see. That's very helpful, thank you!
5. MichaelOwenDyer

   closed this as completedon Mar 5, 2024
6. tsmith023 commented on Mar 5, 2024

   

   tsmith023

   on Mar 5, 2024

   Member

   More actions

   No problem! I think we can improve the error handling for an empty list here rather than having an index out of range error so I will open an issue to cover that