Logo-amall

hi guys! Is someone here that could help me with one small issue ? I installed qdrant to kubernetes services, it deployed just fine. It is exposed as service in ingress route. qdrant is accessible thought REST API (web or insomnia) but when I am using python-client it fails. Sending stack trace. Thanks! ``` File ~/miniforge3/lib/python3.9/site-packages/qdrant_client/qdrant_client.py:1332, in QdrantClient.recreate_collection(self, collection_name, vectors_config, shard_number, replication_factor, write_consistency_factor, on_disk_payload, hnsw_config, optimizers_config, wal_config, timeout) 1329 if isinstance(wal_config, grpc.WalConfigDiff): 1330 wal_config = GrpcToRest.convert_wal_config_diff(wal_config) -> 1332 self.delete_collection(collection_name) 1334 create_collection_request = rest.CreateCollection( 1335 vectors=vectors_config, 1336 shard_number=shard_number, (...) 1342 wal_config=wal_config 1343 ) 1345 self.http.collections_api.create_collection( ... ---> 91 raise UnexpectedResponse.for_response(response) UnexpectedResponse: Unexpected Response: 308 (Permanent Redirect) Raw response content: b'\r\n308 Permanent Redirect\r\n\r\n

308 Permanent Redirect

\r\n
nginx\r\n\r\n\r\n' ```

Last active 3 months ago

56 replies

8 views

  • PR

    hi guys! Is someone here that could help me with one small issue ?

    I installed qdrant to kubernetes services, it deployed just fine. It is exposed as service in ingress route.

    qdrant is accessible thought REST API (web or insomnia) but when I am using python-client it fails. Sending stack trace.
    Thanks!

    File ~/miniforge3/lib/python3.9/site-packages/qdrant_client/qdrant_client.py:1332, in QdrantClient.recreate_collection(self, collection_name, vectors_config, shard_number, replication_factor, write_consistency_factor, on_disk_payload, hnsw_config, optimizers_config, wal_config, timeout)
       1329 if isinstance(wal_config, grpc.WalConfigDiff):
       1330     wal_config = GrpcToRest.convert_wal_config_diff(wal_config)
    -> 1332 self.delete_collection(collection_name)
       1334 create_collection_request = rest.CreateCollection(
       1335     vectors=vectors_config,
       1336     shard_number=shard_number,
       (...)
       1342     wal_config=wal_config
       1343 )
       1345 self.http.collections_api.create_collection(
    ...
    ---> 91 raise UnexpectedResponse.for_response(response)
    
    UnexpectedResponse: Unexpected Response: 308 (Permanent Redirect)
    Raw response content:
    b'\r\n308 Permanent Redirect\r\n\r\n<h1>308 Permanent Redirect</h1>\r\n<hr />nginx\r\n\r\n\r\n'
    
  • AN

    Hi! can you try to open the list in browser?

  • AN

    from the first glance the router in k8s is misconfigured or something like this

  • PR

    Hi! I created a collection through insomnia and when I call in the route/collections/{colName} it works fine.

  • PR

    Browser works too.

  • AN

    hm, only delete collection doesn't?

  • PR

    no. Whole communication from python-client doesnt work.
    just to be sure, I initialised python-client like this.

    client = QdrantClient(host="kubernetes/qdrant", port=6333)
    
  • AN

    kubernetes/qdrant - that doesn't look like a valid host

  • PR

    well. I did not wanna paste our kubernetes address here 🤣 so I used "kubernetes" as placeholder.

  • AN

    i mean, /qdrart should not be there

  • AN

    it expects host, not url

  • PR

    so we need to deploy qdrant to "/" in kubernetes? Because we deployed it "/qdrant"

  • AN

    yeah, i guess so

  • PR

    When I call python-client with only host without "/qdrant" I get time out error

  • AN

    you can do subdomain

  • AN

    qdrant.kubernetes

  • PR

    hmm. But why web api calls with "/qdrant" works?

  • AN

    i think, that’s because of the client logic. it tries to construct url from /, assuming that the host is just host

  • PR

    give me sec. I try move it to "/"

  • PR

    well I tried "/" and "qdrant.kubernetes/" and both failed 😦

  • AN

    could you try just qdrant.kubernetes?

  • AN

    assuming that http://qdrant.kubernetes/ opens in browser with version info

  • PR

    yes

  • PR
    {
        "title": "qdrant - vector search engine",
        "version": "0.10.5"
    }
    
  • PR

    but still same error with python-client. Web works fine

  • AN

    hm, interesting, you are using an old version. What's your client version?

  • PR

    client version

    qdrant-client==0.11.5
    

    and qdrant version I am using is the default from when I installed it with helm install as per instructions on the web site.

  • AN

    could you please try to change the version in your helm chart to v0.11.5

  • AN

    here https://github.com/qdrant/qdrant-helm/blob/main/values.yaml#L6

  • PR

    yeah. give me sec

  • PR

    we updated to v0.11.5. Web and Insomnia calls works fine but when we are trying to access from python-client we get this error.

    File ~/miniforge3/envs/winehunter/lib/python3.9/site-packages/httpcore/_exceptions.py:10, in map_exceptions(map)
          9 try:
    ---&gt; 10     yield
         11 except Exception as exc:  # noqa: PIE786
    
    File ~/miniforge3/envs/winehunter/lib/python3.9/site-packages/httpcore/backends/sync.py:94, in SyncBackend.connect_tcp(self, host, port, timeout, local_address)
         93 with map_exceptions(exc_map):
    ---&gt; 94     sock = socket.create_connection(
         95         address, timeout, source_address=source_address
         96     )
         97 return SyncStream(sock)
    
    File ~/miniforge3/envs/winehunter/lib/python3.9/socket.py:844, in create_connection(address, timeout, source_address)
        843 try:
    --&gt; 844     raise err
        845 finally:
        846     # Break explicitly a reference cycle
    
    File ~/miniforge3/envs/winehunter/lib/python3.9/socket.py:832, in create_connection(address, timeout, source_address)
        831     sock.bind(source_address)
    --&gt; 832 sock.connect(sa)
        833 # Break explicitly a reference cycle
    ...
         96 except Exception as e:
    ---&gt; 97     raise ResponseHandlingException(e)
         98 return response
    
    ResponseHandlingException: timed out
    
  • AN

    hmm, does it work with curl to query something except for GET?

  • AN

    like curl -X DELETE qdrant.k8s/collections/test_collection?

  • PR

    yeah this error.

    308 Permanent Redirect
    
    <h1>308 Permanent Redirect</h1>
    <hr />nginx
    
  • AN

    might it be your nginx config?

  • PR

    I updated this line in ingress config - ingress.kubernetes.io/ssl-redirect: "false"
    curl works fine now but client issue are still present.

  • AN

    are you using https?

  • PR

    yes we are]

  • AN

    try QdrantClient(..., https=True)

  • PR

    still getting same error - ResponseHandlingException: timed out

  • AN

    hmm, and in curl you are using curl -X DELETE qdrant.k8s:6333/collections/test_collection or just qdrant.k8s?

  • PR

    just qdrant.k8s

  • AN

    so you are not using default port?

  • PR

    on qdrant we set up default port 6333

  • AN

    try QdrantClient(..., https=True, port=443)

  • AN

    well, it looks like your ingress overwrote it, cause curl works without the port specified

  • AN

    hi @Shaibal sure, could you please create another thread?

  • AN

    @propher mopher did it work?

  • PR

    yeah! 443 works. Thank you very much. Can I have one more question?

  • AN

    glad to hear that. sure

  • PR

    i tried upsert 12k x 169 vectors
    with

    client.upsert(
    

    and it raised error Unexpected Response: 413 (Request Entity Too Large) do you know where I need to set size for qdrant in k8s? Because on local host it worked without issues

  • FA

    @propher mopher I think the reason for that is nginx

  • FA

    http://nginx.org/en/docs/http/ngxhttpcoremodule.html#clientmaxbodysize

  • FA

    try increasing the clientmaxbody_size

  • AN

    or upload in smaller batches

  • PR

    yes. this is parameter that I was looking for in ingress

    nginx.ingress.kubernetes.io/proxy-body-size: 8m
    

    Thanks guys for help. You are the best.

Last active 3 months ago

56 replies

8 views