Hi, I'm a big fan of qdrant and it solves so many headaches I would have needed to solve myself.
I started to include it in my project and encountered an issue when adding many records to the index.

I reproduced it on v0.11.6 and 0.11.5
x64 system
I'm using docker available from dockerhub

Steps:

• start qdrant service with default config
• add points with payload to index via python client library (prefer grpc = true)
• After a while I get the errors:
  [2022-12-29T14:44:55.289Z ERROR collection::updatehandler] Failed to flush: Service runtime error: Failed to flush idtracker versions: Service runtime error: RocksDB flushcf error: IO error: While open a file for appending: ./storage/collections/t19texts_2022-12-26 13:10:48+00:00/0/segments/6f52a205-b449-4693-aafc-7b39074c8a07/000068.sst: Too many open files

[2022-12-29T14:45:28.858Z ERROR collection::collectionmanager::collectionupdater] Update operation failed: Not recovered from previous error: Service runtime error: RocksDB putcf error: IO error: While open a file for appending: ./storage/collections/t19texts_2022-12-26 13:10:48+00:00/0/segments/452a8371-15c2-4afb-84ab-e43fa0b9e1a8/000179.sst: Too many open files

and many more like that

Hi! are you running qdrant in docker?

yes, on AWS ECS

Hm, that looks strange, usually docker allows to have enough open files. How many segments does you collection have?

there is a way how to increase that - you can run docker with docker run --ulimit nofile=90000:90000

or with docker-compose https://github.com/compose-spec/compose-spec/blob/master/spec.md#ulimits

I just checked how I created the collection.
I do not specify the number of segments, so default?

yeah, should be default. Can you check that in `GET /collection/' api output?

I can try that and see if it just pushes out the problem or if it's solved

Hm, according to https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_Ulimit.html

it has a pretty small default limit. I would recommend to set it to something like 65535

trying this

the collection info, in case it helps.
I'm trying now the ulimits

{
"result": {
"status": "green",
"optimizerstatus": "ok", "vectorscount": 353,
"indexedvectorscount": 0,
"pointscount": 353, "segmentscount": 8,
"config": {
"params": {
"vectors": {
"size": 768,
"distance": "Cosine"
},
"shardnumber": 1, "replicationfactor": 1,
"writeconsistencyfactor": 1,
"ondiskpayload": true
},
"hnswconfig": { "m": 16, "efconstruct": 100,
"fullscanthreshold": 10000,
"maxindexingthreads": 0
},
"optimizerconfig": { "deletedthreshold": 0.2,
"vacuumminvectornumber": 1000, "defaultsegmentnumber": 0, "maxsegmentsize": null, "memmapthreshold": null,
"indexingthreshold": 20000, "flushintervalsec": 5, "maxoptimizationthreads": 1 }, "walconfig": {
"walcapacitymb": 32,
"walsegmentsahead": 0
}
},
"payload_schema": {}
},
"status": "ok",
"time": 0.00004635
}

yeah, "segments_count": 8, is normal, so ulimits should help

thanks for your quick help.
I'm running the process now and see if the crash happens again or not.
I update here in 20mins or so.

thanks again for the quick help.
The process ran through without error and some first checks look ok too.

I set it to nofile limit to 65k.

Should I create a PR to add this to the documentation?
Many linux installs still how low nofiles default.
I'm sure I will no be the only one running into this issue.

a note in documentation might help, yes! Where would you expect to find this configuration?