[Bug]: [benchmark][cluster] The number of growing segments on queryNode gradually increases, the amount of data loaded on queryNode is greater than data actually inserted #40382
Comments
wangting0128
added
kind/bug
|
I found a similar open issue regarding discrepancies in the number of segments and data loaded on the queryNode in Milvus. The issue is titled "[Bug]: count(*) returns a result that exceeds the actual number" and is currently open, but no solution has been found in the comments [1]. Regarding the increase in growing segments, Milvus manages growing and sealed segments using a For the transition from growing to sealed segments, Milvus uses an "autoHandoff" feature. This transition is triggered when a growing segment reaches a predefined threshold, at which point the segment is sealed, and index building begins. If To continue talking to Dosu, mention @dosu. Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other |
|
/assign @wangting0128 |
yanliang567
added
triage/accepted
reproduceargo task: bitmap-corn-1741230000 server: 
|
Is there an existing issue for this?
Environment
Current Behavior
argo task: bitmap-corn-1741143600
server:
Collection shard=16queryNode - Segment Loaded Num
prepare 5 million data, concurrent insert 146 * 10 = 1460, delete 153 * 10 = 1530queryNode - Queryable Entity Num
The amount of data has more than doubled
Expected Behavior
Same case, different image, tested normally
argo task: bitmap-corn-1741057200
image: 2.5-20250303-29c23545-amd64
server:
Steps To Reproduce
concurrent test and calculation of RT and QPS :purpose: `partition_key on scalar int64_1 field`, shards_num=16 verify DQL & DML scenario, which has 1 vector fields(IVF_SQ8) and building `BITMAP` index on all supported 12 scalar fields :test steps: 1. create collection (shards_num=16) with fields: 'float_vector': 128dim 'int64_1': partition_key, num_partitions=1024， data range: 0 ~ 9 all scalar fields: varchar max_length=100, array max_capacity=13 2. build indexes: IVF_SQ8: 'float_vector' BITMAP: all scalar fields 3. insert 5 million data 4. flush collection 5. build indexes again using the same params 6. load collection replica: 1 7. concurrent request: - search - query - hybrid_search - load - insert: batch=10 id = 0 ~ 9 - delete: delete all id = 0 ~ 9 - flush: ignore RateLimiteMilvus Log
No response
Anything else?
test result:
The text was updated successfully, but these errors were encountered: