Optimize MongoDB SCIM attribute queries for large groups using explicit batch sizes

[Copilot]

Retrieving SCIM groups with 20,000+ members takes 16+ minutes due to MongoDB's default cursor batch size (101 documents on first batch). Each member generates multiple attributes, resulting in excessive network round trips.

Changes

• Replace LINQ queries with MongoDB Find API in SCIMRepresentationModel.IncludeAttributes and SCIMRepresentationQueryRepository.FindSCIMRepresentations to explicitly control batch sizes
• Add configurable BatchSize property to MongoDbOptions (default: 10,000) with validation
• Add cancellation token propagation to IncludeAll and IncludeAttributes methods
• Add explicit cursor disposal using using statements

Example

Before:

FlatAttributes = await dbContext.SCIMRepresentationAttributeLst.AsQueryable()
    .Where(a => a.RepresentationId == Id)
    .ToMongoListAsync();

After:

var filter = Builders<SCIMRepresentationAttribute>.Filter.Eq(a => a.RepresentationId, Id);
var findOptions = new FindOptions<SCIMRepresentationAttribute>
{
    BatchSize = dbContext.Options.BatchSize  // 10,000 vs default 101
};
using var cursor = await dbContext.SCIMRepresentationAttributeLst
    .FindAsync(filter, findOptions, cancellationToken);
FlatAttributes = await cursor.ToListAsync(cancellationToken);

Existing RepresentationId index ensures query performance remains unchanged.

Original prompt

There is a performance problem in the scim implementation using MongoDB, when a group with 20 000 users is returned then it takes more than 16 minutes to execute why

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.