perf: skip segment leaf expansion in merge suggestions (CM-1230)

[skwowet]

Summary

Fixes query timeouts on POST /membersToMerge and POST /organizationsToMerge when countOnly: true is called with a large segment scope.

Both endpoints were expanding the selected segment into all leaf subproject IDs and filtering memberSegmentsAgg / organizationSegmentsAgg with a large segmentId IN (...) list. On large groups this caused Postgres to pick expensive scans over hundreds of millions of rows, triggering read timeouts.

Both tables already maintain rollup rows at every hierarchy level (group / project / subproject). Using the selected segment ID directly lets the planner do index-only lookups per merge row instead.

The frontend for organizations was also sending every leaf ID on each request — it now sends the project group ID, matching the existing member behaviour.

Changes

• memberRepository.ts — remove getSegmentSubprojects expansion in findMembersWithMergeSuggestions; use request segment IDs directly
• organizationRepository.ts — same fix in findOrganizationsWithMergeSuggestions
• organization-service.js / organization.api.service.ts — send [projectGroupId] instead of all leaf IDs
• Data quality merge suggestion components (member + org) — drop leaf ID expansion from segments array

No SQL shape changes, no new indexes.

Query plan impact (EXPLAIN ANALYZE on prod)

	Before	After
Members COUNT (large group)	Parallel seq scan on memberSegmentsAgg; >15s timeout	Index-only scan on PK; ~263ms
Members LIST LIMIT 20	~47ms	~30ms
Orgs COUNT (large group)	Seq scan per outer row over leaf set; ~1.3s+, timeout under load	Index-only scan on PK; ~236ms
Orgs LIST LIMIT 20	Per-org segment walk via segments join; ~6.8s	Index-only scan on PK; ~31ms

Before:

Parallel Seq Scan on memberSegmentsAgg
  filter: segmentId IN (hundreds of leaf UUIDs)

After:

Nested Loop on memberToMerge (~125K rows)
  └─ Index Only Scan on memberSegmentsAgg_pkey
       index cond: segmentId = :selectedSegmentId AND memberId = mtm.memberId

Removes one extra DB round-trip per request (subproject expansion query). Rollup rows refresh on a short interval so counts may differ slightly from a live leaf-union — consistent with how other segment-scoped reads work.

[Copilot]

Pull request overview

This PR aims to eliminate Postgres timeouts on POST /membersToMerge and POST /organizationsToMerge (especially with countOnly: true) by avoiding expansion of a selected segment into large leaf-subproject ID lists, and instead relying on rollup rows stored at each segment hierarchy level.

Changes:

• Frontend: stop expanding project groups into leaf subproject IDs when requesting merge suggestions; send only the selected project group ID.
• Backend: remove repository-level getSegmentSubprojects(...) expansion in merge suggestion queries and use SequelizeRepository.getSegmentIds(options) directly.
• Data quality UI: simplify the segments computed arrays to no longer include leaf subproject IDs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
frontend/src/modules/organization/services/organization.api.service.ts	Sends only the selected project group ID for organization merge suggestions.
frontend/src/modules/organization/organization-service.js	Same change for org merge suggestions in the legacy JS service.
frontend/src/modules/data-quality/components/organization/data-quality-organization-merge-suggestions.vue	Drops leaf-subproject expansion from the segments payload.
frontend/src/modules/data-quality/components/member/data-quality-member-merge-suggestions.vue	Drops leaf-subproject expansion from the segments payload.
backend/src/database/repositories/organizationRepository.ts	Removes repository-level leaf expansion for org merge suggestions.
backend/src/database/repositories/memberRepository.ts	Removes repository-level leaf expansion for member merge suggestions.

Comments suppressed due to low confidence (1)

frontend/src/modules/organization/services/organization.api.service.ts:41

• segments can become [undefined] when selectedProjectGroup isn’t set yet. That will still serialize into the request body and can lead to the backend resolving zero segments (or behaving unexpectedly), whereas other methods in this file intentionally send null when there is no selection (see getSelectedProjectGroupId()).

Consider aligning behavior here by sending null (or an empty array) when no project group is selected.

  static async fetchMergeSuggestions(limit: number = 20, offset: number = 0, query: any = {}) {
    const lsSegmentsStore = useLfSegmentsStore();
    const { selectedProjectGroup } = storeToRefs(lsSegmentsStore);

    const segments = [selectedProjectGroup.value?.id];

    const data = {
      limit,
      offset,
      segments,
      ...query,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.