fix(Segment membership PoC): Identity backfill broken due to lack of JSON column support

[khvn26]

Thanks for submitting a PR! Please check the boxes below:

• I have read the Contributing Guide.
• I have added information to docs/ if required so people know about the feature.
• I have filled in the "Changes" section below.
• I have filled in the "How did you test this code" section below.

Changes

Contributes to #5663.

The daily IDENTITIES backfill + per-project refresh have never populated SegmentMembershipCount on staging. Two issues are addressed hiere:

1. clickhouse-driver==0.2.10 has no column class for ClickHouse Cloud 25.12's first-class JSON type, so segment_membership.tasks.backfill_identities_to_clickhouse crashes with KeyError: 'JSON' while encoding the server's sample block.mymarilyn/clickhouse-driver#503 which adds JSON insert + select support. Will switch to a tagged release once that PR merges.
2. ecs-task-definition-migration.json lackes CLICKHOUSE_URL — migration ran manually in staging, and the task definition is fixed here.

How did you test this code?

Smoke tested the exact task code path against staging ClickHouse 25.12.1 from a local Python shell using the patched driver:

• Configured Django with the clickhouse DB alias pointing at the staging CLICKHOUSE_URL (port 9440, ?secure=true).
• Built rows via the real map_identity_document_to_clickhouse_row mapper from synthetic Dynamo-shaped docs carrying str/Decimal/bool traits.
• Created a throwaway table with the same DDL as clickhouse/0001_create_identities, ran cursor.executemany(_INSERT_IDENTITIES_SQL, rows) through connections["clickhouse"].cursor() — the call that was previously failing with KeyError: 'JSON'.
• Queried back with the engine-flavoured shape SELECT environment_id, count() FROM <table> FINAL WHERE environment_id IN %(envs)s AND (traits.plan = 'enterprise' AND traits.age >= 22) GROUP BY environment_id — traits.<key> subcolumn predicates resolved correctly.
• Round-tripped a row: traits returned as a Python dict with Decimal coerced to int and bool preserved.
• Dropped the throwaway table.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
docs	Ignored		May 22, 2026 6:20pm
flagsmith-frontend-preview	Ignored		May 22, 2026 6:20pm
flagsmith-frontend-staging	Ignored		May 22, 2026 6:20pm

[gemini-code-assist]

Code Review

This pull request adds clickhouse-driver as a dependency via a specific git branch and introduces the CLICKHOUSE_URL environment variable to the staging ECS task definition. Feedback indicates that the git-based dependency may necessitate a C compiler in the build environment and could affect performance. Furthermore, the secret ARN for CLICKHOUSE_URL is likely misconfigured, as it lacks the necessary key suffix and is inconsistent with other database URL secrets.

[gemini-code-assist]

Switching clickhouse-driver to a git source removes the pre-built wheels from the lock file. This package contains C extensions, so the build environment (e.g., the Docker build stage) will now require a C compiler and Python development headers. If these are missing, the installation will fail. Additionally, this may lead to a performance regression if the package falls back to the pure-Python implementation due to a failed extension build.

[khvn26]

Bad bot; read the PR description before reviewing

[gemini-code-assist]

The valueFrom ARN for CLICKHOUSE_URL is inconsistent with the ANALYTICS_DATABASE_URL configuration and likely incorrect. While ANALYTICS_DATABASE_URL references a specific key within the shared ECS-API-heAdoB secret, this entry points to a different secret (clickhouse-url-ns26gC) and lacks the required :KEY:: syntax for JSON secrets. If clickhouse-url-ns26gC is a standard Key/Value secret, ECS will inject the entire JSON object as a string, causing the application to fail. Given the PR description states this should use the same value as admin-api, it should likely reference the same secret and key as ANALYTICS_DATABASE_URL.

Suggested change

	"valueFrom": "arn:aws:secretsmanager:eu-west-2:302456015006:secret:clickhouse-url-ns26gC"
	"valueFrom": "arn:aws:secretsmanager:eu-west-2:302456015006:secret:ECS-API-heAdoB:ANALYTICS_DATABASE_URL::"

[khvn26]

bad bot

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.50%. Comparing base (2e16ede) to head (2cd1293).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7584   +/-   ##
=======================================
  Coverage   98.50%   98.50%           
=======================================
  Files        1430     1430           
  Lines       54254    54254           
=======================================
  Hits        53444    53444           
  Misses        810      810           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.