ENG-3000: add Privacy requests tab to integrations (alternative to #8121)#8126
Draft
ENG-3000: add Privacy requests tab to integrations (alternative to #8121)#8126
Conversation
… new Integrations form
ConfigureIntegrationForm relied on two deprecated, system-scoped endpoints
that conflate connection lifecycle with system linking:
* PATCH /system/{fides_key}/connection
* POST /(plus/)system/{fides_key}/connection/instantiate/{type}
Both bake a single system into the connection-create/update call. That
implicitly assumes one-integration-per-system semantics and produces
inconsistent behavior now that a system can have multiple linked
integrations — for example, creating a SaaS integration with a system
selected would error out (the patch path only routes to template
instantiation when secrets is truthy, so SaaS connectors without required
secrets fell through to the standard path and never got a saas_config or
dataset). The deprecation notes on both endpoints already point at the
right replacement: do connection work via the top-level /connection
endpoints, then manage the link via PUT /connection/{key}/system-links.
Refactor handleSubmit to that two-step flow:
1. Create / update the connection without any system context. New SaaS
connections go through POST /connection/instantiate/{type}; everything
else (non-SaaS create, all edits) goes through PATCH /connection.
2. Reconcile the system link via PUT /connection/{key}/system-links —
only when the desired state differs from initialSystemFidesKey, so
common edits (e.g. updating only secrets) skip a redundant idempotent
call. Passing links: [] correctly unlinks, fixing the secondary bug
where clearing the System field on edit silently left the prior link
in place.
Drops the form's dependency on usePatchSystemConnectionConfigsMutation,
useCreatePlusSaasConnectionConfigMutation, and
useCreateSassConnectionConfigMutation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add comments to the model column and the two related Pydantic schemas clarifying that this flag excludes the connection from privacy request (DSR) execution only. It does not affect discovery monitors, connection tests, or any non-DSR consumer; those have their own enable/disable controls. Comments only — no behaviour change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New IntegrationPrivacyRequests component, exposed as a top-level tab
alongside Linked system / Data discovery / etc. on the integration detail
page. Two stacked sections:
* Status — a Switch toggle backed by ConnectionConfig.disabled, immediate
save on flip via PATCH /connection (no confirmation modal). Always
visible. Lets users enable / disable the integration for privacy
request execution without re-opening the connection form.
* Datasets — only rendered for SystemType.DATABASE integrations. Lists
currently linked datasets (router-link to the dataset detail page,
fides_key shown as a monospaced tag, ctl_dataset description as the
secondary line). "Link dataset" opens a search modal of unlinked
datasets. Both flows write through usePutDatasetConfigsMutation; the
PUT endpoint replaces the whole linked-set, so unlink / link are
expressed as a PUT of the new desired key list.
The tab is gated client-side on connection_type, mirroring the
supportsSystemLinking pattern: shown for everything except WEBSITE
(consent only) and DATAHUB (sync only).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ectors
PostgreSQL, RDS MySQL, and Google Cloud SQL for MySQL all extend
SQLConnector and support privacy request execution, but their
integration-type-info marketing tag arrays were missing 'DSR Automation'
(only 'Discovery' / 'Detection' were set). Add the tag so the integration
picker cards accurately advertise the capability and match peer connectors
(BigQuery, MySQL, RDS Postgres, Google Cloud SQL Postgres, etc.).
Reordered postgreSQLInfo tags from ('Detection', 'Discovery') to
('DSR Automation', 'Discovery', 'Detection') for consistency with the
rest of the database connectors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8126 +/- ##
==========================================
+ Coverage 85.19% 85.23% +0.03%
==========================================
Files 638 638
Lines 42008 42011 +3
Branches 4937 4937
==========================================
+ Hits 35788 35807 +19
+ Misses 5111 5096 -15
+ Partials 1109 1108 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
The 'add a new integration associated with a system' test was waiting on
PATCH /api/v1/system/{key}/connection — the deprecated, system-scoped
endpoint this PR routes off of. With the form now creating the connection
via PATCH /connection and reconciling the link via
PUT /connection/{key}/system-links, the test waits on those two requests
instead.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tab order
* Datasets section paragraph: 'Datasets that use this integration to fulfill
privacy requests.' (drops the longer 'are used to traverse and fulfill'
phrasing).
* Both dataset list surfaces (link modal selector and the main linked-
datasets list) now stack the name and the fides_key Tag vertically inside
the title slot, with the Tag using max-w-full so longer fides_keys can use
the row's full width before truncating. Avoids the overflow seen when the
fides_key is long enough to push the name into ellipsis.
* Reorder the Privacy requests tab to sit immediately after Data discovery
(instead of immediately after Linked system). Discovery surfaces the
data; privacy requests act on it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nfig The dataset-config bulk endpoint returns 200 OK even when individual entries fail validation (e.g. 'Dataset for bigquery connection must either have namespace metadata or the connection must have values for the following fields...'). Previously the link/unlink handlers treated any non-error HTTP response as success and showed a 'Dataset linked successfully' toast even when the operation actually failed. Inspect response.data.failed and bail out with the first failure's message before claiming success — same pattern as patchConnectionConfig in the System -> Integrations form. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The new top-level Integrations form's create path doesn't include enabled_actions, so connections it creates land with NULL. The DSR runner treats NULL as 'all actions enabled' for access/erasure but disables consent. Add a comment next to the create payload explaining the trade-off and pointing future readers at the System → Integrations form as the workaround until the field is exposed on the Privacy requests tab. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updated section headings, toggle label, and helper text per latest copy
review:
* Section heading: 'Status' → 'Privacy request automation'
* Toggle label: 'Enable for privacy requests' → 'Automate privacy
requests with this integration'
* Toggle helper: 'When enabled, this integration is used during privacy
request execution.' → 'When off, Fides won't run privacy requests
against data from this integration.'
* Datasets section heading: 'Datasets' → 'Linked datasets'
* Datasets section helper: 'Datasets that use this integration to
fulfill privacy requests.' → 'Choose which datasets Fides traverses
when fulfilling a request. Each dataset can be linked to one
integration.'
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the per-row 'Link' Button + List in the Link datasets modal
with a Table-based bulk-select flow modeled on the Add datasets modal
elsewhere in the product:
* Modal title now shows a 'N selected' Tag pill once the user picks
something. Modal width grows from 520 to 720 to accommodate three
columns.
* Three columns: Dataset (name, falling back to fides_key when null),
Fides key (monospaced, secondary), Description (secondary, em-dash
placeholder when null). All three columns ellipsis with tooltips.
* Antd Table rowSelection drives the checkbox column with built-in
select-all in the header. preserveSelectedRowKeys is true so the
user can search → select → search → select without losing prior
picks. Search now matches name + fides_key + description.
* Modal footer changes from a lone Cancel to Cancel + a primary
'Link N dataset(s)' button (disabled when nothing is selected,
label updates with the selection count). Confirms perform a single
PUT with [...linked, ...selected]; on bulk-write failures the modal
stays open with the first failure's message, so the user can adjust
their selection and retry.
* Pagination defaults to 10 rows, hidden on a single page. The body
is height-bounded with scroll={{ y: 320 }} to keep the modal
compact.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket ENG-3000
Note
This is a sibling PR to #8121, which proposes the same fix with a more scoped UI: an Enable toggle and a datasets picker as inline fields on the create/edit form. This PR keeps everything #8121 does for the bug fix, but moves the privacy-request controls to a dedicated tab and adds tag/marketing fixes that surfaced along the way. Pick one direction; the other can be closed.
Description Of Changes
Three pieces, layered:
1. Stop using deprecated
/system/{key}/connectionendpoints (same as #8121, included here so this PR is self-contained against main).ConfigureIntegrationFormwas wired to two deprecated, system-scoped endpoints that conflate connection lifecycle with system linking and don't compose with multi-integration systems. RefactoredhandleSubmitto a uniform two-step flow:POST /connection/instantiate/{type}, everything else viaPATCH /connection.PUT /connection/{key}/system-links— only when the desired state differs frominitialSystemFidesKey.This fixes the original ENG-3000 bug (SaaS create + system selected fails) and the secondary bug where clearing the System field on edit didn't actually unlink.
2. Add a "Privacy requests" tab on the integration detail page. New
IntegrationPrivacyRequestscomponent, placed afterData discovery(discovery surfaces the data, privacy requests act on it). Two stacked sections:Switchtoggle backed byConnectionConfig.disabled. Always visible. Immediate-save on flip viaPATCH /connection. No confirmation modal — toast on success/error.SystemType.DATABASEintegrations. Lists currently linked datasets (router-link to dataset detail; fides_key shown as a monospacedTagon the line below the name;ctl_dataset.descriptionas the secondary line). "Link dataset" opens a search modal of unlinked datasets. Both flows write throughusePutDatasetConfigsMutation(the PUT endpoint replaces the whole linked-set, so unlink/link are expressed as a PUT of the new desired key list). Bulk responses surface per-entryfailedmessages via toast — same pattern aspatchConnectionConfigin the System → Integrations form.The tab itself is gated client-side on
connection_type, mirroringsupportsSystemLinking— shown for everything exceptWEBSITE(consent only) andDATAHUB(sync only).3. Add
DSR Automationmarketing tag to PostgreSQL, RDS MySQL, and Google Cloud SQL for MySQL. They all extendSQLConnectorand support privacy requests, but their integration-picker tag arrays were missing the tag (peer connectors like BigQuery / MySQL / RDS Postgres / Google Cloud SQL Postgres already had it). Reordered Postgres tags to(DSR Automation, Discovery, Detection)to match the rest.4. Documentation comments on
ConnectionConfig.disabled(model + 2 schemas) clarifying that it gates DSR execution only — not discovery monitors, not connection tests. No behaviour change.Known limitations
enabled_actionsis not exposed in this UI yet. The new top-level Integrations form's create payload doesn't includeenabled_actions, so connections it creates land withNULL. The DSR runner treatsNULLas "all actions enabled" for access/erasure but disables consent (it explicitly requiresenabled_actions IS NOT NULLAND containsconsent— seegraph_task.py:1100-1115). For most integration types this is fine; SaaS consent integrations created through this form need their request types set via the System → Integrations form afterward. Surfacing the field on the Privacy requests tab is a deferred followup (needs a small base-schema addition soPOST /connection/instantiate/{type}andPATCH /connectionacceptenabled_actionsinstead of silently dropping it).Code Changes
clients/admin-ui/src/features/integrations/add-integration/ConfigureIntegrationForm.tsx:PATCH /connection).reconcileSystemLink()helper that fires only whenvalues.system_fides_key !== initialSystemFidesKey, sends[]to unlink.usePatchSystemConnectionConfigsMutation,useCreatePlusSaasConnectionConfigMutation,useCreateSassConnectionConfigMutation.clients/admin-ui/src/features/integrations/IntegrationPrivacyRequests.tsx(new): tab component.clients/admin-ui/src/features/integrations/hooks/useFeatureBasedTabs.tsx: import the component, accept asupportsPrivacyRequestsprop, push the tab after Data discovery.clients/admin-ui/src/pages/integrations/[id]/index.tsx: computesupportsPrivacyRequestsbased onconnection_type.clients/admin-ui/src/features/integrations/integration-type-info/postgreSQLInfo.tsx,rdsMySQLInfo.tsx,googleCloudSQLMySQLInfo.tsx: addDSR Automationtag.src/fides/api/models/connectionconfig.py+src/fides/api/schemas/connection_configuration/connection_config.py: doc comments ondisabled.clients/admin-ui/cypress/e2e/integration-management.cy.ts: update the "add a new integration associated with a system" test to wait onPATCH /connection+PUT /connection/{key}/system-linksinstead of the deprecatedPATCH /system/{key}/connection.Steps to Confirm
POST /api/v1/connection/instantiate/{type}thenPUT /api/v1/connection/{key}/system-links. No call to/system/{key}/connection.GET /api/v1/connection/{key}/system-linksreturns[].setSystemLinksis not called (no-op skip).GET /api/v1/connection/{key}reflects newdisabledvalue. A privacy request that would have hit this connection now skips it (graph_task.skip_if_disabled).[name]on its own line with a[fides_key]tag below and the description underneath. Linking persists viaPUT /api/v1/connection/{key}/datasetconfig. Unlinking shows a confirmation modal.Pre-Merge Checklist
CHANGELOG.mdupdated🤖 Generated with Claude Code