Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
ee437f5
[Azure Logs] Add aadgraphactivitylogs data stream
terrancedejesus May 7, 2026
2c88052
Update changelog for azure.aadgraphactivitylogs dataset
terrancedejesus May 7, 2026
7f60d1d
Apply suggestion from @terrancedejesus
terrancedejesus May 7, 2026
c1a75c0
Apply suggestion from @terrancedejesus
terrancedejesus May 7, 2026
82dd31b
azure/aadgraphactivitylogs: update codeowners
terrancedejesus May 8, 2026
9c4fb1e
Merge branch 'main' into enhancement/azure-ad-graph-activitylogs
terrancedejesus May 8, 2026
357c09f
azure/aadgraphactivitylogs: update codeowners
terrancedejesus May 8, 2026
ecae992
Merge branch 'main' into enhancement/azure-ad-graph-activitylogs
terrancedejesus May 8, 2026
30046ce
azure/aadgraphactivitylogs: remove log YAML HBS file
terrancedejesus May 11, 2026
0a4f697
azure/aadgraphactivitylogs: adding docs and updated README
terrancedejesus May 11, 2026
20ba136
azure/aadgraphactivitylogs: adding separate manifest for aadgraphacti…
terrancedejesus May 11, 2026
8e1dc86
azure/aadgraphactivitylogs: adjust data stream test pipeline events a…
terrancedejesus May 11, 2026
97285cc
azure/aadgraphactivitylogs: fix vale linting warnings
terrancedejesus May 11, 2026
7be1278
azure/aadgraphactivitylogs: adds shared pipeline link
terrancedejesus May 11, 2026
a62a1ac
azure/aadgraphactivitylogs: remove 8.14 workaround for dynamic field …
terrancedejesus May 11, 2026
99758fd
azure/aadgraphactivitylogs: adds aadgraphactivitylogs schema referenc…
terrancedejesus May 11, 2026
7f1b4b6
azure/aadgraphactivitylogs: added Azure AD Graph Activity Logs right …
terrancedejesus May 11, 2026
f7a11a1
Merge branch 'main' into enhancement/azure-ad-graph-activitylogs
terrancedejesus May 11, 2026
c6ba91a
azure/aadgraphactivitylogs: sample event and documentation fix
terrancedejesus May 11, 2026
6575463
Update packages/azure/data_stream/aadgraphactivitylogs/elasticsearch/…
terrancedejesus May 12, 2026
08110a7
Update packages/azure/data_stream/aadgraphactivitylogs/elasticsearch/…
terrancedejesus May 12, 2026
8fae9c5
azure/aadgraphactivitylogs: url.original keep original from fingerprint
terrancedejesus May 12, 2026
f8eb858
Merge branch 'main' into enhancement/azure-ad-graph-activitylogs
terrancedejesus May 12, 2026
f995b6f
Merge branch 'main' into enhancement/azure-ad-graph-activitylogs
terrancedejesus May 13, 2026
bbc07f7
Merge branch 'main' into enhancement/azure-ad-graph-activitylogs
terrancedejesus May 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
/packages/awsfargate @elastic/obs-infraobs-integrations
/packages/awsfirehose @elastic/obs-ds-hosted-services
/packages/azure @elastic/obs-infraobs-integrations @elastic/obs-ds-hosted-services @elastic/security-service-integrations
/packages/azure/data_stream/aadgraphactivitylogs @elastic/security-service-integrations
/packages/azure/data_stream/activitylogs @elastic/obs-infraobs-integrations
/packages/azure/data_stream/application_gateway @elastic/security-service-integrations
/packages/azure/data_stream/auditlogs @elastic/obs-infraobs-integrations
Expand Down
4 changes: 2 additions & 2 deletions packages/azure/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Azure Logs Integration

The Azure Logs integration collects logs for specific Azure services like Microsoft Entra ID (Sign-in, Audit, Identity Protection, and Provisioning logs), Azure Spring Apps, Azure Firewall, Microsoft Graph Activity, and several others using the Activity and Platform logs.
The Azure Logs integration collects logs for specific Azure services like Microsoft Entra ID (Sign-in, Audit, Identity Protection, and Provisioning logs), Azure Spring Apps, Azure Firewall, Microsoft Graph Activity, Azure AD Graph Activity, and several others using the Activity and Platform logs.

You can then visualize that data in Kibana, create alerts if something goes wrong, and reference data when troubleshooting an issue.

Expand All @@ -16,7 +16,7 @@ fail to start due to an exceed quota limit.
The Azure Logs integration collects logs.

**Logs** help you keep a record of events that happen on your Azure account.
Log data streams collected by the Azure Logs integration include Activity, Platform, Microsoft Entra ID (Sign-in, Audit, Identity Protection, Provisioning), Microsoft Graph Activity, and Spring Apps logs.
Log data streams collected by the Azure Logs integration include Activity, Platform, Microsoft Entra ID (Sign-in, Audit, Identity Protection, Provisioning), Microsoft Graph Activity, Azure AD Graph Activity, and Spring Apps logs.

## Requirements

Expand Down
89 changes: 89 additions & 0 deletions packages/azure/_dev/build/docs/aadgraphactivitylogs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Azure AD Graph Activity Logs

Azure AD Graph Activity Logs provide an audit trail of all HTTP requests that the legacy Azure AD Graph service (`graph.windows.net`) has received and processed for a tenant. Although Microsoft has deprecated Azure AD Graph in favor of Microsoft Graph, the API is still actively used by Microsoft first-party tooling, older line-of-business applications, third-party SaaS connectors, and adversary tooling (for example ROADtools, AzureHound v1, AADInternals). Refer to the [AADGraphActivityLogs table reference](https://learn.microsoft.com/en-us/azure/azure-monitor/reference/tables/aadgraphactivitylogs) for the canonical schema.

Tenant administrators can configure the collection and storage destinations of Azure AD Graph Activity Logs through Diagnostic Setting in the Entra Portal. This integration uses Azure Event Hubs destination to stream Azure AD Graph Activity Logs to Elastic.

## Requirements and Setup

### What do I need to use this integration?

The following privileges are required to collect Azure AD Graph Activity Logs:
- A Microsoft Entra ID P1 or P2 tenant license in your tenant.
- A `Security Administrator` or `Global Administrator` Microsoft Entra ID role to configure the diagnostic settings.

### Setup

Refer to the [Azure Logs](https://docs.elastic.co/integrations/azure) page for more information about setting up and using this integration.

### Limitations

- Activities of multi-tenant applications belonging to another tenant are not available.
- In rare cases, events might take up to 2 hours to be delivered to Event Hubs.
- Azure AD Graph is deprecated by Microsoft. New workloads should target Microsoft Graph; this dataset is intended for visibility into legacy traffic that still exists in the tenant.

## Settings

`eventhub` :
_string_
It is a fully managed, real-time data ingestion service. Elastic recommends using only letters, numbers, and the hyphen (-) character for Event Hub names to maximize compatibility. You _can_ use existing Event Hubs having underscores (_) in the Event Hub name. In this case, the integration will replace underscores with hyphens (-) when it uses the Event Hub name to create dependent Azure resources behind the scenes (for example, the storage account container to store Event Hub consumer offsets). Elastic also recommends using a separate event hub for each log type as the field mappings of each log type differ.
Default value `insights-operational-logs`.

`consumer_group` :
_string_
The publish/subscribe mechanism of Event Hubs is enabled through consumer groups. A consumer group is a view (state, position, or offset) of an entire event hub. Consumer groups enable multiple consuming applications to each have a separate view of the event stream, and to read the stream independently at their own pace and with their own offsets.
Default value: `$Default`

`connection_string` :
_string_
The connection string required to communicate with Event Hubs, steps [here](https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-get-connection-string).

A Blob Storage account is required to store, retrieve, and update the offset or state of the eventhub messages. This means that after stopping the filebeat azure module it can start back up at the spot that it stopped processing messages.

`storage_account` :
_string_
The name of the storage account the state/offsets will be stored and updated.

`storage_account_key` :
_string_
The storage account key, this key will be used to authorize access to data in your storage account.

`storage_account_container` :
_string_
The storage account container where the integration stores the checkpoint data for the consumer group. It is an advanced option to use with extreme care. You MUST use a dedicated storage account container for each Azure log type (activity, sign-in, audit logs, and others). DO NOT REUSE the same container name for more than one Azure log type. See [Container Names](https://docs.microsoft.com/en-us/rest/api/storageservices/naming-and-referencing-containers--blobs--and-metadata#container-names) for details on naming rules from Microsoft. The integration generates a default container name if not specified.

`resource_manager_endpoint` :
_string_
Optional. By default the integration uses the Azure public environment. To override, provide a specific resource manager endpoint to use a different Azure environment.

Resource manager endpoints:

```text
# Azure ChinaCloud
https://management.chinacloudapi.cn/

# Azure GermanCloud
https://management.microsoftazure.de/

# Azure PublicCloud
https://management.azure.com/

# Azure USGovernmentCloud
https://management.usgovcloudapi.net/
```

## Logs

### aadgraphactivitylogs

The `aadgraphactivitylogs` data stream of the Azure Logs package collects Azure AD Graph activity events that have been streamed through an Azure event hub. The events ingest pipeline matches `category == "AzureADGraphActivityLogs"` and sets `event.dataset = azure.aadgraphactivitylogs`. The events data stream's routing rules then reroute the document from `logs-azure.events-*` directly to `logs-azure.aadgraphactivitylogs-*`, where this data stream's pipeline applies full ECS field extraction.

Before this data stream existed, AAD Graph events had no specific override in the events router and fell through to the `azure.platformlogs` catch-all, landing in `logs-azure.platformlogs-default` with only generic platform-log parsing. Those previously-indexed events are not backfilled. Only new events are routed to the dedicated dataset.

{{event "aadgraphactivitylogs"}}

**ECS Field Reference**

Refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.

{{fields "aadgraphactivitylogs"}}
5 changes: 5 additions & 0 deletions packages/azure/changelog.yml
Comment thread
chrisberkhout marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
- version: "1.37.0"
changes:
- description: Add azure.aadgraphactivitylogs dataset to ingest legacy Azure AD Graph (graph.windows.net) AzureADGraphActivityLogs diagnostic events.
type: enhancement
link: https://github.com/elastic/integrations/pull/18880
- version: "1.36.1"
changes:
- description: Add missing event.kind pipeline_error handling to ingest pipeline on_failure handlers.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{"category":"AzureADGraphActivityLogs","location":"WestUS","operationName":"AAD Graph Activity","properties":{"__UDI_RequiredFields_EventTime":639140000000000000,"__UDI_RequiredFields_RegionScope":"NA","__UDI_RequiredFields_TenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","__UDI_RequiredFields_UniqueId":"00000001-0001-0001-0001-000000000001","actorType":"User","apiVersion":"1.6","appId":"04b07795-8ddb-461a-bbee-02f9e1bf7b46","callerIpAddress":"81.2.69.143","clientAuthMethod":0,"deviceId":"","directAccessSource":"Gateway","durationMs":59,"env_cloud_role":"restdirectoryservice","httpMethod":"GET","httpStatusCode":200,"identityProvider":"https://sts.windows.net/ab30785b-417f-42a4-b5dc-8f9051718acb/","issuedAt":"5/7/2026 1:50:39 PM","location":"WestUS","requestId":"00000001-0001-0001-0001-000000000001","requestUri":"/v2/ab30785b-417f-42a4-b5dc-8f9051718acb/users","responseSizeBytes":54662,"roles":"","scopes":"62e90394-69f5-4237-9190-012177145e10","servicePrincipalId":"","sessionId":"5a5a5a5a-5a5a-5a5a-5a5a-5a5a5a5a5a5a","signInActivityId":"AAAAAAAAAAAAAAAAAAAAAA==","tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","timeGenerated":"2026-05-07T15:19:33.5368860Z","userAgent":"azure-graph-test-client/1.0","userId":"b37ec517-0a34-4266-b627-f7bb0d679d70","wids":""},"tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb"}
{"category":"AzureADGraphActivityLogs","location":"WestUS","operationName":"AAD Graph Activity","properties":{"__UDI_RequiredFields_EventTime":639140000000000001,"__UDI_RequiredFields_RegionScope":"NA","__UDI_RequiredFields_TenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","__UDI_RequiredFields_UniqueId":"00000002-0002-0002-0002-000000000002","actorType":"User","apiVersion":"1.61-internal","appId":"1b730954-1685-4b74-9bfd-dac224a7b894","callerIpAddress":"81.2.69.143","clientAuthMethod":0,"deviceId":"","directAccessSource":"Gateway","durationMs":25,"env_cloud_role":"restdirectoryservice","httpMethod":"GET","httpStatusCode":403,"identityProvider":"","issuedAt":"5/8/2026 4:11:19 PM","location":"WestUS","requestId":"00000002-0002-0002-0002-000000000002","requestUri":"/v2/ab30785b-417f-42a4-b5dc-8f9051718acb/tenantDetails","responseSizeBytes":-1,"roles":"","scopes":"user_impersonation","servicePrincipalId":"","sessionId":"004d8baa-891d-139e-633f-6c194e566b78","signInActivityId":"BCF50s3Ik0q7Ke-mIBAGAA==","tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","timeGenerated":"2026-05-08T16:37:21.2823997Z","userAgent":"AADInternals","userId":"b37ec517-0a34-4266-b627-f7bb0d679d70","wids":""},"tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb"}
{"category":"AzureADGraphActivityLogs","location":"WestUS","operationName":"AAD Graph Activity","properties":{"__UDI_RequiredFields_EventTime":639141000000000000,"__UDI_RequiredFields_RegionScope":"NA","__UDI_RequiredFields_TenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","__UDI_RequiredFields_UniqueId":"00000003-0003-0003-0003-000000000003","actorType":"Application","apiVersion":"AGSbeta-internal","appId":"65d91a3d-ab74-42e6-8a2f-0add61688c74","callerIpAddress":"81.2.69.144","clientAuthMethod":0,"deviceId":"","directAccessSource":"Gateway","durationMs":44,"env_cloud_role":"restdirectoryservice","httpMethod":"GET","httpStatusCode":200,"identityProvider":"https://sts.windows.net/ab30785b-417f-42a4-b5dc-8f9051718acb/","issuedAt":"5/11/2026 12:35:58 PM","location":"WestUS","requestId":"00000003-0003-0003-0003-000000000003","requestUri":"/v2/ab30785b-417f-42a4-b5dc-8f9051718acb/groups","responseSizeBytes":695,"roles":" Group.Read.All,","scopes":"","servicePrincipalId":"fa5cc200-f464-4068-88b9-c6b8335c8c70","sessionId":"","signInActivityId":"Rr_bbVOVu0WHOTzl0BhgAA==","tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","timeGenerated":"2026-05-11T12:41:10.9430757Z","userAgent":"","userId":"","wids":" 0997a1d0-0d1d-4acb-b408-d5ca73121e90,"},"tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb"}
{"category":"AzureADGraphActivityLogs","location":"SouthCentralUS","operationName":"AAD Graph Activity","properties":{"__UDI_RequiredFields_EventTime":639141000000000001,"__UDI_RequiredFields_RegionScope":"NA","__UDI_RequiredFields_TenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","__UDI_RequiredFields_UniqueId":"00000004-0004-0004-0004-000000000004","actorType":"Application","apiVersion":"1.61-internal","appId":"66244124-575c-4284-92bc-fdd00e669cea","callerIpAddress":"89.160.20.156","clientAuthMethod":0,"deviceId":"","directAccessSource":"Gateway","durationMs":152,"env_cloud_role":"restdirectoryservice","httpMethod":"POST","httpStatusCode":202,"identityProvider":"https://sts.windows.net/ab30785b-417f-42a4-b5dc-8f9051718acb/","issuedAt":"5/9/2026 12:28:05 PM","location":"SouthCentralUS","requestId":"00000004-0004-0004-0004-000000000004","requestUri":"/ab30785b-417f-42a4-b5dc-8f9051718acb/$batch","responseSizeBytes":9580,"roles":"","scopes":"","servicePrincipalId":"4ed697af-4af9-43f7-8bf2-7b2172300068","sessionId":"","signInActivityId":"-TZsVeDWj0e-jamRtuUnAA==","tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","timeGenerated":"2026-05-09T12:33:05.9908603Z","userAgent":"","userId":"","wids":""},"tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb"}
{"category":"AzureADGraphActivityLogs","location":"WestUS","operationName":"AAD Graph Activity","properties":{"__UDI_RequiredFields_EventTime":639142000000000000,"__UDI_RequiredFields_RegionScope":"NA","__UDI_RequiredFields_TenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","__UDI_RequiredFields_UniqueId":"00000005-0005-0005-0005-000000000005","actorType":"User","apiVersion":"1.6","appId":"04b07795-8ddb-461a-bbee-02f9e1bf7b46","callerIpAddress":"81.2.69.143","clientAuthMethod":0,"deviceId":"","directAccessSource":"Gateway","durationMs":71,"env_cloud_role":"restdirectoryservice","httpMethod":"PATCH","httpStatusCode":204,"identityProvider":"https://sts.windows.net/ab30785b-417f-42a4-b5dc-8f9051718acb/","issuedAt":"5/7/2026 1:50:39 PM","location":"WestUS","requestId":"00000005-0005-0005-0005-000000000005","requestUri":"/v2/ab30785b-417f-42a4-b5dc-8f9051718acb/applications/fda2d70f-7df3-4556-8bcb-30acc0a45082","responseSizeBytes":0,"roles":"","scopes":"62e90394-69f5-4237-9190-012177145e10","servicePrincipalId":"","sessionId":"5a5a5a5a-5a5a-5a5a-5a5a-5a5a5a5a5a5a","signInActivityId":"AAAAAAAAAAAAAAAAAAAAAA==","tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb","timeGenerated":"2026-05-07T15:15:44.4909654Z","userAgent":"azure-graph-test-client/1.0","userId":"b37ec517-0a34-4266-b627-f7bb0d679d70","wids":""},"tenantId":"ab30785b-417f-42a4-b5dc-8f9051718acb"}
Loading
Loading