Skip to content

docs: s3: SSE configuration docs #2365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
GabrielYamin wants to merge 3 commits into
base: master
Choose a base branch
from
Open

docs: s3: SSE configuration docs #2365

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
66dde70
docs: s3: SSE configuration docs
GabrielYamin Jan 28, 2026
0ac085e
update key_id to say key ARN only (according to docs)
GabrielYamin Jan 29, 2026
47859f7
Update SSE and KMS key ID descriptions in s3.md
GabrielYamin Jan 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions pipeline/outputs/s3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@
| `s3_key_format` | Format string for keys in S3. This option supports a UUID, strftime time formatters, a syntax for selecting parts of the Fluent log tag using a syntax inspired by the `rewrite_tag` filter. Add `$UUID` in the format string to insert a random string. Add `$INDEX` in the format string to insert an integer that increments each upload. The `$INDEX` value saves in the `store_dir`. Add `$TAG` in the format string to insert the full log tag. Add `$TAG[0]` to insert the first part of the tag in the S3 key. The tag is split into parts using the characters specified with the `s3_key_format_tag_delimiters` option. Add the extension directly after the last piece of the format string to insert a key suffix. To specify a key suffix in `use_put_object` mode, you must specify `$UUID`. See [S3 Key Format](#s3-key-format-and-tag-delimiters). Time in `s3_key` is the timestamp of the first record in the S3 file. | `/fluent-bit-logs/$TAG/%Y/%m/%d/%H/%M/%S` |
| `s3_key_format_tag_delimiters` | A series of characters which will be used to split the tag into `parts` for use with the s3_key_format option. | `.` |
| `send_content_md5` | Send the Content-MD5 header with `PutObject` and UploadPart requests, as is required when Object Lock is enabled. | `false` |
| `sse` | Server-side encryption (SSE) for S3 objects. Set to `AES256` for S3-managed keys (SSE-S3), `aws:kms` for AWS Key Management Service (SSE-KMS), or `aws:kms:dsse` for dual-layer server-side encryption with KMS (DSSE-KMS). | _none_ |

Check warning on line 83 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 83, "column": 257}}}, "severity": "INFO"}

Check warning on line 83 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'DSSE', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'DSSE', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 83, "column": 252}}}, "severity": "INFO"}

Check warning on line 83 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 83, "column": 247}}}, "severity": "INFO"}

Check warning on line 83 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 83, "column": 176}}}, "severity": "INFO"}

Check warning on line 83 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 83, "column": 125}}}, "severity": "INFO"}

Check warning on line 83 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 83, "column": 67}}}, "severity": "INFO"}
Copy link
Copy Markdown
Collaborator

@eschabell eschabell May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GabrielYamin if you enclose the offending word in back-ticks, this will resolve.

| `sse_kms_key_id` | AWS KMS key Resource Name (ARN) for server-side encryption. Only applicable when `sse` is set to `aws:kms` or `aws:kms:dsse`. If not specified, the default AWS-managed KMS key for S3 will be used. | _none_ |

Check warning on line 84 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 84, "column": 211}}}, "severity": "INFO"}

Check warning on line 84 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 84, "column": 47}}}, "severity": "INFO"}
Copy link
Copy Markdown
Collaborator

@eschabell eschabell May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GabrielYamin same back-tick issue here.

| `static_file_path` | Disables behavior where UUID string appends to the end of the S3 key name when `$UUID` isn't provided in `s3_key_format`. `$UUID`, time formatters, `$TAG`, and other dynamic key formatters all work as expected while this feature is set to true. | `false` |
| `store_dir` | Directory to locally buffer data before sending. Plugin uses the S3 Multipart upload API to send data in chunks of 5 MB at a time. | `/tmp/fluent-bit/s3` |
| `store_dir_limit_size` | S3 plugin has its own buffering system with files in the `store_dir`. Use the `store_dir_limit_size` to limit the amount of data S3 buffers in the `store_dir` to limit disk usage. If the limit is reached, data will be discarded. Default is 0 which means unlimited. | `0` |
Expand Down Expand Up @@ -574,8 +576,48 @@
{% endtab %}
{% endtabs %}

An example using SSE-KMS encryption:

Check warning on line 579 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'KMS', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 579, "column": 22}}}, "severity": "INFO"}

Check warning on line 579 in pipeline/outputs/s3.md

View workflow job for this annotation

GitHub Actions / runner / vale 

[vale] reported by reviewdog 🐶
[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.

Raw Output:
{"message": "[FluentBit.Acronyms] Spell out 'SSE', if it's unfamiliar to the audience.", "location": {"path": "pipeline/outputs/s3.md", "range": {"start": {"line": 579, "column": 18}}}, "severity": "INFO"}
Copy link
Copy Markdown
Collaborator

@eschabell eschabell May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GabrielYamin back-ticks will solve this.


{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:

outputs:
- name: s3
match: '*'
bucket: your-bucket
region: us-east-1
store_dir: /home/ec2-user/buffer
total_file_size: 50M
upload_timeout: 10m
sse: aws:kms
sse_kms_key_id: arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[OUTPUT]
Name s3
Match *
bucket your-bucket
region us-east-1
store_dir /home/ec2-user/buffer
total_file_size 50M
upload_timeout 10m
sse aws:kms
sse_kms_key_id arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012
```

{% endtab %}
{% endtabs %}

## AWS for Fluent Bit


Amazon distributes a container image with Fluent Bit and plugins.

### GitHub
Expand Down
Loading