Skip to content

[Dataflow] Added Portable Runner alias to java runners#38411

Merged
shunping merged 5 commits intoapache:masterfrom
TongruiLi:java_portable_runner
May 8, 2026
Merged

[Dataflow] Added Portable Runner alias to java runners#38411
shunping merged 5 commits intoapache:masterfrom
TongruiLi:java_portable_runner

Conversation

@TongruiLi
Copy link
Copy Markdown
Contributor

Added Portable Runner Passing for Java Runners. This includes the following experiment options

  1. enable_portable_runner as a valid option for enabling runner v2.
  2. disable_portable_runner/enable_streaming_java_runner for disabling runner v2.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the Dataflow runner configuration by introducing the 'enable_portable_runner' experiment. This change standardizes how Runner v2 is enabled and managed across the Java, Go, and Python SDKs, while also adding safeguards to prevent invalid configurations by explicitly flagging conflicting experiments like 'disable_portable_runner' and 'enable_streaming_java_runner'.

Highlights

  • Portable Runner Integration: Added 'enable_portable_runner' as a new experiment option to explicitly enable Runner v2 across Java, Go, and Python SDKs.
  • Conflict Management: Introduced 'disable_portable_runner' and 'enable_streaming_java_runner' as experiments that trigger validation errors when conflicting with Runner v2 settings.
  • Cross-SDK Consistency: Updated Dataflow runner logic in Java, Go, and Python to ensure the 'enable_portable_runner' experiment is consistently handled and tested.
New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates the "enable_portable_runner" experiment across the Java, Go, and Python SDKs to support Dataflow Runner v2. It also updates validation logic to include "disable_portable_runner" and "enable_streaming_java_runner". Review feedback highlights a logic error in the Java runner's automatic enablement condition, a typo in an experiment name in the Go SDK, and the need to update error messages to include the new experiments.

Comment thread sdks/go/pkg/beam/runners/dataflow/dataflow.go Outdated
Comment thread sdks/go/pkg/beam/runners/dataflow/dataflow.go
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 7, 2026

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@TongruiLi
Copy link
Copy Markdown
Contributor Author

R: @scwhittle

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

@github-actions github-actions Bot added the java label May 8, 2026
@shunping
Copy link
Copy Markdown
Collaborator

shunping commented May 8, 2026

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the DataflowRunner to include 'enable_portable_runner' as a trigger for the Unified Worker and adds 'disable_portable_runner' and 'enable_streaming_java_runner' to the list of experiments that conflict with it. It also updates corresponding tests and increases a thread pool threshold in UnboundedScheduledExecutorServiceTest. Feedback suggests removing an unused local variable in DataflowRunner and updating an exception message to accurately reflect the newly added experiment flags.

@TongruiLi
Copy link
Copy Markdown
Contributor Author

Addressed all Gemini reviews.

Copy link
Copy Markdown
Collaborator

@shunping shunping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@shunping shunping merged commit 81828fd into apache:master May 8, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants