Add environment variable substitution for reference configuration files

[SalmanDeveloperz]

Implements environment variable substitution for configuration files copied from /usr/share/jenkins/ref/ to $JENKINS_HOME during container initialization.

Problem

Users frequently need to parameterize Jenkins configuration files (XML, properties) with values that vary across environments (URLs, credentials, feature flags). The current implementation copies reference files verbatim, requiring workarounds like Groovy init scripts for dynamic configuration.

Solution

Added substitute_env_vars() function to jenkins-support that processes configuration files after copying. Supports standard shell substitution syntax:

• ${VAR} - Replaced with environment variable value
• ${VAR:-default} - Replaced with env var or default if unset

Applied to .xml, .conf, .properties and .groovy files during copy_reference_file() execution.

Testing done

Manual verification performed with the following scenarios:

Scenario 1: Full environment substitution

docker run -e JENKINS_URL=http://prod.company.com \
           -e SMTP_HOST=smtp.company.com \
           jenkins/jenkins:lts

Result: All placeholders replaced with provided values.

Scenario 2: Partial substitution with defaults

docker run -e JENKINS_URL=http://custom.com jenkins/jenkins:lts

Result: JENKINS_URL substituted, SMTP_HOST used default value from ${SMTP_HOST:-smtp.default.com}.

Scenario 3: No environment variables

docker run jenkins/jenkins:lts

Result: All placeholders used default values from ${VAR:-default} syntax.

Before Fix:

After Fix:

Closes #448

Submitter checklist

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or Jira
• Link to relevant pull requests, esp. upstream and downstream changes
• Ensure you have provided tests that demonstrate the feature works or the issue is fixed

[PsymoNiko]

Good point

[dduportal]

Thanks for this proposed changes.

Since it was sent without an issue explaining the problem you would want to solve, discussion is needed. As such, I'm marking this PR as "Change Requested" until maintainers (at least 2) agree on moving forward.

---

At first sight, you want what Jenkins Configuration as Code (https://www.jenkins.io/projects/jcasc/) is aimed for. JCasC would be the recommended way to solve your initial problem.

That being said, you might have a use case which makes JCasC not an option: can you describe it and explain what is it, so we can all discuss if this code change is worth merging for everyone or refusing because too specific/edgy?

---

Regarding the change itself, if we agree to merge it, I would like it not to be a default behavior because it would break thinks by surprise for current users when they would upgrade to this change (I have patterns of ${VAR} in some files on my test Jenkins and trying this change is breaking them by replacing the value while I don't want to).

Could you make the feature "opt-in" (e.g. you have to pass either a specific flag or set up a specific environment variable to enable the substition?

[SalmanDeveloperz]

Thanks for the review.

On JCasC: yes, it covers most dynamic config needs well. The gap here is for teams that ship a pre-baked Jenkins image with ref XML files and want to parameterize things like URLs or hostnames without pulling in a plugin just for that. The ref/ mechanism is already the standard way to seed config, so making it accept env vars feels like a natural extension rather than a separate concern.

On the opt-in point: makes complete sense. Before I push the update, I want to confirm the approach. My plan is to gate substitution behind something like JENKINS_ENABLE_ENV_SUBST=true so existing users see zero behavior change on upgrade. Does that flag name work for you, or do you have a naming convention you'd prefer I follow? Also, should the flag live as a top-level env var check, or would you rather it be scoped per-file via a naming convention on the ref files themselves??

Want to align on this before pushing so the next revision is closer to mergeable.

[dduportal]

On the opt-in point: makes complete sense. Before I push the update, I want to confirm the approach. My plan is to gate substitution behind something like JENKINS_ENABLE_ENV_SUBST=true so existing users see zero behavior change on upgrade. Does that flag name work for you, or do you have a naming convention you'd prefer I follow? Also, should the flag live as a top-level env var check, or would you rather it be scoped per-file via a naming convention on the ref files themselves??

This proposal looks good to me. I believe you can proceed, while I'm asking other maintainers from another point of view (ping @jenkinsci/team-docker-packaging ) to be sure I'm not missing anything.

As part of your rebase + update of the PR, could you document the variable and its behavior in the README please (it would describe the "problem" to solve)?

Thanks for the testimony. I still recommend using JCasC as it brings many other wanted features in the mix (and adding this well-maintained and strongly recommended plugin is not really an overhead in the Docker images) but it clarifies.

[MarkEWaite]

This proposal looks good to me. I believe you can proceed, while I'm asking other maintainers from another point of view (ping @jenkinsci/team-docker-packaging ) to be sure I'm not missing anything.

I like the approach. I have not evaluated this implementation. I'm using a similar concept in my docker-lfs repository because I stored reference copies of XML files in my ref/jobs directory years ago when I created the repository.

As part of evaluating this pull request, I can attempt to transition my rather dirty hack to use the method proposed by this pull request. I still have the goal to completely transition my private-lfs repository to Job DSL, but that will likely need several more months before the transition is complete.

[dduportal]

Thanks @MarkEWaite ! I believe it gives a positive signal for @SalmanDeveloperz to move this PR forward then 👍

[SalmanDeveloperz]

As part of your rebase + update of the PR, could you document the variable and its behavior in the README please?

Done. Added a Environment variable substitution for reference files section under the ref files part of the README covering the opt-in flag, supported syntax, applicable file types, and a working example.

Could you make the feature opt-in?

Done. Substitution is now gated behind JENKINS_ENABLE_ENV_SUBST=true. Existing users see zero behavior change on upgrade.

Tested locally against three scenarios: full substitution with env vars set, partial substitution falling back to ${VAR:-default} values, and flag disabled confirming placeholders are left untouched.