How include resolution works

The include resolution is the process of finding the file that is referenced by the include keyword.

Terminology

• Virtual CI YAML: Is the GitLab CI YAML that is generated by gitlab-ci-verify to check the effective pipeline configuration. It only exists in memory and is not written to disk.

How gitlab-ci-verify resolves includes

gitlab-ci-verify resolves includes by following these steps:

1. Collect all includes.
2. Make sure each include is unique, while keeping their order.
3. Resolve each local include by reading the file and adding it to the virtual CI YAML.
4. Add all includes to the virtual CI YAML end.

Once this processing is done, the virtual CI YAML is used for checks, including validation by the Pipeline Lint API.

How includes are joined to the Virtual CI YAML

Limitations

As gitlab-ci-verify cant apply the exact same logic as the Gitlab CI YAML resolver, it has some limitations:

• Conditional includes are always resolved
• No deep merge
• Duplicated keys in the virtual CI YAML
• Remote includes are not resolved

You can find more in-depth information about each of these limitations in the following sections.

Conditional includes are always resolved

Gitlab CI allows including files conditionally using rules. This is not supported by gitlab-ci-verify.

include:
  - local: 'file.yml'
    rules:
      - if: $CI_COMMIT_REF_NAME == 'master'

Instead, includes are always resolved, regardless of the rules.

This also means that gitlab-ci-verify could include files for checking that are not included by Gitlab CI.

No deep merge

Gitlab CI uses deep merging for included files. This is not supported by gitlab-ci-verify.

For example, the following configuration would result in key being value in Gitlab CI, but other_value in gitlab-ci-verify:

• file.yml:

  key: value
  

• .gitlab-ci.yml:

  include:
    - local: 'file.yml'
  key: other_value
  

While this is not relevant for most use cases, it is important to keep in mind when using gitlab-ci-verify.

Especially the merged YAML for custom checks might differ from the actual Gitlab CI configuration.

This can include but is not limited to:

• job order in the effective YAML
• variable and script overrides

Duplicated keys in the virtual CI YAML

Gitlab CI allows duplicated keys in the CI YAML. This is not standard YAML behavior.

The last key with the same name will overwrite the previous ones.

For example, the following configuration would result in key being value in Gitlab CI, but other_value in gitlab-ci-verify:

key: value
key: other_value

This is also actively used as the virtual CI YAML is generated by joining all included files.

Remote includes are not resolved

GitLab CI allows you to load files from remote locations. This is not supported by gitlab-ci-verify for several reasons:

• Performance: Fetching remote files can be slow
• Reliability: Fetching remote files can fail, leading to flaky results

Besides that, gitlab-ci-verify assumes that any remote file is already linted by the user, ensuring its stability.