Indirect dependencies in dependent go modules need to be updated #12999

eriksw · 2021-12-07T21:05:19Z

What would you like Renovate to be able to do?

Because go modules include indirect dependencies in the go.mod and go.sum, when Renovate updates a go module, it also needs to update dependent modules.

Scenario:

monorepo/a:
    has a dependency that renovate is updating
monorepo/b:
    has a dependency on monorepo/a
    has a replace rule for monorepo/a => ../a
    does not directly import the library
    has "indirect" go.mod and go.sum entries for the dependencies of a because go adds/requires them now

When renovate updates monorepo/a/... it needs to also update monorepo/go/b/go.* at the same time, in the same commit.

Because renovate currently does not do this, the PR to update module a fails because we have a test that ensures our module files are always tidy. That fails because when the test runs go mod tidy in b, it is stale and updated/rewritten by the go mod tidy invocation in b, which updates b's indirect dependencies in go.mod and go.sum.

Reproduction repo/PR: eriksw/renovate-peer-go-mod-indirect#1

Log:

DEBUG: No dangling containers to remove
INFO: Repository started
{
  "renovateVersion": "29.32.5"
}
DEBUG: Using localDir: /mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect
DEBUG: Repository cache not found
{
  "cacheFileName": "/tmp/renovate-cache/renovate/repository/github/eriksw/renovate-peer-go-mod-indirect.json"
}
DEBUG: initRepo("eriksw/renovate-peer-go-mod-indirect")
DEBUG: Overriding default GitHub endpoint
{
  "endpoint": "https://api.github.com/"
}
DEBUG: eriksw/renovate-peer-go-mod-indirect default branch = main
DEBUG: Using app token for git init
DEBUG: resetMemCache()
DEBUG: Resetting npmrc
DEBUG: detectSemanticCommits()
DEBUG: Initializing git repository into /mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect
DEBUG: Performing blobless clone
DEBUG: git clone completed
{
  "durationMs": 1234
}
DEBUG: latest repository commit
{
  "latestCommit": {
    "hash": "3d7e94578947ca98bffbd997a282b134ebbd1429",
    "date": "2021-12-07T12:53:59-08:00",
    "message": "Initial commit",
    "refs": "HEAD -> main, origin/main, origin/HEAD",
    "body": "",
    "author_name": "Erik Swanson",
    "author_email": "[email protected]"
  }
}
DEBUG: getCommitMessages
DEBUG: Semantic commits detection: unknown
DEBUG: No semantic commits detected
DEBUG: checkOnboarding()
DEBUG: isOnboarded()
DEBUG: findFile(renovate.json)
DEBUG: Config file exists
{
  "fileName": "renovate.json"
}
DEBUG: Retrieving issueList
DEBUG: Retrieved 0 issues
DEBUG: Repo is onboarded
DEBUG: Found renovate.json config file
DEBUG: Repository config
{
  "fileName": "renovate.json",
  "config": {
    "extends": [
      "config:base"
    ],
    "postUpdateOptions": [
      "gomodTidy",
      "gomodUpdateImportPaths"
    ],
    "prConcurrentLimit": 5,
    "rebaseWhen": "conflicted"
  }
}
DEBUG: migrateAndValidate()
DEBUG: No config migration necessary
DEBUG: massaged config
{
  "config": {
    "extends": [
      "github>whitesource/merge-confidence:beta",
      "config:base"
    ],
    "postUpdateOptions": [
      "gomodTidy",
      "gomodUpdateImportPaths"
    ],
    "prConcurrentLimit": 5,
    "rebaseWhen": "conflicted"
  }
}
DEBUG: migrated config
{
  "config": {
    "extends": [
      "github>whitesource/merge-confidence:beta",
      "config:base"
    ],
    "postUpdateOptions": [
      "gomodTidy",
      "gomodUpdateImportPaths"
    ],
    "prConcurrentLimit": 5,
    "rebaseWhen": "conflicted"
  }
}
DEBUG: Setting hostRules from config
DEBUG: Found repo ignorePaths
{
  "ignorePaths": [
    "**/node_modules/**",
    "**/bower_components/**",
    "**/vendor/**",
    "**/examples/**",
    "**/__tests__/**",
    "**/test/**",
    "**/tests/**",
    "**/__fixtures__/**"
  ]
}
DEBUG: No vulnerability alerts found
DEBUG: No vulnerability alerts found
DEBUG: findIssue(Dependency Dashboard)
DEBUG: No baseBranches
DEBUG: extract()
DEBUG: Setting current branch to main
DEBUG: latest commit
{
  "branchName": "main",
  "latestCommitDate": "2021-12-07T12:53:59-08:00"
}
DEBUG: Using file match: (^|/)tasks/[^/]+\.ya?ml$ for manager ansible
DEBUG: Using file match: (^|/)requirements\.ya?ml$ for manager ansible-galaxy
DEBUG: Using file match: (^|/)galaxy\.ya?ml$ for manager ansible-galaxy
DEBUG: Using file match: azure.*pipelines?.*\.ya?ml$ for manager azure-pipelines
DEBUG: Using file match: (^|/)batect(-bundle)?\.yml$ for manager batect
DEBUG: Using file match: (^|/)batect$ for manager batect-wrapper
DEBUG: Using file match: (^|/)WORKSPACE(|\.bazel)$ for manager bazel
DEBUG: Using file match: \.bzl$ for manager bazel
DEBUG: Using file match: (^|/)\.?bitbucket-pipelines\.ya?ml$ for manager bitbucket-pipelines
DEBUG: Using file match: buildkite\.ya?ml for manager buildkite
DEBUG: Using file match: \.buildkite/.+\.ya?ml$ for manager buildkite
DEBUG: Using file match: (^|/)Gemfile$ for manager bundler
DEBUG: Using file match: \.cake$ for manager cake
DEBUG: Using file match: (^|/)Cargo.toml$ for manager cargo
DEBUG: Using file match: (^|/).circleci/config.yml$ for manager circleci
DEBUG: Using file match: (^|/)cloudbuild.ya?ml for manager cloudbuild
DEBUG: Using file match: (^|/)Podfile$ for manager cocoapods
DEBUG: Using file match: (^|/)([\w-]*)composer.json$ for manager composer
DEBUG: Using file match: (^|/)deps\.edn$ for manager deps-edn
DEBUG: Using file match: (^|/)docker-compose[^/]*\.ya?ml$ for manager docker-compose
DEBUG: Using file match: (^|/|\.)Dockerfile$ for manager dockerfile
DEBUG: Using file match: (^|/)Dockerfile\.[^/]*$ for manager dockerfile
DEBUG: Using file match: (^|/).drone.yml$ for manager droneci
DEBUG: Using file match: (^|/).gitmodules$ for manager git-submodules
DEBUG: Using file match: ^(workflow-templates|\.github\/workflows)\/[^/]+\.ya?ml$ for manager github-actions
DEBUG: Using file match: (^|\/)action\.ya?ml$ for manager github-actions
DEBUG: Using file match: \.gitlab-ci\.yml$ for manager gitlabci
DEBUG: Using file match: \.gitlab-ci\.yml$ for manager gitlabci-include
DEBUG: Using file match: (^|/)go.mod$ for manager gomod
DEBUG: Using file match: \.gradle(\.kts)?$ for manager gradle
DEBUG: Using file match: (^|\/)gradle\.properties$ for manager gradle
DEBUG: Using file match: (^|\/)gradle\/.+\.toml$ for manager gradle
DEBUG: Using file match: \.versions\.toml$ for manager gradle
DEBUG: Using file match: (^|/)gradle/wrapper/gradle-wrapper.properties$ for manager gradle-wrapper
DEBUG: Using file match: (^|/)requirements\.yaml$ for manager helm-requirements
DEBUG: Using file match: (^|/)values.yaml$ for manager helm-values
DEBUG: Using file match: (^|/)helmfile.yaml$ for manager helmfile
DEBUG: Using file match: (^|/)Chart.yaml$ for manager helmv3
DEBUG: Using file match: ^Formula/[^/]+[.]rb$ for manager homebrew
DEBUG: Using file match: \.html?$ for manager html
DEBUG: Using file match: (^|/)plugins\.(txt|ya?ml)$ for manager jenkins
DEBUG: Using file match: (^|/)jsonnetfile.json$ for manager jsonnet-bundler
DEBUG: Using file match: (^|/)kustomization\.yaml for manager kustomize
DEBUG: Using file match: (^|/)project\.clj$ for manager leiningen
DEBUG: Using file match: \.pom\.xml$ for manager maven
DEBUG: Using file match: (^|/)pom\.xml$ for manager maven
DEBUG: Using file match: (^|/)package.js$ for manager meteor
DEBUG: Using file match: (^|/)mix\.exs$ for manager mix
DEBUG: Using file match: (^|/).node-version$ for manager nodenv
DEBUG: Using file match: (^|/)package.json$ for manager npm
DEBUG: Using file match: \.(?:cs|fs|vb)proj$ for manager nuget
DEBUG: Using file match: \.(?:props|targets)$ for manager nuget
DEBUG: Using file match: (^|\/)dotnet-tools\.json$ for manager nuget
DEBUG: Using file match: (^|\/)global\.json$ for manager nuget
DEBUG: Using file match: (^|/)\.nvmrc$ for manager nvm
DEBUG: Using file match: (^|/)([\w-]*)requirements\.(txt|pip)$ for manager pip_requirements
DEBUG: Using file match: (^|/)setup.py$ for manager pip_setup
DEBUG: Using file match: (^|/)Pipfile$ for manager pipenv
DEBUG: Using file match: (^|/)pyproject\.toml$ for manager poetry
DEBUG: Using file match: (^|/)\.pre-commit-config\.yaml$ for manager pre-commit
DEBUG: Using file match: (^|/)pubspec\.ya?ml$ for manager pub
DEBUG: Using file match: (^|/).python-version$ for manager pyenv
DEBUG: Using file match: (^|/)\.ruby-version$ for manager ruby-version
DEBUG: Using file match: \.sbt$ for manager sbt
DEBUG: Using file match: project/[^/]*.scala$ for manager sbt
DEBUG: Using file match: (^|/)setup\.cfg$ for manager setup-cfg
DEBUG: Using file match: (^|/)Package\.swift for manager swift
DEBUG: Using file match: \.tf$ for manager terraform
DEBUG: Using file match: (^|/)\.terraform-version$ for manager terraform-version
DEBUG: Using file match: (^|/)terragrunt\.hcl$ for manager terragrunt
DEBUG: Using file match: (^|/)\.terragrunt-version$ for manager terragrunt-version
DEBUG: Using file match: ^.travis.yml$ for manager travis
DEBUG: Matched 1 file(s) for manager github-actions: .github/workflows/go.yml
DEBUG: Matched 2 file(s) for manager gomod: a/go.mod, b/go.mod
DEBUG: No multi-line match: go.uber.org/atomic v1.6.0 // indirect
DEBUG: No multi-line match: go.uber.org/multierr v1.5.0 // indirect
DEBUG: No multi-line match: go.uber.org/atomic v1.6.0 // indirect
DEBUG: No multi-line match: go.uber.org/multierr v1.5.0 // indirect
DEBUG: No multi-line match: go.uber.org/zap v1.16.0 // indirect
DEBUG: Found github-actions package files
DEBUG: Found gomod package files
DEBUG: Found 3 package file(s)
INFO: Dependency extraction complete
{
  "baseBranch": "main",
  "stats": {
    "managers": {
      "github-actions": {
        "fileCount": 1,
        "depCount": 2
      },
      "gomod": {
        "fileCount": 2,
        "depCount": 2
      }
    },
    "total": {
      "fileCount": 3,
      "depCount": 4
    }
  }
}
DEBUG: Goproxy error: trying next URL provided with GOPROXY
{
  "err": {
    "name": "HTTPError",
    "code": "ERR_NON_2XX_3XX_RESPONSE",
    "timings": {
      "start": 1638910494479,
      "socket": 1638910494512,
      "lookup": 1638910494512,
      "connect": 1638910494523,
      "secureConnect": 1638910494542,
      "upload": 1638910494542,
      "response": 1638910494566,
      "end": 1638910494567,
      "phases": {
        "wait": 33,
        "dns": 0,
        "tcp": 11,
        "tls": 19,
        "request": 0,
        "firstByte": 24,
        "download": 1,
        "total": 88
      }
    },
    "message": "Response code 410 (Gone)",
    "stack": "HTTPError: Response code 410 (Gone)\n    at Request.<anonymous> (/home/ubuntu/renovateapp/node_modules/got/dist/source/as-promise/index.js:117:42)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)",
    "options": {
      "headers": {
        "user-agent": "Renovate Bot (GitHub App 2740)",
        "accept-encoding": "gzip, deflate, br"
      },
      "url": "https://proxy.golang.org/github.com/eriksw/renovate-peer-go-mod-indirect/a/@v/list",
      "hostType": "go",
      "username": "",
      "password": "",
      "method": "GET",
      "http2": false
    },
    "response": {
      "statusCode": 410,
      "statusMessage": "Gone",
      "body": "not found: module github.com/eriksw/renovate-peer-go-mod-indirect/a: git ls-remote -q origin in /tmp/gopath/pkg/mod/cache/vcs/c91760e9dcba81fb66c7001ea31c860bab6182581792a0d1aa9ffd8b350085b5: exit status 128:\n\tfatal: could not read Username for 'https://github.com': terminal prompts disabled\nConfirm the import path was entered correctly.\nIf this is a private repository, see https://golang.org/doc/faq#git_https for additional information.",
      "headers": {
        "access-control-allow-origin": "*",
        "cache-control": "public, max-age=60",
        "content-type": "text/plain; charset=UTF-8",
        "date": "Tue, 07 Dec 2021 20:54:54 GMT",
        "expires": "Tue, 07 Dec 2021 20:55:54 GMT",
        "x-content-type-options": "nosniff",
        "x-frame-options": "SAMEORIGIN",
        "x-xss-protection": "0",
        "content-length": "441",
        "alt-svc": "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"",
        "connection": "close"
      },
      "httpVersion": "1.1"
    }
  }
}
DEBUG: Found no results from datasource that look like a version (github.com/eriksw/renovate-peer-go-mod-indirect/a)(dependency="github.com/eriksw/renovate-peer-go-mod-indirect/a")
{
  "result": {
    "sourceUrl": "https://github.com/eriksw/renovate-peer-go-mod-indirect",
    "releases": []
  }
}
DEBUG: GitHub 404
{
  "url": "https://api.github.com/repos/eriksw/renovate-peer-go-mod-indirect/git/refs/tags/v0.0.0-00010101000000-000000000000"
}
DEBUG: Error getting tag commit from GitHub repo
{
  "githubRepo": "eriksw/renovate-peer-go-mod-indirect",
  "err": {
    "name": "HTTPError",
    "code": "ERR_NON_2XX_3XX_RESPONSE",
    "timings": {
      "start": 1638910495000,
      "socket": 1638910495000,
      "lookup": 1638910495000,
      "connect": 1638910495010,
      "secureConnect": 1638910495021,
      "upload": 1638910495021,
      "response": 1638910495176,
      "end": 1638910495177,
      "phases": {
        "wait": 0,
        "dns": 0,
        "tcp": 10,
        "tls": 11,
        "request": 0,
        "firstByte": 155,
        "download": 1,
        "total": 177
      }
    },
    "message": "Response code 404 (Not Found)",
    "stack": "HTTPError: Response code 404 (Not Found)\n    at Request.<anonymous> (/home/ubuntu/renovateapp/node_modules/got/dist/source/as-promise/index.js:117:42)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (internal/process/task_queues.js:95:5)",
    "options": {
      "headers": {
        "user-agent": "Renovate Bot (GitHub App 2740)",
        "accept": "application/vnd.github.machine-man-preview+json",
        "authorization": "***********",
        "accept-encoding": "gzip, deflate, br"
      },
      "url": "https://api.github.com/repos/eriksw/renovate-peer-go-mod-indirect/git/refs/tags/v0.0.0-00010101000000-000000000000",
      "hostType": "github-tags",
      "username": "",
      "password": "",
      "method": "GET",
      "http2": false
    },
    "response": {
      "statusCode": 404,
      "statusMessage": "Not Found",
      "body": {
        "message": "Not Found",
        "documentation_url": "https://docs.github.com/rest"
      },
      "headers": {
        "server": "GitHub.com",
        "date": "Tue, 07 Dec 2021 20:54:55 GMT",
        "content-type": "application/json; charset=utf-8",
        "transfer-encoding": "chunked",
        "x-poll-interval": "300",
        "x-github-media-type": "github.v3; param=machine-man-preview; format=json",
        "x-ratelimit-limit": "5000",
        "x-ratelimit-remaining": "4995",
        "x-ratelimit-reset": "1638910672",
        "x-ratelimit-used": "5",
        "x-ratelimit-resource": "core",
        "access-control-expose-headers": "ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset",
        "access-control-allow-origin": "*",
        "strict-transport-security": "max-age=31536000; includeSubdomains; preload",
        "x-frame-options": "deny",
        "x-content-type-options": "nosniff",
        "x-xss-protection": "0",
        "referrer-policy": "origin-when-cross-origin, strict-origin-when-cross-origin",
        "content-security-policy": "default-src 'none'",
        "vary": "Accept-Encoding, Accept, X-Requested-With",
        "content-encoding": "gzip",
        "x-github-request-id": "9608:73D6:1192E85:122D322:61AFCA1F",
        "connection": "close"
      },
      "httpVersion": "1.1"
    }
  }
}
DEBUG: Package releases lookups complete
{
  "baseBranch": "main"
}
DEBUG: branchifyUpgrades
DEBUG: 1 flattened updates found: go.uber.org/zap
DEBUG: Returning 1 branch(es)
DEBUG: Fetching changelog: https://github.com/uber-go/zap (v1.16.0 -> v1.19.1)
DEBUG: config.repoIsOnboarded=true
DEBUG: packageFiles with updates
{
  "config": {
    "github-actions": [
      {
        "packageFile": ".github/workflows/go.yml",
        "deps": [
          {
            "depName": "actions/checkout",
            "commitMessageTopic": "{{{depName}}} action",
            "datasource": "github-tags",
            "versioning": "docker",
            "depType": "action",
            "replaceString": "actions/checkout@v2",
            "autoReplaceStringTemplate": "{{depName}}@{{#if newDigest}}{{newDigest}}{{#if newValue}} # renovate: tag={{newValue}}{{/if}}{{/if}}{{#unless newDigest}}{{newValue}}{{/unless}}",
            "currentValue": "v2",
            "depIndex": 0,
            "updates": [],
            "warnings": [],
            "sourceUrl": "https://github.com/actions/checkout",
            "currentVersion": "v2",
            "fixedVersion": "v2"
          },
          {
            "depName": "actions/setup-go",
            "commitMessageTopic": "{{{depName}}} action",
            "datasource": "github-tags",
            "versioning": "docker",
            "depType": "action",
            "replaceString": "actions/setup-go@v2",
            "autoReplaceStringTemplate": "{{depName}}@{{#if newDigest}}{{newDigest}}{{#if newValue}} # renovate: tag={{newValue}}{{/if}}{{/if}}{{#unless newDigest}}{{newValue}}{{/unless}}",
            "currentValue": "v2",
            "depIndex": 1,
            "updates": [],
            "warnings": [],
            "sourceUrl": "https://github.com/actions/setup-go",
            "currentVersion": "v2",
            "fixedVersion": "v2"
          }
        ]
      }
    ],
    "gomod": [
      {
        "packageFile": "a/go.mod",
        "constraints": {
          "go": "^1.17"
        },
        "deps": [
          {
            "managerData": {
              "lineNumber": 4
            },
            "depName": "go.uber.org/zap",
            "depType": "require",
            "currentValue": "v1.16.0",
            "datasource": "go",
            "depIndex": 0,
            "updates": [
              {
                "bucket": "non-major",
                "newVersion": "v1.19.1",
                "newValue": "v1.19.1",
                "releaseTimestamp": "2021-09-08T21:32:28.000Z",
                "newMajor": 1,
                "newMinor": 19,
                "updateType": "minor",
                "branchName": "renovate/go.uber.org-zap-1.x"
              }
            ],
            "warnings": [],
            "versioning": "semver",
            "sourceUrl": "https://github.com/uber-go/zap",
            "currentVersion": "v1.16.0",
            "isSingleVersion": true,
            "fixedVersion": "v1.16.0"
          }
        ]
      },
      {
        "packageFile": "b/go.mod",
        "constraints": {
          "go": "^1.17"
        },
        "deps": [
          {
            "managerData": {
              "lineNumber": 4
            },
            "depName": "github.com/eriksw/renovate-peer-go-mod-indirect/a",
            "depType": "require",
            "currentValue": "v0.0.0-00010101000000-000000000000",
            "datasource": "go",
            "currentDigest": "000000000000",
            "digestOneAndOnly": true,
            "depIndex": 0,
            "updates": [],
            "warnings": [],
            "versioning": "semver",
            "sourceUrl": "https://github.com/eriksw/renovate-peer-go-mod-indirect",
            "currentVersion": "v0.0.0-00010101000000-000000000000",
            "fixedVersion": "v0.0.0-00010101000000-000000000000"
          }
        ]
      }
    ]
  }
}
DEBUG: processRepo()
DEBUG: Processing 1 branch: renovate/go.uber.org-zap-1.x
DEBUG: Calculating hourly PRs remaining
DEBUG: Retrieving PR list
DEBUG: Retrieved 0 Pull Requests
DEBUG: currentHourStart=2021-12-07T20:00:00.000+00:00
DEBUG: PR hourly limit remaining: 2
DEBUG: Calculating prConcurrentLimit (5)
DEBUG: getBranchPr(renovate/go.uber.org-zap-1.x)
DEBUG: findPr(renovate/go.uber.org-zap-1.x, undefined, open)
DEBUG: findPr(renovate/go.uber.org-zap-1.x, undefined, closed)
DEBUG: 0 PRs are currently open
DEBUG: PR concurrent limit remaining: 5
DEBUG: Calculated maximum PRs remaining this run
{
  "prsRemaining": 2
}
DEBUG: PullRequests limit = 2
DEBUG: Calculating hourly PRs remaining
DEBUG: currentHourStart=2021-12-07T20:00:00.000+00:00
DEBUG: PR hourly limit remaining: 2
DEBUG: Calculating branchConcurrentLimit (5)
DEBUG: 0 already existing branches found:
DEBUG: Branch concurrent limit remaining: 5
DEBUG: Calculated maximum branches remaining this run
{
  "branchesRemaining": 2
}
DEBUG: Branches limit = 2
DEBUG: Setting current branch to main(branch="renovate/go.uber.org-zap-1.x")
DEBUG: latest commit(branch="renovate/go.uber.org-zap-1.x")
{
  "branchName": "main",
  "latestCommitDate": "2021-12-07T12:53:59-08:00"
}
DEBUG: getBranchPr(renovate/go.uber.org-zap-1.x)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: findPr(renovate/go.uber.org-zap-1.x, undefined, open)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: findPr(renovate/go.uber.org-zap-1.x, undefined, closed)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: branchExists=false(branch="renovate/go.uber.org-zap-1.x")
DEBUG: dependencyDashboardCheck=undefined(branch="renovate/go.uber.org-zap-1.x")
DEBUG: recreateClosed is false(branch="renovate/go.uber.org-zap-1.x")
DEBUG: findPr(renovate/go.uber.org-zap-1.x, Update module go.uber.org/zap to v1.19.1, !open)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: prAlreadyExisted=false(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Checking schedule(at any time, null)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: No schedule defined(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Branch needs creating(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Using reuseExistingBranch: false(branch="renovate/go.uber.org-zap-1.x")
DEBUG: manager.getUpdatedPackageFiles() reuseExistinbranch=false(branch="renovate/go.uber.org-zap-1.x")
DEBUG: gomod.updateDependency: v1.19.1(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Updating go.uber.org/zap in a/go.mod(branch="renovate/go.uber.org-zap-1.x")
DEBUG: gomod.updateArtifacts(a/go.mod)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Adding Git authentication for Go Module retrieval for https://api.github.com/ using token auth.(branch="renovate/go.uber.org-zap-1.x")
DEBUG: go get command included(branch="renovate/go.uber.org-zap-1.x")
{
  "cmd": "go",
  "args": "get -d ./..."
}
DEBUG: go mod tidy command included(branch="renovate/go.uber.org-zap-1.x")
{
  "cmd": "go",
  "args": "mod tidy"
}
DEBUG: additional go mod tidy command included(branch="renovate/go.uber.org-zap-1.x")
{
  "cmd": "go",
  "args": "mod tidy"
}
DEBUG: Using docker to execute(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Found version constraint - checking for a compatible image to use(branch="renovate/go.uber.org-zap-1.x")
{
  "depName": "docker.io/renovate/go",
  "scheme": "npm",
  "constraint": "^1.17"
}
DEBUG: getLabels(https://index.docker.io, renovate/go, latest)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Found compatible image version(branch="renovate/go.uber.org-zap-1.x")
{
  "depName": "docker.io/renovate/go",
  "scheme": "npm",
  "constraint": "^1.17",
  "version": "1.17.4"
}
DEBUG: Resolved tag constraint(branch="renovate/go.uber.org-zap-1.x")
{
  "image": "docker.io/renovate/go",
  "tagConstraint": "^1.17",
  "tagVersioning": "npm",
  "tag": "1.17.4"
}
DEBUG: Fetching Docker image: docker.io/renovate/go:1.17.4(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Finished fetching Docker image(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Executing command(branch="renovate/go.uber.org-zap-1.x")
{
  "command": [
    "docker run --rm --name=renovate_go --label=renovate_child -v \"/mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect\":\"/mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -e GOPATH -e GOPROXY -e GOFLAGS -e CGO_ENABLED -e GIT_CONFIG_KEY_0 -e GIT_CONFIG_VALUE_0 -e GIT_CONFIG_COUNT -e GIT_CONFIG_KEY_1 -e GIT_CONFIG_VALUE_1 -w \"/mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect/a\" docker.io/renovate/go:1.17.4 bash -l -c \"go get -d ./... && go mod tidy && go mod tidy\""
  ]
}
DEBUG: exec completed(branch="renovate/go.uber.org-zap-1.x")
{
  "cmd": "docker run --rm --name=renovate_go --label=renovate_child -v \"/mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect\":\"/mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -e GOPATH -e GOPROXY -e GOFLAGS -e CGO_ENABLED -e GIT_CONFIG_KEY_0 -e GIT_CONFIG_VALUE_0 -e GIT_CONFIG_COUNT -e GIT_CONFIG_KEY_1 -e GIT_CONFIG_VALUE_1 -w \"/mnt/renovate/gh/eriksw/renovate-peer-go-mod-indirect/a\" docker.io/renovate/go:1.17.4 bash -l -c \"go get -d ./... && go mod tidy && go mod tidy\"",
  "durationMs": 7327,
  "stdout": "",
  "stderr": "go: downloading go.uber.org/zap v1.19.1\ngo: downloading go.uber.org/atomic v1.7.0\ngo: downloading go.uber.org/multierr v1.6.0\ngo get: upgraded go.uber.org/atomic v1.6.0 => v1.7.0\ngo get: upgraded go.uber.org/multierr v1.5.0 => v1.6.0\ngo: downloading github.com/pkg/errors v0.8.1\ngo: downloading github.com/stretchr/testify v1.7.0\ngo: downloading go.uber.org/goleak v1.1.11-0.20210813005559-691160354723\ngo: downloading github.com/benbjohnson/clock v1.1.0\ngo: downloading gopkg.in/yaml.v2 v2.2.8\ngo: downloading github.com/davecgh/go-spew v1.1.1\ngo: downloading github.com/pmezard/go-difflib v1.0.0\ngo: downloading gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b\n"
}
DEBUG: Returning updated go.sum(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Found updated go.mod after go.sum update(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Updated 1 package files(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Updated 2 lock files(branch="renovate/go.uber.org-zap-1.x")
{
  "updatedArtifacts": [
    "a/go.sum",
    "a/go.mod"
  ]
}
DEBUG: 2 file(s) to commit(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Committing files to branch renovate/go.uber.org-zap-1.x(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Setting git author name(branch="renovate/go.uber.org-zap-1.x")
{
  "gitAuthorName": "Renovate Bot"
}
DEBUG: Setting git author email(branch="renovate/go.uber.org-zap-1.x")
{
  "gitAuthorEmail": "[email protected]"
}
DEBUG: git commit(branch="renovate/go.uber.org-zap-1.x")
{
  "deletedFiles": [],
  "ignoredFiles": [],
  "result": {
    "author": null,
    "branch": "renovate/go.uber.org-zap-1.x",
    "commit": "8654953",
    "root": false,
    "summary": {
      "changes": 2,
      "insertions": 35,
      "deletions": 0
    }
  }
}
DEBUG: git push(branch="renovate/go.uber.org-zap-1.x")
{
  "result": {
    "pushed": [
      {
        "deleted": false,
        "tag": false,
        "branch": true,
        "new": true,
        "alreadyUpdated": false,
        "local": "refs/heads/renovate/go.uber.org-zap-1.x",
        "remote": "refs/heads/renovate/go.uber.org-zap-1.x"
      }
    ],
    "branch": {
      "local": "renovate/go.uber.org-zap-1.x",
      "remote": "renovate/go.uber.org-zap-1.x",
      "remoteName": "origin"
    },
    "ref": {
      "local": "refs/remotes/origin/renovate/go.uber.org-zap-1.x"
    },
    "remoteMessages": {
      "all": [
        "Create a pull request for 'renovate/go.uber.org-zap-1.x' on GitHub by visiting:",
        "https://github.com/eriksw/renovate-peer-go-mod-indirect/pull/new/renovate/go.uber.org-zap-1.x"
      ],
      "pullRequestUrl": "https://github.com/eriksw/renovate-peer-go-mod-indirect/pull/new/renovate/go.uber.org-zap-1.x"
    }
  }
}
INFO: Branch created(branch="renovate/go.uber.org-zap-1.x")
{
  "commitSha": "8654953"
}
DEBUG: Ensuring PR(branch="renovate/go.uber.org-zap-1.x")
DEBUG: There are 0 errors and 0 warnings(branch="renovate/go.uber.org-zap-1.x")
DEBUG: getBranchPr(renovate/go.uber.org-zap-1.x)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: findPr(renovate/go.uber.org-zap-1.x, undefined, open)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: findPr(renovate/go.uber.org-zap-1.x, undefined, closed)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Creating PR(branch="renovate/go.uber.org-zap-1.x")
{
  "prTitle": "Update module go.uber.org/zap to v1.19.1"
}
DEBUG: Creating PR(branch="renovate/go.uber.org-zap-1.x")
{
  "title": "Update module go.uber.org/zap to v1.19.1",
  "head": "eriksw:renovate/go.uber.org-zap-1.x",
  "base": "main",
  "draft": false
}
DEBUG: PR created(branch="renovate/go.uber.org-zap-1.x")
{
  "pr": 1,
  "draft": false
}
DEBUG: Adding labels '' to #1(branch="renovate/go.uber.org-zap-1.x")
INFO: PR created(branch="renovate/go.uber.org-zap-1.x")
{
  "pr": 1,
  "prTitle": "Update module go.uber.org/zap to v1.19.1"
}
DEBUG: addAssigneesReviewers(pr=1)(branch="renovate/go.uber.org-zap-1.x")
DEBUG: Created Pull Request #1(branch="renovate/go.uber.org-zap-1.x")
DEBUG: PR is not configured for automerge(branch="renovate/go.uber.org-zap-1.x")
DEBUG: getBranchPr(renovate/go.uber.org-zap-1.x)
DEBUG: findPr(renovate/go.uber.org-zap-1.x, undefined, open)
DEBUG: Found PR #1
DEBUG: Returning from graphql open PR list
DEBUG: Ensuring Dependency Dashboard
DEBUG: ensureIssue(Dependency Dashboard)
INFO: Issue created
DEBUG: Removing any stale branches
DEBUG: config.repoIsOnboarded=true
DEBUG: Branch lists
{
  "branchList": [
    "renovate/go.uber.org-zap-1.x"
  ],
  "renovateBranches": [
    "renovate/go.uber.org-zap-1.x"
  ]
}
DEBUG: remainingBranches=
DEBUG: No branches to clean up
DEBUG: Retrieving issueList
DEBUG: Retrieved 1 issues
DEBUG: Repository timing splits (milliseconds)
{
  "splits": {
    "init": 3531,
    "extract": 918,
    "lookup": 902,
    "update": 12877
  },
  "total": 19493
}
DEBUG: http statistics
{
  "urls": {
    "https://api.github.com/graphql (POST)": 5,
    "https://api.github.com/repos/eriksw/renovate-peer-go-mod-indirect/issues (POST)": 1,
    "https://api.github.com/repos/eriksw/renovate-peer-go-mod-indirect/pulls (GET)": 1,
    "https://api.github.com/repos/eriksw/renovate-peer-go-mod-indirect/pulls (POST)": 1,
    "https://api.github.com/repos/eriksw/renovate-peer-go-mod-indirect/releases (GET)": 1,
    "https://api.github.com/repos/eriksw/renovate-peer-go-mod-indirect/tags (GET)": 1,
    "https://api.github.com/repos/whitesource/merge-confidence/contents/beta.json (GET)": 1
  },
  "hostStats": {
    "api.github.com": {
      "requestCount": 11,
      "requestAvgMs": 343,
      "queueAvgMs": 0
    }
  },
  "totalRequests": 11
}
INFO: Repository finished
{
  "durationMs": 19493
}

If you have any ideas on how this should be implemented, please tell us here.

Renovate needs to be aware of relative path replace directives amongst go modules and ensure update/tidy actions propagate following those relationships.

Is this a feature you are interested in implementing yourself?

No

The text was updated successfully, but these errors were encountered:

mowies · 2021-12-16T07:25:46Z

We have the same issue in https://github.com/keptn/keptn :(
Renovate PR: keptn/keptn#6377
Update PR to fix the dependencies: keptn/keptn#6381

oschwald · 2022-01-10T16:47:03Z

We would also like to see this feature. It looks like Renovate used to update indirect dependencies by default, but that was removed in #4650. I don't fully understand the reasoning for why this was done, but it seems like updating indirect dependencies should be an option at the very least.

woohgit · 2022-03-20T03:59:53Z

With golang 1.17 and above the indirect dependencies should be updated too, otherwise it'll fail with:

12:51:56  === Errors
12:51:56  go: updates to go.mod needed; to update it:
12:51:56  	go mod tidy

I hope someone will pick this up and make the necessary changes to make this happen.

rarkins · 2022-03-22T04:45:56Z

@woohgit are you able to create a minimal reproduction for that type of failure?

woohgit · 2022-03-22T05:04:13Z

@rarkins

Yes probably I'll need to create a module (shared) which uses aws-go-sdk for example. Then create 2 more modules that uses the shared module and for go 1.17+ you have to have the indirect deps for the shared for all other modules too, even if they don't directly use the aws-sdk-go.

If you wan't I can create a minimal app with the proper setup to reproduce the issue.

rarkins · 2022-03-22T05:20:08Z

@woohgit thanks, that would be great. I would really like to get this issue solved, and this might be the root cause

woohgit · 2022-03-22T12:52:42Z

@rarkins On it. I'm waiting for the first dependency PR to be created by renovatebot and I can show it to you then.

woohgit · 2022-03-23T00:00:59Z

@rarkins And it happened.

There you go: https://github.com/woohgit/renovate-12999/pull/3/files

This repo consists of a shared module (which depends on aws-sdk-go) and 2 other consumer modules which depend on the shared. For go 1.17+ the go.sum and go.mod has to contain indirect entries for the shared module but that's not updated with the PR. So if you want to build the consumer1 or consumer2 using this PR you'll get the error:

❯ go build .
go: updates to go.mod needed; to update it:
	go mod tidy
❯ pwd
/home/wooh/repos/renovate-12999/consumer1

It think at least it should be a config option to update it everywhere (and run go mod tidy in all relevant directories) and maybe set it to true when the group:monorepo is enabled and the project is golang.

woohgit · 2022-03-31T00:41:24Z

@rarkins Any comment / update / more help needed etc?

rarkins · 2022-04-02T05:16:51Z

@woohgit thanks for your updates to the issue. Previously
I think people had been reporting "I need to run go mod tidy in the same directory one more time". In your case though it sounds like you'd need Renovate to build up a kind of "dependency tree" of each module to know which depends on which and then run "go mod tidy" on each directory which depends on the one which just got updated?

woohgit · 2022-04-04T04:16:49Z

Actually everyone is reporting the same as me:

Look at the original description of this issue: eriksw/renovate-peer-go-mod-indirect#1

This is exactly the same setup as I presented to you.

It's not that we need to run go mod tidy multiple times in the same directory. It's that the //indirect dependencies are not updated, but they should be because from go 1.17+ you need to keep them sync too.

rarkins · 2022-04-04T04:25:21Z

Actually we have a few Go tidying issues which seem to be bugs, some pre-dating 1.17. I thought we may have found the root cause but it seems not.

As discussed above, this will require Renovate to build a dependency tree of related go.mod files, which wasn't necessary before.

Classification as a feature request instead of bug was correct in this case.

woohgit · 2022-04-04T04:26:40Z

@rarkins It's fine! I know it's getting a bit more complex with 1.17. I hope you guys can figure it out quickly how to do it :)

rarkins · 2022-04-04T04:26:42Z

Whoever implements this will need to decide whether the go.mod dependency tree should be built during the extract phase (extractAllPackageFiles) or instead can be determined dynamically during this artifacts updating stage.

viceice · 2022-04-04T04:37:25Z

there is currently ongoing work for nuget, where we need to detect dependent projects. i think we can refactor and reuse some of the code from that PR so solve this issue afterwards.

katexochen · 2022-10-20T07:24:20Z

A simple solution for this issue might be to provide an option that just runs go mod tidy for all modules in the repository, instead of evaluating the replace statements and creating a dependency tree. Currently, the postUpdateOption gomodTidy seem to only update modules that were changed by renovate.

gnuletik · 2023-02-22T10:04:42Z

I'm open to work on a fix for this!

The easiest way I see to implement this is to run go mod tidy inside each go modules.

Does that would save everyone use case?

I'm currently running the following command locally to patch renovate PRs:

for i in $(find . -name go.mod); do cd $(dirname "$i") && go mod tidy && cd -; done && git commit -am "chore: go mod tidy in every go modules" && git push

However, it is a pain to do that inside every PR.

I was thinking about adding an option to renovate to implement a similar behavior, for example a postUpdateOptions: ["goModTidyAll"]. But I can't find where I could implement it.

go mod tidy is currently runs individually in each module here:

renovate/lib/modules/manager/gomod/artifacts.ts

Lines 307 to 337 in 9669ba3

    
           const isGoModTidyRequired = 
        
             !mustSkipGoModTidy && 
        
             (config.postUpdateOptions?.includes('gomodTidy') || 
        
               config.postUpdateOptions?.includes('gomodTidy1.17') || 
        
               config.postUpdateOptions?.includes('gomodTidyE') || 
        
               (config.updateType === 'major' && isImportPathUpdateRequired)); 
        
           if (isGoModTidyRequired) { 
        
             args = 'mod tidy' + tidyOpts; 
        
             logger.debug('go mod tidy command included'); 
        
             execCommands.push(`${cmd} ${args}`); 
        
           } 
        
           if (useVendor) { 
        
             args = 'mod vendor'; 
        
             logger.debug('go mod tidy command included'); 
        
             execCommands.push(`${cmd} ${args}`); 
        
             if (isGoModTidyRequired) { 
        
               args = 'mod tidy' + tidyOpts; 
        
               logger.debug('go mod tidy command included'); 
        
               execCommands.push(`${cmd} ${args}`); 
        
             } 
        
           } 
        
           // We tidy one more time as a solution for #6795 
        
           if (isGoModTidyRequired) { 
        
             args = 'mod tidy' + tidyOpts; 
        
             logger.debug('go mod tidy command included'); 
        
             execCommands.push(`${cmd} ${args}`); 
        
           } 
        
           await exec(execCommands, execOptions);

Is there a package where we could implement "repository wide" operations?

rarkins · 2023-02-22T10:13:11Z

goModTidyAll would be a good way to opt into it.

Check out https://github.com/renovatebot/renovate/blob/main/lib/modules/manager/nuget/package-tree.ts#L16 for an example of how nuget does it

eriksw · 2023-03-10T17:37:38Z

I would love to see a goModTidyAll option. That would indeed solve my use case!

eriksw added priority-5-triage status:requirements Full requirements are not yet known, so implementation should not be started type:feature Feature (new functionality) labels Dec 7, 2021

HonkingGoose added manager:gomod Go Modules reproduction:provided labels Dec 8, 2021

This comment was marked as off-topic.

Sign in to view

This comment was marked as resolved.

Sign in to view

rarkins mentioned this issue Dec 7, 2022

Run go mod tidy in subdirectories #19270

Closed

mpkorstanje mentioned this issue Jan 8, 2023

Setup Renovate with default Configs cucumber/godog#531

Merged

cevich mentioned this issue Mar 16, 2023

bump golang.org/x/net to v0.8.0 containers/skopeo#1938

Merged

cevich mentioned this issue Jun 5, 2023

build(deps): bump github.com/sigstore/rekor from 1.1.2-0.20230508234306-ad288b385a44 to 1.2.0 containers/podman#18709

Closed

This comment was marked as off-topic.

Sign in to view

mtrmac mentioned this issue Jun 7, 2023

Enable gomod ind. deps for vulnerabilities. containers/automation#143

Merged

HonkingGoose mentioned this issue Jun 19, 2023

chore(renovate): enable PRs for all non-kubernetes dependencies crossplane/crossplane#4133

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Indirect dependencies in dependent go modules need to be updated #12999

Indirect dependencies in dependent go modules need to be updated #12999

eriksw commented Dec 7, 2021

mowies commented Dec 16, 2021 •

edited

Loading

oschwald commented Jan 10, 2022 •

edited

Loading

woohgit commented Mar 20, 2022

rarkins commented Mar 22, 2022

woohgit commented Mar 22, 2022

rarkins commented Mar 22, 2022

woohgit commented Mar 22, 2022

woohgit commented Mar 23, 2022 •

edited

Loading

woohgit commented Mar 31, 2022

rarkins commented Apr 2, 2022

woohgit commented Apr 4, 2022

rarkins commented Apr 4, 2022

woohgit commented Apr 4, 2022

rarkins commented Apr 4, 2022

viceice commented Apr 4, 2022

This comment was marked as off-topic.

katexochen commented Oct 20, 2022

This comment was marked as resolved.

gnuletik commented Feb 22, 2023

rarkins commented Feb 22, 2023

eriksw commented Mar 10, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

Indirect dependencies in dependent go modules need to be updated #12999

Indirect dependencies in dependent go modules need to be updated #12999

Comments

eriksw commented Dec 7, 2021

What would you like Renovate to be able to do?

If you have any ideas on how this should be implemented, please tell us here.

Is this a feature you are interested in implementing yourself?

mowies commented Dec 16, 2021 • edited Loading

oschwald commented Jan 10, 2022 • edited Loading

woohgit commented Mar 20, 2022

rarkins commented Mar 22, 2022

woohgit commented Mar 22, 2022

rarkins commented Mar 22, 2022

woohgit commented Mar 22, 2022

woohgit commented Mar 23, 2022 • edited Loading

woohgit commented Mar 31, 2022

rarkins commented Apr 2, 2022

woohgit commented Apr 4, 2022

rarkins commented Apr 4, 2022

woohgit commented Apr 4, 2022

rarkins commented Apr 4, 2022

viceice commented Apr 4, 2022

This comment was marked as off-topic.

katexochen commented Oct 20, 2022

This comment was marked as resolved.

gnuletik commented Feb 22, 2023

rarkins commented Feb 22, 2023

eriksw commented Mar 10, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

mowies commented Dec 16, 2021 •

edited

Loading

oschwald commented Jan 10, 2022 •

edited

Loading

woohgit commented Mar 23, 2022 •

edited

Loading