Source downloader docker-http - Cannot pull from docker registry

[jnfrati]

Hello! Sorry if this is not the proper way to post an issue here or if this is a bad config from my end.

I've been trying to setup a simple nodejs image with some nextjs app to test out and see what could I do, but I cannot find a way to use the docker-http source downlaoder, I've tried with public and private registry and I keep getting the same error:

2024/01/04 18:09:27 registry.ping url=https://registry.digitalocean.com/v2/
2024/01/04 18:09:29 registry.manifest.get url=https://registry.digitalocean.com/v2/node/manifests/latest repository=node reference=latest
ERROR 2024/01/04 18:09:30 Get "https://registry.digitalocean.com/v2/node/manifests/latest": http: non-successful response (status=401 body="{\"errors\":[{\"code\":\"UNAUTHORIZED\",\"message\":\"authentication required\",\"detail\":[{\"Type\":\"repository\",\"Class\":\"\",\"Name\":\"node\",\"Action\":\"pull\"}]}]}\n")

And this is the yaml I'm trying to use:

image:
  distribution: nextjs_example
  description: Nextjs Example
  expiry: 30d
  variant: default
  architecture: amd64

source:
  downloader: docker-http
  url: node

targets:
  lxc:
    create_message: |-
      You just created an {{ image.description }} container.

packages:
  manager: apt
  update: true
  cleanup: true

files:
  - path: /app
    generator: copy
    source: ./test-nextjs/

actions:
- trigger: post-files
  action: |-
    #!/bin/sh

    npm i

    npm run dev

mappings:
  architecture_map: debian

[stgraber]

@monstermunchkin any idea? I've not used the Docker downloader in a long time ;)

[monstermunchkin]

I'd have to investigate this. IIRC I had a similar issue which was caused by either the image not existing or it being in a private repo.

[jnfrati]

Thanks! Is there anything else I can do from my side to help with this?

[Miguel-Rodrigues]

Hello. Hope that you're fine.

I'm having the same issue, trying to connect to dockerhub directly. I have set the environment variables on the system, at /etc/profile, like this:

export DOCKER_REGISTRY_BASE_USER=my_user_name
export DOCKER_REGISTRY_BASE_PASS=personal_access_token

After that I logout and login and try to run the following command, Resulting in an unauthorized status code.

distrobuilder build-lxc ./alpine.yaml ./output/ --cache-dir ./cache/

INFO[2024-06-27T21:23:22Z] Downloading source
2024/06/27 21:23:22 registry.ping url=https://registry-1.docker.io/v2/
2024/06/27 21:23:23 registry.manifest.get url=https://registry-1.docker.io/v2/alpine/manifests/latest repository=alpine reference=latest
ERROR 2024/06/27 21:23:24 Get "https://registry-1.docker.io/v2/alpine/manifests/latest": http: non-successful response (status=401 body="{\"errors\":[{\"code\":\"UNAUTHORIZED\",\"message\":\"authentication required\",\"detail\":[{\"Type\":\"repository\",\"Class\":\"\",\"Name\":\"alpine\",\"Action\":\"pull\"}]}]}\n")

The yaml source code:

image:
  distribution: alpine_linux_example
  description: Alpine Linux Example
  release: latest
  variant: default

source:
  downloader: docker-http
  url: alpine:latest

targets:
  lxc:
    create_message: |-
      Container "{{ image.description }}" created - {{ source.url }}.

packages:
  manager: apk
  update: true

actions:
 - trigger: post-files
   action: |-
     #!/bin/sh
     uname -a

mappings:
  architecture_map: alpinelinux

Am I missing something? Best regards.

[Miguel-Rodrigues]

Found this documentation about how docker and oci registries challenge the authentication:
https://github.com/distribution/distribution/blob/docker/1.13/docs/spec/auth/token.md

In the lines bellow I try to pull the ubuntu/nginx image manifest from dockerhub

I manually solved the challenge using my credentials by calling:
GET https://registry.hub.docker.com/v2/ubuntu/nginx/manifests/latest >

Creates the token based on the provided credentials on the basic authentication header.

GET https://auth.docker.io/token?service=registry.docker.io&scope=repository:ubuntu/nginx:pull
Pulls the image manifest from the registry using the bearer token on the previous json

This may be a problem how the docker-registry-client dependency from docker-companion is mishandling the challenge.

[geaaru]

Hi, reporting my comment added in another issue that could be related to this too.
#822 (comment)

I forked docker-companion to review implementation and using schema v2 (the schema v1 is been removed from Docker Hub i think). We can later remove it and using an alternative to docker-registry-client now archived as second step.

[geaaru]

Here the main change:
geaaru/docker-companion@fbf9ad4

Now distrobuilder works correctly with Macaroni OS docker image, i think also with other.
@jnfrati my suggestion is using the full docker image name (user/repo:tag) and avoid the alias because i don't remember if it's correctly managed by heroku/docker-registry-client package.