[FEATURE] Implement progress bar in stream output during image pull

[D3vil0p3r]

Currently the stream output during pull operation seems to appear like a single element stream in JSON format, for example:

{'stream': 'Trying to pull docker.io/athenaos/base:latest...\n'}
{'stream': 'Getting image source signatures\n'}
{'stream': 'Copying blob sha256:30babffc8f090f7c78c263b9a1b683b34ca5f73da74911ffe942d4d8225dca57\n'}
{'stream': 'Copying config sha256:420f00cb47debe421a8b70b1a2e9ccf16a942cce2f70d1724eff6be764e95fea\n'}
{'stream': 'Writing manifest to image destination\n'}
{'images': ['420f00cb47debe421a8b70b1a2e9ccf16a942cce2f70d1724eff6be764e95fea'], 'id': '420f00cb47debe421a8b70b1a2e9ccf16a942cce2f70d1724eff6be764e95fea'}

Is it possible to implement a more complete stream output and progress bar inside the stream output by default during pull operation as occurs in docker py like:

{'status': 'Pulling from athenaos/base', 'id': 'latest'}
{'status': 'Pulling fs layer', 'progressDetail': {}, 'id': '30babffc8f09'}
{'status': 'Downloading', 'progressDetail': {'current': 527607, 'total': 152137719}, 'progress': '[>                                    
]  527.6kB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Downloading', 'progressDetail': {'current': 1055991, 'total': 152137719}, 'progress': '[>                                   
]  1.056MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Downloading', 'progressDetail': {'current': 1596663, 'total': 152137719}, 'progress': '[>                                   
]  1.597MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Downloading', 'progressDetail': {'current': 2137335, 'total': 152137719}, 'progress': '[>                                   
]  2.137MB/152.1MB', 'id': '30babffc8f09'}
...
...
{'status': 'Downloading', 'progressDetail': {'current': 103782662, 'total': 152137719}, 'progress': 
'[==================================>                ]  103.8MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Downloading', 'progressDetail': {'current': 104319238, 'total': 152137719}, 'progress': 
'[==================================>                ]  104.3MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Downloading', 'progressDetail': {'current': 104851718, 'total': 152137719}, 'progress': 
'[==================================>                ]  104.9MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Downloading', 'progressDetail': {'current': 105392390, 'total': 152137719}, 'progress': 
'[==================================>                ]  105.4MB/152.1MB', 'id': '30babffc8f09'}
...
...
{'status': 'Verifying Checksum', 'progressDetail': {}, 'id': '30babffc8f09'}
{'status': 'Download complete', 'progressDetail': {}, 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 557056, 'total': 152137719}, 'progress': '[>                                     
]  557.1kB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 2228224, 'total': 152137719}, 'progress': '[>                                    
]  2.228MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 3899392, 'total': 152137719}, 'progress': '[=>                                   
]  3.899MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 5570560, 'total': 152137719}, 'progress': '[=>                                   
]  5.571MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 7241728, 'total': 152137719}, 'progress': '[==>                                  
]  7.242MB/152.1MB', 'id': '30babffc8f09'}
...
...
{'status': 'Extracting', 'progressDetail': {'current': 107511808, 'total': 152137719}, 'progress': 
'[===================================>               ]  107.5MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 108068864, 'total': 152137719}, 'progress': 
'[===================================>               ]  108.1MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 109182976, 'total': 152137719}, 'progress': 
'[===================================>               ]  109.2MB/152.1MB', 'id': '30babffc8f09'}
{'status': 'Extracting', 'progressDetail': {'current': 110297088, 'total': 152137719}, 'progress': 
'[====================================>              ]  110.3MB/152.1MB', 'id': '30babffc8f09'}
...
...
{'status': 'Pull complete', 'progressDetail': {}, 'id': '30babffc8f09'}
{'status': 'Digest: sha256:694ae0bf54f96475b2dec18b1dd2c313405a7e73cda8467c71e1a222df15ffc6'}
{'status': 'Status: Downloaded newer image for athenaos/base:latest'}

I am aware currently pull function has progress_bar parameter but it just allows to show an animated progress bar. I think that having so detailed stream output as above as occurs in docker py could be useful.

You can test it by this simple Python script:

import podman
from podman import PodmanClient

import docker
from docker import DockerClient

import json

repository = "docker.io/ohmyzsh/zsh"

## Podman use case
client = podman.from_env()

resp = client.images.pull(repository, stream=True)
for line in resp:
    print(line)

## Docker use case
client = docker.from_env()

resp = client.api.pull(repository, stream=True, decode=True)
for line in resp:
    print(line)

[D3vil0p3r]

This is an issue that is caused by podman API, indeed by running podman API server:

podman system service tcp:localhost:8080 --time=0

and run:

curl -XPOST --unix-socket /run/user/1000/podman/podman.sock -H content-type:application/json http://d/v4.0.0/libpod/images/pull?reference=docker.io/athenaos/base

and we get the podman output shown in the first post:

{"stream":"Trying to pull docker.io/athenaos/base:latest...\n"}
{"stream":"Getting image source signatures\n"}
{"stream":"Copying blob sha256:30babffc8f090f7c78c263b9a1b683b34ca5f73da74911ffe942d4d8225dca57\n"}

I think it should be adapted to docker pull API by showing:

• status
• progressDetails
  • current
  • total
• progress
• id
  instead of stream. It can be useful for those applications that want to manage progress status data (i.e., in Exegol).

Tracked in containers/podman#24887