[QUESTION] How to avoid explicit call to subcommands command #1074

Andrew-Sheridan · 2020-06-13T16:17:56Z

Andrew-Sheridan
Jun 13, 2020

First check

I used the GitHub search to find a similar issue and didn't find it.
I searched the Typer documentation, with the integrated search.
I already searched in Google "How to X in Typer" and didn't find any information.
I already searched in Google "How to X in Click" and didn't find any information.

Description

I'm not sure how to articulate this question without an example. Because of that I had some trouble googling around.

I have two installed packages, foo and bar.

bar can be used independently of foo. Ex: bar PATH # bar does stuff with PATH
foo accepts bar as a plugin. Now foo can use the logic from bar. Ex: foo bar PATH. # foo does bar stuff with PATH
foo can also do other stuff that is built into itself. Ex: foo fizz PATH. # foo does fizz stuff with PATH
I have this working with argparse but can't get foo bar PATH working quite right with typer

So here goes with an example.

In one file bar/src/bar/cli.py I have

import typer

app = typer.Typer()


@app.command()
def main(path: str = typer.Argument(...)):
    typer.echo(f"Doing bar things to {path}")

If I call bar --help I get

Usage: bar [OPTIONS] PATH

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

which is great.

This is installed with this setup.py from bar/setup.py

"""Setup."""

from setuptools import find_packages
from setuptools import setup

PACKAGE_NAME = "bar"

setup(
    name=PACKAGE_NAME,
    version="0.0.1",
    package_dir={"": "src"},
    packages=find_packages(where="src"),
    zip_safe=False,
    include_package_data=True,
    python_requires=">=3.7",
    entry_points={
        "console_scripts": [f"{PACKAGE_NAME} = {PACKAGE_NAME}.cli:app"],
        "foo.cli_plugins": [f"{PACKAGE_NAME} = {PACKAGE_NAME}.cli"],
    },
    install_requires=["typer"],
)

there is also an empty __init__.py in bar/src/bar/

In another file foo/src/foo/cli.py I have

import typer
import pkg_resources

app = typer.Typer()

# real program has lots of plugins
CLI_PLUGINS = {
            entry_point.name: entry_point.load()
            for entry_point in pkg_resources.iter_entry_points("foo.cli_plugins")
        }

for name, entry_point in CLI_PLUGINS.items():
    app.add_typer(entry_point.app, name=name)


@app.command()
def fizz(path: str = typer.Argument(...)):
    typer.echo(f"Fizzy {path}")

This is installed with this setup.py from foo/setup.py

"""Setup."""

from pathlib import Path

from setuptools import find_packages
from setuptools import setup

PACKAGE_NAME = "foo"

setup(
    name=PACKAGE_NAME,
    version="0.0.1",
    package_dir={"": "src"},
    packages=find_packages(where="src"),
    zip_safe=False,
    include_package_data=True,
    python_requires=">=3.7",
    entry_points={
        "console_scripts": [f"{PACKAGE_NAME} = {PACKAGE_NAME}.cli:app"],
    },
    install_requires=["typer"],
)

there is also an empty __init__.py in foo/src/foo/

When I call foo --help I get

Usage: ex-primary [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Commands:
  fizz
  bar

which is also great. This is exactly what I want.

However..

When I call foo bar --help I get

Usage: foo bar [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  main

Which is not what I want. I don't want to have to call main out by name here.

Question

Is there a way to not have main as a command here? I want to be able to call foo bar PATH instead of foo bar main PATH.

What I want for for bar --help is

Usage: foo bar [OPTIONS] PATH

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Additional context

I have tried adding a callback to the add_typer and it almost does what I want.
Ex:
changing

app.add_typer(entry_point.app, name=name)

to

app.add_typer(entry_point.app, name=name, callback=entry_point.main)

Doing this adds the argument of bar to the usage of foo bar --help (yay) but it still leaves the COMMAND [ARGS] and the Commands section. Also I don't think this is the right way to go about this.

I feel like there is a nice way to get this working and I just cannot see it.

I looked at the documentation (which is beautiful) and the example with subcommands doesn't quite match this use case. There the subcommands are designed to be called explicitly.

If there is any more information I can provide please let me know.

Thank you for your great work on typer! Cheers

Answered by patrick91

Dec 3, 2024

This was fixed in Typer 0.15.0, now if you don't pass name to add_typer the commands will merged into the app!

See the docs here: https://typer.tiangolo.com/tutorial/one-file-per-command/

View full answer

Andrew-Sheridan · 2020-06-14T00:55:26Z

Andrew-Sheridan
Jun 14, 2020
Author

I played around with this for a while and came up with a potential solution:

Change this:

for name, entry_point in CLI_PLUGINS.items():
    app.add_typer(entry_point.app, name=name)

To this:

for name, entry_point in CLI_PLUGINS.items():
    app.registered_commands.append(CommandInfo(name=name, callback=entry_point.main))

Which appears to work.

Is there a better way to do this?

Are there any unintended consequences of this approach? I don't see any documentation for registered_commands at https://typer.tiangolo.com/, and there are no docstrings in the source.

Thanks again for your time, apologies for the wall of text.

0 replies

tiangolo · 2020-06-17T17:59:57Z

tiangolo
Jun 17, 2020
Maintainer

Hey @Andrew-Sheridan , here's the thing, in the first case, when you do @app.command(), as there's only one function and nothing else, it is used as a single command, not a group of commands.

On the second case, as it is included as a sub-command of the main application and it is a complete Typer application, then it can no longer assume that it's just a simple command directly and will treat it as a group, no matter if it has one or several subcommands.

In this case, I think you can make it explicitly a group from the beginning, by not using @app.command() but @app.callback().

The callback is the main function of a group (instead of a command), but it is taken as the main function for that group that is always run no matter if there are sub-commands.

That way, it would run directly when you call the first version alone, and should work fine when you use app.add_typer() (without creating the second sub-command).

Maybe the docs here can help: https://typer.tiangolo.com/tutorial/commands/callback/

0 replies

Andrew-Sheridan · 2020-06-17T23:10:21Z

Andrew-Sheridan
Jun 17, 2020
Author

@tiangolo Thank you answering!

Your explanation seems to make sense but I can't actually seem to implement it correctly. I've gone through the docs and the examples but something just must not be clicking.

If I set the main in bar.py (the plugin) as a callback then bar is unusable.

bar.py

import typer

app = typer.Typer()

@app.callback()
def main(path: str = typer.Argument(...)):
    typer.echo(f"Doing bar things to {path}")

help

$ bar --help
Usage: bar [OPTIONS] PATH COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help

usage

$ bar some_path
Usage: bar [OPTIONS] PATH COMMAND [ARGS]...
Try 'bar --help' for help.

Error: Missing command.

You can see it asking for a command when there isn't one. The same issue occurs when calling it with foo bar some_path.

Is there an aspect to this I'm missing or should I fall back to the undocumented app.registered_commands which appears to work?

Thank you again

0 replies

cataerogong · 2020-07-11T11:49:35Z

cataerogong
Jul 11, 2020

@Andrew-Sheridan
I have a solution, maybe not so beautiful 😺

@tiangolo
Maybe the behavior of Typer.add_typer() method could change to this: when the sub typer has only 1 command, add it as a command, not a sub-commands group.

code

sub.py

import typer

app = typer.Typer()

@app.command()
def main(arg: str):
    typer.echo(f'sub(arg={arg})')

if __name__ == "__main__":
    app()

sub2.py

import typer

app = typer.Typer()

@app.command()
def cmd1(arg: str):
    typer.echo(f'sub2.cmd1(arg={arg})')

@app.command()
def cmd2(arg: str):
    typer.echo(f'sub2.cmd2(arg={arg})')

if __name__ == "__main__":
    app()

main.py

import typer
import sub, sub2

app = typer.Typer()

@app.command()
def main(arg: str):
    typer.echo(f'main(arg={arg})')

if __name__ == "__main__":
    if len(sub.app.registered_commands) == 1:
        app.command('sub')(sub.app.registered_commands[0].callback)
    else:
        app.add_typer(sub.app, name='sub')
    if len(sub2.app.registered_commands) == 1:
        app.command('sub2')(sub2.app.registered_commands[0].callback)
    else:
        app.add_typer(sub2.app, name='sub2')
    app()

run

main.py

$ python main.py --help
Usage: main.py [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Commands:
  main
  sub
  sub2

$ python main.py main --help
Usage: main.py main [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py sub --help
Usage: main.py sub [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py sub2 --help
Usage: main.py sub2 [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  cmd1
  cmd2

$ python main.py sub2 cmd1 --help
Usage: main.py sub2 cmd1 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py sub2 cmd2 --help
Usage: main.py sub2 cmd2 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python main.py main sth
main(arg=sth)

$ python main.py sub sth
sub(arg=sth)

$ python main.py sub2 cmd1 sth
sub2.cmd1(arg=sth)

$ python main.py sub2 cmd2 sth
sub2.cmd2(arg=sth)

sub.py

$ python sub.py --help
Usage: sub.py [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

$ python sub.py sth
sub(arg=sth)

sub2.py

$ python sub2.py --help
Usage: sub2.py [OPTIONS] COMMAND [ARGS]...

Options:
  --install-completion [bash|zsh|fish|powershell|pwsh]
                                  Install completion for the specified shell.
  --show-completion [bash|zsh|fish|powershell|pwsh]
                                  Show completion for the specified shell, to
                                  copy it or customize the installation.

  --help                          Show this message and exit.

Commands:
  cmd1
  cmd2

$ python sub2.py cmd1 --help
Usage: sub2.py cmd1 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python sub2.py cmd2 --help
Usage: sub2.py cmd2 [OPTIONS] ARG

Arguments:
  ARG  [required]

Options:
  --help  Show this message and exit.

$ python sub2.py cmd1 sth
sub2.cmd1(arg=sth)

$ python sub2.py cmd2 sth
sub2.cmd2(arg=sth)

0 replies

vlcinsky · 2020-07-12T16:29:03Z

vlcinsky
Jul 12, 2020

Well done @cataerogong

I had similar problem but in my case I knew, that once I want to add command group, another time single subcommand. For the later the callable can be simply registered via app.command(name="whatever")(callable).

user.py

import typer

app = typer.Typer()


@app.command()
def create(name: str):
    """Create user
    """
    typer.echo(f"Creating user: {name}")


@app.command()
def delete(name: str):
    """Delete user
    """
    typer.echo(f"Deleting user: {name}")


if __name__ == "__main__":
    app()

hello.py

import typer

app = typer.Typer()


@app.command(name="hello")
def action(name: str):
    """Say Hello to NAME.
    """
    typer.echo(f"Hello {name}")


if __name__ == "__main__":
    app()

main.py

import typer

import hello
import user

app = typer.Typer()


app.command(name="hello")(hello.action)
app.add_typer(user.app, name="user")


if __name__ == "__main__":
    app()

At Typer is not modifying the callable hello, it can be simply registered with any other Typer app instance.

Implicit expectations

As seen in this issue, there are more users having the same expectation, that app.add_typer on app with just single subcommand will add just one level of command without creating extra command group.

It would be nice, if Typer would work this (implicitly expected) way. So far it was the only case when Typer it surprised me. Great work @tiangolo.

0 replies

andrewasheridan · 2020-07-14T11:03:10Z

andrewasheridan
Jul 14, 2020

Thank you both for your comments and suggestions! I'll be reviewing these in more depth over the weekend.

In the mean time, I'm curious if anyone could comment on my note about using app.registered_commands.append?

0 replies

pmsoltani · 2020-08-31T21:33:47Z

pmsoltani
Aug 31, 2020

Running on Typer v0.3.2, I tried all of the suggestions above for sub-modules with only 1 command. Unfortunately, none of them seem to work 😕

0 replies

andrewasheridan · 2020-08-31T21:57:04Z

andrewasheridan
Aug 31, 2020

What about the app.registered_commands.append(CommandInfo(name=name, callback=entry_point.main)) method that has not been addressed yet?

0 replies

pmsoltani · 2020-09-01T03:22:32Z

pmsoltani
Sep 1, 2020

@andrewasheridan that as well. I started at the top. Here is my project structure:

.
├── main.py (FastAPI)
└── cli
    ├── commands
    │   ├── __init__.py
    │   └── cmd_crawl.py
    ├── __init__.py
    └── cli.py

cmd_crawl.py:

import typer

cli = typer.Typer()

def arg_check(values: List[str]) -> List[str]:
    for value in values:
        if value not in ['wikipedia', 'reddit']:
            raise typer.BadParameter(f"Bad value '{value}'.")
    return values

@cli.command()
def crawl(websites: List[str] = typer.Argument(..., callback=arg_check)):
    typer.echo(websites)

cli.py (initial version):

import typer

from cli.commands import cmd_crawl

cli = typer.Typer()
cli.add_typer(cmd_crawl.cli, name="crawl")

Which would make me type in the terminal: proj crawl crawl wikipedia instead of proj crawl wikipedia, but it works. Your solution (and others):

cli.py (new version):

import typer
from typer.models import CommandInfo

from cli.commands import cmd_crawl


cli = typer.Typer()
cli.registered_commands.append(CommandInfo(name="crawl", callback=cmd_crawl.crawl))

Now, these don't work:
proj crawl wikipedia -> Error: Invalid value for 'ENGINES...': Bad value 'crawl'.
proj crawl crawl wikipedia -> Error: Invalid value for 'ENGINES...': Bad value 'crawl'.

But this one does (not what I want):
proj wikipedia -> Good!

0 replies

pmsoltani · 2020-09-01T12:59:56Z

pmsoltani
Sep 1, 2020

OK... I solved my problem:

cmd_crawl.py:

import typer

# cli = typer.Typer() -> No need to create a Typer app instance

def arg_check(values: List[str]) -> List[str]:
    for value in values:
        if value not in ['wikipedia', 'reddit']:
            raise typer.BadParameter(f"Bad value '{value}'.")
    return values

# @cli.command() -> We just need to have a simple function.
def crawl(websites: List[str] = typer.Argument(..., callback=arg_check)):
    typer.echo(websites)

cli.py (current version):

import typer

from cli.commands import cmd_crawl

cli = typer.Typer()
# cli.add_typer(cmd_crawl.cli, name="crawl")
cli.command(name="crawl")(cmd_crawl.crawl)

Now this works: proj crawl wikipedia. Note that this is different from @vlcinsky's solution, as he turned the sub-module hello.py into a Typer app and I didn't. Anyway, my goal was just to break the code into smaller modules and I didn't necessarily want sub-commands.

0 replies

yobiscus · 2021-03-10T16:01:37Z

yobiscus
Mar 10, 2021

Update: Steps below do not work, COMMAND is still expected.

@Andrew-Sheridan

If I set the main in bar.py (the plugin) as a callback then bar is unusable.

The invoke_without_command option appears to solve this problem.

@app.callback(invoke_without_command=True)
def main(path: str = typer.Argument(...)):
    typer.echo(f"Doing bar things to {path}")

With this option set, the command help appears correctly and it can be executed. I'm not sure if this is a gross misuse of the option, but it works as expected for adding a single subcommand to the command tree.

0 replies

NikosAlexandris · 2023-06-04T11:11:11Z

NikosAlexandris
Jun 4, 2023

Update: Steps below do not work, COMMAND is still expected.

@Andrew-Sheridan

If I set the main in bar.py (the plugin) as a callback then bar is unusable.

The invoke_without_command option appears to solve this problem.
@app.callback(invoke_without_command=True)
def main(path: str = typer.Argument(...)):
    typer.echo(f"Doing bar things to {path}")
With this option set, the command help appears correctly and it can be executed. I'm not sure if this is a gross misuse of the option, but it works as expected for adding a single subcommand to the command tree.

This is just what I was in need for. In my humble view, an example/use-case of this deserves its own section in the tutorial.

ps- https://gitlab.com/NikosAlexandris/typer-command-with-two-subcommands

0 replies

NikosAlexandris · 2023-06-05T09:12:09Z

NikosAlexandris
Jun 5, 2023

Following the above recommendation, there seems to be some annoyance : shouldn't the --help work anywhere and not only right after a subcommand?

I followed the above recommendation and it works. There is one problem, which is reported elsewhere, I recall but cannot find it. Here's one more example, using a custom package after installing it with pip.

Source: prototype

plotdata on  main [!?] via  v3.10.9 (.plotdata_virtual_environment)
❯ plot-data --help

 Usage: plot-data [OPTIONS] COMMAND [ARGS]...

 callback() : Plot time series

╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --verbose                 --no-verbose                                     Verbosity level [default: no-verbose]                                                 │
│ --version             -v                                                   Show the application's version and exit.                                              │
│ --install-completion                      [bash|zsh|fish|powershell|pwsh]  Install completion for the specified shell. [default: None]                           │
│ --show-completion                         [bash|zsh|fish|powershell|pwsh]  Show completion for the specified shell, to copy it or customize the installation.    │
│                                                                            [default: None]                                                                       │
│ --help                                                                     Show this message and exit.                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ outliers                                                              Plot series                                                                                │
│ series                                                                Plot series                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Then, asking for help on the subcommand series works too :

plotdata on  main [!?] via  v3.10.9 (.plotdata_virtual_environment)
❯ plot-data series --help

 Usage: plot-data series [OPTIONS] NETCDF [LONGITUDE] [LATITUDE] COMMAND
                         [ARGS]...

 Plot series

╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ *    netcdf         PATH         Input netCDF file [default: None] [required]                                                                                    │
│      longitude      [LONGITUDE]  Longitude in decimal degrees ranging in [0, 360]. [default: None]                                                               │
│      latitude       [LATITUDE]   Latitude in decimal degrees, south is negative [default: None]                                                                  │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --output-filename                                            PATH  Figure output filename [default: series_in]                                                   │
│ --variable-name-as-suffix    --no-variable-name-as-suffix          Suffix the output filename with the variable [default: variable-name-as-suffix]               │
│ --help                                                             Show this message and exit.                                                                   │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Helpers ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --zero-longitude-center    --no-zero-longitude-center      Convert range of of longitude values to [-180, 180] [default: no-zero-longitude-center]               │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

However, asking for --help in the end of the command line, won't work as expected, i.e. the help for series won't show up. Instead, it's like executing plot-data series without any arguments or options :

plotdata on  main [!?] via  v3.10.9 (.plotdata_virtual_environment)
❯ plot-data series /spacetime/pvgis/from_jeodpp_nextcloud/raster/era5/2009_01_cds_era5_surface_net_solar_radiation.nc -42.419 -22.275 --help
Usage: plot-data series [OPTIONS] NETCDF [LONGITUDE] [LATITUDE] COMMAND
                        [ARGS]...
Try 'plot-data series --help' for help.
╭─ Error ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Missing argument 'NETCDF'.                                                                                                                                       │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

0 replies

patrick91 · 2024-12-03T15:50:49Z

patrick91
Dec 3, 2024
Collaborator

This was fixed in Typer 0.15.0, now if you don't pass name to add_typer the commands will merged into the app!

See the docs here: https://typer.tiangolo.com/tutorial/one-file-per-command/

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QUESTION] How to avoid explicit call to subcommands command #1074

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 14 comments

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

[QUESTION] How to avoid explicit call to subcommands command #1074

First check

Description

Question

Additional context

Replies: 14 comments

Andrew-Sheridan Jun 14, 2020 Author

tiangolo Jun 17, 2020 Maintainer

Andrew-Sheridan Jun 17, 2020 Author

code

sub.py

sub2.py

main.py

run

main.py

sub.py

sub2.py

user.py

hello.py

main.py

Implicit expectations

patrick91 Dec 3, 2024 Collaborator

Andrew-Sheridan
Jun 14, 2020
Author

tiangolo
Jun 17, 2020
Maintainer

Andrew-Sheridan
Jun 17, 2020
Author

patrick91
Dec 3, 2024
Collaborator