Improved discoverability of colcon extensions

[ruffsl]

One of the ideas that came up from today's Open Robotics Infrastructure Community Office Hours was to potential to inform users of available extensions in colcon ecosystem.

https://discourse.ros.org/t/open-robotics-infrastructure-community-office-hours-all-things-colcon-2022-09-26/27390

For example, we could mimic apt's No command X found, did you mean:

$ colcon clean workspace
No verb 'clean' found, did you mean:
 Verb 'clean' from package 'python3-colcon-clean' (ros)
 Verb 'clean' from package 'python3-colcon-cache' (ros)
colcon: verb not found

[cottsay]

Thanks for filing this.

My first question, and I think the hardest problem to solve here, is where the "data" resides. Some thoughts:

1. Create a separate registry similar to mixins. I don't like this because it introduces a discrete "source" for the data.
2. Leverage dnf and/or apt to query for the information we want.
   a. We could look for packages providing .../colcon-.*/verb/foo.py. This is only a heuristic though, and relies on conventions.
   b. We could introduce a new tag to the deb and RPM packages, something like Provides3: colcon-verb(foo). This would reside in the stdeb.py. Still not ideal, but better than a separate repository.

[gavanderhoorn]

2. We could look for packages providing .../colcon-.*/verb/foo.py

if I understand you correctly, this would also only work for packages already installed, unless it'd use something like apt-file (which has its own drawbacks).

[cottsay]

this would also only work for packages already installed

Ah, I think you're right.

[nuclearsandwich]

I think that option 1 would mirror the way that colcon-mixin and colcon-metadata work and would actually a pretty reasonable pattern for an extension registry of some kind.

My concerns with option 2 are that only works with system-wide installations of colcon and we'd have to explicitly add support for each packaging system (apt and dnf/rpm for sure but also pacman, zypper, apk, etc to to mention PyPI) which either creates a large maintenance surface or a very narrow scope of support for who can use the discoverability.

• While there are advantages to creating a self-declared system so that the colcon core team can remain hands-off, I think a counterweight to a self-declared system is the potential for abuse. If colcon in its default configuration is going to be suggesting or enabling discovery of additional extensions I don't think we can justify including packages with zero review unless we add a disclaimer so emphatic it discourages normal use of the feature

A further advantage of a core team maintained extension registry is that it further reduces the need to maintain all extensions within the colcon org on GitHub if that's a policy that we wish to relax.

[nuclearsandwich]

One potentially interesting vector from a ROS perspective would be the ability to recommend colcon extensions that bring support for additional package package.xml build_types.
An example use case: if I have colcon-common-extensions plus our theoretical discovery extension installed and I try to build a workspace with ROS packages which have the build_type meson or ament_meson then colcon could suggest installing the colcon-meson package.

Since package detection and discovery is handled in extensions it would be quite difficult for a non-ROS workspace to make build type recommendations without re-implementing some discovery locally which I think would be a misfeature.

See colcon/colcon-common-extensions#17