Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add release workflow #3

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
name: Create release
on: [push]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
on: [push]
on:
push:
branches:
- 'master'
paths:
- 'pyproject.toml'

maybe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike having the branch hardcoded, as I tend to use main for my branches, but I guess if people use the template they will get this name...

Copy link

@grqz grqz Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    if: format('refs/heads/{0}', github.event.repository.default_branch) == github.ref

maybe use this at job level instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'll make it show up as a run but just be skipped. Your previous suggestion is the way to go I think

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use something like this?

    branches:
      - ${{ github.event.repository.default_branch }}


jobs:
coletdjnz marked this conversation as resolved.
Show resolved Hide resolved
release:
permissions:
contents: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-python@v5
with:
python-version: "3.8"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be 3.9(yt-dlp/yt-dlp#11321)


- name: Install Hatch
run: pipx install hatch

- name: Set variables
id: set_variables
run: |
tag="$(git describe --tags --abbrev=0)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to assume a git tag already exists

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right I forgot I tested with a failed tag existing

Copy link

@grqz grqz Sep 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tag="$(git describe --tags --abbrev=0)"
tag=$(git for-each-ref refs/tags --sort=-creatordate --format='%(refname:strip=2)' --count=1)

This gives an empty string when no tags are present. It works without problem creating the first release of the repo.
Refer to https://stackoverflow.com/a/5261470

echo "::group::Variables"
cat << EOF | tee -a "$GITHUB_OUTPUT"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tag=${tag}
version=v$(hatch project metadata | jq -r .version)
EOF
echo "::endgroup::"

- name: Bundle and create release
env:
GH_TOKEN: ${{ github.token }}
tag: ${{ steps.set_variables.outputs.tag }}
version: ${{ steps.set_variables.outputs.version }}
if: |
env.tag != env.version
run: |
project_name="$(hatch project metadata | jq -r .name)"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move this to the set_variables step?

sources="$(\
hatch run default:pip list --verbose --format json \
| jq -r '.[] | select(.editable_project_location == null) | "\(.name);\(.location)"' \
)"

echo "::group::Dependencies"
printf '%s\n' "${sources}"
echo "::endgroup::"

mkdir bundle/
cp -r yt_dlp_plugins bundle/

while IFS=';' read -r name path; do
if [[ ! "${name}" =~ ^(pip|setuptools|wheel)$ ]]; then
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably also exclude yt-dlp?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested this originally too, but found you'd need to specify all of yt-dlp's dependencies too (until yt-dlp switches to not requiring deps by default)

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: yt-dlp/yt-dlp#11255 has been merged

package_name="$(tr '[:upper:]' '[:lower:]' <<<"${name}" | sed 's/-/_/g')"
cp -r "${path}/${package_name}" bundle/
Copy link

@grqz grqz Sep 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some projects don't have it's code in this path. For example a normal yt-dlp plugin or yt-dlp itself. not sure if we can run pip install . and change installation path to bundle/ and remove .dist-info if necessary, probably like pip install -t bundle/ .?

example log from shell
$ pip install -t bundle/ .; ls -al bundle/
Processing /home/user/tmp
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting yt-dlp-get-pot>=0.0.2
  Using cached yt_dlp_get_pot-0.0.2-py3-none-any.whl (6.8 kB)
Building wheels for collected packages: definitely_not_bgutil-ytdlp-pot-provider_to_distinguish_from_it
  Building wheel for definitely_not_bgutil-ytdlp-pot-provider_to_distinguish_from_it (pyproject.toml) ... done
  Created wheel for definitely_not_bgutil-ytdlp-pot-provider_to_distinguish_from_it: filename=definitely_not_bgutil_ytdlp_pot_provider_to_distinguish_from_it-0.2.22-py3-none-any.whl size=1243 sha256=25788b6ccab2676493a4fdb39808dc3833a7699963d504e1137f2d2fde310794
  Stored in directory: /tmp/pip-ephem-wheel-cache-yh0ca0qt/wheels/c5/5b/dd/eba17e133d5a684b348375bea715fd011a6e07e4630222c11b
Successfully built definitely_not_bgutil-ytdlp-pot-provider_to_distinguish_from_it
Installing collected packages: yt-dlp-get-pot, definitely_not_bgutil-ytdlp-pot-provider_to_distinguish_from_it
Successfully installed definitely_not_bgutil-ytdlp-pot-provider_to_distinguish_from_it-0.2.22 yt-dlp-get-pot-0.0.2
total 20
drwxr-xr-x 5 user user 4096 Sep  2 19:42 .
drwxr-xr-x 5 user user 4096 Sep  2 19:42 ..
drwxr-xr-x 2 user user 4096 Sep  2 19:42 definitely_not_bgutil_ytdlp_pot_provider_to_distinguish_from_it-0.2.22.dist-info
drwxr-xr-x 3 user user 4096 Sep  2 19:42 yt_dlp_get_pot-0.0.2.dist-info
drwxr-xr-x 3 user user 4096 Sep  2 19:42 yt_dlp_plugins

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will look into it; it's def cleaner than copying.

Perhaps we could use something like hatch dep show requirements, or something with --no-deps? There's room to cook something here, I just can't see it quite yet

fi
done <<<"${sources}"

cd bundle/
find . -type f -name '*.py[co]' -delete -o -type d -name __pycache__ -delete
zip -9 --recurse-paths "${project_name}" *
gh release create "${version}" --latest \
--title "${project_name} ${version}" \
"${project_name}.zip"
coletdjnz marked this conversation as resolved.
Show resolved Hide resolved
60 changes: 58 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
This repository contains a sample plugin package for [yt-dlp](https://github.com/yt-dlp/yt-dlp#readme).
This repository contains a sample plugin package for [yt-dlp](https://github.com/yt-dlp/yt-dlp#readme).

See [yt-dlp plugins](https://github.com/yt-dlp/yt-dlp#plugins) for more details.
coletdjnz marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -7,14 +7,70 @@ See [yt-dlp plugins](https://github.com/yt-dlp/yt-dlp#plugins) for more details.

Requires yt-dlp `2023.01.02` or above.

### Install with pip

You can install this package with pip:
```
python3 -m pip install -U https://github.com/yt-dlp/yt-dlp-sample-plugins/archive/master.zip
```

See [installing yt-dlp plugins](https://github.com/yt-dlp/yt-dlp#installing-plugins) for the other methods this plugin package can be installed.
### Install manually

1. Download the latest release zip from [releases](https://github.com/yt-dlp/yt-dlp-sample-plugins/releases)

2. Add the zip to one of the [yt-dlp plugin locations](https://github.com/yt-dlp/yt-dlp#installing-plugins)

- User Plugins
- `${XDG_CONFIG_HOME}/yt-dlp/plugins` (recommended on Linux/MacOS)
- `~/.yt-dlp/plugins/`
- `${APPDATA}/yt-dlp/plugins/` (recommended on Windows)

- System Plugins
- `/etc/yt-dlp/plugins/`
- `/etc/yt-dlp-plugins/`

- Executable location
- Binary: where `<root-dir>/yt-dlp.exe`, `<root-dir>/yt-dlp-plugins/`

For more locations and methods, see [installing yt-dlp plugins](https://github.com/yt-dlp/yt-dlp#installing-plugins)

## Development

See the [Plugin Development](https://github.com/yt-dlp/yt-dlp/wiki/Plugin-Development) section of the yt-dlp wiki.

### Dependencies

Add required dependencies to the `dependencies` section in the `pyproject.toml`.

#### Bundling with release zip

By default, the [release action](.github/workflows/release.yml) will try to bundle the dependencies in the release zip.
For these dependencies to work, they must be pure python modules.

From within the plugin, you will also need to use an import pattern similar to the following:

```py
import sys
import pathlib


import_path = str(pathlib.Path(__file__).parent.parent.parent)

sys.path.append(import_path)
try:
import some_dependency

except ImportError:
some_dependency = None

finally:
sys.path.remove(import_path)
```

If you do not want to bundle the dependencies with the release zip, you can add the dependencies to the exclusion list in the [release action](.github/workflows/release.yml).

## Release

To create a release, simply increment the version in the `pyproject.toml` file.
While convenient, conditional requirements or non pure python modules will most likely not work.
Please edit the `.github/workflows/release.yml` accordingly if you require more control.
24 changes: 20 additions & 4 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.distutils.bdist_wheel]
universal = true
[project]
name = "yt-dlp-sample-plugins"
version = "2023.01.02"
readme = "README.md"
requires-python = ">=3.8"
license = {file = "LICENSE"}
keywords = ["yt-dlp", "yt-dlp-plugin"]
authors = [
{ name = "John Doe", email = "[email protected]" },
]
dependencies = []

[tool.hatch.env.default]
installer = "uv"
path = ".venv"

[tool.hatch.build.targets.wheel]
packages = ["yt_dlp_plugins"]
6 changes: 0 additions & 6 deletions setup.cfg

This file was deleted.