Error: UPGRADE FAILED: post-upgrade hooks failed: 1 error occurred

[Standard-IO]

I'm trying to install a on-premise

I'm using the next versions code

minikube version
minikube version: v1.34.0
commit: 210b148df93a80eb872ecbeb7e35281b3c582c61

kubectl version
Client Version: v1.31.0
Kustomize Version: v5.4.2
Server Version: v1.31.0

helm version
version.BuildInfo{Version:"v3.16.3", GitCommit:"cfd07493f46efc9debd9cc1b02a0961186df7fdf", GitTreeState:"clean", GoVersion:"go1.22.7"}

helm upgrade --install --values my-values.yaml --debug superset superset/superset

But I just get the next error

fault
client.go:693: 2024-12-17 11:55:37.905400633 -0600 CST m=+1.810207368 [debug] Patch StatefulSet "superset-redis-master" in namespace default
client.go:486: 2024-12-17 11:55:37.912612677 -0600 CST m=+1.817419402 [debug] Starting delete for "mytesting-init-db" Job
wait.go:104: 2024-12-17 11:55:37.91791168 -0600 CST m=+1.822718414 [debug] beginning wait for 1 resources to be deleted with timeout of 5m0s
client.go:142: 2024-12-17 11:55:37.961136675 -0600 CST m=+1.865943431 [debug] creating 1 resource(s)
client.go:712: 2024-12-17 11:55:37.967588444 -0600 CST m=+1.872395179 [debug] Watching for changes to Job mytesting-init-db with timeout of 5m0s
client.go:740: 2024-12-17 11:55:37.971198437 -0600 CST m=+1.876005169 [debug] Add/Modify event for mytesting-init-db: ADDED
client.go:779: 2024-12-17 11:55:37.971257601 -0600 CST m=+1.876064327 [debug] mytesting-init-db: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
client.go:740: 2024-12-17 11:55:37.979005069 -0600 CST m=+1.883811804 [debug] Add/Modify event for mytesting-init-db: MODIFIED
client.go:779: 2024-12-17 11:55:37.979048779 -0600 CST m=+1.883855505 [debug] mytesting-init-db: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
upgrade.go:487: 2024-12-17 12:01:41.959031767 -0600 CST m=+301.874667481 [debug] warning: Upgrade "superset" failed: post-upgrade hooks failed: 1 error occurred:
* timed out waiting for the condition
Error: UPGRADE FAILED: post-upgrade hooks failed: 1 error occurred:
* timed out waiting for the condition
helm.go:86: 2024-12-17 12:01:41.980934496 -0600 CST m=+301.896570183 [debug] post-upgrade hooks failed: 1 error occurred:
* timed out waiting for the condition
UPGRADE FAILED
main.newUpgradeCmd.func2
helm.sh/helm/v3/cmd/helm/upgrade.go:243
github.com/spf13/cobra.(*Command).execute
github.com/spf13/[email protected]/command.go:985
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/[email protected]/command.go:1117
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/[email protected]/command.go:1041
main.main
helm.sh/helm/v3/cmd/helm/helm.go:85
runtime.main
runtime/proc.go:271
runtime.goexit
runtime/asm_amd64.s:1695

Reading this log I see the Starting delete for "mytesting-init-db" Job wait.go:104: 2024-12-17 11:55:37.91791168 -0600 CST m=+1.822718414 [debug] beginning wait for 1 resources to be deleted with timeout of 5m0s I'm not sure how to approach this problem.

Any idea?

[dosubot[bot]]

Hey @Standard-IO! 👋 I'm here to help you with your tech issues and questions. Let's squash those bugs together!

I found a similar open issue titled "Unable to install superset using helm and kubernetes" that might be relevant to your problem. However, it doesn't have any comments or solutions provided ^[1].

Regarding your issue with the mytesting-init-db Job timing out, here are some potential areas to investigate:

1. Job Conditions: Ensure that all conditions for the mytesting-init-db Job are met. This includes enabling the job, configuring the service account, setting the correct security context, providing necessary environment variables, and specifying the correct image configuration ^[2].

2. Timeout Settings: The default timeout for Helm operations might not be sufficient for your setup. You can try increasing the timeout by using the --timeout flag in your Helm command, for example, --timeout 10m0s.

3. Logs and Events: Check the logs of the mytesting-init-db Job for any errors or warnings that might indicate why it is not completing successfully. You can use kubectl logs <job-pod-name> to view the logs.

4. Resource Requests and Limits: Ensure that the resource requests and limits for the job are set appropriately. Insufficient resources can cause the job to fail or take longer than expected ^[2].

5. Check for Conflicts: Ensure there are no conflicts with existing resources or configurations that might be causing the job to hang or fail.

By addressing these areas, you may be able to resolve the timeout issue during the Helm upgrade process.

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[Standard-IO]

I found related issues/questions (#28125, #27575, #28782) related to this topic but no one show a clear way how to debug this problem. Maybe because this is very related to k8s and helms charts.

The failing is a Job in k8s to init the postgres db.

We can change the way to install/upgrade the deployment to show more information by just add --debug and install the charts

helm upgrade --install --values my-values.yaml --debug superset superset/superset

After realice that the erros is a Job we can list the related pods by run the next command; a list will be deployed and we choose the job with the errors state:

kubectl get go -A
NAMESPACE              NAME                                        READY   STATUS      RESTARTS      AGE
default                superset-866f4645d7-29r89                   1/1     Running     0             74s
default                superset-init-db-ms85r                      0/1     Error       0             74s
default                superset-postgresql-0                       1/1     Running     0             6h49m
default                superset-redis-master-0                     1/1     Running     0             6h49m
default                superset-worker-59c7866db5-hzwt6            1/1     Running     0             74s

In my case was the superset-init-db-ms85r so is necessary to watch its logs. We do this by using the next command and the Error pod's name .

kubectl logs superset-init-db-ms85r

The displayed errors were no provide a secure SECRET_KEY to sign the request to the backend and not include a module to stablish a connection with postgresql using psycopg2.

1. Create a secure key with the command suggested in the docs security section and put it in the values.yaml file.
2. Add the installation psycopg2 in the bootstrapScript (in the values.yaml file too).

bootstrapScript: |
  #!/bin/bash
  python -m pip install -U psycopg2-binary
  if [ ! -f ~/bootstrap ]; then echo "Running Superset with uid {{ .Values.runAsUser }}" > ~/bootstrap; fi

After this steps everything is working fine.