[Bug]: LiteLLM container gets stuck on starting (Includes fix)

[SeveighTech-Management]

Error Message and Logs

After deploying LiteLLM, it gets stuck on running (starting). Below are the logs:

2024-12-24T07:33:15.122179216Z Traceback (most recent call last):
2024-12-24T07:33:15.122237310Z   File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 732, in lifespan
2024-12-24T07:33:15.122243705Z     async with self.lifespan_context(app) as maybe_state:
2024-12-24T07:33:15.122246906Z                ~~~~~~~~~~~~~~~~~~~~~^^^^^
2024-12-24T07:33:15.122249896Z   File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 608, in __aenter__
2024-12-24T07:33:15.122253363Z     await self._router.startup()
2024-12-24T07:33:15.122257118Z   File "/usr/local/lib/python3.13/site-packages/starlette/routing.py", line 709, in startup
2024-12-24T07:33:15.122260776Z     await handler()
2024-12-24T07:33:15.122264399Z   File "/usr/local/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 3069, in startup_event
2024-12-24T07:33:15.122268545Z     await initialize(**worker_config)
2024-12-24T07:33:15.122272581Z   File "/usr/local/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 2501, in initialize
2024-12-24T07:33:15.122276700Z     ) = await proxy_config.load_config(router=llm_router, config_file_path=config)
2024-12-24T07:33:15.122280861Z         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-24T07:33:15.122284579Z   File "/usr/local/lib/python3.13/site-packages/litellm/proxy/proxy_server.py", line 1642, in load_config
2024-12-24T07:33:15.122288611Z     raise Exception(
2024-12-24T07:33:15.122291921Z         CommonProxyErrors.not_premium_user.value
2024-12-24T07:33:15.122295710Z     )
2024-12-24T07:33:15.122299587Z Exception: You must be a LiteLLM Enterprise user to use this feature. If you have a license please set `LITELLM_LICENSE` in your env. Get a 7 day trial key here: https://www.litellm.ai/#trial. 
2024-12-24T07:33:15.122324776Z Pricing: https://www.litellm.ai/#pricing
2024-12-24T07:33:15.122328715Z 
2024-12-24T07:33:15.124355212Z Application startup failed. Exiting.
2024-12-24T07:33:15.134313733Z Using json logs. Setting log_config to None.

The issue was the docker-compose.yml
After checking the docs, the volume portion is not neccesary as specified here:

volumes:
      -
        type: bind
        source: ./litellm-config.yaml
        target: /app/config.yaml
        content: "general_settings:\n  proxy_batch_write_at: 60\n\nrouter_settings:\n  redis_host: os.environ/REDIS_HOST\n  redis_port: os.environ/REDIS_PORT\n  redis_password: os.environ/REDIS_PASSWORD\n  enable_pre_call_check: true\n\nlitellm_settings:\n  set_verbose: false\n  json_logs: true\n  log_raw_request_response: true\n  # turn_off_message_logging: false\n  # redact_user_api_key_info: false\n  service_callback: [\"prometheus_system\"]\n  drop_params: true\n  # max_budget: 100\n  # budget_duration: 30d\n  num_retries: 3\n  request_timeout: 600\n  telemetry: false\n  cache: true\n  cache_params:\n    type: redis\n    host: os.environ/REDIS_HOST\n    port: os.environ/REDIS_PORT\n    password: os.environ/REDIS_PASSWORD\n    namespace: \"litellm_cache\"\n    ttl: 600\n  success_callback:\n    # - \"langfuse\"\n    - \"prometheus\"\n  failure_callback:\n    # - \"langfuse\"\n    - \"prometheus\"\nmodel_list:\n  # OpenAI\n  - model_name: gpt-4\n    litellm_params:\n      model: openai/gpt-4\n      api_key: os.environ/OPENAI_API_KEY\n      api_base: os.environ/OPENAI_API_BASE\n  - model_name: gpt-4o\n    litellm_params:\n      model: openai/gpt-4o\n      api_key: os.environ/OPENAI_API_KEY\n      api_base: os.environ/OPENAI_API_BASE\n  - model_name: gpt-4o-mini\n    litellm_params:\n      model: openai/gpt-4o-mini\n      api_key: os.environ/OPENAI_API_KEY\n      api_base: os.environ/OPENAI_API_BASE\n  # Anthropic\n  - model_name: claude-3-haiku\n    litellm_params:\n      model: claude-3-haiku-20240307\n      api_key: \"os.environ/ANTHROPIC_API_KEY\"\n      api_base: \"os.environ/ANTHROPIC_API_BASE\"\n  - model_name: claude-3.5-sonnet\n    litellm_params:\n      model: claude-3-5-sonnet-20240620\n      api_key: \"os.environ/ANTHROPIC_API_KEY\"\n      api_base: \"os.environ/ANTHROPIC_API_BASE\"\n  # VoyageAI\n  - model_name: voyage-law-2\n    model_info:\n      output_vector_size: 1024\n    litellm_params:\n      model: voyage/voyage-law-2\n      api_key: \"os.environ/VOYAGE_API_KEY\"\n      api_base: \"os.environ/VOYAGE_API_BASE\"\n      # rpm: 300\n      # tpm: 1000000\n  - model_name: voyage-multilingual-2\n    model_info:\n      mode: embedding\n      max_tokens: 32000\n      max_input_tokens: 32000\n      output_vector_size: 1024\n    litellm_params:\n      model: voyage/voyage-multilingual-2\n      api_key: \"os.environ/VOYAGE_API_KEY\"\n      api_base: \"os.environ/VOYAGE_API_BASE\"\n      input_cost_per_token: 0.00000012\n      output_cost_per_token: 0\n      # rpm: 300\n      # tpm: 1000000"

and in the command portion:

command:
      - '--config'
      - '/app/config.yaml'
      - '--port'
      - '4000'
      - '--num_workers'
      - '8'

config is not necessary.

The complete working fix:

services:
  litellm:
    image: 'ghcr.io/berriai/litellm:main-latest'
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      - SERVICE_FQDN_LITELLM_4000
      - 'LITELLM_LOG=${LITELLM_LOG:-ERROR}'
      - 'LITELLM_MODE=${LITELLM_MODE:-PRODUCTION}'
      - 'LITELLM_MASTER_KEY=${SERVICE_PASSWORD_MASTERKEY}'
      - 'UI_USERNAME=${SERVICE_USER_UI}'
      - 'UI_PASSWORD=${SERVICE_PASSWORD_UI}'
      - 'DATABASE_URL=postgresql://${SERVICE_USER_POSTGRES}:${SERVICE_PASSWORD_POSTGRES}@postgres:5432/${POSTGRES_DB:-litellm}'
      - 'REDIS_HOST=${REDIS_HOST:-redis}'
      - 'REDIS_PORT=${REDIS_PORT:-6379}'
      - 'POSTGRES_USER=${SERVICE_USER_POSTGRES}'
      - 'POSTGRES_PASSWORD=${SERVICE_PASSWORD_POSTGRES}'
      - 'POSTGRES_DB=${POSTGRES_DB:-litellm}'
      - 'OPENAI_API_KEY=${OPENAI_API_KEY}'
      - 'OPENAI_API_BASE=${OPENAI_API_BASE}'
      - 'ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}'
      - 'ANTHROPIC_API_BASE=${ANTHROPIC_API_BASE}'
      - 'VOYAGE_API_KEY=${VOYAGE_API_KEY}'
      - 'VOYAGE_API_BASE=${VOYAGE_API_BASE}'
      - STORE_MODEL_IN_DB=True
    healthcheck:
      test:
        - CMD
        - python
        - '-c'
        - "import requests as r;r.get('http://127.0.0.1:4000/health/liveliness').raise_for_status()"
      interval: 5s
      timeout: 5s
      retries: 3
    command:
      - '--port'
      - '4000'
      - '--num_workers'
      - '8'
  postgres:
    image: 'postgres:16-alpine'
    environment:
      - 'POSTGRES_DB=${POSTGRES_DB:-litellm}'
      - POSTGRES_PASSWORD=$SERVICE_PASSWORD_POSTGRES
      - POSTGRES_USER=$SERVICE_USER_POSTGRES
    volumes:
      - 'pg-data:/var/lib/postgresql/data'
    healthcheck:
      test:
        - CMD-SHELL
        - 'pg_isready -h localhost -U $${POSTGRES_USER} -d $${POSTGRES_DB}'
      interval: 5s
      timeout: 5s
      retries: 3
  redis:
    image: 'redis:7-alpine'
    command: 'redis-server --appendonly yes'
    volumes:
      - 'redis-data:/data'
    healthcheck:
      test:
        - CMD
        - redis-cli
        - ping
      interval: 5s
      timeout: 5s
      retries: 3

Steps to Reproduce

1. Create new project
2. Select LiteLLM resource
3. Deploy

Example Repository URL

No response

Coolify Version

v4.0.0-beta.379

Are you using Coolify Cloud?

No (self-hosted)

Operating System and Version (self-hosted)

Ubuntu 22.04

Additional Information

No response