diff --git a/README.md b/README.md index b40464d..44dc6a1 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,6 @@ - 不对任何原始镜像的有效性做出任何明示或暗示的保证或声明,安全性和风险自查 - 个人仓库,可以Fork后自行更新,但是严禁未经授权,私自删除个人信息后合并发布 - ### 🚀 使用方法 #### 📋 添加脚本到 1Panel 计划任务 @@ -102,7 +101,6 @@ docker run -d --name=nginx -p 80:80 nginx:latest - `README_en.md` - 英文文档 - `logo.png` - 应用图标 - ### 📱 应用列表 以下是当前在本仓库中提供的应用列表及其版本信息,**点击应用名称可查看应用详细介绍文档** @@ -112,7 +110,7 @@ docker run -d --name=nginx -p 80:80 nginx:latest - 支持一键部署AI免费API接口,使用方式请参考应用内**README介绍** - **Free-API系列应用已下架,原项目由于供应链投毒,被植入恶意代码,请及时停止运行并删除这些服务!!!** - 经过几天的排查和重构,已重新上架[GLM-Free-API](https://github.com/xiaoY233/GLM-Free-API)、[MiniMax-Free-API](https://github.com/xiaoY233/MiniMax-Free-API)、[Qwen-Free-API](https://github.com/xiaoY233/Qwen-Free-API)、[Kimi-Free-API](https://github.com/xiaoY233/Kimi-Free-API),[DeepSeek-Free-API](https://github.com/xiaoY233/DeepSeek-Free-API),欢迎各位对源码进行审查,如果不放心,建议还是暂停使用! -- 其他的Free-API系列看情况再搞了,后续主要更新上述几个Free-API兼容Gemini-cli和Claude的API接入。 +- LiteLLM应用原项目又出现了供应链投毒事件,**请及时停止运行并删除该服务,友情提示:为了避免泄漏的API密钥被滥用,请从你的AI供应商中删除或禁止使用相关密钥!!!** @@ -1266,7 +1264,6 @@ AI驱动的开源代码知识库与文档协作平台,支持多模型、多数
- -![Copyright-arch3rPro](https://img.shields.io/badge/Copyright-arch3rPro-ff9800?style=flat&logo=github&logoColor=white) +!\[Copyright-arch3rPro]\( null) diff --git a/apps/litellm/README.md b/apps/litellm/README.md deleted file mode 100644 index c785ee8..0000000 --- a/apps/litellm/README.md +++ /dev/null @@ -1,444 +0,0 @@ -

- 🚅 LiteLLM -

-

-

- Deploy to Render - - Deploy on Railway - -

-

Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.] -
-

-

LiteLLM Proxy Server (LLM Gateway) | Hosted Proxy (Preview) | Enterprise Tier

-

- - PyPI Version - - - Y Combinator W23 - - - Whatsapp - - - Discord - - - Slack - -

- -LiteLLM manages: - -- Translate inputs to provider's `completion`, `embedding`, and `image_generation` endpoints -- [Consistent output](https://docs.litellm.ai/docs/completion/output), text responses will always be available at `['choices'][0]['message']['content']` -- Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - [Router](https://docs.litellm.ai/docs/routing) -- Set Budgets & Rate limits per project, api key, model [LiteLLM Proxy Server (LLM Gateway)](https://docs.litellm.ai/docs/simple_proxy) - -[**Jump to LiteLLM Proxy (LLM Gateway) Docs**](https://github.com/BerriAI/litellm?tab=readme-ov-file#litellm-proxy-server-llm-gateway---docs)
-[**Jump to Supported LLM Providers**](https://github.com/BerriAI/litellm?tab=readme-ov-file#supported-providers-docs) - -🚨 **Stable Release:** Use docker images with the `-stable` tag. These have undergone 12 hour load tests, before being published. [More information about the release cycle here](https://docs.litellm.ai/docs/proxy/release_cycle) - -Support for more providers. Missing a provider or LLM Platform, raise a [feature request](https://github.com/BerriAI/litellm/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.yml&title=%5BFeature%5D%3A+). - -# Usage ([**Docs**](https://docs.litellm.ai/docs/)) - -> [!IMPORTANT] -> LiteLLM v1.0.0 now requires `openai>=1.0.0`. Migration guide [here](https://docs.litellm.ai/docs/migration) -> LiteLLM v1.40.14+ now requires `pydantic>=2.0.0`. No changes required. - - - Open In Colab - - -```shell -pip install litellm -``` - -```python -from litellm import completion -import os - -## set ENV variables -os.environ["OPENAI_API_KEY"] = "your-openai-key" -os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key" - -messages = [{ "content": "Hello, how are you?","role": "user"}] - -# openai call -response = completion(model="openai/gpt-4o", messages=messages) - -# anthropic call -response = completion(model="anthropic/claude-sonnet-4-20250514", messages=messages) -print(response) -``` - -### Response (OpenAI Format) - -```json -{ - "id": "chatcmpl-1214900a-6cdd-4148-b663-b5e2f642b4de", - "created": 1751494488, - "model": "claude-sonnet-4-20250514", - "object": "chat.completion", - "system_fingerprint": null, - "choices": [ - { - "finish_reason": "stop", - "index": 0, - "message": { - "content": "Hello! I'm doing well, thank you for asking. I'm here and ready to help with whatever you'd like to discuss or work on. How are you doing today?", - "role": "assistant", - "tool_calls": null, - "function_call": null - } - } - ], - "usage": { - "completion_tokens": 39, - "prompt_tokens": 13, - "total_tokens": 52, - "completion_tokens_details": null, - "prompt_tokens_details": { - "audio_tokens": null, - "cached_tokens": 0 - }, - "cache_creation_input_tokens": 0, - "cache_read_input_tokens": 0 - } -} -``` - -Call any model supported by a provider, with `model=/`. There might be provider-specific details here, so refer to [provider docs for more information](https://docs.litellm.ai/docs/providers) - -## Async ([Docs](https://docs.litellm.ai/docs/completion/stream#async-completion)) - -```python -from litellm import acompletion -import asyncio - -async def test_get_response(): - user_message = "Hello, how are you?" - messages = [{"content": user_message, "role": "user"}] - response = await acompletion(model="openai/gpt-4o", messages=messages) - return response - -response = asyncio.run(test_get_response()) -print(response) -``` - -## Streaming ([Docs](https://docs.litellm.ai/docs/completion/stream)) - -liteLLM supports streaming the model response back, pass `stream=True` to get a streaming iterator in response. -Streaming is supported for all models (Bedrock, Huggingface, TogetherAI, Azure, OpenAI, etc.) - -```python -from litellm import completion -response = completion(model="openai/gpt-4o", messages=messages, stream=True) -for part in response: - print(part.choices[0].delta.content or "") - -# claude sonnet 4 -response = completion('anthropic/claude-sonnet-4-20250514', messages, stream=True) -for part in response: - print(part) -``` - -### Response chunk (OpenAI Format) - -```json -{ - "id": "chatcmpl-fe575c37-5004-4926-ae5e-bfbc31f356ca", - "created": 1751494808, - "model": "claude-sonnet-4-20250514", - "object": "chat.completion.chunk", - "system_fingerprint": null, - "choices": [ - { - "finish_reason": null, - "index": 0, - "delta": { - "provider_specific_fields": null, - "content": "Hello", - "role": "assistant", - "function_call": null, - "tool_calls": null, - "audio": null - }, - "logprobs": null - } - ], - "provider_specific_fields": null, - "stream_options": null, - "citations": null -} -``` - -## Logging Observability ([Docs](https://docs.litellm.ai/docs/observability/callbacks)) - -LiteLLM exposes pre defined callbacks to send data to Lunary, MLflow, Langfuse, DynamoDB, s3 Buckets, Helicone, Promptlayer, Traceloop, Athina, Slack - -```python -from litellm import completion - -## set env variables for logging tools (when using MLflow, no API key set up is required) -os.environ["LUNARY_PUBLIC_KEY"] = "your-lunary-public-key" -os.environ["HELICONE_API_KEY"] = "your-helicone-auth-key" -os.environ["LANGFUSE_PUBLIC_KEY"] = "" -os.environ["LANGFUSE_SECRET_KEY"] = "" -os.environ["ATHINA_API_KEY"] = "your-athina-api-key" - -os.environ["OPENAI_API_KEY"] = "your-openai-key" - -# set callbacks -litellm.success_callback = ["lunary", "mlflow", "langfuse", "athina", "helicone"] # log input/output to lunary, langfuse, supabase, athina, helicone etc - -#openai call -response = completion(model="openai/gpt-4o", messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}]) -``` - -# LiteLLM Proxy Server (LLM Gateway) - ([Docs](https://docs.litellm.ai/docs/simple_proxy)) - -Track spend + Load Balance across multiple projects - -[Hosted Proxy (Preview)](https://docs.litellm.ai/docs/hosted) - -The proxy provides: - -1. [Hooks for auth](https://docs.litellm.ai/docs/proxy/virtual_keys#custom-auth) -2. [Hooks for logging](https://docs.litellm.ai/docs/proxy/logging#step-1---create-your-custom-litellm-callback-class) -3. [Cost tracking](https://docs.litellm.ai/docs/proxy/virtual_keys#tracking-spend) -4. [Rate Limiting](https://docs.litellm.ai/docs/proxy/users#set-rate-limits) - -## 📖 Proxy Endpoints - [Swagger Docs](https://litellm-api.up.railway.app/) - - -## Quick Start Proxy - CLI - -```shell -pip install 'litellm[proxy]' -``` - -### Step 1: Start litellm proxy - -```shell -$ litellm --model huggingface/bigcode/starcoder - -#INFO: Proxy running on http://0.0.0.0:4000 -``` - -### Step 2: Make ChatCompletions Request to Proxy - - -> [!IMPORTANT] -> 💡 [Use LiteLLM Proxy with Langchain (Python, JS), OpenAI SDK (Python, JS) Anthropic SDK, Mistral SDK, LlamaIndex, Instructor, Curl](https://docs.litellm.ai/docs/proxy/user_keys) - -```python -import openai # openai v1.0.0+ -client = openai.OpenAI(api_key="anything",base_url="http://0.0.0.0:4000") # set proxy to base_url -# request sent to model set on litellm proxy, `litellm --model` -response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [ - { - "role": "user", - "content": "this is a test request, write a short poem" - } -]) - -print(response) -``` - -## Proxy Key Management ([Docs](https://docs.litellm.ai/docs/proxy/virtual_keys)) - -Connect the proxy with a Postgres DB to create proxy keys - -```bash -# Get the code -git clone https://github.com/BerriAI/litellm - -# Go to folder -cd litellm - -# Add the master key - you can change this after setup -echo 'LITELLM_MASTER_KEY="sk-1234"' > .env - -# Add the litellm salt key - you cannot change this after adding a model -# It is used to encrypt / decrypt your LLM API Key credentials -# We recommend - https://1password.com/password-generator/ -# password generator to get a random hash for litellm salt key -echo 'LITELLM_SALT_KEY="sk-1234"' >> .env - -source .env - -# Start -docker compose up -``` - - -UI on `/ui` on your proxy server -![ui_3](https://github.com/BerriAI/litellm/assets/29436595/47c97d5e-b9be-4839-b28c-43d7f4f10033) - -Set budgets and rate limits across multiple projects -`POST /key/generate` - -### Request - -```shell -curl 'http://0.0.0.0:4000/key/generate' \ ---header 'Authorization: Bearer sk-1234' \ ---header 'Content-Type: application/json' \ ---data-raw '{"models": ["gpt-3.5-turbo", "gpt-4", "claude-2"], "duration": "20m","metadata": {"user": "ishaan@berri.ai", "team": "core-infra"}}' -``` - -### Expected Response - -```shell -{ - "key": "sk-kdEXbIqZRwEeEiHwdg7sFA", # Bearer token - "expires": "2023-11-19T01:38:25.838000+00:00" # datetime object -} -``` - -## Supported Providers ([Docs](https://docs.litellm.ai/docs/providers)) - -| Provider | [Completion](https://docs.litellm.ai/docs/#basic-usage) | [Streaming](https://docs.litellm.ai/docs/completion/stream#streaming-responses) | [Async Completion](https://docs.litellm.ai/docs/completion/stream#async-completion) | [Async Streaming](https://docs.litellm.ai/docs/completion/stream#async-streaming) | [Async Embedding](https://docs.litellm.ai/docs/embedding/supported_embedding) | [Async Image Generation](https://docs.litellm.ai/docs/image_generation) | -|-------------------------------------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------|-------------------------------------------------------------------------| -| [openai](https://docs.litellm.ai/docs/providers/openai) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| [Meta - Llama API](https://docs.litellm.ai/docs/providers/meta_llama) | ✅ | ✅ | ✅ | ✅ | | | -| [azure](https://docs.litellm.ai/docs/providers/azure) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| [AI/ML API](https://docs.litellm.ai/docs/providers/aiml) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| [aws - sagemaker](https://docs.litellm.ai/docs/providers/aws_sagemaker) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [aws - bedrock](https://docs.litellm.ai/docs/providers/bedrock) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [google - vertex_ai](https://docs.litellm.ai/docs/providers/vertex) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | -| [google - palm](https://docs.litellm.ai/docs/providers/palm) | ✅ | ✅ | ✅ | ✅ | | | -| [google AI Studio - gemini](https://docs.litellm.ai/docs/providers/gemini) | ✅ | ✅ | ✅ | ✅ | | | -| [mistral ai api](https://docs.litellm.ai/docs/providers/mistral) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [cloudflare AI Workers](https://docs.litellm.ai/docs/providers/cloudflare_workers) | ✅ | ✅ | ✅ | ✅ | | | -| [CompactifAI](https://docs.litellm.ai/docs/providers/compactifai) | ✅ | ✅ | ✅ | ✅ | | | -| [cohere](https://docs.litellm.ai/docs/providers/cohere) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [anthropic](https://docs.litellm.ai/docs/providers/anthropic) | ✅ | ✅ | ✅ | ✅ | | | -| [empower](https://docs.litellm.ai/docs/providers/empower) | ✅ | ✅ | ✅ | ✅ | -| [huggingface](https://docs.litellm.ai/docs/providers/huggingface) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [replicate](https://docs.litellm.ai/docs/providers/replicate) | ✅ | ✅ | ✅ | ✅ | | | -| [together_ai](https://docs.litellm.ai/docs/providers/togetherai) | ✅ | ✅ | ✅ | ✅ | | | -| [openrouter](https://docs.litellm.ai/docs/providers/openrouter) | ✅ | ✅ | ✅ | ✅ | | | -| [ai21](https://docs.litellm.ai/docs/providers/ai21) | ✅ | ✅ | ✅ | ✅ | | | -| [baseten](https://docs.litellm.ai/docs/providers/baseten) | ✅ | ✅ | ✅ | ✅ | | | -| [vllm](https://docs.litellm.ai/docs/providers/vllm) | ✅ | ✅ | ✅ | ✅ | | | -| [nlp_cloud](https://docs.litellm.ai/docs/providers/nlp_cloud) | ✅ | ✅ | ✅ | ✅ | | | -| [aleph alpha](https://docs.litellm.ai/docs/providers/aleph_alpha) | ✅ | ✅ | ✅ | ✅ | | | -| [petals](https://docs.litellm.ai/docs/providers/petals) | ✅ | ✅ | ✅ | ✅ | | | -| [ollama](https://docs.litellm.ai/docs/providers/ollama) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [deepinfra](https://docs.litellm.ai/docs/providers/deepinfra) | ✅ | ✅ | ✅ | ✅ | | | -| [perplexity-ai](https://docs.litellm.ai/docs/providers/perplexity) | ✅ | ✅ | ✅ | ✅ | | | -| [Groq AI](https://docs.litellm.ai/docs/providers/groq) | ✅ | ✅ | ✅ | ✅ | | | -| [Deepseek](https://docs.litellm.ai/docs/providers/deepseek) | ✅ | ✅ | ✅ | ✅ | | | -| [anyscale](https://docs.litellm.ai/docs/providers/anyscale) | ✅ | ✅ | ✅ | ✅ | | | -| [IBM - watsonx.ai](https://docs.litellm.ai/docs/providers/watsonx) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [voyage ai](https://docs.litellm.ai/docs/providers/voyage) | | | | | ✅ | | -| [xinference [Xorbits Inference]](https://docs.litellm.ai/docs/providers/xinference) | | | | | ✅ | | -| [FriendliAI](https://docs.litellm.ai/docs/providers/friendliai) | ✅ | ✅ | ✅ | ✅ | | | -| [Galadriel](https://docs.litellm.ai/docs/providers/galadriel) | ✅ | ✅ | ✅ | ✅ | | | -| [GradientAI](https://docs.litellm.ai/docs/providers/gradient_ai) | ✅ | ✅ | | | | | -| [Novita AI](https://novita.ai/models/llm?utm_source=github_litellm&utm_medium=github_readme&utm_campaign=github_link) | ✅ | ✅ | ✅ | ✅ | | | -| [Featherless AI](https://docs.litellm.ai/docs/providers/featherless_ai) | ✅ | ✅ | ✅ | ✅ | | | -| [Nebius AI Studio](https://docs.litellm.ai/docs/providers/nebius) | ✅ | ✅ | ✅ | ✅ | ✅ | | -| [Heroku](https://docs.litellm.ai/docs/providers/heroku) | ✅ | ✅ | | | | | -| [OVHCloud AI Endpoints](https://docs.litellm.ai/docs/providers/ovhcloud) | ✅ | ✅ | | | | | - -[**Read the Docs**](https://docs.litellm.ai/docs/) - -## Run in Developer mode -### Services -1. Setup .env file in root -2. Run dependant services `docker-compose up db prometheus` - -### Backend -1. (In root) create virtual environment `python -m venv .venv` -2. Activate virtual environment `source .venv/bin/activate` -3. Install dependencies `pip install -e ".[all]"` -4. Start proxy backend `python litellm/proxy_cli.py` - -### Frontend -1. Navigate to `ui/litellm-dashboard` -2. Install dependencies `npm install` -3. Run `npm run dev` to start the dashboard - -# Enterprise -For companies that need better security, user management and professional support - -[Talk to founders](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat) - -This covers: -- ✅ **Features under the [LiteLLM Commercial License](https://docs.litellm.ai/docs/proxy/enterprise):** -- ✅ **Feature Prioritization** -- ✅ **Custom Integrations** -- ✅ **Professional Support - Dedicated discord + slack** -- ✅ **Custom SLAs** -- ✅ **Secure access with Single Sign-On** - -# Contributing - -We welcome contributions to LiteLLM! Whether you're fixing bugs, adding features, or improving documentation, we appreciate your help. - -## Quick Start for Contributors - -This requires poetry to be installed. - -```bash -git clone https://github.com/BerriAI/litellm.git -cd litellm -make install-dev # Install development dependencies -make format # Format your code -make lint # Run all linting checks -make test-unit # Run unit tests -make format-check # Check formatting only -``` - -For detailed contributing guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md). - -## Code Quality / Linting - -LiteLLM follows the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html). - -Our automated checks include: -- **Black** for code formatting -- **Ruff** for linting and code quality -- **MyPy** for type checking -- **Circular import detection** -- **Import safety checks** - - -All these checks must pass before your PR can be merged. - - -# Support / talk with founders - -- [Schedule Demo 👋](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version) -- [Community Discord 💭](https://discord.gg/wuPM9dRgDw) -- [Community Slack 💭](https://www.litellm.ai/support) -- Our numbers 📞 +1 (770) 8783-106 / ‭+1 (412) 618-6238‬ -- Our emails ✉️ ishaan@berri.ai / krrish@berri.ai - -# Why did we build this - -- **Need for simplicity**: Our code started to get extremely complicated managing & translating calls between Azure, OpenAI and Cohere. - -# Contributors - - - - - - - - - - - - - - - diff --git a/apps/litellm/data.yml b/apps/litellm/data.yml deleted file mode 100644 index 73cea5e..0000000 --- a/apps/litellm/data.yml +++ /dev/null @@ -1,27 +0,0 @@ -name: LiteLLM -tags: - - 实用工具 - - AI -title: 使用 OpenAI 格式调用所有 LLM API - [Bedrock、Huggingface、VertexAI、TogetherAI、Azure、OpenAI、Groq 等] -description: - en: Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, - TogetherAI, Azure, OpenAI, Groq etc.] - zh: 使用 OpenAI 格式调用所有 LLM API - [Bedrock、Huggingface、VertexAI、TogetherAI、Azure、OpenAI、Groq 等] -additionalProperties: - key: litellm - name: LiteLLM - tags: - - Tool - - AI - shortDescZh: 使用 OpenAI 格式调用所有 LLM API - [Bedrock、Huggingface、VertexAI、TogetherAI、Azure、OpenAI、Groq 等] - shortDescEn: Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, - VertexAI, TogetherAI, Azure, OpenAI, Groq etc.] - type: website - crossVersionUpdate: true - limit: 0 - website: https://github.com/BerriAI/litellm - github: https://github.com/BerriAI/litellm - document: https://docs.litellm.ai/docs/ diff --git a/apps/litellm/latest/data.yml b/apps/litellm/latest/data.yml deleted file mode 100644 index 17c2a8d..0000000 --- a/apps/litellm/latest/data.yml +++ /dev/null @@ -1,21 +0,0 @@ -additionalProperties: - formFields: - - default: "4000" - envKey: PANEL_APP_PORT_HTTP - label: - en: Port - zh: 端口 - required: true - type: number - edit: true - rule: paramPort - - default: sk-1234 - envKey: LITELLM_MASTER_KEY - label: - en: LITELLM_MASTER_KEY - zh: LITELLM密钥 - required: true - type: text - edit: true - rule: paramCommon - values: [] diff --git a/apps/litellm/latest/docker-compose.yml b/apps/litellm/latest/docker-compose.yml deleted file mode 100644 index b114e3d..0000000 --- a/apps/litellm/latest/docker-compose.yml +++ /dev/null @@ -1,79 +0,0 @@ -services: - litellm: - image: ghcr.io/berriai/litellm:main-stable - container_name: ${CONTAINER_NAME} - ######################################### - ## Uncomment these lines to start proxy with a config.yaml file ## - # volumes: - # - ./config.yaml:/app/config.yaml - # command: - # - "--config=/app/config.yaml" - ############################################## - ports: - - ${PANEL_APP_PORT_HTTP}:4000 # Map the container port to the host, change the host port if necessary - environment: - DATABASE_URL: "postgresql://llmproxy:dbpassword9090@db:5432/litellm" - STORE_MODEL_IN_DB: "True" # allows adding models to proxy via UI - LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY} - depends_on: - - db # Indicates that this service depends on the 'db' service, ensuring 'db' starts first - healthcheck: # Defines the health check configuration for the container - test: [ "CMD-SHELL", "wget --no-verbose --tries=1 http://localhost:4000/health/liveliness || exit 1" ] # Command to execute for health check - interval: 30s # Perform health check every 30 seconds - timeout: 10s # Health check command times out after 10 seconds - retries: 3 # Retry up to 3 times if health check fails - start_period: 40s # Wait 40 seconds after container start before beginning health checks - - networks: - - 1panel-network - labels: - createdBy: "Apps" - db: - image: postgres:16 - restart: always - container_name: ${CONTAINER_NAME}-db - environment: - POSTGRES_DB: litellm - POSTGRES_USER: llmproxy - POSTGRES_PASSWORD: dbpassword9090 - ports: - - "5432:5432" - volumes: - - postgres_data:/var/lib/postgresql/data # Persists Postgres data across container restarts - healthcheck: - test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"] - interval: 1s - timeout: 5s - retries: 10 - - networks: - - 1panel-network - labels: - createdBy: "Apps" - prometheus: - image: prom/prometheus - container_name: ${CONTAINER_NAME}-prometheus - volumes: - - prometheus_data:/prometheus - - ./prometheus.yml:/etc/prometheus/prometheus.yml - ports: - - "9090:9090" - command: - - "--config.file=/etc/prometheus/prometheus.yml" - - "--storage.tsdb.path=/prometheus" - - "--storage.tsdb.retention.time=15d" - restart: always - - networks: - - 1panel-network - labels: - createdBy: "Apps" -volumes: - prometheus_data: - driver: local - postgres_data: - name: litellm_postgres_data # Named volume for Postgres data persistence - -networks: - 1panel-network: - external: true \ No newline at end of file diff --git a/apps/litellm/latest/prometheus.yml b/apps/litellm/latest/prometheus.yml deleted file mode 100644 index 5cb4f90..0000000 --- a/apps/litellm/latest/prometheus.yml +++ /dev/null @@ -1,7 +0,0 @@ -global: - scrape_interval: 15s - -scrape_configs: - - job_name: 'litellm' - static_configs: - - targets: ['litellm:4000'] # Assuming Litellm exposes metrics at port 4000 diff --git a/apps/litellm/logo.png b/apps/litellm/logo.png deleted file mode 100644 index f3f109d..0000000 Binary files a/apps/litellm/logo.png and /dev/null differ diff --git a/apps/litellm/main-v1.82.6-nightly/data.yml b/apps/litellm/main-v1.82.6-nightly/data.yml deleted file mode 100644 index 17c2a8d..0000000 --- a/apps/litellm/main-v1.82.6-nightly/data.yml +++ /dev/null @@ -1,21 +0,0 @@ -additionalProperties: - formFields: - - default: "4000" - envKey: PANEL_APP_PORT_HTTP - label: - en: Port - zh: 端口 - required: true - type: number - edit: true - rule: paramPort - - default: sk-1234 - envKey: LITELLM_MASTER_KEY - label: - en: LITELLM_MASTER_KEY - zh: LITELLM密钥 - required: true - type: text - edit: true - rule: paramCommon - values: [] diff --git a/apps/litellm/main-v1.82.6-nightly/docker-compose.yml b/apps/litellm/main-v1.82.6-nightly/docker-compose.yml deleted file mode 100644 index 9eafe38..0000000 --- a/apps/litellm/main-v1.82.6-nightly/docker-compose.yml +++ /dev/null @@ -1,79 +0,0 @@ -services: - litellm: - image: ghcr.io/berriai/litellm:main-v1.82.6-nightly - container_name: ${CONTAINER_NAME} - ######################################### - ## Uncomment these lines to start proxy with a config.yaml file ## - # volumes: - # - ./config.yaml:/app/config.yaml - # command: - # - "--config=/app/config.yaml" - ############################################## - ports: - - ${PANEL_APP_PORT_HTTP}:4000 # Map the container port to the host, change the host port if necessary - environment: - DATABASE_URL: "postgresql://llmproxy:dbpassword9090@db:5432/litellm" - STORE_MODEL_IN_DB: "True" # allows adding models to proxy via UI - LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY} - depends_on: - - db # Indicates that this service depends on the 'db' service, ensuring 'db' starts first - healthcheck: # Defines the health check configuration for the container - test: [ "CMD-SHELL", "wget --no-verbose --tries=1 http://localhost:4000/health/liveliness || exit 1" ] # Command to execute for health check - interval: 30s # Perform health check every 30 seconds - timeout: 10s # Health check command times out after 10 seconds - retries: 3 # Retry up to 3 times if health check fails - start_period: 40s # Wait 40 seconds after container start before beginning health checks - - networks: - - 1panel-network - labels: - createdBy: "Apps" - db: - image: postgres:16 - restart: always - container_name: ${CONTAINER_NAME}-db - environment: - POSTGRES_DB: litellm - POSTGRES_USER: llmproxy - POSTGRES_PASSWORD: dbpassword9090 - ports: - - "5432:5432" - volumes: - - postgres_data:/var/lib/postgresql/data # Persists Postgres data across container restarts - healthcheck: - test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"] - interval: 1s - timeout: 5s - retries: 10 - - networks: - - 1panel-network - labels: - createdBy: "Apps" - prometheus: - image: prom/prometheus - container_name: ${CONTAINER_NAME}-prometheus - volumes: - - prometheus_data:/prometheus - - ./prometheus.yml:/etc/prometheus/prometheus.yml - ports: - - "9090:9090" - command: - - "--config.file=/etc/prometheus/prometheus.yml" - - "--storage.tsdb.path=/prometheus" - - "--storage.tsdb.retention.time=15d" - restart: always - - networks: - - 1panel-network - labels: - createdBy: "Apps" -volumes: - prometheus_data: - driver: local - postgres_data: - name: litellm_postgres_data # Named volume for Postgres data persistence - -networks: - 1panel-network: - external: true \ No newline at end of file diff --git a/apps/litellm/main-v1.82.6-nightly/prometheus.yml b/apps/litellm/main-v1.82.6-nightly/prometheus.yml deleted file mode 100644 index 5cb4f90..0000000 --- a/apps/litellm/main-v1.82.6-nightly/prometheus.yml +++ /dev/null @@ -1,7 +0,0 @@ -global: - scrape_interval: 15s - -scrape_configs: - - job_name: 'litellm' - static_configs: - - targets: ['litellm:4000'] # Assuming Litellm exposes metrics at port 4000