ghcr.io/huggingface/text-generation-inference:3.2.1 linux/amd64

ghcr.io/huggingface/text-generation-inference:3.2.1 - 国内下载镜像源浏览次数:33

用于文本生成的 Hugging Face Inference 镜像。它旨在提供高性能的文本生成服务，优化了延迟和吞吐量。该镜像支持多种模型，并提供了易于使用的 API，方便部署和扩展。

源镜像	ghcr.io/huggingface/text-generation-inference:3.2.1
国内镜像	swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1
镜像ID	sha256:fc032b59f1dcbe4c4bb7895797e57dbb1b9fac9edabe06edd73e3b8b44c80334
镜像TAG	3.2.1
大小	16.22GB
镜像源	ghcr.io
CMD
启动入口	/tgi-entrypoint.sh
工作目录	/usr/src
OS/平台	linux/amd64
浏览量	33 次
贡献者
镜像创建	2025-03-18T15:32:56.856777273Z
同步时间	2025-03-25 00:58
更新时间	2025-03-28 13:49

环境变量

PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/bin:/usr/src/.venv/bin/ NVARCH=x86_64 NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536 NV_CUDA_CUDART_VERSION=12.4.99-1 NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4 CUDA_VERSION=12.4.0 LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/root/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/ NVIDIA_VISIBLE_DEVICES=all NVIDIA_DRIVER_CAPABILITIES=compute,utility HF_HOME=/data HF_HUB_ENABLE_HF_TRANSFER=1 PORT=80 PYTHON_VERSION=3.11 VIRTUAL_ENV=/usr/src/.venv/ HF_KERNELS_CACHE=/kernels EXLLAMA_NO_FLASH_ATTN=1

镜像标签

NVIDIA CORPORATION <cudatools@nvidia.com>: maintainer 2025-03-18T15:24:42.616Z: org.opencontainers.image.created Large Language Model Text Generation Inference: org.opencontainers.image.description Apache-2.0: org.opencontainers.image.licenses ubuntu: org.opencontainers.image.ref.name 4d28897b4e345f4dfdd93d3434e50ac8afcdf9e1: org.opencontainers.image.revision https://github.com/huggingface/text-generation-inference: org.opencontainers.image.source text-generation-inference: org.opencontainers.image.title https://github.com/huggingface/text-generation-inference: org.opencontainers.image.url 3.2.1: org.opencontainers.image.version

站长正在求职，如果你有北京Linux运维或运维开发的工作机会请联系我🙇‍ 点击了解更多🤗

阿里云高校计划(￥300代金券)领取

阿里云2核2G高性能云服务器 99元/年❤️

华为云2核4G2M云服务器，惊爆价99元1年

腾讯云2核2G云服务器，惊爆价99元1年

Ucloud 2核2G4M 59元/年❤️ 香港专线400G流量/月服务器 93元/年❤️

硅基流动注册送2000万 Tokens！

GPT人工智能60G学习资料❤️

Docker拉取命令无权限下载?点我修复

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1
docker tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1  ghcr.io/huggingface/text-generation-inference:3.2.1

Containerd拉取命令

ctr images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1
ctr images tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1  ghcr.io/huggingface/text-generation-inference:3.2.1

Shell快速替换命令

sed -i 's#ghcr.io/huggingface/text-generation-inference:3.2.1#swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1#' deployment.yaml

Ansible快速分发-Docker

#ansible k8s -m shell -a 'docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1 && docker tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1  ghcr.io/huggingface/text-generation-inference:3.2.1'

Ansible快速分发-Containerd

#ansible k8s -m shell -a 'ctr images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1 && ctr images tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1  ghcr.io/huggingface/text-generation-inference:3.2.1'

镜像构建历史


# 2025-03-18 23:32:56  0.00B 配置容器启动时运行的命令
ENTRYPOINT ["/tgi-entrypoint.sh"]
                        
# 2025-03-18 23:32:56  0.00B 设置环境变量 LD_LIBRARY_PATH
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/root/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/
                        
# 2025-03-18 23:32:56  0.00B 执行命令并创建新的镜像层
RUN /bin/sh -c chmod +x /tgi-entrypoint.sh # buildkit
                        
# 2025-03-18 23:32:56  163.00B 复制新文件或目录到容器中
COPY ./tgi-entrypoint.sh /tgi-entrypoint.sh # buildkit
                        
# 2025-03-18 23:32:56  6.95MB 复制新文件或目录到容器中
COPY /usr/src/target/release-opt/text-generation-launcher /usr/local/bin/text-generation-launcher # buildkit
                        
# 2025-03-18 23:32:56  39.11MB 复制新文件或目录到容器中
COPY /usr/src/target/release-opt/text-generation-router /usr/local/bin/text-generation-router # buildkit
                        
# 2025-03-18 23:00:23  11.83MB 复制新文件或目录到容器中
COPY /usr/src/target/release-opt/text-generation-benchmark /usr/local/bin/text-generation-benchmark # buildkit
                        
# 2025-03-18 19:36:50  220.13MB 执行命令并创建新的镜像层
RUN /bin/sh -c apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends         build-essential         g++         && rm -rf /var/lib/apt/lists/* # buildkit
                        
# 2025-03-18 19:36:40  0.00B 设置环境变量 EXLLAMA_NO_FLASH_ATTN
ENV EXLLAMA_NO_FLASH_ATTN=1
                        
# 2025-03-18 19:36:40  1.31GB 复制新文件或目录到容器中
COPY /usr/src/.venv/lib/python3.11/site-packages/flashinfer/ /usr/src/.venv/lib/python3.11/site-packages/flashinfer/ # buildkit
                        
# 2025-03-18 18:25:33  33.73MB 复制新文件或目录到容器中
COPY /usr/src/causal-conv1d/build/lib.linux-x86_64-cpython-311/ /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 18:25:33  193.79MB 复制新文件或目录到容器中
COPY /usr/src/mamba/build/lib.linux-x86_64-cpython-311/ /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 18:25:33  34.44MB 复制新文件或目录到容器中
COPY /usr/src/lorax-punica/server/punica_kernels/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 18:25:33  9.79MB 复制新文件或目录到容器中
COPY /usr/src/llm-awq/awq/kernels/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 18:25:33  162.67MB 复制新文件或目录到容器中
COPY /usr/src/exllamav2/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 18:25:33  10.20MB 复制新文件或目录到容器中
COPY /usr/src/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 18:25:33  3.04MB 复制新文件或目录到容器中
COPY /usr/src/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 18:25:33  632.45MB 复制新文件或目录到容器中
COPY /usr/src/.venv/lib/python3.11/site-packages/flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 17:50:14  11.04MB 复制新文件或目录到容器中
COPY /usr/src/flash-attention/csrc/rotary/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 17:50:14  736.09MB 复制新文件或目录到容器中
COPY /usr/src/flash-attention/csrc/layer_norm/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 17:50:14  164.61MB 复制新文件或目录到容器中
COPY /usr/src/flash-attention/build/lib.linux-x86_64-cpython-311 /usr/src/.venv/lib/python3.11/site-packages # buildkit
                        
# 2025-03-18 17:40:33  5.72GB 执行命令并创建新的镜像层
RUN /bin/sh -c cd server &&     uv sync --frozen --extra gen --extra bnb --extra accelerate --extra compressed-tensors --extra quantize --extra peft --extra outlines --active --python=${PYTHON_VERSION} &&     uv pip install nvidia-nccl-cu12==2.25.1 &&     pwd &&     text-generation-server --help # buildkit
                        
# 2025-03-18 17:39:54  1.09GB 执行命令并创建新的镜像层
RUN /bin/sh -c cd server && 	uv sync --frozen --extra gen --extra bnb --extra accelerate --extra compressed-tensors --extra quantize --extra peft --extra outlines --no-install-project --active &&     make gen-server-raw &&     kernels download . # buildkit
                        
# 2025-03-18 23:00:23  0.00B 设置环境变量 HF_KERNELS_CACHE
ENV HF_KERNELS_CACHE=/kernels
                        
# 2025-03-18 23:00:23  0.00B 复制新文件或目录到容器中
COPY server/Makefile server/Makefile # buildkit
                        
# 2025-03-18 17:39:37  2.76MB 复制新文件或目录到容器中
COPY server server # buildkit
                        
# 2025-02-14 18:42:37  13.42KB 复制新文件或目录到容器中
COPY proto proto # buildkit
                        
# 2025-02-14 18:42:37  0.00B 设置环境变量 PATH
ENV PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/bin:/usr/src/.venv/bin/
                        
# 2025-02-14 18:42:37  0.00B 设置环境变量 VIRTUAL_ENV
ENV VIRTUAL_ENV=/usr/src/.venv/
                        
# 2025-02-14 18:42:37  61.55MB 执行命令并创建新的镜像层
RUN /bin/sh -c uv python install ${PYTHON_VERSION} # buildkit
                        
# 2025-02-14 18:42:36  0.00B 设置环境变量 PYTHON_VERSION
ENV PYTHON_VERSION=3.11
                        
# 2025-02-14 18:42:36  5.40GB 复制新文件或目录到容器中
COPY /usr/src/.venv /usr/src/.venv # buildkit
                        
# 2025-02-14 18:40:33  0.00B 设置环境变量 PATH
ENV PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/bin
                        
# 2025-02-14 18:40:33  39.85MB 执行命令并创建新的镜像层
RUN /bin/sh -c curl -LsSf https://astral.sh/uv/install.sh | sh # buildkit
                        
# 2025-02-14 18:40:32  85.15MB 执行命令并创建新的镜像层
RUN /bin/sh -c apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends         libssl-dev         ca-certificates         make         curl         git         && rm -rf /var/lib/apt/lists/* # buildkit
                        
# 2025-03-18 23:00:23  0.00B 设置工作目录为/usr/src
WORKDIR /usr/src
                        
# 2025-03-18 23:00:23  0.00B 设置环境变量 HF_HOME HF_HUB_ENABLE_HF_TRANSFER PORT
ENV HF_HOME=/data HF_HUB_ENABLE_HF_TRANSFER=1 PORT=80
                        
# 2024-04-05 07:36:23  0.00B 设置环境变量 NVIDIA_DRIVER_CAPABILITIES
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
                        
# 2024-04-05 07:36:23  0.00B 设置环境变量 NVIDIA_VISIBLE_DEVICES
ENV NVIDIA_VISIBLE_DEVICES=all
                        
# 2024-04-05 07:36:23  17.29KB 复制新文件或目录到容器中
COPY NGC-DL-CONTAINER-LICENSE / # buildkit
                        
# 2024-04-05 07:36:23  0.00B 设置环境变量 LD_LIBRARY_PATH
ENV LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64
                        
# 2024-04-05 07:36:23  0.00B 设置环境变量 PATH
ENV PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
                        
# 2024-04-05 07:36:23  46.00B 执行命令并创建新的镜像层
RUN |1 TARGETARCH=amd64 /bin/sh -c echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf     && echo "/usr/local/nvidia/lib64" >> /etc/ld.so.conf.d/nvidia.conf # buildkit
                        
# 2024-04-05 07:36:23  155.92MB 执行命令并创建新的镜像层
RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update && apt-get install -y --no-install-recommends     cuda-cudart-12-4=${NV_CUDA_CUDART_VERSION}     ${NV_CUDA_COMPAT_PACKAGE}     && rm -rf /var/lib/apt/lists/* # buildkit
                        
# 2024-04-05 07:36:11  0.00B 设置环境变量 CUDA_VERSION
ENV CUDA_VERSION=12.4.0
                        
# 2024-04-05 07:36:11  10.56MB 执行命令并创建新的镜像层
RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update && apt-get install -y --no-install-recommends     gnupg2 curl ca-certificates &&     curl -fsSLO https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/${NVARCH}/cuda-keyring_1.1-1_all.deb &&     dpkg -i cuda-keyring_1.1-1_all.deb &&     apt-get purge --autoremove -y curl     && rm -rf /var/lib/apt/lists/* # buildkit
                        
# 2024-04-05 07:36:11  0.00B 添加元数据标签
LABEL maintainer=NVIDIA CORPORATION <cudatools@nvidia.com>
                        
# 2024-04-05 07:36:11  0.00B 定义构建参数
ARG TARGETARCH
                        
# 2024-04-05 07:36:11  0.00B 设置环境变量 NV_CUDA_COMPAT_PACKAGE
ENV NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4
                        
# 2024-04-05 07:36:11  0.00B 设置环境变量 NV_CUDA_CUDART_VERSION
ENV NV_CUDA_CUDART_VERSION=12.4.99-1
                        
# 2024-04-05 07:36:11  0.00B 设置环境变量 NVIDIA_REQUIRE_CUDA brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand brand
ENV NVIDIA_REQUIRE_CUDA=cuda>=12.4 brand=tesla,driver>=470,driver<471 brand=unknown,driver>=470,driver<471 brand=nvidia,driver>=470,driver<471 brand=nvidiartx,driver>=470,driver<471 brand=geforce,driver>=470,driver<471 brand=geforcertx,driver>=470,driver<471 brand=quadro,driver>=470,driver<471 brand=quadrortx,driver>=470,driver<471 brand=titan,driver>=470,driver<471 brand=titanrtx,driver>=470,driver<471 brand=tesla,driver>=525,driver<526 brand=unknown,driver>=525,driver<526 brand=nvidia,driver>=525,driver<526 brand=nvidiartx,driver>=525,driver<526 brand=geforce,driver>=525,driver<526 brand=geforcertx,driver>=525,driver<526 brand=quadro,driver>=525,driver<526 brand=quadrortx,driver>=525,driver<526 brand=titan,driver>=525,driver<526 brand=titanrtx,driver>=525,driver<526 brand=tesla,driver>=535,driver<536 brand=unknown,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=geforce,driver>=535,driver<536 brand=geforcertx,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=titan,driver>=535,driver<536 brand=titanrtx,driver>=535,driver<536
                        
# 2024-04-05 07:36:11  0.00B 设置环境变量 NVARCH
ENV NVARCH=x86_64
                        
# 2024-02-28 02:52:59  0.00B 
/bin/sh -c #(nop)  CMD ["/bin/bash"]
                        
# 2024-02-28 02:52:58  77.86MB 
/bin/sh -c #(nop) ADD file:21c2e8d95909bec6f4acdaf4aed55b44ee13603681f93b152e423e3e6a4a207b in / 
                        
# 2024-02-28 02:52:57  0.00B 
/bin/sh -c #(nop)  LABEL org.opencontainers.image.version=22.04
                        
# 2024-02-28 02:52:57  0.00B 
/bin/sh -c #(nop)  LABEL org.opencontainers.image.ref.name=ubuntu
                        
# 2024-02-28 02:52:57  0.00B 
/bin/sh -c #(nop)  ARG LAUNCHPAD_BUILD_ARCH
                        
# 2024-02-28 02:52:57  0.00B 
/bin/sh -c #(nop)  ARG RELEASE

镜像信息

{
    "Id": "sha256:fc032b59f1dcbe4c4bb7895797e57dbb1b9fac9edabe06edd73e3b8b44c80334",
    "RepoTags": [
        "ghcr.io/huggingface/text-generation-inference:3.2.1",
        "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference:3.2.1"
    ],
    "RepoDigests": [
        "ghcr.io/huggingface/text-generation-inference@sha256:5f0caa6661f6b30681ff78704c84c2d7b43f529501fc91a700596cde4151c4e5",
        "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/ghcr.io/huggingface/text-generation-inference@sha256:c328913971e8a9e7a22cf0d58118e28d6ac3a5ddd9b53f84a44dc3ee61221b0f"
    ],
    "Parent": "",
    "Comment": "buildkit.dockerfile.v0",
    "Created": "2025-03-18T15:32:56.856777273Z",
    "Container": "",
    "ContainerConfig": null,
    "DockerVersion": "",
    "Author": "",
    "Config": {
        "Hostname": "",
        "Domainname": "",
        "User": "",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/root/.local/bin:/usr/src/.venv/bin/",
            "NVARCH=x86_64",
            "NVIDIA_REQUIRE_CUDA=cuda\u003e=12.4 brand=tesla,driver\u003e=470,driver\u003c471 brand=unknown,driver\u003e=470,driver\u003c471 brand=nvidia,driver\u003e=470,driver\u003c471 brand=nvidiartx,driver\u003e=470,driver\u003c471 brand=geforce,driver\u003e=470,driver\u003c471 brand=geforcertx,driver\u003e=470,driver\u003c471 brand=quadro,driver\u003e=470,driver\u003c471 brand=quadrortx,driver\u003e=470,driver\u003c471 brand=titan,driver\u003e=470,driver\u003c471 brand=titanrtx,driver\u003e=470,driver\u003c471 brand=tesla,driver\u003e=525,driver\u003c526 brand=unknown,driver\u003e=525,driver\u003c526 brand=nvidia,driver\u003e=525,driver\u003c526 brand=nvidiartx,driver\u003e=525,driver\u003c526 brand=geforce,driver\u003e=525,driver\u003c526 brand=geforcertx,driver\u003e=525,driver\u003c526 brand=quadro,driver\u003e=525,driver\u003c526 brand=quadrortx,driver\u003e=525,driver\u003c526 brand=titan,driver\u003e=525,driver\u003c526 brand=titanrtx,driver\u003e=525,driver\u003c526 brand=tesla,driver\u003e=535,driver\u003c536 brand=unknown,driver\u003e=535,driver\u003c536 brand=nvidia,driver\u003e=535,driver\u003c536 brand=nvidiartx,driver\u003e=535,driver\u003c536 brand=geforce,driver\u003e=535,driver\u003c536 brand=geforcertx,driver\u003e=535,driver\u003c536 brand=quadro,driver\u003e=535,driver\u003c536 brand=quadrortx,driver\u003e=535,driver\u003c536 brand=titan,driver\u003e=535,driver\u003c536 brand=titanrtx,driver\u003e=535,driver\u003c536",
            "NV_CUDA_CUDART_VERSION=12.4.99-1",
            "NV_CUDA_COMPAT_PACKAGE=cuda-compat-12-4",
            "CUDA_VERSION=12.4.0",
            "LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/root/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/",
            "NVIDIA_VISIBLE_DEVICES=all",
            "NVIDIA_DRIVER_CAPABILITIES=compute,utility",
            "HF_HOME=/data",
            "HF_HUB_ENABLE_HF_TRANSFER=1",
            "PORT=80",
            "PYTHON_VERSION=3.11",
            "VIRTUAL_ENV=/usr/src/.venv/",
            "HF_KERNELS_CACHE=/kernels",
            "EXLLAMA_NO_FLASH_ATTN=1"
        ],
        "Cmd": null,
        "Image": "",
        "Volumes": null,
        "WorkingDir": "/usr/src",
        "Entrypoint": [
            "/tgi-entrypoint.sh"
        ],
        "OnBuild": null,
        "Labels": {
            "maintainer": "NVIDIA CORPORATION \u003ccudatools@nvidia.com\u003e",
            "org.opencontainers.image.created": "2025-03-18T15:24:42.616Z",
            "org.opencontainers.image.description": "Large Language Model Text Generation Inference",
            "org.opencontainers.image.licenses": "Apache-2.0",
            "org.opencontainers.image.ref.name": "ubuntu",
            "org.opencontainers.image.revision": "4d28897b4e345f4dfdd93d3434e50ac8afcdf9e1",
            "org.opencontainers.image.source": "https://github.com/huggingface/text-generation-inference",
            "org.opencontainers.image.title": "text-generation-inference",
            "org.opencontainers.image.url": "https://github.com/huggingface/text-generation-inference",
            "org.opencontainers.image.version": "3.2.1"
        }
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Size": 16215134681,
    "GraphDriver": {
        "Data": {
            "LowerDir": "/var/lib/docker/overlay2/12532b06c139de8ee980c45ba3e5eabc4b9b0ae5efce0670c98745813d1d2ba5/diff:/var/lib/docker/overlay2/8e564fc4489a284f2a6bac36d4cdd3c211f24e17f2aff16a3ebb7a5fc74a0951/diff:/var/lib/docker/overlay2/b5a591f8c5a00366d6db2adc79c735ff3fbf701bda58d5d79aa5c6a4734d8ced/diff:/var/lib/docker/overlay2/54c95739027bcce1f821ca493988d6f967e51231684df65cc080986caa43edf0/diff:/var/lib/docker/overlay2/2e8fc219facb792acea2a79f66fd4c05652a76fa135ceaa45c0ca1f7c0194f86/diff:/var/lib/docker/overlay2/52209f09ff33797118f325ffacf938d80f5f6e316528b3a42d0193177724b746/diff:/var/lib/docker/overlay2/33411d1897ad0e441ec2cafdde84d6a7ddfdb837153112441bb6a1fbe65df0e7/diff:/var/lib/docker/overlay2/eb22efa45c3061ade468e75ad494b7febee14664feee388f1e429ec4ab21aeff/diff:/var/lib/docker/overlay2/492be1c56ae028fea5de97221a309cd2c58c1cd33eaf1f00e2d265429c27b1cd/diff:/var/lib/docker/overlay2/5ad58327140d719240df68d8b27696bcece066b40a1087f756437dc894657582/diff:/var/lib/docker/overlay2/ea60a6d60310b233c80ff3501844f1ddcc2c48f5b2f343077a401c5e15ec9609/diff:/var/lib/docker/overlay2/757c257bc519786c215d48fc1e8915566ec0e148412295a9a5b633027da4fbe1/diff:/var/lib/docker/overlay2/f151aa5823b79fe94203f3402bf0c6b00a3b16fb23f4e565c95d99ddb3c6f3db/diff:/var/lib/docker/overlay2/f946aa29f47558e8c34ae0b38b03dd7e1d1c9ea401a5bb1aa3b739ba94ca9568/diff:/var/lib/docker/overlay2/304d2837bfd8078565e6573ef69cbd2bb6c70afa955bbfc162cd980f65631113/diff:/var/lib/docker/overlay2/04dad169854d1139d4965ec421ec1270a8c92a74ef094436f75ab0485a460fb9/diff:/var/lib/docker/overlay2/7507bf86d9276ddc8b396c58e79860607af2d312206cde0154b6ad63b165414a/diff:/var/lib/docker/overlay2/7ed5beb93038e8d8de6ca614bc117311c7f1cf9410936e0954641f9cf95e1c71/diff:/var/lib/docker/overlay2/d356a19b18fd7b257b6c49076827cebc74109fac710267480749238a64e71830/diff:/var/lib/docker/overlay2/1bd8db91bfb17834427761d532d102854d2488ce67d7a70cc6489618e74d693d/diff:/var/lib/docker/overlay2/3911734bdee02682992f1cb89b3a4e577c8a7a71b7eb6ec4bdc1a224695aa794/diff:/var/lib/docker/overlay2/b365b0f26d59e054e5e0ad203350ca7ead7b533fb05a21a5862cda9c27db4943/diff:/var/lib/docker/overlay2/7898af1ac9b8c2b6c4d45b6efcaf234579168721a02817ded03a6f0b2448d2e9/diff:/var/lib/docker/overlay2/79c626b83a88462de973b20bb586f6ee670a8bddb4bd31ddfdf1b0e4daea6ba1/diff:/var/lib/docker/overlay2/99b3c2f7ee26f6f3259577fb3a5881d8e377369b9b7b401e71b8298c740147cc/diff:/var/lib/docker/overlay2/555c95404d1db799a142f6a6f93273846ac178a2dc4a14cf65b0226c4213c66d/diff:/var/lib/docker/overlay2/36b3b7936b67764d28d827c2de0e2f768db23bdcf276cf25836d010719e86c27/diff:/var/lib/docker/overlay2/c2079c0f5f03c39ddd131f56455d2769fc1a631cc6f8d2d5800f4a831d1b48ab/diff:/var/lib/docker/overlay2/7b0c185f4e11ff485e243113ca972fe72725fd26d27e72d39a6a53bdeb14ee4a/diff:/var/lib/docker/overlay2/2d5a5b6222fc65702b5e5f61e8029067e7ee38d2c01e3e6d321027c71191e0f0/diff:/var/lib/docker/overlay2/ce14a225036858de6e69e2ff23fff7851c8ea8117b4476138845abf52f3e0e36/diff:/var/lib/docker/overlay2/36afb5f3b33b3915eca7fc7803ea8c9687feff5bf1dafe376b931e39494e6100/diff",
            "MergedDir": "/var/lib/docker/overlay2/57c38ce12408bc84cff425b7e6862e3e062ac24c2144628e56484abbdae3d895/merged",
            "UpperDir": "/var/lib/docker/overlay2/57c38ce12408bc84cff425b7e6862e3e062ac24c2144628e56484abbdae3d895/diff",
            "WorkDir": "/var/lib/docker/overlay2/57c38ce12408bc84cff425b7e6862e3e062ac24c2144628e56484abbdae3d895/work"
        },
        "Name": "overlay2"
    },
    "RootFS": {
        "Type": "layers",
        "Layers": [
            "sha256:5498e8c22f6996f25ef193ee58617d5b37e2a96decf22e72de13c3b34e147591",
            "sha256:4cd4079525948900a02a5734090afd1f3e046fc940dc882c55efcaee0a252dd0",
            "sha256:022bf74291b27404b223ba9ee16a7f3fb067253df9c65e23dfb3339800b28dfa",
            "sha256:eeb5315df33c9e700b3b8b8a3cdd1cf11e13c9dd44bfd946e340573478303349",
            "sha256:e942261d196e5e686398e2326c033119112f910191143b0497f13f78c377fa03",
            "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
            "sha256:64ea17e826dc7e4d48595831d2b74eaedcfbe7714a06ae83e8e60a6999723989",
            "sha256:9e8e2d4f4b3936a6dffc5dddffa3fe7940ea7b897f50d0858743fd024fcf06d0",
            "sha256:0a0b13b3da868f7e1776af9cc80c3f0ce011c85cf001221a926f8b8011c9bab8",
            "sha256:97b94fb17f2ee27ca54c0b4fe4263d6cd2b9adcc0eaa25d0b6bf4b543aa5f666",
            "sha256:8b71c231750c8d396b7a7fda5af0fe14126f7d34473350f2682b6f5f6142cbb1",
            "sha256:5480d129150e33181ce700b193e399215126e681f98367fba45f6c0d13dfe458",
            "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
            "sha256:c68bf6c242b058cc66abbb4812917dbe3161460de7db76c1a779f07492ccf851",
            "sha256:0243404c9d5f3fcc34914990076ca5bee240e3ae338de66219e2026e1d4d6af6",
            "sha256:a1f7d748c97000e15ccdb0571a0e4b2ffa3c4ce5f1ad58947953296e1bb2bb2a",
            "sha256:a07c34da779c21a511cc55a31b3d6cd3e99453bc349a5c425f5f1ef09923652c",
            "sha256:4a4fc12408cd454f95df0d721ae5fe2cac7036bc9b8c3a7a2e6de2100eae6bc3",
            "sha256:389dd944a6e4b9fa47b0eef5a5f54e21fc8ca08f3954164214a83173c1467c96",
            "sha256:ae75ef89a5c646fdceee897b1c936bf493251d7eb46970936b450dd0acc81ed9",
            "sha256:b6eea970777fd54e40311a0e6a08191e3d7c0c8951c40dead4be809ae02541f0",
            "sha256:872c601fc431d446bfd646fdba52054ddabfa8acb89a1815e450ea3ec71ab4f8",
            "sha256:fe3e4a1c06c98623254f2057f8aa0362bc4657233e10b981351aa371763883ed",
            "sha256:6c5d398644c110cd7574dc9a194006ebd4485e9cc666fe82851b50868922120f",
            "sha256:d49e6a2269c8f775a3ec25c51e400cba89f34dff8df8d869631b2d9f71e6c79b",
            "sha256:b2f3c1200de5973be14c2b69158a65ad1ff68287d44b227ce13fcb0ca9a5c728",
            "sha256:c0ad7fa1314a23e1037fa35b6755c46e87e05b9d43ee1a2db8d630b7bf15be64",
            "sha256:841444f95a59e2857fd2f3fc5fcab76b65699716ea5ecdd2d0743c803861e233",
            "sha256:8a04ffc0d3d4dd4c26cf72b5898c8214c3654ab8c418b8893a071e3f14e458f5",
            "sha256:0a1db9e3b6a0e57fef024b53a6398dd1e479a1f5263298c50a2b10fd92bbdee0",
            "sha256:8ed5b1ed642dbb6ced1fb0c5e27bcac605bf9fdec405029b44ae6742a063c61c",
            "sha256:edd608e60c2bfc92a9b2a0f20a2441b02ddbab5b3c97ee0c9703254830158dca",
            "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef"
        ]
    },
    "Metadata": {
        "LastTagTime": "2025-03-25T00:46:10.329525668+08:00"
    }
}

更多版本

ghcr.io/huggingface/text-generation-inference:2.1.1

linux/amd64 ghcr.io10.66GB2024-09-07 04:52

169

ghcr.io/huggingface/text-generation-inference:2.2

linux/amd64 ghcr.io11.37GB2024-09-07 05:20

369

ghcr.io/huggingface/text-generation-inference:2.3.0

linux/amd64 ghcr.io13.75GB2024-09-23 15:50