docker.io/spark:latest linux/amd64

docker.io/spark:latest - 国内下载镜像源 浏览次数:92

温馨提示:此镜像为latest tag镜像，本站无法保证此版本为最新镜像

Apache Spark 是一个快速、通用的、分布式内存处理引擎。

Spark Docker 镜像

这是 Apache Spark 的 Docker 镜像，提供了一个完整的 Spark 集群环境。

  • 基于 OpenJDK 11 和 Java 8
  • 包含 Scala 2.12 和 Python 3.x 运行时
  • 支持 HDFS、HBase 和 Cassandra 等存储系统
  • 可以通过 Docker Compose 或者 Kubernetes 部署
源镜像 docker.io/spark:latest
国内镜像 swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark:latest
镜像ID sha256:595dab9bdaf9d36f7e2e127bcb543f863e4440b24f8887aecb2470eec80458de
镜像TAG latest
大小 984.32MB
镜像源 docker.io
项目信息 Docker-Hub主页 🚀项目TAG 🚀
CMD
启动入口 /opt/entrypoint.sh
工作目录 /opt/spark/work-dir
OS/平台 linux/amd64
浏览量 92 次
贡献者
镜像创建 2024-08-12T09:09:28Z
同步时间 2024-09-25 17:19
更新时间 2024-10-23 14:12
环境变量
PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin JAVA_HOME=/opt/java/openjdk LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8 JAVA_VERSION=jdk-11.0.24+8 SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.5.2/spark-3.5.2-bin-hadoop3.tgz SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.5.2/spark-3.5.2-bin-hadoop3.tgz.asc GPG_KEY=D76E23B9F11B5BF6864613C4F7051850A0AF904D SPARK_HOME=/opt/spark
镜像标签
ubuntu: org.opencontainers.image.ref.name 20.04: org.opencontainers.image.version

Docker拉取命令 无权限下载?点我修复 

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark:latest
docker tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark:latest  docker.io/spark:latest

Containerd拉取命令 

ctr images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark:latest
ctr images tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark:latest  docker.io/spark:latest

Shell快速替换命令 

sed -i 's#spark:latest#swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark:latest#' deployment.yaml

镜像历史

大小 创建时间 层信息
0.00B 2024-08-12 17:09:28 USER spark
303.03MB 2024-08-12 17:09:28 RUN /bin/sh -c set -ex; apt-get update; apt-get install -y python3 python3-pip; rm -rf /var/lib/apt/lists/* # buildkit
0.00B 2024-08-12 17:09:28 USER root
0.00B 2024-08-12 17:09:28 ENTRYPOINT ["/opt/entrypoint.sh"]
0.00B 2024-08-12 17:09:28 USER spark
0.00B 2024-08-12 17:09:28 WORKDIR /opt/spark/work-dir
0.00B 2024-08-12 17:09:28 ENV SPARK_HOME=/opt/spark
4.74KB 2024-08-12 17:09:28 COPY entrypoint.sh /opt/ # buildkit
361.74MB 2024-08-12 17:09:28 RUN |1 spark_uid=185 /bin/sh -c set -ex; export SPARK_TMP="$(mktemp -d)"; cd $SPARK_TMP; wget -nv -O spark.tgz "$SPARK_TGZ_URL"; wget -nv -O spark.tgz.asc "$SPARK_TGZ_ASC_URL"; export GNUPGHOME="$(mktemp -d)"; gpg --batch --keyserver hkps://keys.openpgp.org --recv-key "$GPG_KEY" || gpg --batch --keyserver hkps://keyserver.ubuntu.com --recv-keys "$GPG_KEY"; gpg --batch --verify spark.tgz.asc spark.tgz; gpgconf --kill all; rm -rf "$GNUPGHOME" spark.tgz.asc; tar -xf spark.tgz --strip-components=1; chown -R spark:spark .; mv jars /opt/spark/; mv bin /opt/spark/; mv sbin /opt/spark/; mv kubernetes/dockerfiles/spark/decom.sh /opt/; mv examples /opt/spark/; mv kubernetes/tests /opt/spark/; mv data /opt/spark/; mv python/pyspark /opt/spark/python/pyspark/; mv python/lib /opt/spark/python/lib/; mv R /opt/spark/; chmod a+x /opt/decom.sh; cd ..; rm -rf "$SPARK_TMP"; # buildkit
0.00B 2024-08-12 17:09:28 ENV SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.5.2/spark-3.5.2-bin-hadoop3.tgz SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.5.2/spark-3.5.2-bin-hadoop3.tgz.asc GPG_KEY=D76E23B9F11B5BF6864613C4F7051850A0AF904D
59.91MB 2024-08-12 17:09:28 RUN |1 spark_uid=185 /bin/sh -c set -ex; apt-get update; apt-get install -y gnupg2 wget bash tini libc6 libpam-modules krb5-user libnss3 procps net-tools gosu libnss-wrapper; mkdir -p /opt/spark; mkdir /opt/spark/python; mkdir -p /opt/spark/examples; mkdir -p /opt/spark/work-dir; chmod g+w /opt/spark/work-dir; touch /opt/spark/RELEASE; chown -R spark:spark /opt/spark; echo "auth required pam_wheel.so use_uid" >> /etc/pam.d/su; rm -rf /var/lib/apt/lists/* # buildkit
64.84KB 2024-08-12 17:09:28 RUN |1 spark_uid=185 /bin/sh -c groupadd --system --gid=${spark_uid} spark && useradd --system --uid=${spark_uid} --gid=spark spark # buildkit
0.00B 2024-08-12 17:09:28 ARG spark_uid=185
0.00B 2024-08-12 17:09:28 ENTRYPOINT ["/__cacert_entrypoint.sh"]
4.74KB 2024-08-12 17:09:28 COPY --chmod=755 entrypoint.sh /__cacert_entrypoint.sh # buildkit
0.00B 2024-08-12 17:09:28 RUN /bin/sh -c set -eux; echo "Verifying install ..."; echo "java --version"; java --version; echo "Complete." # buildkit
140.96MB 2024-08-12 17:09:28 RUN /bin/sh -c set -eux; ARCH="$(dpkg --print-architecture)"; case "${ARCH}" in amd64) ESUM='e0c1938093da3780e4494d366a4e6b75584dde8d46a19acea6691ae11df4cda5'; BINARY_URL='https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.24%2B8/OpenJDK11U-jre_x64_linux_hotspot_11.0.24_8.tar.gz'; ;; arm64) ESUM='1fe97cdaad47d7d108f329c6e4560b46748ef7f2948a1027812ade0bbc2a3597'; BINARY_URL='https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.24%2B8/OpenJDK11U-jre_aarch64_linux_hotspot_11.0.24_8.tar.gz'; ;; armhf) ESUM='bf893085627c6ec484e63aa1290276b23bcfee547459da6b0432ae9c5c1be22a'; BINARY_URL='https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.24%2B8/OpenJDK11U-jre_arm_linux_hotspot_11.0.24_8.tar.gz'; ;; ppc64el) ESUM='8ee351314182df93fbad96139bb74b97814944d66197896e388404a1ecfa06b3'; BINARY_URL='https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.24%2B8/OpenJDK11U-jre_ppc64le_linux_hotspot_11.0.24_8.tar.gz'; ;; s390x) ESUM='5b331f093bb03126334bbbc24f05f60681baeda461d860e4e2cdb693ee54e0ed'; BINARY_URL='https://github.com/adoptium/temurin11-binaries/releases/download/jdk-11.0.24%2B8/OpenJDK11U-jre_s390x_linux_hotspot_11.0.24_8.tar.gz'; ;; *) echo "Unsupported arch: ${ARCH}"; exit 1; ;; esac; wget --progress=dot:giga -O /tmp/openjdk.tar.gz ${BINARY_URL}; echo "${ESUM} */tmp/openjdk.tar.gz" | sha256sum -c -; mkdir -p "$JAVA_HOME"; tar --extract --file /tmp/openjdk.tar.gz --directory "$JAVA_HOME" --strip-components 1 --no-same-owner ; rm -f /tmp/openjdk.tar.gz ${JAVA_HOME}/lib/src.zip; find "$JAVA_HOME/lib" -name '*.so' -exec dirname '{}' ';' | sort -u > /etc/ld.so.conf.d/docker-openjdk.conf; ldconfig; java -Xshare:dump; # buildkit
0.00B 2024-08-12 17:09:28 ENV JAVA_VERSION=jdk-11.0.24+8
45.80MB 2024-08-12 17:09:28 RUN /bin/sh -c set -eux; apt-get update; DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends curl wget fontconfig ca-certificates p11-kit tzdata locales ; echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen; locale-gen en_US.UTF-8; rm -rf /var/lib/apt/lists/* # buildkit
0.00B 2024-08-12 17:09:28 ENV LANG=en_US.UTF-8 LANGUAGE=en_US:en LC_ALL=en_US.UTF-8
0.00B 2024-08-12 17:09:28 ENV PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
0.00B 2024-08-12 17:09:28 ENV JAVA_HOME=/opt/java/openjdk
0.00B 2024-08-12 17:09:28 /bin/sh -c #(nop) CMD ["/bin/bash"]
72.81MB 2024-08-12 17:09:28 /bin/sh -c #(nop) ADD file:e7cff353f027ecf0a2cb1cdd51714de3b083a11a0d965f104489f9a7e6925056 in /
0.00B 2024-08-12 17:09:28 /bin/sh -c #(nop) LABEL org.opencontainers.image.version=20.04
0.00B 2024-08-12 17:09:28 /bin/sh -c #(nop) LABEL org.opencontainers.image.ref.name=ubuntu
0.00B 2024-08-12 17:09:28 /bin/sh -c #(nop) ARG LAUNCHPAD_BUILD_ARCH
0.00B 2024-08-12 17:09:28 /bin/sh -c #(nop) ARG RELEASE

镜像信息 

{
    "Id": "sha256:595dab9bdaf9d36f7e2e127bcb543f863e4440b24f8887aecb2470eec80458de",
    "RepoTags": [
        "spark:latest",
        "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark:latest"
    ],
    "RepoDigests": [
        "spark@sha256:fe675ae4606f8dd7a22a5d261c544f2a2af3678d060bfd910619d3c1173a0774",
        "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/spark@sha256:e617a3c4c2cb65c906729d82cc01180a4d9800378f71e4ad4591e5fc73f5f505"
    ],
    "Parent": "",
    "Comment": "buildkit.dockerfile.v0",
    "Created": "2024-08-12T09:09:28Z",
    "Container": "",
    "ContainerConfig": null,
    "DockerVersion": "",
    "Author": "",
    "Config": {
        "Hostname": "",
        "Domainname": "",
        "User": "spark",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "JAVA_HOME=/opt/java/openjdk",
            "LANG=en_US.UTF-8",
            "LANGUAGE=en_US:en",
            "LC_ALL=en_US.UTF-8",
            "JAVA_VERSION=jdk-11.0.24+8",
            "SPARK_TGZ_URL=https://archive.apache.org/dist/spark/spark-3.5.2/spark-3.5.2-bin-hadoop3.tgz",
            "SPARK_TGZ_ASC_URL=https://archive.apache.org/dist/spark/spark-3.5.2/spark-3.5.2-bin-hadoop3.tgz.asc",
            "GPG_KEY=D76E23B9F11B5BF6864613C4F7051850A0AF904D",
            "SPARK_HOME=/opt/spark"
        ],
        "Cmd": null,
        "Image": "",
        "Volumes": null,
        "WorkingDir": "/opt/spark/work-dir",
        "Entrypoint": [
            "/opt/entrypoint.sh"
        ],
        "OnBuild": null,
        "Labels": {
            "org.opencontainers.image.ref.name": "ubuntu",
            "org.opencontainers.image.version": "20.04"
        }
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Size": 984322619,
    "GraphDriver": {
        "Data": {
            "LowerDir": "/var/lib/docker/overlay2/e544297a99531065632cce8aa3e37f3b01da879b68a2faf46221193cbc328baa/diff:/var/lib/docker/overlay2/3ea20322e6314701aa0c1e2a5bb274b90ca8e06b30436a62256355b9679c1076/diff:/var/lib/docker/overlay2/cbef94a2b2fad77b0652ae05dd26a7de37a375a56304b8851100958117e1fb0c/diff:/var/lib/docker/overlay2/75cfd6bc2fc370497ae6be90600dfd7101746e5cda4508870bd7c2ff9732cda5/diff:/var/lib/docker/overlay2/d80e520eb18813ad0d978c026d23d66283274e906127f40a8a4bc79399d16b76/diff:/var/lib/docker/overlay2/c901f9033188e728bd32c1775996380edc6e4d26e685cb007870eee40654e910/diff:/var/lib/docker/overlay2/b23409f12e66219dcc2564deb40729639ebf4fdfa3975f525b2eaae23c7f3b76/diff:/var/lib/docker/overlay2/cb2cd5ea16c92ff97522fc7a6a782bcdee5f49bca94e784d1d079548bef666a1/diff:/var/lib/docker/overlay2/201451fb0875aadf9a588fc82b2e1ab0d088a738d1bd0ab7c596fe49ca5be0d0/diff:/var/lib/docker/overlay2/65a71f3d6fa2d3ed5132e17de9d09a24ccaae8dac76a730d0389a0d449b96e0f/diff",
            "MergedDir": "/var/lib/docker/overlay2/aa8c3640d4662a1b1f21b6a235b34346245e14c7ff1af3fbb9b591db63a453c0/merged",
            "UpperDir": "/var/lib/docker/overlay2/aa8c3640d4662a1b1f21b6a235b34346245e14c7ff1af3fbb9b591db63a453c0/diff",
            "WorkDir": "/var/lib/docker/overlay2/aa8c3640d4662a1b1f21b6a235b34346245e14c7ff1af3fbb9b591db63a453c0/work"
        },
        "Name": "overlay2"
    },
    "RootFS": {
        "Type": "layers",
        "Layers": [
            "sha256:3ec3ded77c0ce89e931f92aed086b2a2c774a6fbd51617853decc8afa4e1087a",
            "sha256:8cdab93a842f1ebf4bfcd2da131a8b77a3d34a77c27fc239e0c0ef1803705f6d",
            "sha256:2f30f6654b4ec617de7f4513be9a6bbbb91f18194b0481496a76b04c3e112590",
            "sha256:4a5ab425d6e141aeea4423031c339b6b9a735ecb5f2d55952cc6704cf9ed8b3c",
            "sha256:b2650c6d730e3b3e60b27ee239244e841d56c85165215f2f557f2fae3d772276",
            "sha256:1467e36f28f8591d34563f2930f833d5c8528ffbc6c63653928d02e4986d5be9",
            "sha256:46baf3a0b9e1e132cd18b752f5b42d79b90347f6e815bf8b881f71456c31ce05",
            "sha256:a0e90269d992f7f1101584dac015029c0e8ef8c47635d8cbbe7802f320278f91",
            "sha256:899ae76fe3721825be5df54af2c2622acea60baedd88c7649932b88321273f3a",
            "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
            "sha256:304142e61c3b118428dfee9cc052945d610140c25a99883f2099f20b07564d1f"
        ]
    },
    "Metadata": {
        "LastTagTime": "2024-09-25T17:17:51.434595616+08:00"
    }
}

更多版本