docker.io/s1mplecc/spark-hadoop:3 linux/amd64

docker.io/s1mplecc/spark-hadoop:3 - 国内下载镜像源 浏览次数:77
## Docker镜像描述信息 ### Docker镜像基本描述 * **名称:** docker.io/s1mplecc/spark-hadoop * **描述:** 本镜像为Apache Spark和Hadoop集成的镜像，提供了一个可以直接运行Spark应用程序的环境。 * **包含功能:** + Apache Spark：支持多种编程模型，如Scala、Python等 + Hadoop：提供了一个分布式数据处理框架 * **使用场景:** + 大数据分析和处理 + 实时计算和流式处理 + 批处理和离线计算
源镜像 docker.io/s1mplecc/spark-hadoop:3
国内镜像 swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3
镜像ID sha256:52386000b26894054e2ff26525e8c6c4b5d78868495d7d31d792175a2b7ccfcc
镜像TAG 3
大小 2.91GB
镜像源 docker.io
项目信息 Docker-Hub主页 🚀项目TAG 🚀
CMD /opt/bitnami/scripts/spark/run.sh
启动入口 /opt/bitnami/scripts/spark/entrypoint.sh
工作目录 /opt
OS/平台 linux/amd64
浏览量 77 次
贡献者
镜像创建 2021-12-24T08:54:40.286535815Z
同步时间 2024-10-16 12:28
更新时间 2024-11-22 17:29
环境变量
PATH=/opt/hadoop/hadoop/sbin:/opt/hadoop/bin:/opt/bitnami/python/bin:/opt/bitnami/java/bin:/opt/bitnami/spark/bin:/opt/bitnami/spark/sbin:/opt/bitnami/common/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOME=/ OS_ARCH=amd64 OS_FLAVOUR=debian-10 OS_NAME=linux BITNAMI_APP_NAME=spark BITNAMI_IMAGE_VERSION=3.2.0-debian-10-r54 JAVA_HOME=/opt/bitnami/java LD_LIBRARY_PATH=/opt/bitnami/python/lib/:/opt/bitnami/spark/venv/lib/python3.8/site-packages/numpy.libs/: LIBNSS_WRAPPER_PATH=/opt/bitnami/common/lib/libnss_wrapper.so NSS_WRAPPER_GROUP=/opt/bitnami/spark/tmp/nss_group NSS_WRAPPER_PASSWD=/opt/bitnami/spark/tmp/nss_passwd SPARK_HOME=/opt/bitnami/spark HADOOP_HOME=/opt/hadoop HADOOP_CONF_DIR=/opt/hadoop/etc/hadoop HADOOP_LOG_DIR=/var/log/hadoop
镜像标签
Docker image with Spark (3.1.2) and Hadoop (3.2.0), based on bitnami/spark:3. For more information, please visit https://github.com/s1mplecc/spark-hadoop-docker.: description s1mplecc <s1mple951205@gmail.com>: maintainer
镜像安全扫描 查看Trivy扫描报告

系统OS: debian 10.11 扫描引擎: Trivy 扫描时间: 2024-10-27 20:29

低危漏洞:241 中危漏洞:340 高危漏洞:661 严重漏洞:252
docker.io/s1mplecc/spark-hadoop:3 (debian 10.11) (debian) Java (jar) Python (python-pkg) opt/bitnami/common/bin/gosu (gobinary) /etc/ssh/ssh_host_rsa_key () /root/.ssh/id_rsa () /etc/ssh/ssh_host_ecdsa_key () /etc/ssh/ssh_host_ed25519_key ()

Docker拉取命令 无权限下载?点我修复 

docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3
docker tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3  docker.io/s1mplecc/spark-hadoop:3

Containerd拉取命令 

ctr images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3
ctr images tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3  docker.io/s1mplecc/spark-hadoop:3

Shell快速替换命令 

sed -i 's#s1mplecc/spark-hadoop:3#swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3#' deployment.yaml

Ansible快速分发-Docker 

#ansible k8s -m shell -a 'docker pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3 && docker tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3  docker.io/s1mplecc/spark-hadoop:3'

Ansible快速分发-Containerd 

#ansible k8s -m shell -a 'ctr images pull swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3 && ctr images tag  swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3  docker.io/s1mplecc/spark-hadoop:3'

镜像历史

大小 创建时间 层信息
0.00B 2021-12-24 16:54:40 CMD ["/opt/bitnami/scripts/spark/run.sh"]
0.00B 2021-12-24 16:54:40 ENTRYPOINT ["/opt/bitnami/scripts/spark/entrypoint.sh"]
785.00B 2021-12-24 16:54:40 RUN /bin/sh -c sed -i '1 a /etc/init.d/ssh start > /dev/null &' /opt/bitnami/scripts/spark/entrypoint.sh # buildkit
678.00B 2021-12-24 16:34:38 RUN /bin/sh -c hdfs namenode -format # buildkit
8.61KB 2021-12-24 16:34:36 RUN /bin/sh -c chmod +x /opt/start-hadoop.sh && chmod +x $HADOOP_HOME/sbin/start-dfs.sh && chmod +x $HADOOP_HOME/sbin/start-yarn.sh # buildkit
94.00B 2021-12-24 16:23:26 COPY start-hadoop.sh /opt/start-hadoop.sh # buildkit
22.50KB 2021-12-24 16:23:26 RUN /bin/sh -c mv /tmp/ssh_config /root/.ssh/config && mv /tmp/hadoop-env.sh $HADOOP_CONF_DIR/hadoop-env.sh && mv /tmp/hdfs-site.xml $HADOOP_CONF_DIR/hdfs-site.xml && mv /tmp/core-site.xml $HADOOP_CONF_DIR/core-site.xml && mv /tmp/mapred-site.xml $HADOOP_CONF_DIR/mapred-site.xml && mv /tmp/yarn-site.xml $HADOOP_CONF_DIR/yarn-site.xml && mv /tmp/workers $HADOOP_CONF_DIR/workers # buildkit
28.65KB 2021-12-24 16:23:26 COPY config/* /tmp/ # buildkit
0.00B 2021-12-24 16:07:14 RUN /bin/sh -c mkdir -p /root/hdfs/namenode && mkdir -p /root/hdfs/datanode # buildkit
858.26MB 2021-12-24 16:07:14 RUN /bin/sh -c tar -xzvf hadoop-3.2.0.tar.gz && mv hadoop-3.2.0 hadoop && rm -rf hadoop-3.2.0.tar.gz && mkdir /var/log/hadoop # buildkit
345.63MB 2021-12-24 16:07:01 RUN /bin/sh -c curl -OL https://archive.apache.org/dist/hadoop/common/hadoop-3.2.0/hadoop-3.2.0.tar.gz # buildkit
2.63KB 2021-12-24 16:02:24 RUN /bin/sh -c ssh-keygen -t rsa -f /root/.ssh/id_rsa -P '' && cat /root/.ssh/id_rsa.pub >> /root/.ssh/authorized_keys # buildkit
49.51MB 2021-12-24 16:02:24 RUN /bin/sh -c apt-get update && apt-get install -y openssh-server # buildkit
0.00B 2021-12-24 15:50:00 WORKDIR /opt
0.00B 2021-12-24 15:50:00 ENV PATH=/opt/hadoop/hadoop/sbin:/opt/hadoop/bin:/opt/bitnami/python/bin:/opt/bitnami/java/bin:/opt/bitnami/spark/bin:/opt/bitnami/spark/sbin:/opt/bitnami/common/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
0.00B 2021-12-24 15:50:00 ENV HADOOP_LOG_DIR=/var/log/hadoop
0.00B 2021-12-24 15:50:00 ENV HADOOP_CONF_DIR=/opt/hadoop/etc/hadoop
0.00B 2021-12-24 15:50:00 ENV HADOOP_HOME=/opt/hadoop
0.00B 2021-12-24 15:50:00 USER root
0.00B 2021-12-24 15:50:00 LABEL description=Docker image with Spark (3.1.2) and Hadoop (3.2.0), based on bitnami/spark:3. For more information, please visit https://github.com/s1mplecc/spark-hadoop-docker.
0.00B 2021-12-24 15:50:00 LABEL maintainer=s1mplecc <s1mple951205@gmail.com>
0.00B 2021-12-23 18:36:20 /bin/sh -c #(nop) CMD ["/opt/bitnami/scripts/spark/run.sh"]
0.00B 2021-12-23 18:36:20 /bin/sh -c #(nop) ENTRYPOINT ["/opt/bitnami/scripts/spark/entrypoint.sh"]
0.00B 2021-12-23 18:36:20 /bin/sh -c #(nop) USER 1001
0.00B 2021-12-23 18:36:19 /bin/sh -c #(nop) WORKDIR /opt/bitnami/spark
0.00B 2021-12-23 18:36:19 /bin/sh -c #(nop) ENV BITNAMI_APP_NAME=spark BITNAMI_IMAGE_VERSION=3.2.0-debian-10-r54 JAVA_HOME=/opt/bitnami/java LD_LIBRARY_PATH=/opt/bitnami/python/lib/:/opt/bitnami/spark/venv/lib/python3.8/site-packages/numpy.libs/: LIBNSS_WRAPPER_PATH=/opt/bitnami/common/lib/libnss_wrapper.so NSS_WRAPPER_GROUP=/opt/bitnami/spark/tmp/nss_group NSS_WRAPPER_PASSWD=/opt/bitnami/spark/tmp/nss_passwd SPARK_HOME=/opt/bitnami/spark
0.00B 2021-12-23 18:36:19 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c /opt/bitnami/scripts/java/postunpack.sh
494.53MB 2021-12-23 18:36:18 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c /opt/bitnami/scripts/spark/postunpack.sh
13.89KB 2021-12-23 18:36:11 /bin/sh -c #(nop) COPY dir:bb778896e5c85acbfd8ac3a5350fb463771f82e694c9b837e595acdcc78e5ddc in /
0.00B 2021-12-23 18:36:10 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c chmod g+rwX /opt/bitnami
2.30MB 2021-12-23 18:36:10 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c . /opt/bitnami/scripts/libcomponent.sh && component_unpack "gosu" "1.14.0-0" --checksum 3e6fc37ca073b10a73a804d39c2f0c028947a1a596382a4f8ebe43dfbaa3a25e
735.76MB 2021-12-23 18:36:09 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c . /opt/bitnami/scripts/libcomponent.sh && component_unpack "spark" "3.2.0-3" --checksum 311b3644dc49a0965f49eb046bd4c0ccf3d37e941f0ac7f678bd5a85b0112e05
210.82MB 2021-12-23 18:35:53 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c . /opt/bitnami/scripts/libcomponent.sh && component_unpack "java" "1.8.312-0" --checksum a7e034898281dff05591e74de285ecd69899ddaaff4ce1ea9c09556ac89c9c72
133.52MB 2021-12-23 18:35:48 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c . /opt/bitnami/scripts/libcomponent.sh && component_unpack "python" "3.8.12-5" --checksum 18aaf8247baa258f4e16de86673a3e799cd50bbef2824f0d2ae805c9c2068f11
15.99MB 2021-12-23 18:35:44 |1 JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security /bin/sh -c install_packages acl ca-certificates curl gzip libbz2-1.0 libc6 libffi6 libgcc1 liblzma5 libncursesw6 libreadline7 libsqlite3-0 libssl1.1 libstdc++6 libtinfo6 procps tar zlib1g
65.67KB 2021-12-23 18:35:38 /bin/sh -c #(nop) COPY dir:5524866e4ec8c9ef8bcd410d2c2146f08d3454f7079fe80b957968ee46b1de0a in /
0.00B 2021-12-23 18:35:38 /bin/sh -c #(nop) ARG JAVA_EXTRA_SECURITY_DIR=/bitnami/java/extra-security
0.00B 2021-12-23 18:35:38 /bin/sh -c #(nop) ENV HOME=/ OS_ARCH=amd64 OS_FLAVOUR=debian-10 OS_NAME=linux PATH=/opt/bitnami/python/bin:/opt/bitnami/java/bin:/opt/bitnami/spark/bin:/opt/bitnami/spark/sbin:/opt/bitnami/common/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
0.00B 2021-12-23 18:35:37 /bin/sh -c #(nop) LABEL maintainer=Bitnami <containers@bitnami.com>
67.51MB 2021-12-15 08:07:19

镜像信息 

{
    "Id": "sha256:52386000b26894054e2ff26525e8c6c4b5d78868495d7d31d792175a2b7ccfcc",
    "RepoTags": [
        "s1mplecc/spark-hadoop:3",
        "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop:3"
    ],
    "RepoDigests": [
        "s1mplecc/spark-hadoop@sha256:abe419489019c720baab1aa944c40072a276055ef9db356d10a45c14a9c242e9",
        "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/s1mplecc/spark-hadoop@sha256:abe419489019c720baab1aa944c40072a276055ef9db356d10a45c14a9c242e9"
    ],
    "Parent": "",
    "Comment": "buildkit.dockerfile.v0",
    "Created": "2021-12-24T08:54:40.286535815Z",
    "Container": "",
    "ContainerConfig": null,
    "DockerVersion": "",
    "Author": "",
    "Config": {
        "Hostname": "",
        "Domainname": "",
        "User": "root",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "PATH=/opt/hadoop/hadoop/sbin:/opt/hadoop/bin:/opt/bitnami/python/bin:/opt/bitnami/java/bin:/opt/bitnami/spark/bin:/opt/bitnami/spark/sbin:/opt/bitnami/common/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "HOME=/",
            "OS_ARCH=amd64",
            "OS_FLAVOUR=debian-10",
            "OS_NAME=linux",
            "BITNAMI_APP_NAME=spark",
            "BITNAMI_IMAGE_VERSION=3.2.0-debian-10-r54",
            "JAVA_HOME=/opt/bitnami/java",
            "LD_LIBRARY_PATH=/opt/bitnami/python/lib/:/opt/bitnami/spark/venv/lib/python3.8/site-packages/numpy.libs/:",
            "LIBNSS_WRAPPER_PATH=/opt/bitnami/common/lib/libnss_wrapper.so",
            "NSS_WRAPPER_GROUP=/opt/bitnami/spark/tmp/nss_group",
            "NSS_WRAPPER_PASSWD=/opt/bitnami/spark/tmp/nss_passwd",
            "SPARK_HOME=/opt/bitnami/spark",
            "HADOOP_HOME=/opt/hadoop",
            "HADOOP_CONF_DIR=/opt/hadoop/etc/hadoop",
            "HADOOP_LOG_DIR=/var/log/hadoop"
        ],
        "Cmd": [
            "/opt/bitnami/scripts/spark/run.sh"
        ],
        "ArgsEscaped": true,
        "Image": "",
        "Volumes": null,
        "WorkingDir": "/opt",
        "Entrypoint": [
            "/opt/bitnami/scripts/spark/entrypoint.sh"
        ],
        "OnBuild": null,
        "Labels": {
            "description": "Docker image with Spark (3.1.2) and Hadoop (3.2.0), based on bitnami/spark:3. For more information, please visit https://github.com/s1mplecc/spark-hadoop-docker.",
            "maintainer": "s1mplecc \u003cs1mple951205@gmail.com\u003e"
        }
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Size": 2913980838,
    "GraphDriver": {
        "Data": {
            "LowerDir": "/var/lib/docker/overlay2/6be73b3fdc69ee80e5b722838686f24915a31ecde0c7619e0b294f2052d67215/diff:/var/lib/docker/overlay2/be3a829eee9b0ebd59810c6f9a0e749c79ccff76341388219dc6064f94334f48/diff:/var/lib/docker/overlay2/c3e5f95f0855e4f6789ef4cb7f5c71253c2f4041791d5e5f2d0d430caf5f87ae/diff:/var/lib/docker/overlay2/53bdc5d9170426dd1ef58d4a839b8e3e0cad1e81140ea64edcda15a8b73a7270/diff:/var/lib/docker/overlay2/48ebce1fde22d5ff749e358ef1215f864d56b678d3a1c899443f6b5edc965ca4/diff:/var/lib/docker/overlay2/2e4c3f6799c13a713cac921d99c85f432b28deaa7a3d97c4b82fa777737f0459/diff:/var/lib/docker/overlay2/168f59d622bdf0ec4d53efdc840fe04e5fc9595f564179b9691d49429248d7f8/diff:/var/lib/docker/overlay2/b00fb5300fcf1645a17410b75b9789a9d602a2daccc6701197ba967bbe20a675/diff:/var/lib/docker/overlay2/85a643ffbf687c938570d12d2869e16968587e878d12d1c0e4025dd52b6280d1/diff:/var/lib/docker/overlay2/3bd21b06bc257b5cbeba6f2da11ff8a137a438c423972f45491a34a4505dcc76/diff:/var/lib/docker/overlay2/7837161ca8c8cbfb04d721bfa36ef14ef0b1b5012e291f803e34e2e166365c4c/diff:/var/lib/docker/overlay2/20bc3fb459d555376d454edae5a91e6337e9b64ce9c9a2652921eb6c65be8e48/diff:/var/lib/docker/overlay2/5bbee98be3bb93e383d6d0372ef523bf85c60043fc547e6eb1f4503996cd8976/diff:/var/lib/docker/overlay2/2895a7d3abdc10bce20efa6af4b8d1fe6a3e707e7a54dccb048ca874eb433061/diff:/var/lib/docker/overlay2/87033cfc7ba146fd00172a09ddbfd84f6a95c336a0310b74eae1a0638fe79046/diff:/var/lib/docker/overlay2/b764b1080e5af88020b6f5549016dcea34090618a83fd5a5a722f956347564b2/diff:/var/lib/docker/overlay2/2ec0509bc8a80836ed51dc3453350a4091f4d9b262a103f57ce360246f8e7730/diff:/var/lib/docker/overlay2/51c97f7318731eae996ce4d9070f00e53cadac8e97f9e8495b52642b94d9b298/diff:/var/lib/docker/overlay2/94b71db461df4ded8ff6efa30974ddee24b2f175a26cd1201d94ca7986ff2e4b/diff:/var/lib/docker/overlay2/265398e5104322c72fe5738753cd67c5b2551cbb767e68c9fa348c5ed361e28c/diff:/var/lib/docker/overlay2/68bce41a9f7c7412c2463999408cd0821eb6673c17721517f9948a585dbf42c6/diff",
            "MergedDir": "/var/lib/docker/overlay2/6eb112eacd7c6416b63084bdd2303b123711bdab518340ae9e1908b817191b64/merged",
            "UpperDir": "/var/lib/docker/overlay2/6eb112eacd7c6416b63084bdd2303b123711bdab518340ae9e1908b817191b64/diff",
            "WorkDir": "/var/lib/docker/overlay2/6eb112eacd7c6416b63084bdd2303b123711bdab518340ae9e1908b817191b64/work"
        },
        "Name": "overlay2"
    },
    "RootFS": {
        "Type": "layers",
        "Layers": [
            "sha256:77281d05f1695b48636054c963cc40acbd5b040327ba0cd887a9d579817e0772",
            "sha256:2810ae202bca391df9908a7cec6dfcb9918364ca417e2b56419a265f0ae28aab",
            "sha256:63df53be8c264c25480623aa6fe501a0ca137f67557f1d3eafe26b3414a20762",
            "sha256:43234a7c1aeda6d4a6ebdd6bdd3490c55e4994ba362a457fead323d58d6d010a",
            "sha256:7b68fa14abd43b5fa333939ddee0dac2370598345c5bf5df1f7b559e9b3d6b57",
            "sha256:3f7779d38ca5635ccac170b7c728d5a423f3b22c7a8b13017a31f6123a35cdae",
            "sha256:7a955f50310d3409075b908eb7da1ea27b89568dac42d159f967d4b9798699cc",
            "sha256:7e3854e457060e96b24395279727d7fff8770976b4c7582ad86266cdc3e19be5",
            "sha256:8a01050b5e458e9a5cefd61e8bcdbf2e98eb982c0826a32420e2cc556b0cadc8",
            "sha256:3c5021f2880169e51a6c993319bd9179692942e18bfa55adeb7eaf1ad1c3de91",
            "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef",
            "sha256:b59a5308473e910c714c3fc01d931280d8fda7b66e76b03aea7633705e87b289",
            "sha256:9b7d94d1424bd273220f2ed998246ed133dc50582bbc635d285d3beddd3ca134",
            "sha256:c0a2e56f801b24761ef62c8fc229426a2c6d966cccc4d399db49165d4ef727d3",
            "sha256:1cc443b1ded3397d53423eb85d150157746a94b499995de5ee1bdee041bd92f1",
            "sha256:4c5cff6554e7e9e3f09553024c2521129a7ee88ebf23b29352da9bf0c4a5f2e5",
            "sha256:3b61ae1e1ca9219dab84f1da27bfb0d54db6c34c83a573a8664a08f5c3952ba3",
            "sha256:6e167367202b7295c27fd086408630a70f2e2123794e42aa339880db0953d819",
            "sha256:6722b6d4525d080c61f510f8ff884f35d57b53837d998d2c8e14a94c8abdeacb",
            "sha256:04f480452df053ecab75867565674b5eeffc50be298542c69d74d381ee1d5b81",
            "sha256:9d9ea3521f93df2a09012f3bbaf001f86130f874aab0f86f5e7d43cdcaf75260",
            "sha256:4f4366c7a7c94e0b071bd08bcb952051494964715b7b511020b7830fb2b9941c"
        ]
    },
    "Metadata": {
        "LastTagTime": "2024-10-16T12:23:56.58222821+08:00"
    }
}

更多版本

docker.io/s1mplecc/spark-hadoop:3

linux/amd64 docker.io2.91GB2024-10-16 12:28
76