TEZ-4682: [Cloud] Tez AM docker image#456
TEZ-4682: [Cloud] Tez AM docker image#456Aggarwal-Raghav wants to merge 3 commits intoapache:masterfrom
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
|
@abstractdog , I was able to start DagAppMaster with ZK on local. Attaching logs for the container docker_logs.txt But this PR has lot of open items and I need some advice on the following:
|
Aggarwal-Raghav
force-pushed
the
TEZ-4682
branch
from
ffceca5 to
856875a
Compare
This comment was marked as outdated.
This comment was marked as outdated.
very good, very good, let me check this in detail sometime this week, here are some pointers in the meantime, responding your questions:
there is no Yarn NodeManager in a k8s environment, so the reader of the entrypoint.sh should see a clear code distinguishing between needed env vars and legacy/backward-compatible env vars, that's what should be handled with care in my opinion
|
|
Thanks for the pointers @abstractdog .
Few additional things:
|
Aggarwal-Raghav
force-pushed
the
TEZ-4682
branch
from
856875a to
bfebf9e
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Aggarwal-Raghav
force-pushed
the
TEZ-4682
branch
from
c371557 to
9efce66
Compare
Using tez.local.mode=true, solves this as it will use |
This comment was marked as outdated.
This comment was marked as outdated.
Aggarwal-Raghav
changed the title
Aggarwal-Raghav
force-pushed
the
TEZ-4682
branch
from
9efce66 to
551075b
Compare
|
💔 -1 overall
This message was automatically generated. |
|
@abstractdog , Can you please help with review? |
let me get back to this next week |
sure |
|
💔 -1 overall
This message was automatically generated. |
tez-dist/src/docker/build-docker.sh
Outdated
| DIST_DIR=${DIST_DIR:-"$SCRIPT_DIR/../.."} | ||
| PROJECT_ROOT=${PROJECT_ROOT:-"$SCRIPT_DIR/../../.."} | ||
|
|
||
| repo=${REPO:-apache} |
There was a problem hiding this comment.
what about reusing REPO, I don't think we need a new assignment, instead:
REPO=${REPO:-apache}
tez-dist/src/docker/entrypoint.sh
Outdated
| find "${TEZ_CUSTOM_CONF_DIR}" -type f -exec \ | ||
| ln -sf {} "${TEZ_CONF_DIR}"/ \; | ||
|
|
||
| # Remove template keyword if it exist |
There was a problem hiding this comment.
in the long run, we're going to have separate entrypoint for AM and Tasks, it's worth naming then now accordingly
the same separation can be done to Dockerfile also (maybe with using a common tez docker base to prevent duplication as much as possible)
tez-dist/src/docker/README.md
Outdated
| 1. Building the docker image: | ||
|
|
||
| ```bash | ||
| mvn clean install -DskipTests -Pdocker,tools |
There was a problem hiding this comment.
is the tools profile needed for the docker image to build successfully?
| mvn clean install -DskipTests -Pdocker,tools | ||
| ``` | ||
|
|
||
| 2. Install zookeeper in mac by: |
There was a problem hiding this comment.
can you add ubuntu steps? we might be so kind to let linux users' life be easier
|
|
||
| repo=${REPO:-apache} | ||
| WORK_DIR="$(mktemp -d)" | ||
| CACHE_DIR="$SCRIPT_DIR/cache" |
There was a problem hiding this comment.
I can see this folder polluting git tree (after a failed build), please add it to gitignore:
tez-dist/src/docker/cache/
tez-dist/src/docker/tez.env
Outdated
|
|
||
| # Tez AM Container Environment Configuration | ||
|
|
||
| HADOOP_USER_NAME=tez |
There was a problem hiding this comment.
nitpicking, can you order the env vars here the same as they are ordered in the entrypoint script?
tez-dist/pom.xml
Outdated
| <argument>${project.basedir}/src/docker/build-docker.sh</argument> | ||
| <argument>-hadoop</argument> | ||
| <argument>${hadoop.version}</argument> | ||
| <argument>-tez</argument> | ||
| <argument>${project.version}</argument> | ||
| <argument>-repo</argument> | ||
| <argument>apache</argument> |
There was a problem hiding this comment.
can you make it similar to what I can see in Hive? much less verbose, e.g.
<arguments>
<argument>.... .sh</argument>
<argument>-hadoop ${hadoop.version}</argument>
<argument>-tez ${tez.version}</argument>
</arguments>
|
Thanks for the thorough review @abstractdog . I'll address the review comments shortly, you can continue the review in the meantime. I hope you are able to build the image and start tez am standalone process 😅 I still believe we can get rid of hadoop tarball dependency completely as the hadoop dependent required jars are already part of tez tarball. It might unnecessary increase docker image size. Also please suggest should I use eclipse-temurin:21.0.3_9-jre-ubi9-minimal or jdk, in case we want to take jstack or other java debugging tools, jkd image is required. |
|
tez-dist/src/docker/build-docker.sh
Outdated
| # HADOOP FETCH LOGIC # | ||
| ###################### | ||
| HADOOP_FILE_NAME="hadoop-$HADOOP_VERSION.tar.gz" | ||
| HADOOP_URL=${HADOOP_URL:-"https://archive.apache.org/dist/hadoop/core/hadoop-$HADOOP_VERSION/$HADOOP_FILE_NAME"} |
There was a problem hiding this comment.
what about using this first:
https://dlcdn.apache.org/hadoop/common/hadoop-3.4.2/hadoop-3.4.2.tar.gz
and then fall back to archive
archive.apache.org is crazy slow for me at the moment (not the first time), maybe it would worth discovering dlcdn.apache.org
tez-dist/src/docker/README.md
Outdated
| docker run \ | ||
| -p 10001:10001 -p 8042:8042 \ | ||
| --name tez-am \ | ||
| apache/tez-am:1.0.0-SNAPSHOT |
There was a problem hiding this comment.
I would introduce a TEZ_VERSION env var beforehand and refer to it: this would make it clear what's going to become obsolete in this doc, and what's more permanent :)
tez-dist/src/docker/README.md
Outdated
| docker run \ | ||
| -p 10001:10001 -p 8042:8042 \ | ||
| --name tez-am \ | ||
| apache/tez-am:1.0.0-SNAPSHOT |
There was a problem hiding this comment.
I'm trying the steps here and while the docker run I get:
2026-02-25 15:08:52,107 ERROR app.DAGAppMaster: Error starting DAGAppMaster
java.io.FileNotFoundException: /opt/tez/tez-conf.pb (No such file or directory)
at java.base/java.io.FileInputStream.open0(Native Method)
at java.base/java.io.FileInputStream.open(Unknown Source)
at java.base/java.io.FileInputStream.<init>(Unknown Source)
at org.apache.tez.common.TezUtilsInternal.readUserSpecifiedTezConfiguration(TezUtilsInternal.java:83)
at org.apache.tez.frameworkplugins.yarn.YarnServerFrameworkService$YarnAMExtensions.loadConfigurationProto(YarnServerFrameworkService.java:73)
at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2435)
also earlier I get this:
/entrypoint.sh: line 34: hostname: command not found
I believe this happens in the entrypoint, so should not be related to my host machine
can you advise what could possible cause these? I mean, I can debug it for sure, but maybe it's more straightforward for you, given you're the author
There was a problem hiding this comment.
For /opt/tez/tez-conf.pb in top commit i removed it because it was not failing for me. please ensure you are using the same tez-site.xml as in the PR or you can revert the last commit entrypoint.sh .
hostname one is know issue the command doesn't exist in the base docker image. i'll fix it, i forgot to remove it :-(
There was a problem hiding this comment.
cd tez-dist/src/docker and run the command.
docker run --rm \
-p 10001:10001 -p 5005:5005 \
--env-file tez.env \
--name tez-am \
apache/tez-am:1.0.0-SNAPSHOT
that tez-conf.pb is required in YARN mode not in zk mode. please give -e STANDALONE_ZOOKEEPER or tez.env and ensure that hadoop is also running or remove the followning propety. I was testing this with TEZ-4686
<property>
<name>fs.defaultFS</name>
<value>hdfs://host.docker.internal:9000</value>
</property>
There was a problem hiding this comment.
your stacktrace also suggests its going in default YARN mode.
There was a problem hiding this comment.
makes sense, but I think docker run command should contain tez.env right? I just copy-pasted the command, that was the problem, I can see only:
```bash
docker run \
-p 10001:10001 -p 8042:8042 \
--name tez-am \
apache/tez-am:1.0.0-SNAPSHOT
There was a problem hiding this comment.
yes, the README.md needs some change. will update it. Please let me know if you face any more issues in tez-am startup. I'm fully available until your tez-am image works :-)
I added the tez.env in point4 but didn't updated the older headings
There was a problem hiding this comment.
absolutely, thanks! making the image work on my side is crucial part of the review 🚀
3 participants
No description provided.