Closed
Conversation
Tested this Databricks Runtime 7.1 and 7.2 using a minimal image created in this folder along with env.yml
houqp
reviewed
Sep 15, 2020
|
|
||
| #This image required an update to start with the base of 18.04 created from containers/ubuntu/minimal this is attached as another dockerfile ubuntu18.04-minimal | ||
| #Example from previous build: FROM databricksruntime/minimal:latest | ||
| FROM nadroj09/test_jordan:minimal |
There was a problem hiding this comment.
why is this base image hosted under your personal namespace?
There was a problem hiding this comment.
Looks like this PR is blocked/depends on PR #35 to create the databricksruntime/minimal:latest image, and the personal namespace is a placeholder for that.
Contributor
Author
There was a problem hiding this comment.
Correct, you need to update the databricksruntime/minimal:latest prior to creating this Dockerfile based on the reference.
Contributor
Author
|
When I created the base image using the databricksruntime/minimal:latest
this would still include the ubuntu version of 16.04. Therefore I had to
create a base minimal image including the libraries installed as standard
for 7.x as well as upgrade the image to use ubuntu 18.04.
The other docker file that I used to create that image is in the other pull
request.
It is a copy of the databricksruntime/minimal:latest that I created hosted
in a public repository to build the file for testing.
…On Tue, Sep 15, 2020 at 1:38 PM QP Hou ***@***.***> wrote:
***@***.**** commented on this pull request.
------------------------------
In experimental/ubuntu/Databricks Runtime 7/ubuntu18.04Python
<#34 (comment)>:
> + libdigest-sha-perl \
+ bzip2 \
+ strace
+
+# Download miniconda 4.5.12, then upgrade it to 4.6.12.
+RUN wget --quiet --output-document miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-4.5.12-Linux-x86_64.sh \
+ && (echo '866ae9dff53ad0874e1d1a60b1ad1ef8 miniconda.sh' | md5sum -c) \
+ && (echo 'e5e5b4cd2a918e0e96b395534222773f7241dc59d776db1b9f7fedfcb489157a miniconda.sh' | shasum -a 256 -c) \
+ # Conda must be installed at /databricks/conda
+ && /bin/bash miniconda.sh -b -p /databricks/conda \
+ && rm miniconda.sh \
+ && /databricks/conda/bin/conda install --name base conda=4.6.12
+
+#This image required an update to start with the base of 18.04 created from containers/ubuntu/minimal this is attached as another dockerfile ubuntu18.04-minimal
+#Example from previous build: FROM databricksruntime/minimal:latest
+FROM nadroj09/test_jordan:minimal
why is this base image hosted under your personal namespace?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#34 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANRXI4FZWFO3FQUVT27E7KLSF6YARANCNFSM4REAOURQ>
.
--
*Jordan Hicks*
*Technical Solutions Engineer*
*jordan.hicks@databricks.com <jordan.hicks@databricks.com> *
[image: email_signature_logo_sm]
|
Contributor
Author
|
The base image databricksruntime/minimal:latest does not include ubuntu
18.04 to create the image from.
FROM ubuntu:16.04
RUN apt-get update \
&& apt-get install --yes \
openjdk-8-jdk \
iproute2 \
bash \
sudo \
coreutils \
procps \
&& /var/lib/dpkg/info/ca-certificates-java.postinst configure \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
Yet, since there is not one created, I created one based directly on the
databricksruntime/minimal:latest and that is the reference in the
Dockerfile.
That is included in the second pull request.
Please let me know if that makes sense.
On Tue, Sep 15, 2020 at 2:24 PM Jordan Hicks <jordan.hicks@databricks.com>
wrote:
… When I created the base image using the databricksruntime/minimal:latest
this would still include the ubuntu version of 16.04. Therefore I had to
create a base minimal image including the libraries installed as standard
for 7.x as well as upgrade the image to use ubuntu 18.04.
The other docker file that I used to create that image is in the other
pull request.
It is a copy of the databricksruntime/minimal:latest that I created
hosted in a public repository to build the file for testing.
On Tue, Sep 15, 2020 at 1:38 PM QP Hou ***@***.***> wrote:
> ***@***.**** commented on this pull request.
> ------------------------------
>
> In experimental/ubuntu/Databricks Runtime 7/ubuntu18.04Python
> <#34 (comment)>:
>
> > + libdigest-sha-perl \
> + bzip2 \
> + strace
> +
> +# Download miniconda 4.5.12, then upgrade it to 4.6.12.
> +RUN wget --quiet --output-document miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-4.5.12-Linux-x86_64.sh \
> + && (echo '866ae9dff53ad0874e1d1a60b1ad1ef8 miniconda.sh' | md5sum -c) \
> + && (echo 'e5e5b4cd2a918e0e96b395534222773f7241dc59d776db1b9f7fedfcb489157a miniconda.sh' | shasum -a 256 -c) \
> + # Conda must be installed at /databricks/conda
> + && /bin/bash miniconda.sh -b -p /databricks/conda \
> + && rm miniconda.sh \
> + && /databricks/conda/bin/conda install --name base conda=4.6.12
> +
> +#This image required an update to start with the base of 18.04 created from containers/ubuntu/minimal this is attached as another dockerfile ubuntu18.04-minimal
> +#Example from previous build: FROM databricksruntime/minimal:latest
> +FROM nadroj09/test_jordan:minimal
>
> why is this base image hosted under your personal namespace?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <#34 (review)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ANRXI4FZWFO3FQUVT27E7KLSF6YARANCNFSM4REAOURQ>
> .
>
--
*Jordan Hicks*
*Technical Solutions Engineer*
***@***.*** ***@***.***> *
[image: email_signature_logo_sm]
--
*Jordan Hicks*
*Technical Solutions Engineer*
*jordan.hicks@databricks.com <jordan.hicks@databricks.com> *
[image: email_signature_logo_sm]
|
Collaborator
|
Done by #49 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tested this Databricks Runtime 7.1 and 7.2 using a minimal image created in this folder along with env.yml