openshift/install e2e-aws add run smoke tests

[sallyom]

This PR depends on openshift/installer#223 for Dockerfiles

@bbguimaraes @stevekuznetsov
EDITED:
Ran this from local system, from openshift/ci-operator dir:
and I've placed 3 files in a local installer/secret dir: license, pull-secret, and .aws/credentials
Running this cmd:

./ci-operator -template /path/to/modified-for-local-dev/cluster-launch-installer-e2e.yaml \
 -config /path/to/release/ci-operator/config/openshift/installer/master.json \
 -secret-dir=/path/to/local/openshfit.yaml \
 -secret-dir=/local/path/installer/secret  \
 -git-ref=sallyom/installer@mybranch \
 -namespace=myns

And then, creating like so (noting here so I have record/don't forget):
1.oc create secret generic dev-cluster-profile --from-file=/local/path/to/aws/openshift.yaml -n myns
2.oc create secret -n myns generic cluster-secrets-aws --from-file=/path/to/secret/pull-secret --from-file=/path/to/secret/license --from-file=/path/to/secret/credentials -o yaml --dry-run | oc apply -n myns -f -
3. oc create -f release/cluster/ci/config/installer-origin-release-bazel.yaml -n openshift (tested in my ns)

I have tested this several times locally, all looks good, hoping it works same way w/in the presubmit job

[stevekuznetsov]

you should be able to just bazel run

[sallyom]

oh good

[stevekuznetsov]

Why does this vary per job?

[sallyom]

nope, it should not

[stevekuznetsov]

This seems wrong

[sallyom]

WIP lol It was wrong.. I've updated (since I was kind of able to test locally) I've made the change to the correct template now

[sallyom]

@stevekuznetsov This is only to get the bazel binary available, to run bazel build smoke_tests . If we can add that to the base image, we won't need this.

[sallyom]

@stevekuznetsov I need to run above 2 lines from directory where openshift/installer source code is.. is this where it is?? I haven't gotten that far in my testing to find out...

[stevekuznetsov]

You want a cluster-launch-installer-src template, look at cluster-launch-src for examples

[sallyom]

thanks

[sallyom]

@stevekuznetsov This is for local testing using rh-dev account.. will change it back or put in an if_local_dev block or something

[stevekuznetsov]

Let's remove the local dev bits

[sallyom]

yup done

[wking]

We can remove "(if post remove etcd)", since both this repo and the installer repo are past that point.

[sallyom]

ok, there was talk that the bootstrap instance would eventually not be included in the overall node count though? if it's terminated after the bootstrap is complete? we'll have to keep an eye on that number, it definitely shouldn't be hardcoded the way it is

[wking]

...there was talk that the bootstrap instance would eventually not be included in the overall node count though?

I'm hoping that when that happens, we'll have terminated the bootstrap node by tge time the installer exits. We may need parallel release/installer PRs for that, but the etcd comment is stale ;).

[sallyom]

ok, got it

[wking]

What's this about? Are you suggesting calculating this from inputs.yaml?

[sallyom]

yea, or keeping it in a variable rather than hardcoded there

[wking]

nit: we should be able to drop this. cp can create the target directory on its own:

$ mkdir -p a/b
$ cd a
$ cp -vR . /tmp/installer-src
‘.’ -> ‘/tmp/installer-src’
‘./b’ -> ‘/tmp/installer-src/b’

Once we drop the mkdir, the only thing left is the cp call. So we should be able to drop the shell entirely, and go straight to cp:

command:
- cp
- -R
- .
- /tmp/installer-src

or something like that.

[sallyom]

ok, will do

[wking]

There's previous discussion of source access here. But using parallel containers for the copy and Bazel tests seems pretty complicated. I'm fine with this approach to get this PR landed, but it would be nice if there was an easier way...

[sallyom]

it would be nice! took me quite awhile to figure that out lol

[wking]

What's this which doing? Can we drop it?

[sallyom]

yea, it was also debugging, fixing..

[wking]

Does the cache need to be mounted into the container? We don't mount it here, and the tarball build seems to work. After a successful build, you're already copying the smoke-test binary out of its symlinked location into /tmp/smoke. I expect all other Bazel files could die with the container.

[sallyom]

yes, I can def delete the other files, meant to do that.. I was stumped with the .cache stuff, I'm not sure what's going on there, yea, our other job does not require that, I know! I kept getting Permission Denied, can't create .cache/bazel-something so creating a writable path for that solved the issue.. anyways, I'll clean up the other files we don't need...

[sallyom]

cleaned up bazel-bin

[wking]

I kept getting Permission Denied, can't create .cache/bazel-something so creating a writable path for that solved the issue...

Ah, probably because of the random UID Prow uses and the lack of $HOME. More on this issue in #1178 and #1185. Try something like this (and if you also go that route for USER, you can drop your internal export from the script).

[wking]

Can we drop the "${@}"? This script isn't going to be called with arguments, is it?

[sallyom]

yup, fixing

[wking]

These two lines feel like debugging information that we can drop, now that you have the wait loop set up.

[sallyom]

yea, right. fixing..

[smarterclayton]

/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sallyom, smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [smarterclayton]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@sallyom: Updated the following 2 configmaps:

• job-config configmap using the following files:
  • key openshift-installer-presubmits.yaml using file ci-operator/jobs/openshift/installer/openshift-installer-presubmits.yaml
• ci-operator-openshift-installer configmap using the following files:
  • key master.yaml using file ci-operator/config/openshift/installer/master.yaml

Details

In response to this:

This PR depends on openshift/installer#223 for Dockerfiles

@bbguimaraes @stevekuznetsov
EDITED:
Ran this from local system, from openshift/ci-operator dir:
and I've placed 3 files in a local installer/secret dir: license, pull-secret, and .aws/credentials
Running this cmd:

./ci-operator -template /path/to/modified-for-local-dev/cluster-launch-installer-e2e.yaml \
-config /path/to/release/ci-operator/config/openshift/installer/master.json \
-secret-dir=/path/to/local/openshfit.yaml \
-secret-dir=/local/path/installer/secret  \
-git-ref=sallyom/installer@mybranch \
-namespace=myns

And then, creating like so (noting here so I have record/don't forget):
1.oc create secret generic dev-cluster-profile --from-file=/local/path/to/aws/openshift.yaml -n myns
2.oc create secret -n myns generic cluster-secrets-aws --from-file=/path/to/secret/pull-secret --from-file=/path/to/secret/license --from-file=/path/to/secret/credentials -o yaml --dry-run | oc apply -n myns -f -
3. oc create -f release/cluster/ci/config/installer-origin-release-bazel.yaml -n openshift (tested in my ns)

I have tested this several times locally, all looks good, hoping it works same way w/in the presubmit job

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@sallyom: The following updates succeeded:

• /usr/bin/make applyTemplate WHAT=projects/origin-release/pipeline.yaml

  
  $ /usr/bin/make applyTemplate WHAT=projects/origin-release/pipeline.yaml
  oc process -f projects/origin-release/pipeline.yaml | oc apply -f -
  imagestream "origin-release" configured
  buildconfig "origin-release-golang-1.8" configured
  buildconfig "origin-release-golang-1.9" configured
  buildconfig "origin-release-golang-1.10" configured
  buildconfig "origin-release-golang-1.11" configured
  buildconfig "origin-release-bazel" created
  buildconfig "origin-release-nodejs-8" configured
  buildconfig "origin-release-nodejs-8-browser-tests" configured
  

Details

In response to this:

This PR depends on openshift/installer#223 for Dockerfiles

@bbguimaraes @stevekuznetsov
EDITED:
Ran this from local system, from openshift/ci-operator dir:
and I've placed 3 files in a local installer/secret dir: license, pull-secret, and .aws/credentials
Running this cmd:

./ci-operator -template /path/to/modified-for-local-dev/cluster-launch-installer-e2e.yaml \
-config /path/to/release/ci-operator/config/openshift/installer/master.json \
-secret-dir=/path/to/local/openshfit.yaml \
-secret-dir=/local/path/installer/secret  \
-git-ref=sallyom/installer@mybranch \
-namespace=myns

And then, creating like so (noting here so I have record/don't forget):
1.oc create secret generic dev-cluster-profile --from-file=/local/path/to/aws/openshift.yaml -n myns
2.oc create secret -n myns generic cluster-secrets-aws --from-file=/path/to/secret/pull-secret --from-file=/path/to/secret/license --from-file=/path/to/secret/credentials -o yaml --dry-run | oc apply -n myns -f -
3. oc create -f release/cluster/ci/config/installer-origin-release-bazel.yaml -n openshift (tested in my ns)

I have tested this several times locally, all looks good, hoping it works same way w/in the presubmit job

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wking]

This is giving us errors like:

error: could not resolve inputs: could not determine inputs for step [input:base-smoke]: could not resolve base image: imagestreamtags.image.openshift.io "origin-release" not found

This is possibly because the origin-release image is not pushed to Quay?

[wking]

imagestreamtags.image.openshift.io "origin-release" not found

@sallyom has filed #1428 working on this.

[wking]

And also #1431.

[wking]

And also #1433.

[wking]

And also #1434.