diff --git a/README.md b/README.md index 6785baf6d..6a2e121fa 100644 --- a/README.md +++ b/README.md @@ -12,9 +12,9 @@ OSMT is written in Kotlin with Spring Boot, and Angular. It uses backend-instanc OSMT uses Elasticsearch, Redis, and MySQL as back-end dependencies. OSMT also requires an OAuth2 provider. Local / non-production configurations can be stood up using only Docker. See additional notes below for [Configuration](README.md#configuration). ## Getting started -Follow the steps in the [Pre-requisites](README.md#pre-requisites) and [Running the Quickstart](README.md#running-the-quickstart) sections to bootstrap a local Quickstart OSMT instance (you can see more details in the [Quickstart Configuration](README.md#quickstart-configuration) section). -* NOTE: A Quickstart OSMT instance should be for demo purposes only. The Docker images are fixed/pinned, and the data is stored on a Docker volume. Unless your organization has taken technical steps to ensure the backing data will remain available, you should consider Quickstart data to be temporary. If you create RSDs or Collections, you should export them before shutting down the Quickstart. See the [How-To](https://osmt.io/docs.html) section of osmt.io for more information on exporting. +Follow the steps in the [Pre-requisites](README.md#pre-requisites) section. * Please have the right technical people in your organization deploy a production OSMT instance before making any real investment in effort for building skills. +* For latest updates, reach out in the [Discussion](https://github.com/wgu-opensource/osmt/discussions) boards in GitHub. ### Using the OSMT CLI utility (`osmt_cli.sh`) The OSMT source code includes a utility named `osmt_cli.sh`, in the project root directory. `osmt_cli.sh` simplifies setting up a local OSMT environment and doing routine tasks. It uses BASH, and works on MacOS and Linux. We have had some success with a BASH interpreter in Windows. You can run `./osmt_cli.sh -h` for the help text. @@ -22,24 +22,13 @@ The OSMT source code includes a utility named `osmt_cli.sh`, in the project root ### Pre-requisites By default, OSMT is wired up for Okta as an OAuth2 provider. While you can change this, these documents will only address configuring Okta. -1. Obtain a "free developer" Okta account. This is required to log in to a local OSMT. Please follow the steps in [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) below, and return here when complete. You can refer to [Environment files for Quickstart and Development Stacks](README.md#environment-files-for-quickstart-and-development-stacks) for more details. +1. Obtain a "free developer" Okta account. This is required to log in to a local OSMT. Please follow the steps in [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) below, and return here when complete. You can refer to [Environment file for Development and API Tests Stack](README.md#environment-files-for-development-and-api-tests-stacks) for more details. 2. Initialize your environment files. After running this, update the OAUTH2/OIDC values in the env files (replace the `xxxxxx` values with the correct values from your Okta account) ``` ./osmt_cli.sh -i ``` -### Running the Quickstart -1. Start the Quickstart configuration. This will take several minutes to download resources and build the software. If you continue to loop on errors that report "Retrying in 10 seconds...", there's something wrong. Search and reach out in the [Discussion](https://github.com/wgu-opensource/osmt/discussions) boards in GitHub. - ``` - ./osmt_cli.sh -q - ``` - -2. Open your browser to `http://localhost:8080`. - -3. You can exit the Quickstart configuration by pressing [Ctrl-C] from your terminal window. - * If you don't clean up the Docker images and volumes, Quickstart should start very quickly when re-running `osmt_cli.sh -q`. - ### Running the Development configuration 1. Validate your local environment (for Docker, Java, and other SDKs / runtimes). If this command reports error, you will need to resolve them before running the Development configuration. See more in the [Requirements to Build OSMT](README.md#requirements-to-build-osmt) section ``` @@ -77,7 +66,7 @@ By default, OSMT is wired up for Okta as an OAuth2 provider. While you can chang ``` ### Housekeeping with `osmt_cli.sh` -You can surgically clean up OSMT-related Docker images and data volumes. This step **will** delete data from local OSMT Quickstart and Development configurations. It does not remove the mysql/redis/elasticsearch Docker images, as those may be available locally for other purposes. +You can surgically clean up OSMT-related Docker images and data volumes. This step **will** delete data from Development and API Test configurations. It does not remove the mysql/redis/elasticsearch Docker images, as those may be available locally for other purposes. ``` ./osmt_cli.sh -c @@ -140,20 +129,7 @@ OSMT is a multi-module Maven project. pom.xml files exist in the project root, ` The [API](api/README.md) and [UI](ui/README.md) modules have their own README.md files with more specific information about those modules. ### Configuration -This project makes use of 2 similar local configurations, "Quickstart" and "Development". These both rely on docker-compose and environment files. Follow the steps in [Pre-requisites](README.md#pre-requisites) above. See [Environment files for Quickstart and Development Stacks](README.md#environment-files-for-quickstart-and-development-stacks) for more details. - -### Quickstart Configuration -The Quickstart configuration uses the `docker-compose.quickstart.yml` file in the project root to stand up a non-production OSMT stack. This stack uses containers for the Spring application and the back-end dependencies (MySQL, ElasticSearch, and Redis). -* The Quickstart configuration is opinionated, in that all services and volumes are stood up on a single machine. That topology may or may not be appropriate for your organization. It could inform how to configure a production OSMT instance, but it is not intended for a production deployment. -* Quickstart uses the `osmt-quickstart.env` file to provide your OAuth2 secrets, and override default values. If you break your working copy of this env file, you can refer to `osmt-quickstart.env.example` for the default values. -* By default, the Quickstart configuration automatically imports BLS and O*NET job code metadata, and reindexes ElasticSearch. You can alter this default behavior with the `REINDEX_ELASTICSEARCH` and `SKIP_METADATA_IMPORT` environment variables (export them in your shell, or uncomment them in `osmt-quickstart.env`). -* By default, Quickstart uses an image named `wguopensource/osmt-app:latest` as the Spring application. It will pull the image tagged `latest` from DockerHub. - * You can override the tagged version number with the `OSMT_APP_IMAGE_TAG` environment variable (export it in your shell, or uncomment it in `osmt-quickstart.env`). - * You can build a new Docker image for the Spring application, and tag it with `wguopensource/osmt-app:latest`. Quickstart will use it, instead of pulling from DockerHub. -* You are not required to use `osmt_cli.sh`. You can run this command from the project root to stand up the Quickstart configuration. - ``` - docker-compose --file docker-compose.quickstart.yml --env-file osmt-quickstart.env --project-name osmt_quickstart up - ``` +This project does "Development" configuration. It relies on docker-compose and environment files. Follow the steps in [Pre-requisites](README.md#pre-requisites) above. ### Development Configuration The Development configuration uses the `dev-stack.yml` docker-compose file in the `docker` directory, for standing up just the back-end dependencies. This facilitates doing active development in the Spring or Angular layers. You can use `osmt_cli.sh` in the steps given in [Running the Development configuration](README.md#running-the-development-configuration) to simplify starting and stopping the Docker services and Spring application. @@ -199,8 +175,9 @@ When using Okta, you will use the `oauth2-okta` profile for Spring Boot, which w If you want to enable OSMT user permissions by roles, see additional details in [Role-based Access in OSMT](README.md#role-based-access-in-osmt). -### Environment Files for Quickstart and Development Stacks -There are many ways to provide environment values to a Spring application. That said, you should never push secrets to GitHub, so you should never store secrets in source code. The OSMT project is configured to git ignore files named `osmt*.env`, and we recommend you follow this approach. The OSMT source code includes example environment files for the Quickstart and Development configurations (`osmt-quickstart.env.example` and `api/osmt-dev-stack.env.example`). Running `./osmt_cli.sh -i` will create env files for you, but you will need to replace the 'xxxxxx' values with your OAUTH2/OIDC values, following the guidance in the [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) section. +### Environment Files for Development and API Tests Stacks +There are many ways to provide environment values to a Spring application. That said, you should never push secrets to GitHub, so you should never store secrets in source code. The OSMT project is configured to git ignore files named `osmt*.env`, and we recommend you follow this approach. +The OSMT source code includes example environment files for the Development configuration (`api/osmt-dev-stack.env.example`) and API Tests configuration (`test/api/osmt-apitest.env.example`). Running `./osmt_cli.sh -i` will create env files for you, but you will need to replace the 'xxxxxx' values with your OAUTH2/OIDC values, following the guidance in the [OAuth2 and Okta Configuration](README.md#oauth2-and-okta-configuration) section. #### Alternate approaches for Providing Environment Variables * Provide these OAuth2 values as program arguments when starting your Spring Boot app (`-Dokta.oauth2.clientId="123456qwerty"`). @@ -215,7 +192,7 @@ There are many ways to provide environment values to a Spring application. That ``` ### Post-installation (BLS, O*NET, etc) -The Quickstart configuration automatically imports the default BLS and O*NET job code metadata. For a Development configuration, you can import the metadata with this command: +For Development configuration, you can import the metadata with this command: ``` osmt_cli.sh -m ``` diff --git a/docker-compose.quickstart.yml b/docker-compose.quickstart.yml deleted file mode 100644 index ad0b0d313..000000000 --- a/docker-compose.quickstart.yml +++ /dev/null @@ -1,78 +0,0 @@ -# A dockerized stack for the non-production Quickstart configuration -version: '3.3' -services: - app: - image: "wguopensource/osmt-app:${OSMT_APP_IMAGE_TAG:-latest}" - volumes: - - ./import:/opt/osmt/import:rw - environment: - - "ENVIRONMENT=${ENVIRONMENT:-dev,apiserver,oauth2-okta}" - - "BASE_DOMAIN=${BASE_DOMAIN}" - - "FRONTEND_URL=${FRONTEND_URL}" - - "DB_NAME=${DB_NAME}" - - "DB_USER=${DB_USER}" - - "DB_PASSWORD=${DB_PASSWORD}" - - "DB_URI=${DB_USER}:${DB_PASSWORD}@db:3306" - - "MIGRATIONS_ENABLED=${MIGRATIONS_ENABLED:-true}" - - "REINDEX_ELASTICSEARCH=${REINDEX_ELASTICSEARCH:-true}" - - "SKIP_METADATA_IMPORT=${SKIP_METADATA_IMPORT:-false}" - - "ELASTICSEARCH_URI=${ELASTICSEARCH_URI}" - - "REDIS_URI=${REDIS_URI}" - - "OAUTH_ISSUER=${OAUTH_ISSUER}" - - "OAUTH_CLIENTID=${OAUTH_CLIENTID}" - - "OAUTH_CLIENTSECRET=${OAUTH_CLIENTSECRET}" - - "OAUTH_AUDIENCE=${OAUTH_AUDIENCE}" - depends_on: - - db - - elasticsearch - - redis - ports: - - "8080:8080" - networks: - - osmt - - db: - image: library/mysql:8.0 - command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci - volumes: - - sql_db:/var/lib/mysql:rw - - ./docker/mysql-init:/docker-entrypoint-initdb.d - environment: - - "MYSQL_PASSWORD=${DB_PASSWORD}" - - "MYSQL_USER=${DB_USER}" - - "MYSQL_DATABASE=${DB_NAME}" - - "MYSQL_HOST=0.0.0.0" - - "MYSQL_ROOT_PASSWORD=password" - ports: - - "3306:3306" - networks: - - osmt - - elasticsearch: - image: docker.elastic.co/elasticsearch/elasticsearch:7.17.4 - volumes: - - elastic_data:/usr/share/elasticsearch/data - environment: - - "discovery.type=single-node" - - "net=host" - - "xpack.security.enabled=false" - ports: - - "9200:9200" - - "9300:9300" - networks: - - osmt - - redis: - image: library/redis:6.0.6 - ports: - - "6379:6379" - networks: - - osmt - -networks: - osmt: - driver: bridge - -volumes: - sql_db: - elastic_data: diff --git a/osmt-quickstart.env.example b/osmt-quickstart.env.example deleted file mode 100644 index f82a2bbbb..000000000 --- a/osmt-quickstart.env.example +++ /dev/null @@ -1,22 +0,0 @@ -ENVIRONMENT=dev,apiserver,oauth2-okta -BASE_DOMAIN=localhost:8080 -FRONTEND_URL=http://localhost:8080 - -DB_NAME=osmt_db -DB_USER=osmt_db_user -DB_PASSWORD=password -MYSQL_ROOT_PASSWORD=root_password - -# These flags have defaults in docker-compose.quickstart.yml. Uncomment them to override default behavior -#OSMT_APP_IMAGE_TAG=x.y.z -#MIGRATIONS_ENABLED=false -#REINDEX_ELASTICSEARCH=false -#SKIP_METADATA_IMPORT=true - -ELASTICSEARCH_URI=http://osmt_quickstart_elasticsearch_1:9200 -REDIS_URI=osmt_quickstart_redis_1:6379 - -OAUTH_ISSUER=https://xxxxxx.okta.com/oauth2/default -OAUTH_CLIENTID=xxxxxx -OAUTH_CLIENTSECRET=xxxxxx -OAUTH_AUDIENCE=xxxxxx diff --git a/osmt_cli.sh b/osmt_cli.sh index 3f4299fd9..32b0a5103 100755 --- a/osmt_cli.sh +++ b/osmt_cli.sh @@ -4,7 +4,6 @@ set -u -declare QUICKSTART_ENV_FILE declare DEV_ENV_FILE declare APITEST_ENV_FILE @@ -23,7 +22,6 @@ declare -r OSMT_SECURITY_PROFILE="${OSMT_SECURITY_PROFILE:-}" init_osmt_env_files() { - _init_osmt_env_file "Quickstart" "${QUICKSTART_ENV_FILE}" || return 1 _init_osmt_env_file "Development" "${DEV_ENV_FILE}" || return 1 _init_osmt_env_file "API Tests" "${APITEST_ENV_FILE}" || return 1 } @@ -40,7 +38,6 @@ validate_osmt_dev_environment() { echo echo_info "Checking environment files used in local OSMT instances..." - _validate_env_file "${QUICKSTART_ENV_FILE}" || is_environment_valid+=1 _validate_env_file "${DEV_ENV_FILE}" || is_environment_valid+=1 _validate_env_file "${APITEST_ENV_FILE}" || is_environment_valid+=1 @@ -51,29 +48,6 @@ validate_osmt_dev_environment() { fi } -start_osmt_quickstart() { - local -i rc - _validate_git || return 1 - _validate_docker_version - rc=$? - if [[ $rc -ne 0 ]]; then - echo_err "Aborting OSMT Quickstart. Exiting..." - return 1 - fi - - _cd_osmt_project_dir || return 1 - _validate_env_file "${QUICKSTART_ENV_FILE}" - rc=$? - if [[ $rc -ne 0 ]]; then - echo_err "Aborting OSMT Quickstart. Exiting..." - return 1 - fi - - echo - echo_info "Starting OSMT Quickstart with docker-compose using osmt-quickstart.env" - docker-compose --file docker-compose.quickstart.yml --env-file "${QUICKSTART_ENV_FILE}" --project-name osmt_quickstart up -} - import_osmt_metadata() { local stack_name="${1}" echo @@ -231,13 +205,12 @@ start_osmt_api_tests() { _remove_osmt_docker_artifacts() { remove_osmt_docker_artifacts_for_stack "osmt_dev" remove_osmt_docker_artifacts_for_stack "osmt_api_test" - remove_osmt_docker_artifacts_for_stack "osmt_quickstart" } cleanup_osmt_docker_artifacts() { local prompt_msg; prompt_msg="$(cat <<-EOF ${INDENT}Do you want to clean up OSMT-related Docker images and volumes? -${INDENT}This step will delete data from local OSMT Quickstart and Development configurations. +${INDENT}This step will delete data from Development and API Test configurations. ${INDENT}Please answer 'y' to proceed? EOF @@ -264,11 +237,8 @@ A command line utility to simplify onboarding with OSMT development instances. T Usage: osmt_cli.sh [accepts a single option] - -i Initialize environment files for Quickstart and Development configurations. + -i Initialize environment files for Development and API Test configurations. -v Validate local environment and dependencies for development. - -q Start the Quickstart configuration. Application and services are containerized and - managed by docker-compose. The docker-compose stack will attach to the console, with containers - named "osmt_quickstart". -d Start the backend Development Docker stack (MySQL, ElasticSearch, Redis). docker-compose stack will be detached, with containers named "osmt_dev". You can review status with 'docker ps'. -e Stop the detached backend Development Docker stack (MySQL, ElasticSearch, Redis). @@ -281,8 +251,7 @@ Usage: -r Start the local Spring app to reindex ElasticSearch. -a Start the local API tests for OSMT. This requires a valid OSMT jar file (from a 'mvn package') -m Import default BLS and O*NET metadata into local Development instance. - -c Surgically clean up OSMT-related Docker images and data volumes. This step will delete data from local OSMT - Quickstart and Development configurations. It does not remove the mysql/redis/elasticsearch images, as + -c Surgically clean up OSMT-related Docker images and data volumes. This step will delete data from Development and API Test configurations. It does not remove the mysql/redis/elasticsearch images, as those may be available locally for other purposes. -h Show this help message. @@ -295,7 +264,6 @@ EOF declare script_dir; script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) source "${script_dir}/bin/lib/common.sh" || exit 135 -QUICKSTART_ENV_FILE="${PROJECT_DIR}/osmt-quickstart.env" DEV_ENV_FILE="${PROJECT_DIR}/api/osmt-dev-stack.env" APITEST_ENV_FILE="${PROJECT_DIR}/test/osmt-apitest.env" @@ -325,10 +293,6 @@ while getopts "ivqdelrasmch" flag; do validate_osmt_dev_environment || exit 135 exit 0 ;; - q) - start_osmt_quickstart || exit 135 - exit 0 - ;; d) start_osmt_docker_stack "${OSMT_STACK_NAME}" || exit 135 exit 0