2: How to setup your dev environment on Docker

Setting up your development environment on Docker is a straightforward, but multi-step process. My hope is to eventually make this as simple as 1 command.

Before you get started

1. Install Docker for Mac, or Docker for Linux, or Docker for Windows from https://docs.docker.com/. This should install docker, as well as Kitematic.

2. Get the VPN key and config environment folder from your manager. You will need this to download data from production for your development server. It has keys in it, so its not included in the repo.

3. Get and validate Github repo access. Ensure you have Github access to the OMAlab repos. You should have access to oma, oma-models, oma-processors, audienti, and some others.

4. Generate a Github Oauth Token, that will be used for pushing repos. This token can be generated in Github at: https://github.com/settings/tokens/new

5. Ensure Rancher access. Can you access http://rancher.audienti.com:8080/. You can login with your Github account.

6. Install rancher-compose and the rancher CLI and ensure these are operational. This can either be downloaded from the rancher server, or you can download from the public repos. Your environment will need to be setup for these to work.

7. Make sure you have answers for the questions below. You can get these from your manager.

Questions you will need to answer during setup

• What is your home folder? (i.e. /Users/wflanagan)

• What is your name (i.e. William Flanagan)?

• What is your github and docker email ([e-mail])?

• What is your github token? (see https://github.com/settings/tokens)?

• What is your Docker/Dockerhub username?

• What is your Docker/Dockerhub password?

• What is your Quay.io username?

• What is your supplied Rancher API URL?

• What is your supplied Rancher Access Key?

• What is your supplied Rancher Secret Key?

• What is your Drone.io Url?

• What is your Drone access key?

• What is your Drone secret key?

• What command do you use to login to the bastion server?

Machine Setup

1. Change into your main developer_root folder. This should be on your machine ~/containers/ folder to follow OMAlab conventions.

2. Clone the "audienti" project. This project has all the configuration data and scripts needed to deploy our back end application on Docker.

3. Type $>bundle installto bundle your gems that are required for development setup.

4. Type $>ruby ./cli.rb config. This is the command line interface to the repo. It provides a LOT of wrapping around commands to help you, and you will use it every day. You can type ./cli.rb without any arguments to get a list of available commands.

5. The script will ask you these questions, and build a .sh or .zsh file that will match your environment. If this is not configured correctly, nothing else will work. You will need your ENV to have the right data in it to move forward from this step.

6. Type $>audiand ensure you are in the ROOT of your /audienti development environment. If you are not, then you are not setup correctly.

7. Type $>clito ensure you have the audienti command line available. The ENV modifications you installed wrap the ruby ./cli.rb x to use the simpler cli command. On Mac, this can be entered as simply "cli". But, on other systems, you might need to enter ./cli.rb to run. You should see a help screen for the command line with a list of commands for use in managing your configuration. If you don't see this, then you're not setup correctly.

8. Go to github.com/omalab/oma-config and download the config files you'll need for your docker environment (all of them). Place all of them in $AUDIENTI_ROOT/config/environments. This is a manual step for now, as the storage location for these has recently changed.

9. Type $>cli clone_repos. This will clone the repos into the correct folders and folder locations. It copies the primary folders.

10. Type $>docker loginto login to Docker.

11. Type $>dcdr oma bash --loginto enter the oma repo image in development mode. This SHOULD pull a bunch of repos, start the database servers, and then land you at a bash prompt in the oma repo folder.

12. Type $>ruby ./create_database.rband create theoma_testdatabase, so you can run tests.

13. Type $>ruby ./create_database.rband create theoma_developmentdatabase, so you can run development.

14. Type $>bundle exec rake db:schema:loadto load the schema into the postgres database. Then exit the container by typing $>exit.

15. Type $>dctr oma bash --loginand login to the container in test mode. Type $>bundle install. Then typebundle exec rake db:schema:loadto load your test environment schema.

16. Type $>bundle exec rspecand ensure your tests pass. This will ensure you have a working database environment.

Where things are

• There are essentially 2 applications: oma and oma-processors. Both applications use oma-models. When you are in development mode (using docker-compose and the docker-compose override), you will have them in a Docker container mapped to/var/app/omaand/var/app/oma-processors. In both cases, oma-models will be mapped to/var/app/oma-models.

• If you need to work on oma-models, you can simply cd ../oma-models from the folder you are in when you login to the container. If you are making changes to both the main appliation, and oma-models, you can update the Gemfile in the application folder. There will be a commented out version that points to the local path for this file.

Notes about development

1. When you are developing, your oma-models folder is mapped into the development environment, so you can make changes there if necessary. OMA is mapped to/var/app/omaand oma-models is mapped to/var/app/oma-models. All of this mapping is done in the docker-compose.override.yml file if you want to see what else is mapped, or make any mapping changes.

2. If you are developing on the front end, then you need to map service ports. You do this by starting with: $> dcdr --service-ports oma bash --login. Then, you can type bundle exec thin start -p 3000 and it should work. There is also a command that is dcdb oma bash that will do the mapping for you.

3. You might need a static entry in your hosts file to map to the IP of your docker machine. Your development environment is no longer on localhost. This file is on /etc/hosts.

4. You change environment by changing environment variables. To access different environments, type:dcdr oma bash --login for development, dcplr oma bash --loginfor production local, etc.

5. Pushing and pulling to Github should be the same inside an image. Follow our traditional instructions.

6. When you enter an image, it will be using the last time the image was built. So you will need to bundle install.

7. IF you need to do development on our crawler or other supplemental repos, you can clone them into the $AUDIENTI_ROOT folder, then add them to the docker-compose.override.yml to map them into your environment in the development environment (you can also use the cli command clone_supplemental to do this).

Notes about Docker

1. Docker development is different than your traditional development. Your databases are on different IP addresses. So, changes have had to be made with RSpec and tests to work with it. From any machine, you can check if your machine has access to another machine by pinging it from the command line. $>ping dbwill check if you can access postgres for example. The names used in this case, are the names listed in the docker-compose.override.yml file.

2. If you run out of disk space, go into Kitematic and terminate some of your old images. Every time you boot your machine, you create a new image. You have to go out occasionally and clean them out. There are oma cli commands docker nuke that will wipe everything. See the cli help for more assistance on that.