How to Setup Your Local Machine

Setting up your development environment on Docker is a straightforward, but multi-step process. My hope is to eventually make this as simple as 1 command.

Before you get started

1. Install Docker for Mac, or Docker for Linux, or Docker for Windows from https://docs.docker.com/. This should install docker, as well as Kitematic.

2. Get the VPN key and config environment folder from your manager. You will need this to download data from production for your development server. It has keys in it, so it's not included in the repo.

3. Get and validate Github repo access. Ensure you have Github access to the OMAlab repos. You should have access to oma, oma-models, oma-processors, audienti, backend, front, and some others.

4. Generate a Github Oauth Token, that will be used for pushing repos. This token can be generated on the Github website at: https://github.com/settings/tokens/new

For pushing to production, and CI you'll need in your developer.yml file:

1. Ensure Rancher access. Can you access http://rancher.audienti.com:8080/. You can login with your Github account. From Rancher, get your developer tokens.

2. Install rancher-compose and the rancher CLI and ensure these are operational. This can either be downloaded from the rancher server, or you can download from the public repos. Your environment will need to be setup for these to work.

3. Drone keys and tokens

4. Your administrator has given you bastion access (access to the production VLAN).

You should be able to answer everything in this list. This is a .yml file that will be placed in your setup to configure it for development.

---
developer_home: "/Users/wflanagan"
audienti_path: "/Users/wflanagan/containers/audienti"
inspire_path: "/Users/wflanagan/containers/inspire"
developer_name: William Flanagan
developer_email: [e-mail]
developer_token: 2khdsksdghi3hig34349gh34g93hgki34hgi34gh
docker_username: wflanagan
docker_password: password123
rancher_url: http://rancher.audienti.com:8080
rancher_access_key: skgskdhgkh234t23h23ihgi
rancher_secret_key: 239h293fh02egh02h04ihg3i4gh04igh
bastion_server: ssh -i ~/.ssh/william.flanagan.pem william.flanagan@138.201.22.22
drone_server: http://drone.audienti.com:8000
drone_token: kdgsdkgsdoghsdokgksdoghsdghsodgkhsdoghsodkghogiwieohsieghseoihgsoueighsoughsourghsorughosurhgosurhgoj

Machine Setup

1. Change into your main developer_root folder. This is the folder where you keep projects and check out new code. This should be on your machine ~/projects/ folder to follow OMAlab conventions.

2. Go to Github.com, and navigate to the omalab/audienti project. Clone the "audienti" project to this folder. Ensure the folder name is called "audienti". This project has all the configuration data and scripts needed to deploy our back end application on Docker.

3. Type $>bundle installto bundle your gems that are required for development setup.

4. Put your developer.yml in the config/developer folder of the audienti folder you just cloned. This file will be read to prime/setup the system.

5. From the root of the audienti proejct, type bin/setup. This command will read your developer.yml file and place a file called custom_aliases.sh into the root of the audienti project folder. This should be placed into our .profile or bashrc or other config so that it is read every time you open a new terminal window. This will be unique to your local development environment, so it's outside the scope of this guide as to how to do it.

6. When loaded correctly, you should be able to type $>audi and change to the directory of your /audienti development environment. If you are not, then you are not setup correctly.

7. The bin/cli is a command line for utilities for the audienti project. The ENV modifications you installed wrap the ruby ./cli.rb x to use the simpler cli command. On Mac, this can be entered as simply "cli". But, on other systems, you might need to enter ./cli.rb to run. You should see a help screen for the command line with a list of commands for use in managing your configuration. If you don't see this, then you're not setup correctly.

8. Type $> bin/clone_config. This will clone the configuration files needed for accessing the system in various environments.

9. Go into ./config/environments/dc1/front and ./config/environments/dc1/back and modify the config files to match your local development hostname. If you follow OMAlab convention, you have added a www.omaengine.dev to point to your localhost. If you do this, these files should NOT require modification.

10. Type $>bin/clone_repos. This will clone the repos into the correct folders and folder locations. It copies the primary folders.

11. Type $>docker loginto login to Docker.

12. Type $>dcdr api bashto enter the next gen Audienti repo image in development mode. This SHOULD pull a bunch of repos, start the database servers, and then land you at a bash prompt in the Rails app repo folder.

13. Type $> ./setup_new_container.sh to setup a new container. This will bundle, create databases, load schema, and otherwise get you up and running. If you want to do things manually instead, follow the steps in this script.

14. Type $> ./rd.sh to load all the runtime dependencies.. This will run bundler, yarn, and npm.

15. Log out. Login with your test environment and do the same thing as above.

16. Log out. Now, do dcdp app bash. This will load the app (React) front end application.

17. Type $> yarn to bundle the node assets.

18. Type $> yarn startto start the application. It should be serving on port 8080.

Notes about development

• There are 2 main repos you deal with for the application: front and backend. In the docker-compose, they map to the "app" and the "api". You start and stop them with "app" an "api" container commands.

• The legacy application front end is the "oma" repo. The legacy models for the Rails app are in the oma-models repo. The old back end processing logic was in the oma-processors repo.

• Once you're up and running, in development mode, you can run dcdup and this will start all the services and push you back to your command prompt. Then, to attach to an existing, running service with a bash prompt, you can type dattachaudienti_api or dattach audienti_app.

• You might need a static entry in your hosts file to map to the IP of your docker machine. Your development environment is no longer on localhost. This file is on /etc/hosts.

• Changing running environment is done with a dc commands. They are made up of 4 commands:

• dc = docker compose

• {environment} = d for development, t for test, p for production

• {run | run port mapped} = r to run and not map ports for your local, p to run and map ports to your local

• So the command becomes dcdr (dc development in port mode) dcdp (dc development in port mapped mode), dctr (dc test in port mapped mode).

• Pushing and pulling to Github should be the same inside an image. Follow our traditional instructions.

• When you enter an image, it will be using the last time the image was built. So you will need to bundle install.

• IF you need to do development on our crawler or other supplemental repos, you can clone them into the $AUDIENTI_ROOT folder, then add them to the docker-compose.override.yml to map them into your environment in the development environment (you can also use the cli command clone_supplemental to do this).

Notes about Docker

1. Docker development is different than your traditional development. Your databases are on different IP addresses. So, changes have had to be made with RSpec and tests to work with it. From any machine, you can check if your machine has access to another machine by pinging it from the command line. $>ping dbwill check if you can access postgres for example. The names used in this case, are the names listed in the docker-compose.override.yml file.

2. If you run out of disk space, go into Kitematic and terminate some of your old images. Every time you boot your machine, you create a new image. You have to go out occasionally and clean them out. There are oma cli commands docker nuke that will wipe everything. See the cli help for more assistance on that.