guide
  • Introduction
  • Guiding Principles
    • Mission Statement
    • Conflict Resolution Process
  • Operating Model
    • Working Together
    • Holacracy
      • Meetings
      • Specific Roles
      • Terms and Definitions
      • Finer Points
      • Holacracy-Asana Key
    • Getting Things Done
      • Daily, Weekly, Monthly, and Annual Reviews
      • GTD-Asana Key
    • Transparency
    • Language
    • Budgeting
    • By Department
      • Engineering Operations
  • General Guidelines
  • Employment Policies
    • Equal Opportunity Employment
    • At-Will Employment
    • Code of Conduct in the Community
    • Complaint Policy
    • Drug and Alcohol Policy
    • Vacation, Holiday, and Paid Time Off (PTO) Policy
    • Supplemental Policies for Remote Employees and Contractors
    • Supplemental Policy for Bonus, Commissions, and other Performance-based Payments
    • Supplemental Policies for Hourly International Contractors or Workers
    • Supplemental Policies for Hourly International Contractors or Workers
    • Disputes and Arbitration
  • Benefits and Perks
    • Health Care
    • Vacation, Holiday and Paid Time Off (PTO) Policy
    • Holiday List
  • Hiring Documents
    • Acknowledgement of Receipt
    • Partner Proprietary Information and Inventions Agreement
  • Engineering Wiki
    • Code Snippets
      • Front End Code Snippets
    • Setup
      • 1: Overview of development using Audienti
      • 2: How to setup your dev environment on Docker
      • 2a: Setting up on our cloud your dev server
      • 3: Connect to Production using the VPN
      • 4: Import data into your development environment
    • Deployment
      • Docker based deployment of back end (manual)
    • Culture
      • How our development team works
      • Code Best Practices
    • Tips
      • Setting up a new development machine
      • Importing data to Development environment
      • GIT workflow and work tracking
      • Using Slack
      • Using Rubocop
      • Our Code Standards
      • General suggested best practices
      • Tracking your time
      • Naming Iterations
    • Migrations
      • Postgres
      • ElasticSearch
      • Redis
    • Database and System Maintenance
      • Redis Howtos
      • Elasticsearch HowTos
      • Postgres HowTos
      • Administration recipes
      • App maintenance crash course notes
    • Front End
      • 2016 Plan
      • Deploy
      • Assets
      • SearchLogic
      • How to create UI components
      • OMA Standard Tables
    • Monitoring and Alerting
      • Monitoring Systems
      • Monitoring individual controller actions
      • Get notified when a metric reaches a certain threshold
      • Instrumenting your models using Oma Stats
      • Configuring Graphite Charts
      • Tracking your results with StatsD
      • Logging Fields
      • Updating Kibana Filtering
    • Testing
      • Coverage
      • Elasticsearch mapping config synchronization
      • Testing Gotchas
      • Rspec Preloader
      • Test Best Practices
    • Models
      • Backlinks
    • Queueing and Worker System
      • Queueing and Job Overview
    • Processors
      • Rebuilding Spot Instances
      • Deploying processors
      • Running processors in development
      • Reverting to the previous build on a failed deployment
    • Processors / Opportunity Pipeline
      • Opportunity Pipeline
      • Diagram
    • Processors / Enrichment Pipeline
      • Diagram
      • Clustering
    • Processors / Backlink Pipeline
      • Diagram
      • Backlink Pipeline external APIs
      • Backlink pipeline logic
    • Processors / Automation Pipeline
      • Diagram
      • Automation Pipeline Overview
      • Agents
      • Running in development
    • Messaging and Social Accounts
      • Overview
    • API
      • Audienti API
    • Algorithms
    • Troubleshooting
      • Elasticsearch
    • Big Data Pipeline Stuff
      • Spark
    • Our Product
      • Feature synopsis of our product
    • Research
      • Backend framework comparison
      • Internet marketing Saas companies
    • Code snippets
      • Commonly Used
      • Not Used
    • Miscellaneous
      • Proxies and Bax
    • Legacy & Deprecated
      • Search criteria component
      • Classes list
      • Target Timeline
      • Twitter processor
      • Asset compilation
      • Test related information
      • Interface to EMR Hadoop jobs
      • Mongo Dex Indexes to be Built
      • Mongodb errors
      • Opportunity pipeline scoring
      • Graph Page
      • Lead scoring
      • Insights
      • Shard keys
      • Setting up OMA on local
      • Clone project to local machine
      • Getting around our servers in AWS
  • Acknowledgements
  • Documents That Receiving Your First Payment Triggers Acknowledgement and Acceptanace
Powered by GitBook
On this page
  • Producing a new oma-processors image.
  • Build the image
  • Build rancher-compose and docker-compose yml files for deployment
  • Push to Rancher
  • Check that it's done
  • Other notes
  1. Engineering Wiki
  2. Deployment

Docker based deployment of back end (manual)

Note: This is based on a deployment environment based on the Audienti repo, and that your development environment is setup correctly for development using Docker.

In general, the steps are:

  • Produce a new oma-processors image with a new tag on Dockerhub.

  • Build new rancher stacks docker-compose and rancher-compose files.

  • Push the stacks to Rancher for redeployment.

Producing a new oma-processors image.

Once you have a release to push, in your oma-processors repo, you type:

./release.sh

This will ask you for a new release version, tagging the current branch as that release. It also updates the VERSION and CHANGES files in the root of the repo. Once this is done, $>git push origin masterto push this tag to the server.

Build the image

cli build oma-processors

This will build the image, and push it to DockerHub. It will use the current tag, and will overwrite the tag if there is a duplicate tag on dockerhub. Ensure you are logged into dockerhub before you issue this command, or it will fail. It takes about 20 minutes currently.

Build rancher-compose and docker-compose yml files for deployment

cli write_rancher_stacks docker dc1

This command will delete and re-create the deploy folder's scripts. In the deploy folder there is a folder for each "stack" in rancher. Inside, there is a rancher-compose and a docker-compose file for each stack. These are what are pushed by the deploy commands. If you do NOT update these, you will push an old version.

Push to Rancher

cli deploy_all

This will deploy all the stacks. You can monitor progress on the command line, and the rancher web UI. This can take 2 to 5 hours. It's slow. But, you can be doing development in the background at the same time.

Note you can also do individual stack deployments by typing $>cli deploy automation-pipelinefor example. You can list them out too, like $>cli deploy automation-pipeline global opportunity-pipelineif you want to deploy a subset of the stacks.

Check that it's done

Sometimes there will a container or two that finish their upgrade, but don't complete (removing the old one). Go through each stack and ensure they are running. Click finish upgrade to finish.

Other notes

  • Scaling. There is a file in the deploy folder that stays, which is the scale. You can adjust this file to adjust the default number of each service that is deployed. If you delete it, the when you issue the command $>cli write_rancher_stacksit will build it, and abort with a message. Edit the config, type the command again, and it will build the stacks.

PreviousDeploymentNextCulture

Last updated 7 years ago