Infrastructure
Audienti uses a number of different tools and technologies.
Tooling
Docker - This is our main "container" which is used for all things Audienti. Our common development environment (CDE) is based on it (and docker compose), as well as our production environment and deployments.
Docker Compose - A companion to Docker, docker compose enables us to, with one command, launch all the databases and related services required to run a development environment.
Rancher - Rancher is the orchestration platform we use for managing deployments in production. Deployments go through CI and then are pushed to Rancher's Catalog as a proprietary catalog. This catalog enables one click upgrades of our environments.
Drone - Go-based CI testing. Uses our docker container, and tests the system with it. If it works, then it creates and pushes a new version of the app to Dockerhub, rewrites the catalog, and pushes all of it up to production.
Cloud9 IDE and Our Proprietary DinD setup - We have built a custom cloud IDE based on Cloud9. This runs on Rancher. This allows us to spin up a new development server, with all its dependencies met, in about 20 minutes. The developer can work right way, on a machine that is pretty powerful (64GB RAM, 1TB SSD).
Development Command Line Interface (CLI). Our common things we need to do are written into scripts and merged into a command line interface. This CLI is a git repo that we share and manage.
Github. All our code is stored here. The majority is in private repos.
VPN - Accessing production can only be done through our VPN. You will need to be setup as a user of the VPN. The VPN is a service provided by Rancher.
This Wiki - We try to document as much as we can using this Wikis.
api.audienti.com - This is an API proxy we use to proxy through to various services. This takes a single key, and then translates to our own proprietary keys.
Graphite & Seyren (and statsd). We use StatsD for tracking metrics in our applications. Graphite provides a graphical UI to look at charts and graphs related to this. Seyren provides alerts that flow into our Slack #engineering channel.
Slack - Used for chats, dialog, and system updates all along the path.
Databases
Our databases are not stored in the virtualized layer. Rather, they are bare metal at our hosting.
PostgreSQL - 9.5 version of our datatabase. This is our main ACID data store. We use extensions like hstore to store JSON and other things in this table as well.
Redis - Used for caching and queuing (via Sidekiq)
Cassandra - This is emerging as a primary data store for us as well. Our v2 pipelines are all based on this.
ElasticSearch - Was the dominant store in our old system. Still determined if v2 will use this at all.
Kafka - In flight queues for the system. So far, I have not used this in v2.
Hosting
We use Hetzner for our hosting. We have a dozen (12) bare metal servers that have the same config generally: 64GB RAM 8 core and 1-3TB of disk. About half have 1TB SSD on them, which are used for the bare metal deployments of databases.
Last updated