guide
  • Introduction
  • Guiding Principles
    • Mission Statement
    • Conflict Resolution Process
  • Operating Model
    • Working Together
    • Holacracy
      • Meetings
      • Specific Roles
      • Terms and Definitions
      • Finer Points
      • Holacracy-Asana Key
    • Getting Things Done
      • Daily, Weekly, Monthly, and Annual Reviews
      • GTD-Asana Key
    • Transparency
    • Language
    • Budgeting
    • By Department
      • Engineering Operations
  • General Guidelines
  • Employment Policies
    • Equal Opportunity Employment
    • At-Will Employment
    • Code of Conduct in the Community
    • Complaint Policy
    • Drug and Alcohol Policy
    • Vacation, Holiday, and Paid Time Off (PTO) Policy
    • Supplemental Policies for Remote Employees and Contractors
    • Supplemental Policy for Bonus, Commissions, and other Performance-based Payments
    • Supplemental Policies for Hourly International Contractors or Workers
    • Supplemental Policies for Hourly International Contractors or Workers
    • Disputes and Arbitration
  • Benefits and Perks
    • Health Care
    • Vacation, Holiday and Paid Time Off (PTO) Policy
    • Holiday List
  • Hiring Documents
    • Acknowledgement of Receipt
    • Partner Proprietary Information and Inventions Agreement
  • Engineering Wiki
    • Code Snippets
      • Front End Code Snippets
    • Setup
      • 1: Overview of development using Audienti
      • 2: How to setup your dev environment on Docker
      • 2a: Setting up on our cloud your dev server
      • 3: Connect to Production using the VPN
      • 4: Import data into your development environment
    • Deployment
      • Docker based deployment of back end (manual)
    • Culture
      • How our development team works
      • Code Best Practices
    • Tips
      • Setting up a new development machine
      • Importing data to Development environment
      • GIT workflow and work tracking
      • Using Slack
      • Using Rubocop
      • Our Code Standards
      • General suggested best practices
      • Tracking your time
      • Naming Iterations
    • Migrations
      • Postgres
      • ElasticSearch
      • Redis
    • Database and System Maintenance
      • Redis Howtos
      • Elasticsearch HowTos
      • Postgres HowTos
      • Administration recipes
      • App maintenance crash course notes
    • Front End
      • 2016 Plan
      • Deploy
      • Assets
      • SearchLogic
      • How to create UI components
      • OMA Standard Tables
    • Monitoring and Alerting
      • Monitoring Systems
      • Monitoring individual controller actions
      • Get notified when a metric reaches a certain threshold
      • Instrumenting your models using Oma Stats
      • Configuring Graphite Charts
      • Tracking your results with StatsD
      • Logging Fields
      • Updating Kibana Filtering
    • Testing
      • Coverage
      • Elasticsearch mapping config synchronization
      • Testing Gotchas
      • Rspec Preloader
      • Test Best Practices
    • Models
      • Backlinks
    • Queueing and Worker System
      • Queueing and Job Overview
    • Processors
      • Rebuilding Spot Instances
      • Deploying processors
      • Running processors in development
      • Reverting to the previous build on a failed deployment
    • Processors / Opportunity Pipeline
      • Opportunity Pipeline
      • Diagram
    • Processors / Enrichment Pipeline
      • Diagram
      • Clustering
    • Processors / Backlink Pipeline
      • Diagram
      • Backlink Pipeline external APIs
      • Backlink pipeline logic
    • Processors / Automation Pipeline
      • Diagram
      • Automation Pipeline Overview
      • Agents
      • Running in development
    • Messaging and Social Accounts
      • Overview
    • API
      • Audienti API
    • Algorithms
    • Troubleshooting
      • Elasticsearch
    • Big Data Pipeline Stuff
      • Spark
    • Our Product
      • Feature synopsis of our product
    • Research
      • Backend framework comparison
      • Internet marketing Saas companies
    • Code snippets
      • Commonly Used
      • Not Used
    • Miscellaneous
      • Proxies and Bax
    • Legacy & Deprecated
      • Search criteria component
      • Classes list
      • Target Timeline
      • Twitter processor
      • Asset compilation
      • Test related information
      • Interface to EMR Hadoop jobs
      • Mongo Dex Indexes to be Built
      • Mongodb errors
      • Opportunity pipeline scoring
      • Graph Page
      • Lead scoring
      • Insights
      • Shard keys
      • Setting up OMA on local
      • Clone project to local machine
      • Getting around our servers in AWS
  • Acknowledgements
  • Documents That Receiving Your First Payment Triggers Acknowledgement and Acceptanace
Powered by GitBook
On this page
  1. Engineering Wiki
  2. Processors / Automation Pipeline

Agents

Agents are the main component of the Automation Pipeline. They are the tool that receives events (in most cases) and processes them, emitting events.

The general Agent has the following configuration information:

  • description (required) = a description that will be shown on the front end

  • schedule (required) = how often that agent should run

  • keep_events_for (required) = how long before we delete its events

  • criteria (agent-dependent) = the criteria that is configured by the end user.

  • group_criteria (agent-dependent) = whether the agent subscribes to only certain events.

Agents have a "process" method that, in all cases, performs all of their processing. It is expected that, unless necessary, they will have an "process_event" as well. This will process each event. It is expected that the event's payload will be passed on downstream as well (unless the proces_event returns a false).

Agents, unless customized, use a "search_query" that will find their events to process. This query is run, and a list of "results" is produced. They are fed into a loop that runs the process_event on each of them.

Given all this "commonality", a child agent will have often just a "process_event" implemented. the rest can be inherited by the basic system.

Segment-based Agents

For segment-based agents (InfluencerSegment), they also take a "segment_id" as part of their configuration. They do not receive events. They only produce them. By default, they emit a new event every time a new object (Influencer, Mention, Backlink, Page) matches their segment criteria.

Conditional/Grooming Agents

There a a number of agents that are built to split events, limit the reate of events, filter events, etc. These are built as agents as well.

Update and Edit Information Agents

Currently there are Update Field (updates an object's field) and AddField (updates an event with data that is passed downstream). These are so we can update information. There is also a "Add to Negatives List" that will add an Influencer to the project's negatives list. This should have the consequence that it will take it out of processing across all project pipelines.

Service-based Agents

Based on our Service models (Twitter, Facebook GooglePlus, etc.) we can take actions with an object based on usage of this service (follow, unfollow, send messages, etc.). These use our existing message templates.

Connections

Agents are connected using a "connection". Both agents have to be saved first (so they have ids). These connections are what the agent uses to determine what events to listen for.

PreviousAutomation Pipeline OverviewNextRunning in development

Last updated 7 years ago