guide
  • Introduction
  • Guiding Principles
    • Mission Statement
    • Conflict Resolution Process
  • Operating Model
    • Working Together
    • Holacracy
      • Meetings
      • Specific Roles
      • Terms and Definitions
      • Finer Points
      • Holacracy-Asana Key
    • Getting Things Done
      • Daily, Weekly, Monthly, and Annual Reviews
      • GTD-Asana Key
    • Transparency
    • Language
    • Budgeting
    • By Department
      • Engineering Operations
  • General Guidelines
  • Employment Policies
    • Equal Opportunity Employment
    • At-Will Employment
    • Code of Conduct in the Community
    • Complaint Policy
    • Drug and Alcohol Policy
    • Vacation, Holiday, and Paid Time Off (PTO) Policy
    • Supplemental Policies for Remote Employees and Contractors
    • Supplemental Policy for Bonus, Commissions, and other Performance-based Payments
    • Supplemental Policies for Hourly International Contractors or Workers
    • Supplemental Policies for Hourly International Contractors or Workers
    • Disputes and Arbitration
  • Benefits and Perks
    • Health Care
    • Vacation, Holiday and Paid Time Off (PTO) Policy
    • Holiday List
  • Hiring Documents
    • Acknowledgement of Receipt
    • Partner Proprietary Information and Inventions Agreement
  • Engineering Wiki
    • Code Snippets
      • Front End Code Snippets
    • Setup
      • 1: Overview of development using Audienti
      • 2: How to setup your dev environment on Docker
      • 2a: Setting up on our cloud your dev server
      • 3: Connect to Production using the VPN
      • 4: Import data into your development environment
    • Deployment
      • Docker based deployment of back end (manual)
    • Culture
      • How our development team works
      • Code Best Practices
    • Tips
      • Setting up a new development machine
      • Importing data to Development environment
      • GIT workflow and work tracking
      • Using Slack
      • Using Rubocop
      • Our Code Standards
      • General suggested best practices
      • Tracking your time
      • Naming Iterations
    • Migrations
      • Postgres
      • ElasticSearch
      • Redis
    • Database and System Maintenance
      • Redis Howtos
      • Elasticsearch HowTos
      • Postgres HowTos
      • Administration recipes
      • App maintenance crash course notes
    • Front End
      • 2016 Plan
      • Deploy
      • Assets
      • SearchLogic
      • How to create UI components
      • OMA Standard Tables
    • Monitoring and Alerting
      • Monitoring Systems
      • Monitoring individual controller actions
      • Get notified when a metric reaches a certain threshold
      • Instrumenting your models using Oma Stats
      • Configuring Graphite Charts
      • Tracking your results with StatsD
      • Logging Fields
      • Updating Kibana Filtering
    • Testing
      • Coverage
      • Elasticsearch mapping config synchronization
      • Testing Gotchas
      • Rspec Preloader
      • Test Best Practices
    • Models
      • Backlinks
    • Queueing and Worker System
      • Queueing and Job Overview
    • Processors
      • Rebuilding Spot Instances
      • Deploying processors
      • Running processors in development
      • Reverting to the previous build on a failed deployment
    • Processors / Opportunity Pipeline
      • Opportunity Pipeline
      • Diagram
    • Processors / Enrichment Pipeline
      • Diagram
      • Clustering
    • Processors / Backlink Pipeline
      • Diagram
      • Backlink Pipeline external APIs
      • Backlink pipeline logic
    • Processors / Automation Pipeline
      • Diagram
      • Automation Pipeline Overview
      • Agents
      • Running in development
    • Messaging and Social Accounts
      • Overview
    • API
      • Audienti API
    • Algorithms
    • Troubleshooting
      • Elasticsearch
    • Big Data Pipeline Stuff
      • Spark
    • Our Product
      • Feature synopsis of our product
    • Research
      • Backend framework comparison
      • Internet marketing Saas companies
    • Code snippets
      • Commonly Used
      • Not Used
    • Miscellaneous
      • Proxies and Bax
    • Legacy & Deprecated
      • Search criteria component
      • Classes list
      • Target Timeline
      • Twitter processor
      • Asset compilation
      • Test related information
      • Interface to EMR Hadoop jobs
      • Mongo Dex Indexes to be Built
      • Mongodb errors
      • Opportunity pipeline scoring
      • Graph Page
      • Lead scoring
      • Insights
      • Shard keys
      • Setting up OMA on local
      • Clone project to local machine
      • Getting around our servers in AWS
  • Acknowledgements
  • Documents That Receiving Your First Payment Triggers Acknowledgement and Acceptanace
Powered by GitBook
On this page
  1. Engineering Wiki
  2. Legacy & Deprecated

Mongo Dex Indexes to be Built

(by william, last updated: mar 2013)

{
    "runStats": {
        "linesRecommended": 711108, 
        "linesProcessed": 715011, 
        "linesPassed": 9497091
    }, 
    "results": [

        // parallel arrays 
        {
            "index": "{'status': 1, 'domain_id': 1, 'last_crawled': 1, 'potential_keywords': 1, 'tags': 1}", 
            "queryCount": 42, 
            "namespace": "marketfugraph_production.graph_pages", 
            "totalTimeMillis": 1439641, 
            "avgTimeMillis": 34277
        },

        // already exists (15 second query)
        {
            "index": "{'domain_id': 1, 'status': 1}", 
            "queryCount": 73, 
            "namespace": "marketfugraph_production.graph_pages", 
            "totalTimeMillis": 1142394, 
            "avgTimeMillis": 15649
        }, 

        // cannot index parallel arrays 
        {
            "index": "{'market_rank': 1, 'tags': 1, 'domain_id': 1, 'anchor_words': 1}", 
            "queryCount": 32, 
            "namespace": "marketfugraph_web_production.backlinks", 
            "totalTimeMillis": 16583, 
            "avgTimeMillis": 518
        },

        // already exists (3 second queries)
        {
            "index": "{'domain_id': 1, 'issue_category': 1, 'last_crawled': 1}", 
            "queryCount": 28, 
            "namespace": "marketfugraph_production.graph_pages", 
            "totalTimeMillis": 86210, 
            "avgTimeMillis": 3078
        }, 

        // cannot index parallel arrays 
        {
            "index": "{'project_ids': 1, 'ignore_project_ids': 1}", 
            "queryCount": 12, 
            "namespace": "marketfugraph_web_production.potential_keywords", 
            "totalTimeMillis": 14092, 
            "avgTimeMillis": 1174
        }, 

        // multi arrays
        {
            "index": "{'tag': 1, 'project_ids': 1, 'ignore_project_ids': 1}", 
            "queryCount": 9, 
            "namespace": "marketfugraph_web_production.potential_keywords", 
            "totalTimeMillis": 5068, 
            "avgTimeMillis": 563
        }
    ]
}
PreviousInterface to EMR Hadoop jobsNextMongodb errors

Last updated 7 years ago