guide
  • Introduction
  • Guiding Principles
    • Mission Statement
    • Conflict Resolution Process
  • Operating Model
    • Working Together
    • Holacracy
      • Meetings
      • Specific Roles
      • Terms and Definitions
      • Finer Points
      • Holacracy-Asana Key
    • Getting Things Done
      • Daily, Weekly, Monthly, and Annual Reviews
      • GTD-Asana Key
    • Transparency
    • Language
    • Budgeting
    • By Department
      • Engineering Operations
  • General Guidelines
  • Employment Policies
    • Equal Opportunity Employment
    • At-Will Employment
    • Code of Conduct in the Community
    • Complaint Policy
    • Drug and Alcohol Policy
    • Vacation, Holiday, and Paid Time Off (PTO) Policy
    • Supplemental Policies for Remote Employees and Contractors
    • Supplemental Policy for Bonus, Commissions, and other Performance-based Payments
    • Supplemental Policies for Hourly International Contractors or Workers
    • Supplemental Policies for Hourly International Contractors or Workers
    • Disputes and Arbitration
  • Benefits and Perks
    • Health Care
    • Vacation, Holiday and Paid Time Off (PTO) Policy
    • Holiday List
  • Hiring Documents
    • Acknowledgement of Receipt
    • Partner Proprietary Information and Inventions Agreement
  • Engineering Wiki
    • Code Snippets
      • Front End Code Snippets
    • Setup
      • 1: Overview of development using Audienti
      • 2: How to setup your dev environment on Docker
      • 2a: Setting up on our cloud your dev server
      • 3: Connect to Production using the VPN
      • 4: Import data into your development environment
    • Deployment
      • Docker based deployment of back end (manual)
    • Culture
      • How our development team works
      • Code Best Practices
    • Tips
      • Setting up a new development machine
      • Importing data to Development environment
      • GIT workflow and work tracking
      • Using Slack
      • Using Rubocop
      • Our Code Standards
      • General suggested best practices
      • Tracking your time
      • Naming Iterations
    • Migrations
      • Postgres
      • ElasticSearch
      • Redis
    • Database and System Maintenance
      • Redis Howtos
      • Elasticsearch HowTos
      • Postgres HowTos
      • Administration recipes
      • App maintenance crash course notes
    • Front End
      • 2016 Plan
      • Deploy
      • Assets
      • SearchLogic
      • How to create UI components
      • OMA Standard Tables
    • Monitoring and Alerting
      • Monitoring Systems
      • Monitoring individual controller actions
      • Get notified when a metric reaches a certain threshold
      • Instrumenting your models using Oma Stats
      • Configuring Graphite Charts
      • Tracking your results with StatsD
      • Logging Fields
      • Updating Kibana Filtering
    • Testing
      • Coverage
      • Elasticsearch mapping config synchronization
      • Testing Gotchas
      • Rspec Preloader
      • Test Best Practices
    • Models
      • Backlinks
    • Queueing and Worker System
      • Queueing and Job Overview
    • Processors
      • Rebuilding Spot Instances
      • Deploying processors
      • Running processors in development
      • Reverting to the previous build on a failed deployment
    • Processors / Opportunity Pipeline
      • Opportunity Pipeline
      • Diagram
    • Processors / Enrichment Pipeline
      • Diagram
      • Clustering
    • Processors / Backlink Pipeline
      • Diagram
      • Backlink Pipeline external APIs
      • Backlink pipeline logic
    • Processors / Automation Pipeline
      • Diagram
      • Automation Pipeline Overview
      • Agents
      • Running in development
    • Messaging and Social Accounts
      • Overview
    • API
      • Audienti API
    • Algorithms
    • Troubleshooting
      • Elasticsearch
    • Big Data Pipeline Stuff
      • Spark
    • Our Product
      • Feature synopsis of our product
    • Research
      • Backend framework comparison
      • Internet marketing Saas companies
    • Code snippets
      • Commonly Used
      • Not Used
    • Miscellaneous
      • Proxies and Bax
    • Legacy & Deprecated
      • Search criteria component
      • Classes list
      • Target Timeline
      • Twitter processor
      • Asset compilation
      • Test related information
      • Interface to EMR Hadoop jobs
      • Mongo Dex Indexes to be Built
      • Mongodb errors
      • Opportunity pipeline scoring
      • Graph Page
      • Lead scoring
      • Insights
      • Shard keys
      • Setting up OMA on local
      • Clone project to local machine
      • Getting around our servers in AWS
  • Acknowledgements
  • Documents That Receiving Your First Payment Triggers Acknowledgement and Acceptanace
Powered by GitBook
On this page
  • Running a ping on all pages in the site
  • Miscellaneous
  • External site access during test run
  • Tracker for pages that cause timeouts, memory exceeded errors and slow loading
  1. Engineering Wiki
  2. Legacy & Deprecated

Test related information

PreviousAsset compilationNextInterface to EMR Hadoop jobs

Last updated 7 years ago

Running a ping on all pages in the site

Follow these steps to run a quick fetch of all the pages of the system. The test runs against the production website for approximately 15 minutes.

  • git clone https://github.com/omalab/omatest.git

  • bundle install

  • cucumber feature/project/whole_shebang.feature:3

Once the test is finished, make sure to visit the PaperTrail log and see if any of the web dynos have exceeded their memory. If so, restart them.

To change the target from the production website to the staging website, goto features/step_definitions/login_common.rb and changevisit 'http://omatestuser75inc.omaengine.com'tovisit 'http://omatestuser75inc.omamatic.com'

Miscellaneous

External site access during test run

The following external sites are accessed during test run

  • lsapi.seomoz.com/linkscape/url-metrics/

  • us.api.semrush.com/?type=phrase_this

  • us.api.semrush.com/?type=phrase_related

  • us.api.semrush.com/?type=domain_organic

  • us.fullsearch-api.semrush.com/?type=phrase_fullsearch

  • us.fullsearch-api.semrush.com/?type=phrase_this

  • us.fullsearch-api.semrush.com/?type=url_organic

  • api.ahrefs.com/get_backlinks_count_with_anchor.php

  • api.url2png.com/v3/

  • Google Trends/Insights

Tracker for pages that cause timeouts, memory exceeded errors and slow loading

May 29 2013 - Removed n+1 queries

  • Number of times memory exceeded during test suite - 4

  • H12 timeouts - 4

  • Total processing time for test suite - 613s

  • Slowest pages

/projects/241/domains/872/traffic_box   30003
/projects/241/domains/872/trust_box 30003
/projects/241/domains/872/fact_box  30000
/projects/241/charts/common_sites?method=get&project_id=241&type=common_sites   30000
/projects/241/keywords/13536    15676
/projects/241/domains/872/load_data_links?&type=anchors 14424
/projects/241/charts/domains_backlink_symmetry?project_id=241   14074
/projects/241/domains/872/links/all 11698
/projects/241/domains/872/links/momentum    11277
/projects/241/domains/872/links/relevance   11013
/projects/241/domains/872/links/authority   10560
/projects/241/mentions/load_data?channel=forums 10038
/projects/241/domains/872/links/new_links   9623
/projects/241/competitor_research/individual_domain_analysis    9450
/projects/241/mentions/load_data?channel=twitter    7143
/projects/241/insights/load_tab?&tab=place%26people 4044
/projects/241/charts/related_keywords?method=get&project_id=241&type=related_keywords   3084
/projects/241/leads 3016
/projects/241/charts/domain_keyword_power?id=13536&method=get&project_id=241&type=backlinks 2968

May 28 2013 - Baseline

  • Number of times memory exceeded during test suite - 7

  • H12 timeouts - 7

  • Total processing time for test suite - (783s - first run; 550s - second run)

  • Slowest pages

/projects/241/domains/872/traffic_box   30007
/projects/241/domains/872/trust_box 30002
/projects/241/domains/872/fact_box  30000
/projects/241/charts/common_sites?method=get&project_id=241&type=common_sites   30000
/projects/241/charts/domains_backlink_symmetry?project_id=241   30000
/projects/241/insights/load_tab?&tab=place%26people 30000
/projects/241/leads 30000
/projects/241/contacts/load_list?&type=new_contacts 26592
/projects/241/domains/872/load_data?type=overview   25572
/projects/241/mentions/load_data?channel=web    25498
/projects/241/competitor_research/common_backlinks  15406
/projects/241/domains/872/links/all 14849
/projects/241/domains/872/links/strategic   14138
/projects/241/insights/load_tab?&tab=sentiment  10914
/projects/241/domains/872/links/authority   10645
/projects/241/keyword_research/research_keywords    9284
/projects/241/domains/872/links/new_links   8909
/projects/241/contacts/load_list?&type=contacts 8093
www.alexa.com/siteinfo
www.gravatar.com/avatar.php
www.google.com/s2/favicons