Close
Glad You're Ready. Let's Get Started!

Let us know how we can contact you.

Thank you!

We'll respond shortly.

PIVOTAL LABS
Standup 03/03/2009

Interesting Things

  • Somebody was seeing mongrels hang when using an older copy of the S3 gem. It turned out the older version had the option for persistent connections defaulting to true. Setting :persistent => false or using a newer version that has false as the default fixed their problem

  • One of our sites was seeing a unbalanced distribution of requests despite the fact that the load balancer was evenly distributing connections. One host typically had 2x the traffic of the others, and it would switch every few hours to be a different host. It turned out to be the Google crawler, which uses a keepalive, getting stuck on a single host and making a lot of requests. The load balancer is only able to balance TCP connections, which Google is only using a single one of. The likely solution will be haproxy or something similar in front of the hosts to better distribute traffic.

Comments
  1. JGeiger says:

    Another option for the bot issue is to dedicate a single host to just bots. That way you make sure everything gets crawled, but also you can then tweak that host (less horsepower) to make sure your big servers keep dealing with the customer load.

Post a Comment

Your Information (Name required. Email address will not be displayed with comment.)

* Copy This Password *

* Type Or Paste Password Here *