Request queuing on engine yard explained (SSL Edition)
Developer resources
Table of content
We believe that performance is a critical measurement for any web application. Users tend to use fast applications more and are in general more satisfied when it feels snappy. The statement “performance is a feature” says it all (and for example page speed is a factor in Google’s ranking, for a reason). That is why we measure many different metrics to know how LingoHub “feels” like. In our constant battle to make LingoHub faster, stronger, better, we always came across “Request Queuing” time:
Requesting Queuing explained in simple terms
“The time the request spends before actually being processed.” In our case, the time the request sits on NGINX and waits to get processed by Rails. There are many good articles that explain it in much more detail and better and more accurately than I did:
http://blog.newrelic.com/2013/01/22/understanding-new-relic-queuing/
https://support.cloud.engineyard.com/entries/21382793-New-Relic-Best-Practices
Request Queuing, performance and SSL encryption
As you can see in the above diagram, request queuing takes almost as much time as the actual request processing. Seems like a low hanging fruit, right? So something can be done there. Queuing means that there are not enough Rails instances to handle our requests. Hence throwing in more servers to handle the load will fix the problem. Sadly this wasn’t the case, because in our investigation with the help of EngineYard Support (who are awesome by the way), we realized that the reason behind this constant request queuing is usage of HTTPS for all requests. In times like these (Hi there PRISM) encryption is a reliable way to keep out prying eyes. LingoHub was 100% SSL secured from the get go. A secured connection requires several round-trips because of the required handshake. Such a handshake takes around 50-300ms depending on the location, ping time, etc. (worse for mobile). So the request is waiting at NGINX to finish the handshake before it is getting delivered to Rails.
So what can we do? Certainly not sacrificing security over performance, instead we have to look at other places to get out more performance. We will keep you posted on our battle for performance. Please share your ideas in the comments as well.
Related articles
Dev Bit: Auto generate UUID with Postgres and Rails 4
Learn how to auto-generate UUIDs with PostgreSQL and Rails 4 using the uuid-ossp extension and database-level defaults.
Dev Bit: How to add plugins to elasticsearch under Mac/Brew
Learn how to install Elasticsearch plugins on Mac with Homebrew, find the right libexec path, and verify plugins like Kopf or Elastic HQ.
Dev Bit: How to reuse matched value of regex in JavaScript's replace() function
JavaScript's replace() function has special $ references that can be used with regular expressions to replace (sub)strings. In this article, we provide a detailed overview of how it works.
Dev Bit: SQL word count & character count with Postgres
LingoHub uses Postgres and Elasticsearch to manage its extensive collection of text data. If you're wondering whether there's a way to conduct an SQL word count for translations, the answer is yes. In this article, we'll provide you with two SQL queries that can be used to determine the character and word count of a text column.