High concurrency in Web
Have you ever wondered what kind of request volume or how many users a high-concurrency web application can support? An application that can only handle several hundred or several thousand PVs per day is not considered a high-concurrency system.
However, there is no clear definition online as to what constitutes high concurrency. Note that the term concurrency here does not refer to concurrency in programming. For information on concurrency and the difference between concurrency and parallelism in programming, please refer to this article. In the context of the internet or mobile internet, high concurrency refers to the number of users accessing your service at the same time.
Here is a post by Bird Brother that was found on Weibo:
According to a previous experience on Weibo, a simple business could handle about 200 QPS on a single machine, and after applying for about 100 machines, it could handle about 20,000 QPS.
First, let’s go over some concepts: QPS (TPS): the number of requests processed per second by a single process; TPS: the number of transactions processed per second by a single process; Throughput: the number of requests processed in a unit of time (usually determined by QPS and concurrency, note that QPS!=concurrency); Response Time: the average time taken by a system to respond to a request.
Concurrency refers to the number of visits that arrive simultaneously at a certain moment in time, while QPS refers to the number of requests responded to per second.
Here are a few formulas for a single machine: Total PV per day = QPS * 3600 * 6 Total PV per day = QPS * 3600 * 8 Number of servers = ceil(Total PV per day / Total PV per day per server)
Peak QPS and machine calculation formulas: 80% of visits occur in 20% of the day, so:
Peak QPS (per second) = (Total PV * 80%) / (Seconds in a day * 20%) Number of machines required for peak hours = Peak QPS / QPS per machine
Suppose that 3 million PVs occur on a single machine every day. Then, the QPS that this machine needs to achieve is: (3000000 * 0.8) / (86400 * 0.2) = 139. Suppose the QPS per machine is 90, then we would need (139/90) = 2 machines to support it.
In actual situations, server configuration and the complexity of business logic can significantly impact single-machine QPS.
So, can we estimate that Weibo’s peak PV during Spring Festival is approximately 400 million, calculated as 20,000 * (86400 * 0.2)?
How to Improve Concurrency Hardware solution: Add more machines
If one machine can’t handle it, we can add more machines. Then, technologies such as DB master-slave, read-write separation, and load balancing come into play.
The core idea is to distribute the pressure that was previously concentrated in one place. This method is simple, crude, and effective, but it is costly.
Software solution: Increase single-machine performance
In fact, how much a single machine’s performance can be increased depends on the machine’s configuration and how complex the service is.
Common ways to improve single-machine performance include: data caching, enabling php’s opcache, optimizing code (such as n+1 problem, nested loops, deep recursion, etc.), DB table optimization (such as indexes, sharding, and partitioning), and so on.