Producer-consumer Model Based Thread Pool Design

The concurrency of the server has a crucial impact on the efficiency of processing requests on the server side, so the thread pool technology has been widely applied to the server. Different thread pools play an important role in improving the performance of highly concurrent servers. This article first introduces socket technology, thread pools, etc., which are high-concurrency server development technologies in Linux systems, then proposes a thread pool model based on the producer consumer model, and gives key codes. Finally, the performance is compared through experiments . Experimental results show that lightweight servers can achieve higher concurrency by using this thread pool model.


Introduction
With the development of information technology, countless users communicate on the Internet every day, and the server plays an important role in it. Therefore, the server receives more and more requests. And how to allow the server to handle a large number of concurrent connections in a short time while also ensuring the reliability of the server has become the first problem to be solved in the server development process.
Since the advent of the Internet, server technology has evolved along with the development of the Internet industry. The early server was a traditional Unix model, which is a single-process server, and a new process was created to process each new request. Today's mainstream multi-process servers derive a certain amount of processes in advance, and wait for the request to be directly allocated to the process. Typical multi-process servers include Apache and Microsoft IIS. With the advent of multi-core processors, multi-threaded server technology has also been developed. The Nginx server is a typical multi-threaded server. Domestic Taobao, NetEase and other companies have built a server framework based on Nginx.
For light-weight servers, multi-process servers consume more memory and have lower concurrent processing power. Frequent creation of multi-threads also consumes more light servers. Therefore, traditional multi-process / multi-thread based concurrent servers can no longer meet the requirements of high concurrency and high reliability. To deal with the shortcomings of traditional concurrent servers, this paper proposes a thread pool based on the ideas of producers and consumers, which can keep lightweight servers in a most efficient state. Socket is an interface specification used to directly access the communication protocol, the program can directly access TCP, UDP and other protocols to receive and send data through the socket. According to the definition of RFC793, the socket is composed of the port number and IP.

High concurrent server development technology
2. 1. 2. Socket programming framework. Server communication process:(1) Call the socket () function to create a socket and return the file descriptor "listenfd".
(2) Call the bind (listenfd, server address port) function to bind the socket with the local address and port number.
(3) Call the listen (listenfd, connection queue length) function to declare the socket as listening mode.
(4) Call the accept (listenfd, client address port) function. After the three-way handshake, the server calls this function to block waiting for the client to connect, and returns a new socket "connfd" for communication with the client.
(5) Call the read (connfd, buf, size) and write (connfd, buf, size) functions to communicate with the client.
(6) Call the close (connfd) function to close the socket. Client communication process: (1) Call the socket () function to create a socket and return the file descriptor "fd".
(2) Call the connect (fd, server address port) function, send a request to the server, and establish a connection.
(3) Call write (fd, buf, size) and read (fd, buf, size) functions to communicate with the server.
(4) Call the close (fd) function to close the socket. The specific flowchart is shown in Figure 1:

Producer-consumer model
The producer-consumer model is a model that solves the problem of strong coupling between producers and consumers by using a blocking queue. There is no direct data communication between the two, but communication through blocking queues. After the producer produces the data, it is not submitted to the consumer, but directly placed in the blocking queue, and then the consumer directly fetches the data from the blocking queue. The blocking queue provides a buffer that can balance the processing power of producers and consumers, and can be used to decouple producers and consumers. In general, the producer-consumer model has the advantages of decoupling, support for concurrency, and support for uneven busyness.

3. 1. Processes and threads.
A process is a basic unit for resource allocation in an operating system, and a thread is an execution unit within a process, and is a basic unit that is independently scheduled and dispatched by the operating system. All resources owned by the process are shared among threads. Compared with processes, threads have the advantages of low resource consumption and fast creation speed. Therefore, in the process of creating highly concurrent servers, multi-threaded programming has gradually become the mainstream.

3. Thread pool technology.
Although the resource consumption required to create a thread is greatly reduced compared to the creation process, if the server needs to process a task with a short execution time and frequent requests, the server will continuously create and destroy threads. In this process, a lot of resources are wasted. Therefore, it is not suitable for high concurrency situations. The thread pool technology creates a certain amount of threads in advance, which saves the resources for creating and destroying threads, making the server respond faster and more efficient.

1. Creation of thread pool model
The thread pool model based on the producer-consumer model is shown in Figure 2. For the client and the task queue, the client is the producer and the task queue is the consumer, then the client puts a request in the task queue; for the thread pool and the task queue, the thread pool is the consumer and the task queue is the producer, then the thread pool takes the request sent by the client from the task queue. (1) The task queue is responsible for accepting client connection requests and placing them in the task queue. When the task queue is full, it blocks clients to put requests into the task queue; when the task queue is empty, it blocks clients to take the task from the task queue.
(2) The thread pool includes the manager thread and the worker thread. The manager thread is not responsible for processing tasks. Instead, it queries the task queue every time whether the task queue is empty. If there are tasks in the task queue and there are no idle threads or there are too many working threads, the manager thread will create a certain amount of threads, also it can destroy idle threads according to a certain ratio; the worker thread is only responsible for processing tasks. When there is a task in the task queue, the blocked worker thread is woken up and the worker thread takes the task from the task queue and removes the task from the task queue.
The specific workflow is shown in Figure 3. First, the worker thread and the manager thread are created by initializing the thread pool. The number of initial worker threads can be given a value according to actual needs. Then when the task queue is empty, the thread blocks and waits for the task queue to receive the request. When the task queue is not empty, the blocked thread is awakened, the worker thread takes the task from the head of the task queue, and finally the task processing function of the worker thread starts to process the obtained request.

2. Thread pool code implementation
Some key codes of thread pool creation are as follows. (2) Initialize the task queue: initialize the relevant parameters of the task queue, the maximum number of nodes is determined according to actual needs. taskQueue* taskqueue_init(int maxsize) { taskQueue* queue = (taskQueue*)malloc(sizeof(taskQueue)); queue->size = 0; queue->maxsize = maxsize; queue->front = 0; queue->tail = 0; return queue; } (3) Add tasks to the task queue: call this function in the main function, through the callback function parameter void * (* function) (void * arg) of the function, you can directly insert the task into the task queue and use pthread_cond_wait () and pthread_cond_signal () function to block and wake up the thread.

Thread pool test results and analysis
Siege is a high-performance stress testing tool that can be used for stress testing and performance evaluation of servers. Therefore, in order to test the performance of the thread pool proposed in this paper, a simple web server was written, one does not use optimization, and the other uses thread pool optimization. The siege tool simulates multiple concurrent access servers. The response time and the number of successful transmissions of the two are collected. Some performance data are shown in Table  1 and  It can be seen from Figure 4 that when the number of concurrent connections simulated by the siege tool is increasing, the response time of the server is also increasing. When the number of concurrency is not high, the difference between the response time of the original server and the thread pool server is not large, but once the number of concurrency is higher than about 400, the response time of the original server will rise rapidly, while the increment of the thread pool server remains unchanged, and the overall response time is lower than the original server.
Other performance changes, such as the number of successful transmissions and transaction volume per second, are generally similar to the response time. At low concurrent numbers, the performance difference between the two servers is not large, but when the concurrent number is too high, the original server 's Performance will drop dramatically, and the thread pool server will remain stable.
Based on the above experimental results, compared with the original server, the thread pool model studied in this paper not only improves the performance, but also improves the stability. Therefore, it can be applied to a lightweight and highly concurrent server.

Conclusion
This article reveals some of the problems with traditional servers by introducing modern highconcurrency server development techniques. In order to solve these problems, this paper proposes a thread pool model based on producer-consumer model. High concurrency is achieved as much as possible and the server's operating efficiency is optimized. Through the analysis of experimental results , the lightweight server based on this thread pool has good stability and high reliability.