Often getting empty $_POST data and proxy_fcgi error / Apache2 / PHP-FPM

I have an Apache2 server with PHP-FPM (working on Ubuntu 20) and often I getting empty $_POST data from the clients, but Content-Length in header is > 0. I have not a small traffic (~20 req/sec (max) on PHP-FPM) and from some clients I receive an empty POST request (it can be 1 in hour or sometimes > 10). Here is the example header that I receiving:

[Content-Length] => 454
[Content-Type] => application/x-www-form-urlencoded
[User-Agent] => BestHTTP/2 v2.3.1
[Te] => identity
[Keep-Alive] => timeout=21
[Connection] => Keep-Alive, TE
[Accept-Encoding] => gzip, identity
[Host] => example.com

And some info from $_SERVER:

[USER] => www-data
[HOME] => /var/www
[SCRIPT_NAME] => some/path/set_score.php
[REQUEST_URI] => some/path/set_score.php
[REQUEST_METHOD] => POST
[SERVER_PROTOCOL] => HTTP/1.1
[GATEWAY_INTERFACE] => CGI/1.1
[REMOTE_PORT] => 43852
[SCRIPT_FILENAME] => //var/www/example.com/public_html/some/path/set_score.php
[SERVER_ADMIN] => webmaster@localhost
[CONTEXT_DOCUMENT_ROOT] => /var/www/example.com/public_html/
[REQUEST_SCHEME] => https
[DOCUMENT_ROOT] => /var/www/example.com/public_html/
[REMOTE_ADDR] => **.**.***.**
[SERVER_PORT] => 443
[SERVER_ADDR] => ***.***.***.***
[SERVER_NAME] => example.com
[SERVER_SOFTWARE] => Apache
[SERVER_SIGNATURE] => <address>Apache Server at example.com Port 443</address>

[PATH] => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
[CONTENT_LENGTH] => 454
[CONTENT in /var/www/example.com/public_html/some/path/set_score.php on line 3

As you can see Content-Length is > 0, but $_POST data array is empty. At the same time, this empty POST request often following with proxy_fcgi error:

[Sun May 02 04:17:59.777441 2021] [proxy_fcgi:error] [pid 105827:tid 140667812566784] (70007)The timeout specified has expired: [client ***.***.***.***:40171] AH01075: Error dispatching request to : (reading input brigade)

Also when this error occurs, in the access.log I see 408 code. And I don’t know the problem is the Apache configuration or PHP-FPM, because POST data that I sending from clients is small and it’s only text data. Strange that on the previous server (cloudways) I didn’t have those problems. But now when I did my own server, this magic error happens. Maybe they just ignore these types of errors or the timeout value was to big. My scripts execute time is < 0.1 seconds and I don’t need big timeout values (I guess).

Here is some important settings in my configurations:

apache2.conf

Timeout 60
KeepAlive On
MaxKeepAliveRequests 164
KeepAliveTimeout 5

HostnameLookups Off

<Directory />
        Options FollowSymLinks
        AllowOverride None
        Require all denied
</Directory>

<Directory /usr/share>
        AllowOverride None
        Require all granted
</Directory>

<Directory /var/www/>
        Options FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

/apache2/sites-available/example.com.conf

<VirtualHost *:443>
        ServerAdmin webmaster@localhost
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/example.com/public_html/

        SSLEngine on
        SSLProtocol -all +TLSv1.2 +TLSv1.3
        SSLCertificateFile /etc/ssl/certs/example.com.crt
        SSLCertificateKeyFile /etc/ssl/private/example.com.key
        SSLCertificateChainFile /etc/ssl/certs/ca-example.com.crt

        # tried to fix that "Error dispatching request to"
        <IfModule reqtimeout_module>
                RequestReadTimeout header=20-40,minrate=150
                RequestReadTimeout body=20,minrate=150
        </IfModule>

        <FilesMatch ".+.ph(ar|p|tml)$">
                SetHandler "proxy:unix:/run/php/php7.4-fpm-example.com.sock|fcgi://localhost/"
        </FilesMatch>
</VirtualHost>

<VirtualHost *:80>
        ServerName example.com
        ServerAlias www.example.com
        Redirect 301 / https://example.com/
</VirtualHost>

/php/7.4/fpm/pool.d/example.com.conf

[example.com]
user = www-data
group = www-data
listen = /run/php/php7.4-fpm-example.com.sock
listen.owner = www-data
listen.group = www-data

pm = dynamic
pm.max_children = 64
pm.start_servers = 16
pm.min_spare_servers = 8
pm.max_spare_servers = 16

php_admin_value[error_log] = /var/www/example.com/logs/error-fpm-php.log
php_admin_flag[log_errors] = on

Answer

Fixed the appearance of timeout errors by switching from HTTP/1.1 protocol to HTTP/2. Empty POST requests now come rarely, but they do.