Apache php-fpm json response sometimes truncated

Recently we’ve changed over from EC2 to ECS Fargate. Both of these were run through an AWS application balancer. We’ve run into an issue on one of our endpoints where the JSON is being truncated at random, missing the end of the response, or sometimes it behaves just fine. Interestingly enough the response always has a length of 44149. The response still gives us a 200 either way.

To the best of my knowledge we’re trying to use the same PHP and Apache settings on our new system as we did previously. However on the new ECS system we’ve changed to using PHP-FPM instead of an apache mod.

Versions: PHP-FPM 7.2, Apache 2.4.29

New ECS system truncated response, transferred 42.40 kB (113.36 kB size), headers:

HTTP/2 200 OK
date: Tue, 07 Jul 2020 15:02:39 GMT
content-type: application/json
content-length: 44149
server: Apache/2.4.29 (Ubuntu)
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
x-frame-options: SAMEORIGIN
vary: Accept-Encoding
content-encoding: gzip
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
referrer-policy: origin-when-cross-origin
content-security-policy: default-src *  data: blob: 'unsafe-inline' 'unsafe-eval'
strict-transport-security: max-age=31536000
X-Firefox-Spdy: h2

New ECS system working response, transferred 43.68 kB (116.67 kB size), has the same headers as when truncated

Old EC2 system working response, transferred 43.67 kB (116.67 kB size), headers:

HTTP/2 200 OK
date: Tue, 07 Jul 2020 15:18:54 GMT
content-type: application/json
content-length: 44149
server: Apache
x-frame-options: SAMEORIGIN
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
vary: Accept-Encoding
content-encoding: gzip
x-xss-protection: 1; mode=block
x-content-type-options: nosniff
referrer-policy: origin-when-cross-origin
content-security-policy: default-src *  data: blob: 'unsafe-inline' 'unsafe-eval'
strict-transport-security: max-age=31536000
X-Firefox-Spdy: h2

I’m completely lost as to what to do to fix this, all of the Apache and PHP settings I’ve checked are the same on both the old and new system.

There is a similar stack overflow question to this one already, however it’s directed towards Nginx and other infrastructure that’s different to mine. The provided solutions on there either don’t work or aren’t applicable.

Answer

The issue was due to a bug in AWS load balancer, one of their engineers applied a fix for me which involved upgrading the balancer to use larger nodes. Case id 7164651631 if anyone else is having the same issue and wishes to reference the case when speaking to AWS support themselves.