PHPUnit Stalling in CI

We’re having an issue with PHPUnit sometimes failing to correctly run our test suite in CI. The behaviour manifests as roughly 1/3 tests failing to start and the CI step timing out. Interestingly enough, we don’t see this behaviour:

Locally on our machines
Every time we run the tests
Occur with the dusk tests

Our setup is to run the tests using docker compose. We specifically start the dependencies of the application container first and pause for 30 seconds to wait for them to come up (i.e. the database).

Some things I have tried to fix this that haven’t made a difference:

Running the tests with process isolation on.
Splitting the test suite into two to run less tests (both splits have the same problem)

The tests are running in Github Actions, this makes around ~7 gigs of ram available to the machines, along with multiple cores.

What is also interesting is that the dusk based tests have never stalled like this, despite them using the same docker-compose file but with additional services (like a Selenium).

I’m not sure if this has a clear answer, but I’m at the point where I’m out of ideas of what to try next (short of hosting the GitHub action runners ourselves).

Output from a failed test looks like:

+ sleep 30
+ docker-compose -f docker-compose.ci.yml run --rm postgres pg_isready -h postgres -d testing -U testing -t 30
postgres:5432 - accepting connections
+ docker-compose -f docker-compose.ci.yml run --no-deps backend phpunit '--filter=/::test[J-Z|j-z]/' --testdox
Pulling backend (***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5)...
acb548c6125ad97e57e8070c414644c7e6f385f5: Pulling from builder
Digest: sha256:c84a53dbb201b809a2325a93c52d4a362eefa6de64ab98f759e43c5dd363aac2
Status: Downloaded newer image for ***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5
PHPUnit 9.1.5 by Sebastian Bergmann and contributors.

##[error]The operation was canceled.

A successul run:

+ sleep 30
+ docker-compose -f docker-compose.ci.yml run --rm postgres pg_isready -h postgres -d testing -U testing -t 30
postgres:5432 - accepting connections
+ docker-compose -f docker-compose.ci.yml run --no-deps backend phpunit '--filter=/::test[A-I|a-i]/' --testdox
Pulling backend (***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5)...
acb548c6125ad97e57e8070c414644c7e6f385f5: Pulling from builder
Digest: sha256:c84a53dbb201b809a2325a93c52d4a362eefa6de64ab98f759e43c5dd363aac2
Status: Downloaded newer image for ***.dkr.ecr.ap-southeast-2.amazonaws.com/builder:acb548c6125ad97e57e8070c414644c7e6f385f5
PHPUnit 9.1.5 by Sebastian Bergmann and contributors.

Audit Controller (TestsFeatureAdminAuditController)
 ✔ Audits index

(I’ve truncated the full test output)

Sometimes I will see it reach the first test before stalling.

Answer

I had similar (but not the same) problem on Codeception framework for tests but I was not using Github Actions and docker.

The issue was in the way how database was deployed to the server during tests. Because of that database was deployed in long long time but after that, tests starts. We didn’t have limit on server so there was no time out.

On Local machine it was working fine.

As I see you are running functional/acceptance tests with database, so I propose to check module for database.

Advertisement

Answer