¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io

Re: [TDD] Long running test suite


 

Hi John,

I like your list of ideas, but I take a small exception with your comment about running tests on multiple threads.

I agree that it sometimes introduces non-deterministic results, but that's generally due to faults in the tests. Ideally, it should be possible to run any set of unit tests (i.e. microtests) in random order on parallel threads. In fact, it ought to be possible to surrender all control of ordering and parallelism to the runner. I grant that this is a hard goal to achieve.

Charlie

On Mon, Jun 6, 2016 at 6:38 PM, John Carter john.carter@... [testdrivendevelopment] <testdrivendevelopment@...> wrote:

?

Things I do to handle this....

* Benchmark. Every build spits out an order list of how long each test took. Attack the long ones first.
* Parallelism via Processes not threads. Multithreaded unit testing is a recipe for non-determinism and flakiness.
* No sleep rule. Find every test with a "sleep" in it and fix it. Sleeps, delays, timeouts are test smells of the first order.
* Unreal time. Wrap all time services and create a test interface that allows you to "roll time forward" and trigger all events in order.
* Some tests want to be able to run for days to get better state space coverage. Make that a "developer only" option. Run a tiny / fast subset for unit test checkin coverage and let developer run exhaustive test himself when needed.


On Mon, May 23, 2016 at 3:25 PM, Joselito D Moreno joenmoreno@... [testdrivendevelopment] <testdrivendevelopment@...> wrote:
?

Hello,

To those who have experienced test suites that run long, say ~25 minutes, what are some techniques you have used to mitigate it?? We still would like to be able to check in often and run the test suites prior to checking our code in to our repository but these long running tests makes us stretch what we mean with "often".

Joen

__._,




--
John Carter? ? ? ? ? ? ? ? ? ? ? ? ? ?? Phone : (64)(3) 358 6639
Tait Electronics? ? ? ? ? ? ? ? ? ? ? ? Fax?? : (64)(3) 359 4632
PO Box 1645 Christchurch? ? ? ? ? ? ? ? Email : john.carter@...
New Zealand



This Communication is Confidential. We only send and receive email on the
basis of the terms set out at


Join [email protected] to automatically receive all group messages.