paper: Test-Driven Development Benefits Beyond Design Quality: Flow State and Developer Experience
Hey everyone,
Some months ago I asked for help in the group and some of you gently answered my survey; now the research paper is out!
You can check the preprint here
And I will be glad to discuss future work and evolutions of what is discussed here.
Merry Christmas,
Pedro Calais
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
I've not taken the time to look at the survey - sorry. However I did think the OP might be interested in a recent item from?Christiaan Verwijs:?
BTW Christiaan has been doing survey work towards his PhD for some years.
Cheers
Mark - a human
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
What do you mean by polarization?
I went on your personal page and discovered you've worked on polarization, biases and misinformation.
Like Steve I was surprised by the content of the survey oriented "feelings" and wondered if your goal was to measure how biased tdd practioners are or something like that.
Practicing tdd or not, polarize dev because it's hard to have tdd practioners and non-practioners working on the same source code.
Also I think, but I may be wrong, once you've been in tdd, you won't go back to writing tests after or no test at all.? I don't want to talk for others so personnaly, I'd immediatly refuse a project where I won't be able to do tdd (among other practices, this one is mandatory) It might seem an extreme position that might have interests for research.
Regards,? Gregory
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
Software development is a people problem, not a technical problem.? Understand that you are actually researching people and how they work, so do not ignore hundreds of years of social science research techniques.?
On Tue, Sep 13, 2022 at 10:08 AM Steven Gordon < sgordonphd@...> wrote: However, if you survey the people you observe, then you have something useful.? If you are not doing observations of how people are actually working, it?does not matter what your other dimensions are because it is divorced from reality,? Most people do not actually know how they work and/or answer?survey questions based on rationalizations.? Involve at least one real social scientist in your research group to avoid wasting your efforts.
Uncontrolled surveys are not just limited - they are deceptive and can easily move your research in a useless direction.
Hi Steve!
I see your point on the limits of surveys; that's why this is just on of the 3 dimensions we are looking at! And I think the survey is probably the weakest evidence due to the reasons you just pointed out... I think it has its value as a complement though.
This may be a nice read:
Survey Guidelines in Software Engineering: An Annotated Review
Pedro
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
However, if you survey the people you observe, then you have something useful.? If you are not doing observations of how people are actually working, it?does not matter what your other dimensions are because it is divorced from reality,? Most people do not actually know how they work and/or answer?survey questions based on rationalizations.? Involve at least one real social scientist in your research group to avoid wasting your efforts.
Uncontrolled surveys are not just limited - they are deceptive and can easily move your research in a useless direction.
Hi Steve!
I see your point on the limits of surveys; that's why this is just on of the 3 dimensions we are looking at! And I think the survey is probably the weakest evidence due to the reasons you just pointed out... I think it has its value as a complement though.
This may be a nice read:
Survey Guidelines in Software Engineering: An Annotated Review
Pedro
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
Hi Steve!
I see your point on the limits of surveys; that's why this is just on of the 3 dimensions we are looking at! And I think the survey is probably the weakest evidence due to the reasons you just pointed out... I think it has its value as a complement though.
This may be a nice read:
Survey Guidelines in Software Engineering: An Annotated Review
Pedro
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
Hi Gregory!
I will for sure share the results when the paper is ready.
What do you mean by polarization?
Best,
Pedro
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
Sorry, but as a retired academician and developer, such "promiscuous" surveys are totally worthless as a research tool.? No control for biases, especially self-selection biases.??
Please, work directly with software development organizations to observe what people do instead of what they say they do, and then survey only?the people you observe.? Why do computer scientists have no knowledge of social science research precepts?
Hey folks, how are you?
I am a member of a group of researchers interested in studying the impact of testing practices on developer's productivity and well-being.
?
Our goal is to improve the software engineering community understanding on how and why practices such as unit testing and test-driven development are effective beyond the impact on code design.
?
If you are willing to help, we have 12 quick questions to you!
Best,
Pedro
Computer Scientist P.hD.
|
Re: Survey: Impact of TDD on developer's productivity and well-being.
Hi Pedro, Is this work related to polarization ?
Would you share with us your results ?
Regards, ³Ò°ù¨¦²µ´Ç°ù²â
Hey folks, how are you?
I am a member of a group of researchers interested in studying the impact of testing practices on developer's productivity and well-being.
?
Our goal is to improve the software engineering community understanding on how and why practices such as unit testing and test-driven development are effective beyond the impact on code design.
?
If you are willing to help, we have 12 quick questions to you!
Best,
Pedro
Computer Scientist P.hD.
|
Survey: Impact of TDD on developer's productivity and well-being.
Hey folks, how are you? I am a member of a group of researchers interested in studying the impact of testing practices on developer's productivity and well-being.
?
Our goal is to improve the software engineering community understanding on how and why practices such as unit testing and test-driven development are effective beyond the impact on code design.
?
If you are willing to help, we have 12 quick questions to you!
Best, Pedro Computer Scientist P.hD.
|
Re: Please critique this TDD flow
Hi Mike,
?
my first impression is that you have some interesting logic that is now coupled to your GraphQL engine.? Are you sure you want this?? Since the business logic around your use case might be non-trivial, then implementing it all in GraphQL could be non-trivial and the resulting code might by hard to change.? It's all fair if the logic is very stable and you need every drop of performance.? An alternative way would be to query GraphQL for the minimal set of data that your logic needs to make a decision, and then implement the interesting business logic in a separate function, that is pure and synchronous.
I have a similar impression. You will probably notice, if you continue this way, that you will eventually be duplicating a lot of silly details about GraphQL in your tests, even though the code you write ends up being mostly plain JavaScript to make the new tests pass. At some point, you might decide that this feels wrong somehow and it would help you to isolate the plain JavaScript behavior from the GraphQL parts. This would make the tests simpler and execute more quickly, which improves your feedback loop and helps you change the "core business" behavior more easily.
I also question whether you really want a single function that does querying, business logic, and saving an updated state on the DB.? Perhaps you would be better served by 3 separate functions.
Again, I agree that this will probably eventually happen. The same advice applies: if you continue by putting everything in one place, then eventually you'll notice that some of your tests have copy/paste duplication of distracting details, and if you separate these three bits of behavior into separate pieces, then the distracting details go away and the remaining details feel much more relevant and meaningful. I interpret this as a good sign, generally.
Good luck. -- J. B. (Joe) Rainsberger :: ?:: ::
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: Please critique this TDD flow
Hi Mike,
?
my first impression is that you have some interesting logic that is now coupled to your GraphQL engine.? Are you sure you want this?? Since the business logic around your use case might be non-trivial, then implementing it all in GraphQL could be non-trivial and the resulting code might by hard to change.? It's all fair if the logic is very stable and you need every drop of performance.? An alternative way would be to query GraphQL for the minimal set of data that your logic needs to make a decision, and then implement the interesting business logic in a separate function, that is pure and synchronous.
?
I also question whether you really want a single function that does querying, business logic, and saving an updated state on the DB.? Perhaps you would be better served by 3 separate functions.
?
Hope this?helps a bit,
?
Matteo
|
Please critique this TDD flow
I have a domain entity called a "Booking Window" that has a Start Date and End Date, and defines a period in which a Member can make a Booking.? The business is a mobile car wash, and Members can book a wash every 14 days. I have a means of establishing the first Booking Window - at the point of the Member signing up.? Now I feel like the next most useful function is one that finds all Booking Windows closing "today" and makes new ones. So I wrote a test like this (kinda pseudo coded)
test("nothing to do when no booking windows at all", async () => {
? ? const client = stubGraphQlClientContainingNothing()
? ? const clock = new FixedClock(new Date())
? ? const windows = await makeTodaysBookingWindows(client, clock)
? ? expect(windows).toHaveLength(0)
})
and my function was just
export async function makeTodaysBookingWindows(client: GraphQlClient, clock: Clock): Promise<BookingWindow[]> {
? ? return []
?
}
Then I decided I needed a test of the case that had bookings, but none ending today:
test("nothing to do when no booking windows end today", async () => {
? ? const client = stubGraphQlClientContaining(bookingWindowEnding(tomorrow()))
? ? const clock = new FixedClock(new Date())
? ? const windows = await makeTodaysBookingWindows(client, clock)
? ? expect(windows).toHaveLength(0)
?
})
and the same function implementation allowed this to pass.
Now I decided I need to do a real case, of one Booking Window ending today:
test("create a new booking window when one expires today", async () => {
? ? const client = stubGraphQlClientContaining(bookingWindowEnding(today()))
? ? const clock = new FixedClock(new Date())
? ? const windows = await makeTodaysBookingWindows(client, clock)
? ? expect(windows).toHaveLength(1)
? ? expect(window[0].startDate).toBe(today())
? ? expect(window[0].endDate).toBe(today() + 14 days)
?
})
and that forced me to make some production code.
Now I am left thinking, what next?? The code I wrote to make the above test pass does not loop, so I need to cover the case where there are many Booking Windows ending today, so I write this:
test("create a new booking window for every window ending today", async () => {
? ? const client = stubGraphQlClientContaining(bookingWindowEnding(today()), bookingWindowEnding(today()))
? ? const clock = new FixedClock(new Date())
? ? const windows = await makeTodaysBookingWindows(client, clock)
? ? expect(windows).toHaveLength(2)
? ? expect(window[0].startDate).toBe(today())
? ? expect(window[0].endDate).toBe(today() + 14 days)
? ? expect(window[1].startDate).toBe(today())
? ? expect(window[1].endDate).toBe(today() + 14 days)
?
})
Now I probably have the production function I need, but the "create a new booking window when one expires today" test case is now irrelevant, so I guess the right thing to do is remove it.
Anyway, thats enough to communicate my approach to this function.? It doesn't "feel" great, so I'd appreciate knowing how others would approach this.
Thanks!
|
Re: Subclass to Test in JavaScript?
Here are two techniques for injecting a dependency using the previous example with minimal modification:
1. A function that returns the Product function
// the "real" production price logic function calculatePrice(sku, quantity) { ... } // Sort of a "constructor constructor" that allows // injecting alternate implementation function makeProduct(calculatePrice) { ? return function Product(sku) { ? ? let quantity, sku; ? ? return { ? ? ? setQuantity(val) { ? ? ? ? quantity = val ? ? ? }, ? ? ? getPrice() { ? ? ? ? return calculatePrice(sku, quantity) ? ? ? } ? ? } ? } } // in production, use the "real" implementation const Product = makeProduct(calculatePrice) // in test, something like: const TestProduct = makeProduct(calculatePriceTestDouble)
2. Optional argument with default value
function calculatePrice(sku, quantity) {...}
// default argument value that is overridable
function Product(sku, calculator = calculatePrice) {
? let quantity, sku;
? return {
? ? setQuantity(val) {
? ? ? quantity = val
? ? },
? ? getPrice() {
? ? ? return calculator(sku, quantity)
? ? }
? }
}
// in production, the default arg is used const realProduct = Product('abc-123')
// in test, override
const testProduct = Product('abc-123', calculatePriceTestDouble)
On Fri, Jun 10, 2022 at 5:16 AM J. B. Rainsberger < me@...> wrote: On Tue, Jun 7, 2022 at 5:23 AM Avi Kessner < akessner@...> wrote: I looked up Subclass to Test here,? and I'm still not sure if I'm understanding the use case correctly... However.?
One thing I like about JavaScript is the ability to just replace a function or property on an object with another one. I do this alot with reporting code.? const oldLog = console.log console.log = myNewFunction .... console.log = oldLog?
This lets me do some quick mocking for just a subsection of object without having to replicate everything.?
This is what I had expected to do, but since I was working with methods that changed the state of variables closed inside the object... I was just very very wrong about the whole thing. ?
The other thing we sometimes do is extract a new function which returns the function we are working on, and then allow our outer function to modify or provide some context variables.?
Ooh. Can you throw together a quick example of that? I'm not _quite_ getting it yet. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
Re: Subclass to Test in JavaScript?
On Tue, Jun 7, 2022 at 5:23 AM Avi Kessner < akessner@...> wrote: I looked up Subclass to Test here,? and I'm still not sure if I'm understanding the use case correctly... However.?
One thing I like about JavaScript is the ability to just replace a function or property on an object with another one. I do this alot with reporting code.? const oldLog = console.log console.log = myNewFunction .... console.log = oldLog?
This lets me do some quick mocking for just a subsection of object without having to replicate everything.?
This is what I had expected to do, but since I was working with methods that changed the state of variables closed inside the object... I was just very very wrong about the whole thing. ?
The other thing we sometimes do is extract a new function which returns the function we are working on, and then allow our outer function to modify or provide some context variables.?
Ooh. Can you throw together a quick example of that? I'm not _quite_ getting it yet. -- J. B. (Joe) Rainsberger :: ?:: ::
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: Subclass to Test in JavaScript?
? I don't believe that using the prototype chain quite accomplishes what is being asked for. "Objects, but not classes" implies the usage of closures to encapsulate data, and I *think* that what is being asked for is to have access to the bits that are "closed" similar to how one might?access protected members in a class in java. The?closed values are not accessible to "subclasses" on the prototype chain.
Indeed, this was the conceptual mistake I made on stage. I was overriding a method that queried a closed variable in the SUT and simultaneously expecting the underlying value to change later. It was a mess and I hope I was just tired. :P
I'm imagining that the legacy code you are talking about is structured something like?this:
// a function that returns an object function Product(sku) {
? // these values are "closed" aka "private" ? let quantity, sku;
? // this is also "private" / closed ? function calculatePrice() { ? ? ?// ... some logic here that we want to either: ? ? ?//? ?1. test directly or? ? ? ?//.? 2. stub to allow us to test other parts of this object ? ? ?// this function's behavior depends on the closed values ? }
? // the object returned ? return { ? ? setQuantity(val) { ? ? ? quantity = val ? ? }.? ? ? getPrice() { ? ? ? calculatePrice() ? ? } ? } }
In this case, there is not?a good way to get access to the "innards" without modifying the SUT.
Thank you for confirming this. I think I understand better now. -- J. B. (Joe) Rainsberger :: ?:: ::
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: Subclass to Test in JavaScript?
I looked up Subclass to Test here,? and I'm still not sure if I'm understanding the use case correctly... However.?
One thing I like about JavaScript is the ability to just replace a function or property on an object with another one. I do this alot with reporting code.? const oldLog = console.log console.log = myNewFunction .... console.log = oldLog?
This lets me do some quick mocking for just a subsection of object without having to replicate everything.?
The other thing we sometimes do is extract a new function which returns the function we are working on, and then allow our outer function to modify or provide some context variables.?
And lastly, we might refactor a module and what it exports and requires to modify dependencies that go in or out.?
On Sun, 5 Jun 2022, 15:42 J. B. Rainsberger, < me@...> wrote: Hi, folks. I embarrassed myself in a Surviving Legacy Code training course recently by fumbling with Subclass to Test in front of a bunch of JavaScript programmers. It occurred to me that I don't know the object-based analog to this approach. I waved my hands a little, then thankfully, I ran the clock out without much of a resolution.
I'd like to make it up to them and I need your help.
What is the corresponding technique when working in a language with objects but not classes? In JavaScript, for example, how do I achieve the same effect of being able to create a "testable" (more-inspectable) object in my test with minimal changes to the legacy code object definition? In JavaScript, can I do the equivalent of making a property visible to "subclasses" without making it widely-visible to all legacy clients? Or should I think about it a different way?
My goal is to make minimally-disruptive changes to the existing code, while adding tests for it that would support refactoring. I want the equivalent of overriding _this small part_ so that I can simulate/control it in order to test that other part of the same object.
I feel like I'm missing something obvious, since I'm barely functionally literate in JavaScript.
Thanks. J. B. Rainsberger :: :: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
Re: Subclass to Test in JavaScript?
Sadly Dave is right here - the contents of a closure are, well,
closed. Hence the name.
Luckily Crockford's Module Pattern is a rare sight these days, I can
count on the fingers of one mouse the number of times I've seen it in
the wild in the last ten years. These days the focus tends to be far
more on 'faux-O' Javascript, 'Haskell-wannabe Typescript', or some
functional-style code that looks like a mashup of Clojure, Ruby &
Absinthe.
If the Module Pattern _is_ your problem, then a refactoring like
'extract method to helper object' is a reasonable choice, and the path
you'd take would be very similar to that in a typical OO language
environment.
Fox
---
toggle quoted message
Show quoted text
On Sun, 5 Jun 2022 at 17:42, Dave Foley <davidmfoley@...> wrote:
I don't believe that using the prototype chain quite accomplishes what is being asked for.
"Objects, but not classes" implies the usage of closures to encapsulate data, and I *think* that what is being asked for is to have access to the bits that are "closed" similar to how one might access protected members in a class in java.
The closed values are not accessible to "subclasses" on the prototype chain.
I'm imagining that the legacy code you are talking about is structured something like this:
// a function that returns an object
function Product(sku) {
// these values are "closed" aka "private"
let quantity, sku;
// this is also "private" / closed
function calculatePrice() {
// ... some logic here that we want to either:
// 1. test directly or
//. 2. stub to allow us to test other parts of this object
// this function's behavior depends on the closed values
}
// the object returned
return {
setQuantity(val) {
quantity = val
}.
getPrice() {
calculatePrice()
}
}
}
In this case, there is not a good way to get access to the "innards" without modifying the SUT.
On Sun, Jun 5, 2022 at 8:41 AM Sleepyfox <sleepyfox@...> wrote:
Hi JB,
This is trivially simple in JS, simply create a new object (your "subclass to test") that has its prototype set to your SUT object.
All of your SUT methods and attributes will now be accessible in your child object.
Simples!
---
a = {
f: (x) => x + 1,
z: 2
}
b = {}
b.__proto__ = a
console.log(b.z) // 2
console.log(b.f(4)) // 5
On Sun, 5 Jun 2022, 13:43 J. B. Rainsberger, <me@...> wrote:
Hi, folks. I embarrassed myself in a Surviving Legacy Code training course recently by fumbling with Subclass to Test in front of a bunch of JavaScript programmers. It occurred to me that I don't know the object-based analog to this approach. I waved my hands a little, then thankfully, I ran the clock out without much of a resolution.
I'd like to make it up to them and I need your help.
What is the corresponding technique when working in a language with objects but not classes? In JavaScript, for example, how do I achieve the same effect of being able to create a "testable" (more-inspectable) object in my test with minimal changes to the legacy code object definition? In JavaScript, can I do the equivalent of making a property visible to "subclasses" without making it widely-visible to all legacy clients? Or should I think about it a different way?
My goal is to make minimally-disruptive changes to the existing code, while adding tests for it that would support refactoring. I want the equivalent of overriding _this small part_ so that I can simulate/control it in order to test that other part of the same object.
I feel like I'm missing something obvious, since I'm barely functionally literate in JavaScript.
Thanks.
J. B. Rainsberger :: :: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
Re: Subclass to Test in JavaScript?
I don't believe that using the prototype chain quite accomplishes what is being asked for. "Objects, but not classes" implies the usage of closures to encapsulate data, and I *think* that what is being asked for is to have access to the bits that are "closed" similar to how one might?access protected members in a class in java. The?closed values are not accessible to "subclasses" on the prototype chain.
I'm imagining that the legacy code you are talking about is structured something like?this:
// a function that returns an object function Product(sku) {
? // these values are "closed" aka "private" ? let quantity, sku;
? // this is also "private" / closed ? function calculatePrice() { ? ? ?// ... some logic here that we want to either: ? ? ?//? ?1. test directly or? ? ? ?//.? 2. stub to allow us to test other parts of this object ? ? ?// this function's behavior depends on the closed values ? }
? // the object returned ? return { ? ? setQuantity(val) { ? ? ? quantity = val ? ? }.? ? ? getPrice() { ? ? ? calculatePrice() ? ? } ? } }
In this case, there is not?a good way to get access to the "innards" without modifying the SUT.
Hi JB,
This is trivially simple in JS, simply create a new object (your "subclass to test") that has its prototype set to your SUT object.
All of your SUT methods and attributes will now be accessible in your child object.
Simples! --- a = { ? f: (x) => x + 1, ? z: 2 }
b = {} b.__proto__ = a console.log(b.z)? ? ?// 2 console.log(b.f(4))? // 5
On Sun, 5 Jun 2022, 13:43 J. B. Rainsberger, < me@...> wrote: Hi, folks. I embarrassed myself in a Surviving Legacy Code training course recently by fumbling with Subclass to Test in front of a bunch of JavaScript programmers. It occurred to me that I don't know the object-based analog to this approach. I waved my hands a little, then thankfully, I ran the clock out without much of a resolution.
I'd like to make it up to them and I need your help.
What is the corresponding technique when working in a language with objects but not classes? In JavaScript, for example, how do I achieve the same effect of being able to create a "testable" (more-inspectable) object in my test with minimal changes to the legacy code object definition? In JavaScript, can I do the equivalent of making a property visible to "subclasses" without making it widely-visible to all legacy clients? Or should I think about it a different way?
My goal is to make minimally-disruptive changes to the existing code, while adding tests for it that would support refactoring. I want the equivalent of overriding _this small part_ so that I can simulate/control it in order to test that other part of the same object.
I feel like I'm missing something obvious, since I'm barely functionally literate in JavaScript.
Thanks. J. B. Rainsberger :: :: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
Re: Subclass to Test in JavaScript?
Hi JB,
This is trivially simple in JS, simply create a new object (your "subclass to test") that has its prototype set to your SUT object.
All of your SUT methods and attributes will now be accessible in your child object.
Simples! --- a = { ? f: (x) => x + 1, ? z: 2 }
b = {} b.__proto__ = a console.log(b.z)? ? ?// 2 console.log(b.f(4))? // 5
On Sun, 5 Jun 2022, 13:43 J. B. Rainsberger, < me@...> wrote: Hi, folks. I embarrassed myself in a Surviving Legacy Code training course recently by fumbling with Subclass to Test in front of a bunch of JavaScript programmers. It occurred to me that I don't know the object-based analog to this approach. I waved my hands a little, then thankfully, I ran the clock out without much of a resolution.
I'd like to make it up to them and I need your help.
What is the corresponding technique when working in a language with objects but not classes? In JavaScript, for example, how do I achieve the same effect of being able to create a "testable" (more-inspectable) object in my test with minimal changes to the legacy code object definition? In JavaScript, can I do the equivalent of making a property visible to "subclasses" without making it widely-visible to all legacy clients? Or should I think about it a different way?
My goal is to make minimally-disruptive changes to the existing code, while adding tests for it that would support refactoring. I want the equivalent of overriding _this small part_ so that I can simulate/control it in order to test that other part of the same object.
I feel like I'm missing something obvious, since I'm barely functionally literate in JavaScript.
Thanks. J. B. Rainsberger :: :: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
Pedro Calais
Mark Levison
Gregory Salvan
J. B. Rainsberger
Sleepyfox