Re: "The design signals are weak"
On Fri, Jul 21, 2023 at 9:56?PM Vaughn Cato via <vaughn_cato= [email protected]> wrote: ? I find that the design signals that I get directly during refactoring are actually pretty weak. There are code smells, but these aren't the signals. The code smells are things that you or other people have determined generally tend to make the design better, but this is just something that you recognize from experience.
What makes this "weak" for you? The signals can only say "Pay attention here!" I'm not sure they could ever say anything stronger than that.
But then, the signals from test runs can only say "Pay attention here!" as well. What makes them stronger for you than the "code smell"/"design risk" signals? ? I find the actual design signals come about when trying to create a test or to make a test pass. I get strong signals such as "It's not clear to me how to make this pass", or "I'm having to make changes in a lot of places to make this pass," or "I'm having to write a lot of test code to show what I'm trying to achieve." This leads to me doing more preparatory refactoring than other kinds of refactoring, because the signals are strongest then.
I don't find those signals stronger nor weaker, but merely different. What makes them feel stronger for you? J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: "The design signals are weak"
I find this pretty provocative, so of course, that interests me immediately
Allow me to begin with a disclaimer: that the thoughts that follow are still in something of a draft phase, and may indeed lead nowhere useful when pursued to their logical end.? K?
Excellent! That's what we're here for. :) Which "survey literature" has drawn conclusions about the strength of design error signals?
None that I know of - as far as I can tell, the question doesn't even exist yet outside of this thread.
The conclusions (currently weakly held) are mine, based on what twenty? twenty-five? of reading without finding much of anything that I recognize as addressing the topic.
Got it. What kind of "weak" is your "weak" here?
Consider the refactoring task in the TDD cycle: make a behavior preserving change, run the tests, loop until some stop condition is reached.
If we make a mistake, the change that we intended to be behavior preserving may not be.? But when we run the tests, the tests detect the change, and give us a strong signal that an error has occurred.? If we are running the tests frequently (as the discipline tells us to do) then the tests further tell us that the error was recent.? Taken to an extreme, the mistake must have occurred in the last change you made (and the remedy is trivial - revert and try again).
I would call this a strong signal: tests fail, processes return error statuses, green pixels turn read, the cloister bell sounds.? Really hard to argue that you are "doing TDD" if you miss the signals.
Now, consider instead a different sort of mistake; the intended change is behavior preserving, and the change is executed as intended by the programmer, but the change makes the design worse.
- What, if any, signals do we get to alert us to the fact that the design has been made worse?
- How much time passes before the programmer is aware of the signal
- How does TDD improve these answers, in comparison to either
- just doing it
- the next best alternative
The answer certainly isn't the execution of the tests, because the tests will bless any behavior preserving change.
I follow this so far. I haven't read the rest of this conversation yet, so it's possible that someone else will have asked the same questions or raised the same points and you've answered them.
# "Weak" compared to what?
When you call this a "weak signal", are you calling it "weak" only in comparison to the obviously "strong signal" of "tests should have passed, but they failed"? or are you judging them "weak" by some other standard? I think I'd like to know about that standard.
# Other signals
I identify at least two kinds of signals coming from practising TDD:
- The result of running the tests, often reduced to "all pass" or "some fail". If it's not the result I expected, then that reminds me to investigate further. - The annoyance of reading or writing the tests. If that feels strange, then that reminds me to reconsider my design choices.
You seem to classify the second kind of signal as "actually pretty weak", which I'd like to understand further. Certainly I agree that, on average, Signals of the Second Kind are weaker than Signals of the First Kind.
Regarding only Signals of the Second Kind, I tend to interpret them most often as "design risk signals". TDD taught me to adopt this principle: If the tests seem poorly designed, then assume that the cause lies in the design of the production code. I consider this a kind of primary principle that distinguishes TDD from test-first programming.
Some design risk signals seem quite strong to me, such as "Why do I need to parse text in my assertion to check arithmetic?!" and others feel quite weak to me, such as "These side-effects seem related to each other, so maybe it's time to introduce an intermediate abstraction that simplifies the interaction. Maybe they're 3 implementations of the same interface." I could imagine someone claiming that these signals are weak on average, but they don't feel weak on average to me. Some of them feel weak (I don't always react to them when I see them) and some of them feel strong (this is obviously a problem and I will fix it unless I have a very good reason not to). Do you mean "weak" and "strong" in similar ways to this?
I could also imagine someone saying "I don't need TDD to notice these problems", which must be obviously true. Do you mean "weak" in a way similar to this? I don't consider this to point to the strength of the signals, but rather whether the programmer already gets those signals "strongly enough" from somewhere else.
Am I anywhere close to what you mean by "weak" here? If not, then I'd like to know what it's like for you.
Thanks! J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: Which comes first: design skill or TDD?
On Wed, Jul 19, 2023 at 3:55?PM Vaughn Cato via <vaughn_cato= [email protected]> wrote: ? One thing that helped me with design by using TDD was how it encourages making constant changes to the code. It makes changes necessary because you haven't thought too far ahead, and it makes changes safe because of the quality of the tests and how fast you can run them. Because you are constantly changing the code, this leads to wanting to find ways to make those changes easier, and code that is easy and safe to change strongly overlaps with well-designed code. Reducing coupling and increasing cohesion is one of those things that you find makes code easier to change.
This is perhaps one aspect of Evolutionary Design in general that I emphasize: it normalizes changing code; it helps reverse the belief that changing code is a failure of prediction and that there is a "first time" to "get it right".
I recently sidestepped an argument on Reddit about this. It had been a long time since I'd encountered someone who genuinely believed that one must always get the design right the first time and every other way to work is wrong. They framed it as something like "Why would you waste time building it the wrong way just so that you can build it the right way later?!" They seemed unable to see that it might not be possible to see "the right way" and "the wrong way" in advance or that those labels might change or that those labels are not helpful in the first place.
I don't know where people are learning this, apart from generic "life lessons" of perfectionism drilled into them when they were very young.
J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: "The design signals are weak"
Olof,
I found that much of my design pressure came from the REEN "cat" state. After I made the test pass, I often noticed that the code that did so didn't communicate well. Often it was coupled with something else, or had two concepts intermingled in a what that wasn't cohesive. This is what drove my refactoring.
- George
toggle quoted message
Show quoted text
On 7/24/23 7:17 AM, Olof Bjarnason wrote: I have a couple of opinions/experiences I want to share on the?topic of design pressure and TDD.
TDD has three 'states' which I think of as animals or colors:
?BLUE/Owl: "add a test" (understand the problem / see the big picture)
?RED/Rabbit: "make it pass" (as quickly as possible / run for your life)
?GREEN/Cat: "make it right" (improve readability / enjoy life)
We start in the blue state, adding a test. This is also the state we finish in, when we are 'done' with some features/story/thing for now, at least.
The highest design pressure comes from BLUE state - formulating the tests first in TDD, not the implementation (RED/"make it pass!") and only slightly from refactoring* (GREEN/"make it right!"). Why? Well the pressure?to think "how can I test (automatically/transparently/economically) this thing I'm building right now, before it's visible to the user/end-to-end?" immediately?instead of after hours/days/weeks of development, is what focuses me to write things in what I consider to be better design.
I have several times, especially from senior developers, heard that this is "just one side of design" or "focusing only on testability" or similar sentiments. While I philosophically?agree, I think that having a design which is not automatically testable in an economically viable way (unit tests are much cheaper than end-to-end tests e.g) isn't really serious?or product oriented. Such designs are valuable for short-term throw-away code or proof-of-concepts/spikes, not so much for products or long-lived software (and it can be hard to decide what sort of code you are working on at times, but that is a separate discussion).
I re-invented my own view of what good design is when I started out with TDD in 2006, and I think that requires an effort of curiosity?and time. It is not comfortable if you are stuck in your own ideas of what?good design is -- especially if that design isn't very testable. Prime example: letting go of patterns which makes testing harder, like the Singleton pattern from GoF.
*GREEN/refactoring state of TDD does give you a chance to improve things, both in test code and production code. Especially for the 'internal' quality of the production code (implementation details). The API/external design is also possible to change, but at a higher cost than from writing the first test / API design. That is why I think the highest pressure is in BLUE rather than GREEN state.
/Olof
On Sat, 22 Jul 2023 at 18:04, Steve Gordon <sgordonphd@... <mailto:sgordonphd@...>> wrote:
Promiscuous pairing and code reviews are remedies you can find in
the literature that try to address this problem,? If you find any
deterministic rules for determining whether a design is good, I
would be extremely skeptical of those deterministic rules. Otherwise, there would be software that you could run on the code
base after every change to see if the design got worse.
I would add if proposed requirement changes are sometimes pushed off
or rejected due to how much work it would entail, then that is a
good signal that the design might be poor (or that the changes were
ill conceived).
On Fri, Jul 21, 2023 at 5:34?PM
<groups-io.20191212@...
<mailto:groups-io.20191212@...>> wrote:
I find this pretty provocative, so of course, that interests
me immediately
Allow me to begin with a disclaimer: that the thoughts that
follow are still in something of a draft phase, and may indeed
lead nowhere useful when pursued to their logical end.? K?
Which "survey literature" has drawn conclusions about the
strength of design error signals?
None that I know of - as far as I can tell, the question doesn't
even exist yet outside of this thread.
The conclusions (currently weakly held) are mine, based on what
twenty? twenty-five? of reading without finding much of anything
that I recognize as addressing the topic.
What kind of "weak" is your "weak" here?
Consider the refactoring task in the TDD cycle: make a behavior
preserving change, run the tests, loop until some stop condition
is reached.
If we make a mistake, the change that we intended to be behavior
preserving may not be.? But when we run the tests, the tests
detect the change, and give us a strong signal that an error has
occurred.? If we are running the tests frequently (as the
discipline tells us to do) then the tests further tell us that
the error was recent.? Taken to an extreme, the mistake must
have occurred in the last change you made (and the remedy is
trivial - revert and try again).
I would call this a strong signal: tests fail, processes return
error statuses, green pixels turn read, the cloister bell
sounds.? Really hard to argue that you are "doing TDD" if you
miss the signals.
Now, consider instead a different sort of mistake; the intended
change is behavior preserving, and the change is executed as
intended by the programmer, but the change makes the design /worse.
/
1. What, if any, signals do we get to alert us to the fact that
the design has been made worse?
2. How much time passes before the programmer is aware of the
signal
3. How does TDD improve these answers, in comparison to either
1. just doing it
2. the next best alternative
The answer certainly isn't the execution of the tests, because
the tests will bless any behavior preserving change.
-- Danil
--
----------------------------------------------------------------------
* George Dinwiddie *
Software Development
Consultant and Coach
----------------------------------------------------------------------
|
Re: "The design signals are weak"
I advocate postponing refactoring to any design pattern if related stories are still in the backlog,?especially if they are slices of the same bigger story.? The implementation of the later stories may reveal a better pattern.
I have a couple of opinions/experiences I want to share on the?topic of design pressure and TDD.
TDD has three 'states' which I think of as animals or colors:
?BLUE/Owl: "add a test" (understand the problem / see the big picture) ?RED/Rabbit: "make it pass" (as quickly as possible / run for your life) ?GREEN/Cat: "make it right" (improve readability / enjoy life)
We start in the blue state, adding a test. This is also the state we finish in, when we are 'done' with some features/story/thing for now, at least.
The highest design pressure comes from BLUE state - formulating the tests first in TDD, not the implementation (RED/"make it pass!") and only slightly from refactoring* (GREEN/"make it right!"). Why? Well the pressure?to think "how can I test (automatically/transparently/economically) this thing I'm building right now, before it's visible to the user/end-to-end?" immediately?instead of after hours/days/weeks of development, is what focuses me to write things in what I consider to be better design.
I have several times, especially from senior developers, heard that this is "just one side of design" or "focusing only on testability" or similar sentiments. While I philosophically?agree, I think that having a design which is not automatically testable in an economically viable way (unit tests are much cheaper than end-to-end tests e.g) isn't really serious?or product oriented. Such designs are valuable for short-term throw-away code or proof-of-concepts/spikes, not so much for products or long-lived software (and it can be hard to decide what sort of code you are working on at times, but that is a separate discussion).
I re-invented my own view of what good design is when I started out with TDD in 2006, and I think that requires an effort of curiosity?and time. It is not comfortable if you are stuck in your own ideas of what?good design is -- especially if that design isn't very testable. Prime example: letting go of patterns which makes testing harder, like the Singleton pattern from GoF.
*GREEN/refactoring state of TDD does give you a chance to improve things, both in test code and production code. Especially for the 'internal' quality of the production code (implementation details). The API/external design is also possible to change, but at a higher cost than from writing the first test / API design. That is why I think the highest pressure is in BLUE rather than GREEN state.
/Olof
Promiscuous pairing and code reviews are remedies you can find in the literature that try to address this problem,? If you find any deterministic rules for determining whether a design is good, I would be extremely skeptical of those deterministic rules.? Otherwise, there would be software that you could run on the code base after every change to see if the design got worse.
I would add if proposed requirement changes are sometimes pushed off or rejected due to how much work it would entail, then that is a good signal that the design might be poor (or that the changes were ill conceived).
I find this pretty provocative, so of course, that interests me immediately
Allow me to begin with a disclaimer: that the thoughts that follow are still in something of a draft phase, and may indeed lead nowhere useful when pursued to their logical end.? K?
Which "survey literature" has drawn conclusions about the strength of design error signals?
None that I know of - as far as I can tell, the question doesn't even exist yet outside of this thread.
The conclusions (currently weakly held) are mine, based on what twenty? twenty-five? of reading without finding much of anything that I recognize as addressing the topic.
What kind of "weak" is your "weak" here?
Consider the refactoring task in the TDD cycle: make a behavior preserving change, run the tests, loop until some stop condition is reached.
If we make a mistake, the change that we intended to be behavior preserving may not be.? But when we run the tests, the tests detect the change, and give us a strong signal that an error has occurred.? If we are running the tests frequently (as the discipline tells us to do) then the tests further tell us that the error was recent.? Taken to an extreme, the mistake must have occurred in the last change you made (and the remedy is trivial - revert and try again).
I would call this a strong signal: tests fail, processes return error statuses, green pixels turn read, the cloister bell sounds.? Really hard to argue that you are "doing TDD" if you miss the signals.
Now, consider instead a different sort of mistake; the intended change is behavior preserving, and the change is executed as intended by the programmer, but the change makes the design worse.
- What, if any, signals do we get to alert us to the fact that the design has been made worse?
- How much time passes before the programmer is aware of the signal
- How does TDD improve these answers, in comparison to either
- just doing it
- the next best alternative
The answer certainly isn't the execution of the tests, because the tests will bless any behavior preserving change.
-- Danil
|
Re: "The design signals are weak"
I have a couple of opinions/experiences I want to share on the?topic of design pressure and TDD.
TDD has three 'states' which I think of as animals or colors:
?BLUE/Owl: "add a test" (understand the problem / see the big picture) ?RED/Rabbit: "make it pass" (as quickly as possible / run for your life) ?GREEN/Cat: "make it right" (improve readability / enjoy life)
We start in the blue state, adding a test. This is also the state we finish in, when we are 'done' with some features/story/thing for now, at least.
The highest design pressure comes from BLUE state - formulating the tests first in TDD, not the implementation (RED/"make it pass!") and only slightly from refactoring* (GREEN/"make it right!"). Why? Well the pressure?to think "how can I test (automatically/transparently/economically) this thing I'm building right now, before it's visible to the user/end-to-end?" immediately?instead of after hours/days/weeks of development, is what focuses me to write things in what I consider to be better design.
I have several times, especially from senior developers, heard that this is "just one side of design" or "focusing only on testability" or similar sentiments. While I philosophically?agree, I think that having a design which is not automatically testable in an economically viable way (unit tests are much cheaper than end-to-end tests e.g) isn't really serious?or product oriented. Such designs are valuable for short-term throw-away code or proof-of-concepts/spikes, not so much for products or long-lived software (and it can be hard to decide what sort of code you are working on at times, but that is a separate discussion).
I re-invented my own view of what good design is when I started out with TDD in 2006, and I think that requires an effort of curiosity?and time. It is not comfortable if you are stuck in your own ideas of what?good design is -- especially if that design isn't very testable. Prime example: letting go of patterns which makes testing harder, like the Singleton pattern from GoF.
*GREEN/refactoring state of TDD does give you a chance to improve things, both in test code and production code. Especially for the 'internal' quality of the production code (implementation details). The API/external design is also possible to change, but at a higher cost than from writing the first test / API design. That is why I think the highest pressure is in BLUE rather than GREEN state.
/Olof
Promiscuous pairing and code reviews are remedies you can find in the literature that try to address this problem,? If you find any deterministic rules for determining whether a design is good, I would be extremely skeptical of those deterministic rules.? Otherwise, there would be software that you could run on the code base after every change to see if the design got worse.
I would add if proposed requirement changes are sometimes pushed off or rejected due to how much work it would entail, then that is a good signal that the design might be poor (or that the changes were ill conceived).
I find this pretty provocative, so of course, that interests me immediately
Allow me to begin with a disclaimer: that the thoughts that follow are still in something of a draft phase, and may indeed lead nowhere useful when pursued to their logical end.? K?
Which "survey literature" has drawn conclusions about the strength of design error signals?
None that I know of - as far as I can tell, the question doesn't even exist yet outside of this thread.
The conclusions (currently weakly held) are mine, based on what twenty? twenty-five? of reading without finding much of anything that I recognize as addressing the topic.
What kind of "weak" is your "weak" here?
Consider the refactoring task in the TDD cycle: make a behavior preserving change, run the tests, loop until some stop condition is reached.
If we make a mistake, the change that we intended to be behavior preserving may not be.? But when we run the tests, the tests detect the change, and give us a strong signal that an error has occurred.? If we are running the tests frequently (as the discipline tells us to do) then the tests further tell us that the error was recent.? Taken to an extreme, the mistake must have occurred in the last change you made (and the remedy is trivial - revert and try again).
I would call this a strong signal: tests fail, processes return error statuses, green pixels turn read, the cloister bell sounds.? Really hard to argue that you are "doing TDD" if you miss the signals.
Now, consider instead a different sort of mistake; the intended change is behavior preserving, and the change is executed as intended by the programmer, but the change makes the design worse.
- What, if any, signals do we get to alert us to the fact that the design has been made worse?
- How much time passes before the programmer is aware of the signal
- How does TDD improve these answers, in comparison to either
- just doing it
- the next best alternative
The answer certainly isn't the execution of the tests, because the tests will bless any behavior preserving change.
-- Danil
|
Re: "The design signals are weak"
Promiscuous pairing and code reviews are remedies you can find in the literature that try to address this problem,? If you find any deterministic rules for determining whether a design is good, I would be extremely skeptical of those deterministic rules.? Otherwise, there would be software that you could run on the code base after every change to see if the design got worse.
I would add if proposed requirement changes are sometimes pushed off or rejected due to how much work it would entail, then that is a good signal that the design might be poor (or that the changes were ill conceived).
I find this pretty provocative, so of course, that interests me immediately
Allow me to begin with a disclaimer: that the thoughts that follow are still in something of a draft phase, and may indeed lead nowhere useful when pursued to their logical end.? K?
Which "survey literature" has drawn conclusions about the strength of design error signals?
None that I know of - as far as I can tell, the question doesn't even exist yet outside of this thread.
The conclusions (currently weakly held) are mine, based on what twenty? twenty-five? of reading without finding much of anything that I recognize as addressing the topic.
What kind of "weak" is your "weak" here?
Consider the refactoring task in the TDD cycle: make a behavior preserving change, run the tests, loop until some stop condition is reached.
If we make a mistake, the change that we intended to be behavior preserving may not be.? But when we run the tests, the tests detect the change, and give us a strong signal that an error has occurred.? If we are running the tests frequently (as the discipline tells us to do) then the tests further tell us that the error was recent.? Taken to an extreme, the mistake must have occurred in the last change you made (and the remedy is trivial - revert and try again).
I would call this a strong signal: tests fail, processes return error statuses, green pixels turn read, the cloister bell sounds.? Really hard to argue that you are "doing TDD" if you miss the signals.
Now, consider instead a different sort of mistake; the intended change is behavior preserving, and the change is executed as intended by the programmer, but the change makes the design worse.
- What, if any, signals do we get to alert us to the fact that the design has been made worse?
- How much time passes before the programmer is aware of the signal
- How does TDD improve these answers, in comparison to either
- just doing it
- the next best alternative
The answer certainly isn't the execution of the tests, because the tests will bless any behavior preserving change.
-- Danil
|
Re: "The design signals are weak"
I find that the design signals that I get directly during refactoring are actually pretty weak. There are code smells, but these aren't the signals. The code smells are things that you or other people have determined generally tend to make the design better, but this is just something that you recognize from experience. I find the actual design signals come about when trying to create a test or to make a test pass. I get strong signals such as "It's not clear to me how to make this pass", or "I'm having to make changes in a lot of places to make this pass," or "I'm having to write a lot of test code to show what I'm trying to achieve." This leads to me doing more preparatory refactoring than other kinds of refactoring, because the signals are strongest then.
? - Vaughn
On Friday, July 21, 2023 at 08:34:18 PM EDT, <groups-io.20191212@...> wrote:
I find this pretty provocative, so of course, that interests me immediately
Allow me to begin with a disclaimer: that the thoughts that follow are still in something of a draft phase, and may indeed lead nowhere useful when pursued to their logical end.? K?
Which "survey literature" has drawn conclusions about the strength of design error signals?
None that I know of - as far as I can tell, the question doesn't even exist yet outside of this thread. The conclusions (currently weakly held) are mine, based on what twenty? twenty-five? of reading without finding much of anything that I recognize as addressing the topic.
What kind of "weak" is your "weak" here?
Consider the refactoring task in the TDD cycle: make a behavior preserving change, run the tests, loop until some stop condition is reached. If we make a mistake, the change that we intended to be behavior preserving may not be.? But when we run the tests, the tests detect the change, and give us a strong signal that an error has occurred.? If we are running the tests frequently (as the discipline tells us to do) then the tests further tell us that the error was recent.? Taken to an extreme, the mistake must have occurred in the last change you made (and the remedy is trivial - revert and try again). I would call this a strong signal: tests fail, processes return error statuses, green pixels turn read, the cloister bell sounds.? Really hard to argue that you are "doing TDD" if you miss the signals. Now, consider instead a different sort of mistake; the intended change is behavior preserving, and the change is executed as intended by the programmer, but the change makes the design worse.
- What, if any, signals do we get to alert us to the fact that the design has been made worse?
- How much time passes before the programmer is aware of the signal
- How does TDD improve these answers, in comparison to either
- just doing it
- the next best alternative
The answer certainly isn't the execution of the tests, because the tests will bless any behavior preserving change. -- Danil
|
Re: "The design signals are weak"
I find this pretty provocative, so of course, that interests me immediately
Allow me to begin with a disclaimer: that the thoughts that follow are still in something of a draft phase, and may indeed lead nowhere useful when pursued to their logical end.? K?
Which "survey literature" has drawn conclusions about the strength of design error signals?
None that I know of - as far as I can tell, the question doesn't even exist yet outside of this thread. The conclusions (currently weakly held) are mine, based on what twenty? twenty-five? of reading without finding much of anything that I recognize as addressing the topic.
What kind of "weak" is your "weak" here?
Consider the refactoring task in the TDD cycle: make a behavior preserving change, run the tests, loop until some stop condition is reached. If we make a mistake, the change that we intended to be behavior preserving may not be.? But when we run the tests, the tests detect the change, and give us a strong signal that an error has occurred.? If we are running the tests frequently (as the discipline tells us to do) then the tests further tell us that the error was recent.? Taken to an extreme, the mistake must have occurred in the last change you made (and the remedy is trivial - revert and try again). I would call this a strong signal: tests fail, processes return error statuses, green pixels turn read, the cloister bell sounds.? Really hard to argue that you are "doing TDD" if you miss the signals. Now, consider instead a different sort of mistake; the intended change is behavior preserving, and the change is executed as intended by the programmer, but the change makes the design worse.
- What, if any, signals do we get to alert us to the fact that the design has been made worse?
- How much time passes before the programmer is aware of the signal
- How does TDD improve these answers, in comparison to either
- just doing it
- the next best alternative
The answer certainly isn't the execution of the tests, because the tests will bless any behavior preserving change. -- Danil
|
Re: Does an AI assistant help with learning/using TDD?
Thanks for the scientific report on AI generation of unit test Mauricio - very interesting.? David Koontz
?? ? Email: ?david@... ?? ? (360) 259-8380 ? ? ?http://about.me/davidakoontz
toggle quoted message
Show quoted text
On Jul 20, 2023, at 5:40 AM, Avi Kessner <akessner@...> wrote:
? What interests me more is if the LLM can write code based on the tests I write, rather than generating tests for the code I write. brought to you by the letters A, V, and I
and the number 47 This paper () shows that these
models are still quite bad at writing good tests.
Nevertheless, as an assistant, maybe they can help. We're about to
start more formal experiments around this topic.
Cheers,
--
Mauricio Aniche
Author of Effective Software Testing: A Developer's Guide
On Wed, Jul 19, 2023 at 10:56?PM David Koontz <david@...> wrote:
>
> Here is my attempt at AI unit test creation...
>
> Well the quick story - it didn't write good tests - but it jump started my writing test for my calculator app I'm building in Swift.
>
> I feel that with better prompts (prompt-engineering is a thing) the tool would produce Good Tests.
>
>
>
>
|
Re: Does an AI assistant help with learning/using TDD?
What interests me more is if the LLM can write code based on the tests I write, rather than generating tests for the code I write. brought to you by the letters A, V, and I
and the number 47
This paper () shows that these
models are still quite bad at writing good tests.
Nevertheless, as an assistant, maybe they can help. We're about to
start more formal experiments around this topic.
Cheers,
--
Maur¨ªcio Aniche
Author of Effective Software Testing: A Developer's Guide
On Wed, Jul 19, 2023 at 10:56?PM David Koontz <david@...> wrote:
>
> Here is my attempt at AI unit test creation...
>
> Well the quick story - it didn't write good tests - but it jump started my writing test for my calculator app I'm building in Swift.
>
> I feel that with better prompts (prompt-engineering is a thing) the tool would produce Good Tests.
>
>
>
>
|
Re: Does an AI assistant help with learning/using TDD?
This paper () shows that these
models are still quite bad at writing good tests.
Nevertheless, as an assistant, maybe they can help. We're about to
start more formal experiments around this topic.
Cheers,
--
Maur¨ªcio Aniche
Author of Effective Software Testing: A Developer's Guide
toggle quoted message
Show quoted text
On Wed, Jul 19, 2023 at 10:56?PM David Koontz <david@...> wrote:
Here is my attempt at AI unit test creation...
Well the quick story - it didn't write good tests - but it jump started my writing test for my calculator app I'm building in Swift.
I feel that with better prompts (prompt-engineering is a thing) the tool would produce Good Tests.
|
Re: Does an AI assistant help with learning/using TDD?
Here is my attempt at AI unit test creation... ?
Well the quick story - it didn't write good tests - but it jump started my writing test for my calculator app I'm building in Swift.
I feel that with better prompts (prompt-engineering is a thing) the tool would produce Good Tests.
|
Re: Which comes first: design skill or TDD?
One thing that helped me with design by using TDD was how it encourages making constant changes to the code. It makes changes necessary because you haven't thought too far ahead, and it makes changes safe because of the quality of the tests and how fast you can run them. Because you are constantly changing the code, this leads to wanting to find ways to make those changes easier, and code that is easy and safe to change strongly overlaps with well-designed code. Reducing coupling and increasing cohesion is one of those things that you find makes code easier to change.
On Wednesday, July 19, 2023 at 09:34:33 AM EDT, J. B. Rainsberger <me@...> wrote:
? But TDD doesn't teach you about coupling and cohesion, it doesn't teach you about 'clean code' (whatever you determine that to mean), it doesn't even teach you about removal of duplication - that's Simple Design.
I don't think I've ever encountered exactly this point before. To the extent that it's true, I find it useless. :) Even so, I appreciate its attempt to be precise and to highlight the potential for confusion due to differences in understanding of terms.
I claim that practising TDD guides one to learn these things, because one can't _drive_ one's development using tests while ignoring what happens in the mind when you try. :) One thing that happens: one gets bored from typing the same thing over and over again and seeks a way to reduce the effort, which seems quite likely to at least encourage managing duplication somehow. If not, then what could possibly be happening during the "refactor" step?
A hypothetical programmer who is entirely unmoved by seeing duplication, who seems content literally to duplicate code all day long, is perhaps not refactoring, and therefore couldn't practise TDD.
This leads me, however, to wonder idly about what refactoring looks like if we use principles other than the Four Elements of Simple Design to inform our refactoring choices. I don't think I'm capable of imagining how that would actually happen, even though I can imagine that it's theoretically possible. I suppose someone who wanted to refactor, but didn't know those things, would be forced to find them, and therefore TDD would be the trigger for learning those things.
J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
On Tue, Jul 4, 2023 at 12:13?PM Tim Ottinger < tottinge@...> wrote: ? Freedom to refactor, knowing you can check your work after each variable rename to know if you've made a mistake.?
This sounds like "Freedom from getting it right the first time". Is there more than that to this particular freedom for you? J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
"The design signals are weak"
? But signals about design errors?? My experience, and my survey of the literature, suggests those signals are actually pretty weak.
The Koss/Martin bowling game exercise offers one candidate example, where a compiler error triggered a re-evaluation of the use of "value objects" in the API; you'll have to consider whether reverting to a general purpose data structure was a design improvement or not....
I find this pretty provocative, so of course, that interests me immediately. :)
Two questions leap to mind:
- What kind of "weak" is your "weak" here? - Which "survey literature" has drawn conclusions about the strength of design error signals? Most of the attempts I've seen have measured what happens when people who don't know much about design try to write tests first, notably undergraduate computer science students.
To be clear, I routinely experience design signals that are relatively weak individually, but become stronger in aggregate, such as "I see 3 reasons to do refactoring X, so now I feel confident that doing it will help". I also notice that, as I become more experienced, I react to the signals more softly, meaning as suggestions or hints, rather than as urgent calls to action. I don't consider that a weakening of the signal, but rather an indication that I have become less dogmatic in my practice.
To provide examples of how I think of "strong" or "weak", I can think of at least two common examples of design signals that I'd label "strong":?
- scattered duplication of behavior focused on a naked, primitive data type (a number or some text) which points towards introducing a Whole Value type - rampant duplication in application request handlers which points towards introducing Parsers and extracting Standard Workflows
If you didn't know those options existed, I'd expect duplication in either production code or tests to provide the motivation to seek them out. J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: Which comes first: design skill or TDD?
? But TDD doesn't teach you about coupling and cohesion, it doesn't teach you about 'clean code' (whatever you determine that to mean), it doesn't even teach you about removal of duplication - that's Simple Design.
I don't think I've ever encountered exactly this point before. To the extent that it's true, I find it useless. :) Even so, I appreciate its attempt to be precise and to highlight the potential for confusion due to differences in understanding of terms.
I claim that practising TDD guides one to learn these things, because one can't _drive_ one's development using tests while ignoring what happens in the mind when you try. :) One thing that happens: one gets bored from typing the same thing over and over again and seeks a way to reduce the effort, which seems quite likely to at least encourage managing duplication somehow. If not, then what could possibly be happening during the "refactor" step?
A hypothetical programmer who is entirely unmoved by seeing duplication, who seems content literally to duplicate code all day long, is perhaps not refactoring, and therefore couldn't practise TDD.
This leads me, however, to wonder idly about what refactoring looks like if we use principles other than the Four Elements of Simple Design to inform our refactoring choices. I don't think I'm capable of imagining how that would actually happen, even though I can imagine that it's theoretically possible. I suppose someone who wanted to refactor, but didn't know those things, would be forced to find them, and therefore TDD would be the trigger for learning those things.
J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: Which comes first: design skill or TDD?
On Tue, Jul 4, 2023 at 11:06?AM Kevin Rutherford < kevin@...> wrote: JB,
I'm quite surprised by your second claim, that "One can learn design
skill by practising TDD".
Can you support that?
I did it, so it must be possible. :) I'm not sure what seems surprising about the claim. What do you find surprising in it? J. B. (Joe) Rainsberger :: ?:: ::
Replies from this account routinely take a few days, which allows me to reply thoughtfully. I reply more quickly to messages that clearly require answers urgently. If you need something from me and are on a deadline, then let me know how soon you need a reply so that I can better help you to get what you need. Thank you for your consideration.
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
Re: Searching for a book that helps introduce JavaScript or Java Programming?
This I think ropes you into being a
collaborator on the update Mark. Welcome aboard!
:-)
|
| Jeff Langr / +1-719-287-4335
|
|
|
|
|
|
toggle quoted message
Show quoted text
Josue
- thanks this a gem. Since Jeff is on this (I think) we can just ask
him to update his book for my daughter. Jeff - Thanks in advance for
updating your book for a 16yr :-)
Cheers
Mark
|
Re: Searching for a book that helps introduce JavaScript or Java Programming?
Thanks for the recommendation Josue!
The book indeed is on long-EOL'd Java; it was written in 2004 and timed
to coincide with the release of Java 1.5.
I've had a number of folks request an update, but I'm currently working
on updating Pragmatic Unit Testing in Java for the same reason. So--no
time to do so currently. I also think it would warrant a complete
overhaul, which'd be significant effort.
It'd be easy enough for a sharp dev to learn the old version, then learn
what's new in Java since, but I think that's asking a lot.
I'm open to a collaboration effort on a rewrite next year. At least one
other person had expressed an interest.
Cheers,
Jeff
|
| Jeff Langr / +1-719-287-4335
|
|
|
|
|
|
toggle quoted message
Show quoted text
For Java:
Agile Java: Crafting Code with
Test-Driven Development -??
It is the one that I indicate when they ask for a
book to learn Java. As I think to learn any language it is important to
start together?with tests (because code without tests is legacy code), I
tend to appreciate this kind of book.
The
problem is that it seems there is no updated?version of the book. It's
very old Java. When I was teaching using this book as a base, I helped
the?trainers to change the language?constructs to more modern ones.
Maybe it could be a problem for a beginner?alone.
-- Abra?os,
´³´Ç²õ³Ü¨¦
|
J. B. Rainsberger
David Koontz