On Tue, Mar 29, 2022 at 5:31 PM Russell Gold < russ@...> wrote:
On Mar 29, 2022, at 2:58 PM, J. B. Rainsberger < me@...> wrote:
On Tue, Mar 29, 2022 at 1:57 PM Russell Gold < russ@...> wrote: I¡¯ve never heard of the ¡°Mikado method,¡± but that¡¯s pretty much the way I learned to do TDD.
Saff¡¯s technique seems interesting, and I will keep it in mind for the future; I¡¯m not sure yet exactly where it would help. Most of the problems I deal with are errors that wind up being caught in integration testing, because we missed a unit test. There, the problem is that it is not immediately obvious what test is missing, because we haven¡¯t even reproduced the problem other than the integration test: and it doesn¡¯t directly call the production code, so we cannot inline anything useful.
That's _exactly_ what the Saff Squeeze does: it starts with a failing, bigger unit test and produces a missing, smaller unit test. It merely uncovers that test systematically instead of relying on your intuition to imagine it.
But I don¡¯t have a failing unit test. I have a failing integration test (and one that typically takes close to an hour to run through all setup and initialization each time).?
I infer that you're using the term "integration test" to mean "unit test for a bigger unit". It's still a unit, even if it's a large one. :) (Yes, I'm trying to rehabilitate the original meaning of "unit": any independently-inspectable part of the system. And yes, that means that the entire system is a unit.)
If you're starting with a failing test that takes 1 hour to run, then the Saff Squeeze remains perfectly effective, but perhaps you can't afford the time to do it that way. Or you merely might not have the patience for it. I wouldn't blame you. In that case, you might need to guess at some smaller failing unit tests, then Squeeze from there.
Even so, I propose two possibilities:
1. I can imagine situations in which Saff Squeeze would lead you to discovering the cause of the defect sooner than groping in the dark, hoping to find the failing part of the code, even if the first few iterations of the Squeeze took an entire day.
2. The Saff Squeeze might be helpful, even if you can't afford to _run_ the tests for the first few iterations. Perhaps even systematically inlining would work better (and feel better) than using a debugger or other, less-systematic techniques.
? When I get a failing unit test, I generally just revert the change that caused it and try again. If that¡¯s not practical for some reason, I will try this.
You're assuming that you know the change that caused it. The whole point of this technique is to help find the cause of a defect when we don't know the cause of the defect by other means. -- J. B. (Joe) Rainsberger :: ?:: ::
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
On Tue, Mar 29, 2022 at 4:51 PM Avi Kessner < akessner@...> wrote: ? Thanks all more clear now.? We want to do a copy/paste inline. The difference being that the IDE will delete the original function when inlining, not make an inlined copy.
I realized later that you might have meant this. I don't use the words this way, so I didn't grasp the difference immediately.
IntelliJ IDEA, for example, doesn't assume that "inline" means "...and delete the original". You have to decide that when you inline a function.
Indeed, I meant what you're calling "copy/paste inline": inline only this invocation without deleting the original function. The Saff Squeeze would destroy the production code otherwise.
-- J. B. (Joe) Rainsberger :: ?:: ::
?
On Tue, 29 Mar 2022, 21:40 J. B. Rainsberger, < me@...> wrote: On Mon, Mar 28, 2022 at 2:52 PM Avi Kessner < akessner@...> wrote: ? Copy/paste inline, or actually inline?
What's the difference?
-- J. B. (Joe) Rainsberger :: ?:: ::
? It's in the tests, because you start with inlining the code tested in the failing test.? IIUC everything happens in the tests, until you get to the point where you uncover that "missing test" JB is speaking of.? --?
Arnaud Bailly - @dr_c0d3
On Mon, Mar 28, 2022 at 7:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
If that¡¯s your situation, your first step must be to figure out a how to run a segment of your program through a Saff loop before your can even consider applying the technique. ?In my case, it might take 15 minutes to build everything, but my most complicated tests don¡¯t take more than a couple minutes and don¡¯t require a full rebuild if I change something.
toggle quoted message
Show quoted text
On Mar 29, 2022, at 16:31, Russell Gold <russ@...> wrote:
But I don¡¯t have a failing unit test. I have a failing integration test (and one that typically takes close to an hour to run through all setup and initialization each time).?
|
To be clear, I am the original poster; I have plenty of experience with TDD but minimal experience with Asynchronous web-based testing. ?Everything I tested before now was a function call that does a bunch of stuff and gives me [access to] values I can assert against. ?I was merely lost how to get past the web-portal login for testing my use of external API¡¯s. ?That question has largely been resolved in my mind. ?(For now.)
toggle quoted message
Show quoted text
On Mar 29, 2022, at 12:29, Steve Gordon <sgordonphd@...> wrote:
? Given the original question where the poster is catching up with his legacy code, which implies no experience doing TDD, the Saff's technique seems to be overkill that discourages getting started.? Keep it simple until enough experience?is gained to warrant getting more sophisticated. On Tue, Mar 29, 2022 at 9:16 AM Alan Baljeu via <alanbaljeu= [email protected]> wrote: This looks like a useful method.? Another approach I learned as the Mikado method:? 1. Write a failing test. 2. Change the code to pass it. 3. If you can't _simply_ do this, then: a. identify where you get stuck b. revert c. refactor so you're not stuck, and goto 1 (or 2).
In the context to match Saff's technique, the refactoring would slowly expose the heart of the troublesome code until the problem is identified and solved.? That said, I can see Saff's route may be more efficient at times.? It especially looks useful if the problem ultimately lies in a 3rd party's library and you need to send a small failing test for them to fix your problem.
|
On Mar 29, 2022, at 2:58 PM, J. B. Rainsberger < me@...> wrote:
On Tue, Mar 29, 2022 at 1:57 PM Russell Gold < russ@...> wrote: I¡¯ve never heard of the ¡°Mikado method,¡± but that¡¯s pretty much the way I learned to do TDD.
Saff¡¯s technique seems interesting, and I will keep it in mind for the future; I¡¯m not sure yet exactly where it would help. Most of the problems I deal with are errors that wind up being caught in integration testing, because we missed a unit test. There, the problem is that it is not immediately obvious what test is missing, because we haven¡¯t even reproduced the problem other than the integration test: and it doesn¡¯t directly call the production code, so we cannot inline anything useful.
That's _exactly_ what the Saff Squeeze does: it starts with a failing, bigger unit test and produces a missing, smaller unit test. It merely uncovers that test systematically instead of relying on your intuition to imagine it.
But I don¡¯t have a failing unit test. I have a failing integration test (and one that typically takes close to an hour to run through all setup and initialization each time).?
This is why I¡¯d really want some concrete examples.
You don't need them. You already understand it. Go do it the next time you encounter an unexpectedly-failing test and the cause of the failure isn't already obvious.
When I get a failing unit test, I generally just revert the change that caused it and try again. If that¡¯s not practical for some reason, I will try this. ----------------- Author, HttpUnit <> and SimpleStub <> Now blogging at <>
Have you listened to Edict Zero <>? If not, you don¡¯t know what you¡¯re missing!
|
Thanks all more clear now.? We want to do a copy/paste inline. The difference being that the IDE will delete the original function when inlining, not make an inlined copy.
On Tue, 29 Mar 2022, 21:40 J. B. Rainsberger, < me@...> wrote: On Mon, Mar 28, 2022 at 2:52 PM Avi Kessner < akessner@...> wrote: ? Copy/paste inline, or actually inline?
What's the difference?
-- J. B. (Joe) Rainsberger :: ?:: ::
? It's in the tests, because you start with inlining the code tested in the failing test.? IIUC everything happens in the tests, until you get to the point where you uncover that "missing test" JB is speaking of.? --?
Arnaud Bailly - @dr_c0d3
On Mon, Mar 28, 2022 at 7:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
On Tue, Mar 29, 2022 at 1:57 PM Russell Gold < russ@...> wrote: I¡¯ve never heard of the ¡°Mikado method,¡± but that¡¯s pretty much the way I learned to do TDD.
Saff¡¯s technique seems interesting, and I will keep it in mind for the future; I¡¯m not sure yet exactly where it would help. Most of the problems I deal with are errors that wind up being caught in integration testing, because we missed a unit test. There, the problem is that it is not immediately obvious what test is missing, because we haven¡¯t even reproduced the problem other than the integration test: and it doesn¡¯t directly call the production code, so we cannot inline anything useful.
That's _exactly_ what the Saff Squeeze does: it starts with a failing, bigger unit test and produces a missing, smaller unit test. It merely uncovers that test systematically instead of relying on your intuition to imagine it.
This is why I¡¯d really want some concrete examples.
You don't need them. You already understand it. Go do it the next time you encounter an unexpectedly-failing test and the cause of the failure isn't already obvious. -- J. B. (Joe) Rainsberger :: ?:: ::
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
? Given the original question where the poster is catching up with his legacy code, which implies no experience doing TDD, the Saff's technique seems to be overkill that discourages getting started.? Keep it simple until enough experience?is gained to warrant getting more sophisticated.
I'm surprised by this framing. I don't know what the Saff Squeeze has to do with experience with TDD. The Squeeze doesn't care whether you've written the test first nor whether you're doing evolutionary design. The only novel part is learning what "inline" means in this context.
The entire technique is "write a failing test, then (inline & prune dead code) until you can spot the cause with the naked eye". That sounds pretty simple to me. I don't see what's sophisticated in it. Indeed, the whole point is that it's mechanical and easy to learn, if tedious, compared to the delicate heuristics needed to step through code with a debugger; decide when to step over, step in, or step out; judge which variables to watch and how closely; and decide when enough is enough. *That* seems sophisticated to me. -- J. B. (Joe) Rainsberger :: ?:: ::
? On Tue, Mar 29, 2022 at 9:16 AM Alan Baljeu via <alanbaljeu= [email protected]> wrote: This looks like a useful method.? Another approach I learned as the Mikado method:? 1. Write a failing test. 2. Change the code to pass it. 3. If you can't _simply_ do this, then: a. identify where you get stuck b. revert c. refactor so you're not stuck, and goto 1 (or 2).
In the context to match Saff's technique, the refactoring would slowly expose the heart of the troublesome code until the problem is identified and solved.? That said, I can see Saff's route may be more efficient at times.? It especially looks useful if the problem ultimately lies in a 3rd party's library and you need to send a small failing test for them to fix your problem.
On Monday, March 28, 2022, 05:28:05 p.m. EDT, Rodolfo Carvalho < rhcarvalho@...> wrote:
Hi Alan,
On Mon, Mar 28, 2022 at 7:54 PM Alan Baljeu via <alanbaljeu= [email protected]> wrote: I'm confused by language in that post.? I think "commit" means locally post to Source control your newly changed code.??
[...] - > A) Maybe squash all these commits, [...]
In other words, delete the commit and TDD a new test which we make pass. - > B) Throw away all those commits, then test-drive the missing behavior using the notes [...]
In other words, delete the commit and TDD a new test which we make pass.
Do you see my confusion?
I see how this can be confusing.
The difference is in case A you don't simply "delete" commits, "squashing" means you replace several commits with a single commit that applies the same patch as all the previous commits together. In this case, you accept that you're done and change history for the sake of having a clean history without the step-by-step commits.
Conversely, in case B you indeed throw away commits, and rely solely on the fresh knowledge to do it all again. You may or may not end up with the same patch as in case A.
Cheers,
Rodolfo Carvalho
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
On Mon, Mar 28, 2022 at 2:54 PM Alan Baljeu via <alanbaljeu= [email protected]> wrote: I'm confused by language in that post.? I think "commit" means locally post to Source control your newly changed code.??
Then we see this: > Don¡¯t rush. Take a moment. Write things down. Breathe. Now decide how to proceed: - > A) Maybe squash all these commits, because they add up to ¡°here¡¯s a failing test we should have written before;?
- > we¡¯ll make it pass now¡±. The project history makes it look the same as if we¡¯d always planned to test-drive this?
- > behavior today.
In other words, delete the commit and TDD a new test which we make pass.
? - > B) Throw away all those commits, then test-drive the missing behavior using the notes you wrote when you?
- > documented the cause of the defect and how to fix it.
In other words, delete the commit and TDD a new test which we make pass.
Do you see my confusion?
Yes.
You are describing an equivalent result of (A), but not the path of (A). I don't delete the commit in order to write a new test; I refactor towards increasingly small tests, then delete the intermediate steps.
When I do (B), I might think differently, stop, delete everything, and then write a completely new test, different from the one that I arrive at when I do (A). I have certainly done this when I scream "NOW I know how I wanted to design this!!"
Does that help? -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Monday, March 28, 2022, 01:47:43 p.m. EDT, Arnaud Bailly < arnaud.oqube@...> wrote:
It's in the tests, because you start with inlining the code tested in the failing test.? IIUC everything happens in the tests, until you get to the point where you uncover that "missing test" JB is speaking of.? --?
Arnaud Bailly - @dr_c0d3 On Mon, Mar 28, 2022 at 7:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
On Mon, Mar 28, 2022 at 2:52 PM Avi Kessner < akessner@...> wrote: ? Copy/paste inline, or actually inline?
What's the difference?
-- J. B. (Joe) Rainsberger :: ?:: ::
? It's in the tests, because you start with inlining the code tested in the failing test.? IIUC everything happens in the tests, until you get to the point where you uncover that "missing test" JB is speaking of.? --?
Arnaud Bailly - @dr_c0d3
On Mon, Mar 28, 2022 at 7:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
On Mon, Mar 28, 2022 at 2:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
You're deleting code from the test as you inline parts of the test. You're only changing production code in order to make the test work (making non-public parts public, mostly). -- J. B. (Joe) Rainsberger :: ?:: ::
? On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
I¡¯ve never heard of the ¡°Mikado method,¡± but that¡¯s pretty much the way I learned to do TDD.
Saff¡¯s technique seems interesting, and I will keep it in mind for the future; I¡¯m not sure yet exactly where it would help. Most of the problems I deal with are errors that wind up being caught in integration testing, because we missed a unit test. There, the problem is that it is not immediately obvious what test is missing, because we haven¡¯t even reproduced the problem other than the integration test: and it doesn¡¯t directly call the production code, so we cannot inline anything useful.
This is why I¡¯d really want some concrete examples.
----------------- Author, HttpUnit <> and SimpleStub <> Now blogging at <>
Have you listened to Edict Zero <>? If not, you don¡¯t know what you¡¯re missing!
toggle quoted message
Show quoted text
This looks like a useful method.? Another approach I learned as the Mikado method:? 1. Write a failing test. 2. Change the code to pass it. 3. If you can't _simply_ do this, then: a. identify where you get stuck b. revert c. refactor so you're not stuck, and goto 1 (or 2).
In the context to match Saff's technique, the refactoring would slowly expose the heart of the troublesome code until the problem is identified and solved.? That said, I can see Saff's route may be more efficient at times.? It especially looks useful if the problem ultimately lies in a 3rd party's library and you need to send a small failing test for them to fix your problem.
On Monday, March 28, 2022, 05:28:05 p.m. EDT, Rodolfo Carvalho < rhcarvalho@...> wrote:
Hi Alan,
On Mon, Mar 28, 2022 at 7:54 PM Alan Baljeu via <alanbaljeu= [email protected]> wrote: I'm confused by language in that post.? I think "commit" means locally post to Source control your newly changed code.??
[...] - > A) Maybe squash all these commits, [...]
In other words, delete the commit and TDD a new test which we make pass. - > B) Throw away all those commits, then test-drive the missing behavior using the notes [...]
In other words, delete the commit and TDD a new test which we make pass.
Do you see my confusion?
I see how this can be confusing.
The difference is in case A you don't simply "delete" commits, "squashing" means you replace several commits with a single commit that applies the same patch as all the previous commits together. In this case, you accept that you're done and change history for the sake of having a clean history without the step-by-step commits.
Conversely, in case B you indeed throw away commits, and rely solely on the fresh knowledge to do it all again. You may or may not end up with the same patch as in case A.
Cheers,
Rodolfo Carvalho
|
Given the original question where the poster is catching up with his legacy code, which implies no experience doing TDD, the Saff's technique seems to be overkill that discourages getting started.? Keep it simple until enough experience?is gained to warrant getting more sophisticated.
On Tue, Mar 29, 2022 at 9:16 AM Alan Baljeu via <alanbaljeu= [email protected]> wrote: This looks like a useful method.? Another approach I learned as the Mikado method:? 1. Write a failing test. 2. Change the code to pass it. 3. If you can't _simply_ do this, then: a. identify where you get stuck b. revert c. refactor so you're not stuck, and goto 1 (or 2).
In the context to match Saff's technique, the refactoring would slowly expose the heart of the troublesome code until the problem is identified and solved.? That said, I can see Saff's route may be more efficient at times.? It especially looks useful if the problem ultimately lies in a 3rd party's library and you need to send a small failing test for them to fix your problem.
On Monday, March 28, 2022, 05:28:05 p.m. EDT, Rodolfo Carvalho < rhcarvalho@...> wrote:
Hi Alan,
On Mon, Mar 28, 2022 at 7:54 PM Alan Baljeu via <alanbaljeu= [email protected]> wrote: I'm confused by language in that post.? I think "commit" means locally post to Source control your newly changed code.??
[...] - > A) Maybe squash all these commits, [...]
In other words, delete the commit and TDD a new test which we make pass. - > B) Throw away all those commits, then test-drive the missing behavior using the notes [...]
In other words, delete the commit and TDD a new test which we make pass.
Do you see my confusion?
I see how this can be confusing.
The difference is in case A you don't simply "delete" commits, "squashing" means you replace several commits with a single commit that applies the same patch as all the previous commits together. In this case, you accept that you're done and change history for the sake of having a clean history without the step-by-step commits.
Conversely, in case B you indeed throw away commits, and rely solely on the fresh knowledge to do it all again. You may or may not end up with the same patch as in case A.
Cheers,
Rodolfo Carvalho
|
This looks like a useful method.? Another approach I learned as the Mikado method:? 1. Write a failing test. 2. Change the code to pass it. 3. If you can't _simply_ do this, then: a. identify where you get stuck b. revert c. refactor so you're not stuck, and goto 1 (or 2).
In the context to match Saff's technique, the refactoring would slowly expose the heart of the troublesome code until the problem is identified and solved.? That said, I can see Saff's route may be more efficient at times.? It especially looks useful if the problem ultimately lies in a 3rd party's library and you need to send a small failing test for them to fix your problem.
On Monday, March 28, 2022, 05:28:05 p.m. EDT, Rodolfo Carvalho <rhcarvalho@...> wrote:
Hi Alan,
On Mon, Mar 28, 2022 at 7:54 PM Alan Baljeu via <alanbaljeu= [email protected]> wrote: I'm confused by language in that post.? I think "commit" means locally post to Source control your newly changed code.??
[...] - > A) Maybe squash all these commits, [...]
In other words, delete the commit and TDD a new test which we make pass. - > B) Throw away all those commits, then test-drive the missing behavior using the notes [...]
In other words, delete the commit and TDD a new test which we make pass.
Do you see my confusion?
I see how this can be confusing.
The difference is in case A you don't simply "delete" commits, "squashing" means you replace several commits with a single commit that applies the same patch as all the previous commits together. In this case, you accept that you're done and change history for the sake of having a clean history without the step-by-step commits.
Conversely, in case B you indeed throw away commits, and rely solely on the fresh knowledge to do it all again. You may or may not end up with the same patch as in case A.
Cheers,
Rodolfo Carvalho
|
Hi Alan,
On Mon, Mar 28, 2022 at 7:54 PM Alan Baljeu via <alanbaljeu= [email protected]> wrote: I'm confused by language in that post.? I think "commit" means locally post to Source control your newly changed code.??
[...] - > A) Maybe squash all these commits, [...]
In other words, delete the commit and TDD a new test which we make pass. - > B) Throw away all those commits, then test-drive the missing behavior using the notes [...]
In other words, delete the commit and TDD a new test which we make pass.
Do you see my confusion?
I see how this can be confusing.
The difference is in case A you don't simply "delete" commits, "squashing" means you replace several commits with a single commit that applies the same patch as all the previous commits together. In this case, you accept that you're done and change history for the sake of having a clean history without the step-by-step commits.
Conversely, in case B you indeed throw away commits, and rely solely on the fresh knowledge to do it all again. You may or may not end up with the same patch as in case A.
Cheers,
Rodolfo Carvalho
|
I'm confused by language in that post.? I think "commit" means locally post to Source control your newly changed code.??
Then we see this: > Don¡¯t rush. Take a moment. Write things down. Breathe. Now decide how to proceed: - > A) Maybe squash all these commits, because they add up to ¡°here¡¯s a failing test we should have written before;?
- > we¡¯ll make it pass now¡±. The project history makes it look the same as if we¡¯d always planned to test-drive this?
- > behavior today.
In other words, delete the commit and TDD a new test which we make pass. - > B) Throw away all those commits, then test-drive the missing behavior using the notes you wrote when you?
- > documented the cause of the defect and how to fix it.
In other words, delete the commit and TDD a new test which we make pass.
Do you see my confusion?
On Monday, March 28, 2022, 01:47:43 p.m. EDT, Arnaud Bailly <arnaud.oqube@...> wrote:
It's in the tests, because you start with inlining the code tested in the failing test.? IIUC everything happens in the tests, until you get to the point where you uncover that "missing test" JB is speaking of.? --?
Arnaud Bailly - @dr_c0d3
On Mon, Mar 28, 2022 at 7:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
Copy/paste inline, or actually inline?
It's in the tests, because you start with inlining the code tested in the failing test.? IIUC everything happens in the tests, until you get to the point where you uncover that "missing test" JB is speaking of.? --?
Arnaud Bailly - @dr_c0d3
On Mon, Mar 28, 2022 at 7:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
It's in the tests, because you start with inlining the code tested in the failing test.? IIUC everything happens in the tests, until you get to the point where you uncover that "missing test" JB is speaking of.? --?
Arnaud Bailly - @dr_c0d3
On Mon, Mar 28, 2022 at 7:07 PM Avi Kessner < akessner@...> wrote: Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
Very nice, but one piece confuses me. Am I supposed to delete if statements in the production code, or if statements in the tests? Also, can this help with bugs related to third party libraries?
On Mon, 28 Mar 2022, 16:27 J. B. Rainsberger, < me@...> wrote: On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
|
On Sat, Mar 26, 2022 at 4:32 AM Jacqueline Lee < jmasonlee@...> wrote: Hey J.B.?
Do you have this description of a Saff squeeze posted anywhere? I have a blog post that I'm writing that references a Saff squeeze, and I'd love to just link to a description rather than writing it up, but I can't find a good description elsewhere. Thanks!
I simply haven't published it yet. And that's funny, because I've been doing it _often_ the past few days.
I finally published something about it. Let's see how the world reacts:
Thank you for prompting me finally to write it. :) -- J. B. (Joe) Rainsberger :: ?:: ::
?
On Fri, Mar 11, 2022 at 12:55 PM J. B. Rainsberger < me@...> wrote: One reasonable way to get from a few slow-running end-to-end tests to some strategic unit testing is to do the following whenever you encounter a bug:
[...]
Another suggestion: apply the Saff Squeeze. Think "debugging with automated tests".
Start with a failing test.
1. Inline the action of the test: that's the central function that you invoke to make the test _do_ something. 2. Make invisible things visible (private -> public in many languages). 3. Prune away dead code, such as the branches of an `if` statement that this test's input won't run. Maybe unroll a loop.
4. Add assertions to check intermediate results, especially the conditions of `if` statements or the stopping conditions of loops. Do this until you add an assertion that fails. 5. After you add a new failing assertion, delete the rest of the test; it won't execute, anyway.
Now you have a smaller test that fails for the same reason as the original test. Commit. Maybe now it's obvious what caused the defect. Yes? Fix it. No? Repeat the process.
This is one way to uncover smaller unit tests that would have helped you. Over time, you'll get to know which kinds of smaller unit tests you want to practise writing. -- J. B. (Joe) Rainsberger :: ?:: ::
--
J. B. (Joe) Rainsberger :: :: ::
Teaching evolutionary design and TDD since 2002
-- J. B. (Joe) Rainsberger :: :: :: Teaching evolutionary design and TDD since 2002
|
J. B. Rainsberger