Thinking through what I need to do for the plot capability I'm writing, I've had a realization. I thought I was writing a library. That might, in fact, still be the form that the plot stuff takes. But thinking about how plot might be used, there's a bifurcation. On the one hand, I might want to construct a plot instance for each cycle, do the drawing, and throw it away. On the other hand, I might want a plot instance that persists and can be updated. I realized that if I take the first route, plot doesn't have a state -- it's really a function, not an object. How did I get to this conclusion? Because writing tests forced me to construct a plot, and think about how it is used rather than just focusing on the object capabilities.
Of course, it's impossible to tell whether I would have arrived at the same conclusion without doing TDD. But even if I had arrived there, I'm not sure it would have happened so early on.
I've been revising my specifications for the plot capability. I'm not sure about format, but here's the current version.
Top-level story: A user wants to add a plot to an display for a sim variable.
Constraints: Existing architecture provides a data source abstraction and a user interface. The user interface will need to change to use plotlib to request a plot drawing. UI will provide plot specifications including an image to draw into. Plot must construct the image using the primitive drawing functions provided by the image interface, according to the specifications.
Specifications include:
1) axes
2) titles (are these user-specifiable?)
3) legends
4) one or more variable to plot
5) styles (color, line style, fonts etc.)
6) a data source, provided as a data river instance
Desirable:
- Plots should be cross-platform
The specifications might not all be relevant to plot. For example, perhaps font needs to be handled at a different level, leaving plot with simply a writeText(string) function, or a setFontSize(int), or even setFontSize(FontSizes) using an enum such as normal, large, small etc. I'm not going to worry about this for now. That's down the road for sure.
I think once I get used to the "run and see the tests pass" I might actually like it. What's surprising me at the moment is that the tests are driving the design to some degree. Based on the great advice I received on the TDD board, I'm feeling free to think about design on both large and small scales, while always coming back to "OK, but what's the next test?", and "how does that spec translate into a test?" One of the advantages of not looking ahead is that I'm not having to carry everything around in my head all the time. I don't have to think "I'm going to write the Plot constructor, and it will have a Spec, and the Spec will need to have an Image, and the Image will need to be constructed, and the Spec will also have Axes which might be an concrete instantiation of some abstract PlotObject abstract class, oh, and ..." If I want to throw up some test balloons like this to help me see where I might be going, I do. But I also realize that I need to get to them via the tests, because the tests show what's necessary on a practical level to get the objects working.
So I suppose that this means I am starting to see the tests playing a positive role in developing the design, and also in refactoring. I've done a bit of the latter already, to reflect my updated specs, and yes, it was nice being able to run the unit tests and see them pass even though at this stage they are fairly trivial. It reminds me of why I prefer using static rather than dynamic languages. Just as the C++ compiler catches all sorts of type errors that might make it though in a dynamic language, so the tests catch all sorts of logic errors that might make it through a compilation without them. Writing the unit tests is like building a customized "logic compiler" for my code.
One big remaining question is the amount of time spent refactoring tests and the quality of the tests that I end up with. TDD advocates tend to minimize this issue, but I don't believe them. There's actually a book on this subject (xUnit Test Patterns: Refactoring Test Code) where the author writes the following:
We started doing eXtreme Programming "by the book" using pretty much all of the practices it recommended, including pair programming, collective ownership, and test-driven development. Of course, we encountered a few challenges in figuring out how to test some aspects of the behavior of the application, but we still managed to write tests for most of the code. Then, as the project progressed, I started to notice a disturbing trend: It was taking longer and longer to implement seemingly similar tasks.
I explained the problem to the developers and asked them to record on each task card how much time had been spent writing new tests, modifying existing tests, and writing the production code. Very quickly, a trend emerged. While the time spent writing new tests and writing the production code seemed to be staying more or less constant, the amount of time spent modifying existing tests was increasing and the developers' estimates were going up as a result. When a developer asked me to pair on a task and we spent 90% of the time modifying existing tests to accommodate a relatively minor change.
The problem is that the people who invent methods use a lot of tacit knowledge as they develop. This is noticeable in the books they write. When I was reading Kent Beck's "Test-Driven Development by Example," there were several occasions when I thought "OK, I can see that the way he goes is a legitimate way to go, but it's not the way I would have gone. I wonder why he chose it?" It's one of those Alistair Cockburn things where we don't know what we know, and therefore can't tell whether or not everything that needs to be expressed has been expressed. even if we could express it.
I will probably need to get that book on xUnit refactoring at some point. But elsewhere on a forum I saw someone else say something to the effect of "if the team doesn't use this book, they will get into trouble." Of course, that individual could be wrong, and certainly he is speaking in a context. But the fact remains: there's no royal road to "clean and working code" developed quickly that's easy to maintain. TDD might start with two sentences worth of rules, but the outworking of the rules is still many books worth of material and experience and that's no bad thing; it implies to me that TDD has enough to it to stand a chance of working across a range of projects.
Enough for now. Off to write some tests.
Thursday, January 26, 2012
TDD with QtCreator
I'm feeling better about TDD. I might even be seeing some real benefits even though there's hardly any code yet.
To begin with, let's get some setup out of the way. One challenge was to get gtest working with QtCreator. A question I've had is whether to have one executable containing all my tests, or whether to split tests into multiple executables. This is somewhat significant, since my goal is to type CTRL-R in QtCreator and have the tests build and run -- since the TDD methodology means running tests often, it has to be easy and fast. On the other hand, it seems logical to split tests up into separate executables in case I only want to run one set (e.g. the image tests). I came up with the following solution.
1) Have the makefile produce multiple executables.
2) Have the makefile produce a run_test.sh script that runs all the executables, stopping if there's a breakage and echoing "ALL TESTS PASS" if not. This makes it easy to see when all the tests pass (hopefully the majority of the time). The rule to make the script is quite simple:
make_exec_script: $(TESTS)
echo '#!/bin/sh' > $(EXEC_SCRIPT)
for i in $(TESTS); do \
echo "./$$i && \\" >> $(EXEC_SCRIPT); \
done
echo 'echo " " && \\' >> $(EXEC_SCRIPT); \
echo 'echo \* \* \* ALL TESTS PASSED \* \* \*' >> $(EXEC_SCRIPT)
chmod 755 $(EXEC_SCRIPT)
Now I just set the project up to execute the script, and I'm done. Granted, this won't work on Windows. Poor Windows. Always the oddball.
To begin with, let's get some setup out of the way. One challenge was to get gtest working with QtCreator. A question I've had is whether to have one executable containing all my tests, or whether to split tests into multiple executables. This is somewhat significant, since my goal is to type CTRL-R in QtCreator and have the tests build and run -- since the TDD methodology means running tests often, it has to be easy and fast. On the other hand, it seems logical to split tests up into separate executables in case I only want to run one set (e.g. the image tests). I came up with the following solution.
1) Have the makefile produce multiple executables.
2) Have the makefile produce a run_test.sh script that runs all the executables, stopping if there's a breakage and echoing "ALL TESTS PASS" if not. This makes it easy to see when all the tests pass (hopefully the majority of the time). The rule to make the script is quite simple:
make_exec_script: $(TESTS)
echo '#!/bin/sh' > $(EXEC_SCRIPT)
for i in $(TESTS); do \
echo "./$$i && \\" >> $(EXEC_SCRIPT); \
done
echo 'echo " " && \\' >> $(EXEC_SCRIPT); \
echo 'echo \* \* \* ALL TESTS PASSED \* \* \*' >> $(EXEC_SCRIPT)
chmod 755 $(EXEC_SCRIPT)
Now I just set the project up to execute the script, and I'm done. Granted, this won't work on Windows. Poor Windows. Always the oddball.
Saturday, January 21, 2012
Shu-ha-ri and the art of learning
I have waited too long to write this entry.
I thought that I needed time for ideas to sink in, and since I'd posted a question on the GOOS board, I thought it would be a good idea to give time for other people to answer. Then, looking over what I've written as a summary of GOOS so far, I felt I had not done a good job of summarizing the GOOS approach, but possibly lacked the background to understand what they were after. This is always an issue with getting into something new, of course: where there is a community, there is a community language and assumed background which is not always easy for the outsider to pick up on.
To get some background, I've started reading "Agile Software Development: The Cooperative Game" (CG) by Alistair Cockburn. I've always like Cockburn's stuff -- it was his view of the human side of software development that drew me to Agile methods in the first place. I was initially going to look at a different book of his, referenced in GOOS, but I think CG is a book that I need to read. In short, it is a discussion of theories of programming, and more broadly of epistemology and communication in a programming setting.
Cockburn begins with a familiar couple of epistemic problems. Can we know what we are experiencing? And (taking other minds for granted), can we express what we know? Cockburn answers both questions in the negative. I found his discussion interesting, but not always coherent. At some level, if it is not possible to know what we experience, it is hard to see how we can then express it to ourselves. And if we cannot express it to ourselves, and cannot express it to others, then why write a book about it? The method employed is not that of philosophical discussion, building cases from axioms, or syllogisms, for example, but rather attempting to convince through stories and reflections on what he takes from those stories.
In his first story, for example, the author turns up at a party with a bottle of red wine, which the hostess insists is white, even though the label clearly says it is red. Later on, when he points out the mistake, the hostess again insists the wine is white and even points to the label, finding out only when she reads it out loud that it says "red." From this, Cockburn argues that we are subject to making mistakes when we think we know something that we don't know, and therefore we can end up producing requirements that contain observational errors. Fair enough. But this does not really address whether we can express what we know, or whether we can know what we experience. Rather, it argues that we may be mistaken about what we think we know, and may therefore end up conveying mistakes to others. One could argue that, on the contrary, it is precisely the fact that we can communicate and can understand our experiences that allows the hostess to realize a mistake has been made, and to laugh together with the author about it.
I don't think that Cockburn had a epistemological treatise in mind, though, when he drafted this book. His audience is programmers, and pragmatic ones at that. He encourages those who do not like "abstract" discussion to skip the first chapter altogether. His advice, then, should probably be seen as practical rather than theoretical, even in "abstract" chapters. Looked at this way, there is a lot to like.
Practically, our communication suffers from a lot of problems.
1) Our comprehension of our own experiences is limited by our ability to interpret those experiences.
2) Our ability to interpret is limited by many factors including language, presuppositions, eager interpretation (judging too early), and level of mastery.
3) In communicating with others, we need to establish a common vocabulary. This is impossible to do perfectly since understanding is layered in terms of learning and experience and everyone is unique in that regard. At best, we look for sufficiently similar experiences, which might involve the equivalent of an experience English speaker adopting a very simple vocabulary to talk to a child.
Expanding on point 3, Cockburn brings in an idea of learning mastery built on Aikido's concept Shu-Ha-Ri. Mastery occurs in three stages. In Shu (learn), we start from the beginning and learn one particular path or technique. Trying to learn different techniques at this stage of mastery leads only to confusion. In Ha (detach), we come to see that our technique does not always work well, and that there are other techniques that work better in certain circumstances. We look for boundaries that define when to use one technique rather than another. In Ri (transcend), we come to see techniques as means to an end, not an end in themselves, and roll our own ways of getting there specific to the task, using the knowledge of the techniques without being restricted to them.
According to Cockburn, this causes problems with a level 3 (Ri) person talks to a newbie. Ri people say things like "do what works," by which they mean something like "there's no perfect answer, but there is a multiplicity of good ones so there's no need to be prescriptive." What a newbie might hear, though, is "it doesn't matter how you code, so long as it works," or "I'm not going to help you figure out how to do it" (I'm interpolating here -- these are my words, not Cockburn's). Beginners need to know that they are getting something right.
There's an obvious parallel to my experience with GOOS and TDD so far. I may (or may not) have written this explicitly in a previous post, but what I'm looking for is ONE way to get into TDD. Looking around the web, there are plenty of people arguing for their interpretations of TDD. That's fine, but I need context first. I recognize the danger of seeing GOOS as "the one, authentic, right method." I think I have enough experience to avoid making that mistake. But I do want to understand GOOS at a deep level, deeper than just "make a slice, code a test, code the initial behavior then fill out from there." I sense that there is more to GOOS than this -- assumptions that are more or less tacit that make GOOS a good fit for Mocks rather than Mocks simply being one technology the authors have decided to employ.
Clearly, what I was doing in my post to the GOOS message board was an attempt to draw on common background, to phrase GOOS ideas in terminology I have seen before. This is good. The first step towards communication is trying to find what is in common, like two modems negotiating a baud rate. But unlike a modem, whose limitations are inherent in the hardware, I can work my way up from my current 300baud state to 56k, and who knows, maybe to T1 some day.
I thought that I needed time for ideas to sink in, and since I'd posted a question on the GOOS board, I thought it would be a good idea to give time for other people to answer. Then, looking over what I've written as a summary of GOOS so far, I felt I had not done a good job of summarizing the GOOS approach, but possibly lacked the background to understand what they were after. This is always an issue with getting into something new, of course: where there is a community, there is a community language and assumed background which is not always easy for the outsider to pick up on.
To get some background, I've started reading "Agile Software Development: The Cooperative Game" (CG) by Alistair Cockburn. I've always like Cockburn's stuff -- it was his view of the human side of software development that drew me to Agile methods in the first place. I was initially going to look at a different book of his, referenced in GOOS, but I think CG is a book that I need to read. In short, it is a discussion of theories of programming, and more broadly of epistemology and communication in a programming setting.
Cockburn begins with a familiar couple of epistemic problems. Can we know what we are experiencing? And (taking other minds for granted), can we express what we know? Cockburn answers both questions in the negative. I found his discussion interesting, but not always coherent. At some level, if it is not possible to know what we experience, it is hard to see how we can then express it to ourselves. And if we cannot express it to ourselves, and cannot express it to others, then why write a book about it? The method employed is not that of philosophical discussion, building cases from axioms, or syllogisms, for example, but rather attempting to convince through stories and reflections on what he takes from those stories.
In his first story, for example, the author turns up at a party with a bottle of red wine, which the hostess insists is white, even though the label clearly says it is red. Later on, when he points out the mistake, the hostess again insists the wine is white and even points to the label, finding out only when she reads it out loud that it says "red." From this, Cockburn argues that we are subject to making mistakes when we think we know something that we don't know, and therefore we can end up producing requirements that contain observational errors. Fair enough. But this does not really address whether we can express what we know, or whether we can know what we experience. Rather, it argues that we may be mistaken about what we think we know, and may therefore end up conveying mistakes to others. One could argue that, on the contrary, it is precisely the fact that we can communicate and can understand our experiences that allows the hostess to realize a mistake has been made, and to laugh together with the author about it.
I don't think that Cockburn had a epistemological treatise in mind, though, when he drafted this book. His audience is programmers, and pragmatic ones at that. He encourages those who do not like "abstract" discussion to skip the first chapter altogether. His advice, then, should probably be seen as practical rather than theoretical, even in "abstract" chapters. Looked at this way, there is a lot to like.
Practically, our communication suffers from a lot of problems.
1) Our comprehension of our own experiences is limited by our ability to interpret those experiences.
2) Our ability to interpret is limited by many factors including language, presuppositions, eager interpretation (judging too early), and level of mastery.
3) In communicating with others, we need to establish a common vocabulary. This is impossible to do perfectly since understanding is layered in terms of learning and experience and everyone is unique in that regard. At best, we look for sufficiently similar experiences, which might involve the equivalent of an experience English speaker adopting a very simple vocabulary to talk to a child.
Expanding on point 3, Cockburn brings in an idea of learning mastery built on Aikido's concept Shu-Ha-Ri. Mastery occurs in three stages. In Shu (learn), we start from the beginning and learn one particular path or technique. Trying to learn different techniques at this stage of mastery leads only to confusion. In Ha (detach), we come to see that our technique does not always work well, and that there are other techniques that work better in certain circumstances. We look for boundaries that define when to use one technique rather than another. In Ri (transcend), we come to see techniques as means to an end, not an end in themselves, and roll our own ways of getting there specific to the task, using the knowledge of the techniques without being restricted to them.
According to Cockburn, this causes problems with a level 3 (Ri) person talks to a newbie. Ri people say things like "do what works," by which they mean something like "there's no perfect answer, but there is a multiplicity of good ones so there's no need to be prescriptive." What a newbie might hear, though, is "it doesn't matter how you code, so long as it works," or "I'm not going to help you figure out how to do it" (I'm interpolating here -- these are my words, not Cockburn's). Beginners need to know that they are getting something right.
There's an obvious parallel to my experience with GOOS and TDD so far. I may (or may not) have written this explicitly in a previous post, but what I'm looking for is ONE way to get into TDD. Looking around the web, there are plenty of people arguing for their interpretations of TDD. That's fine, but I need context first. I recognize the danger of seeing GOOS as "the one, authentic, right method." I think I have enough experience to avoid making that mistake. But I do want to understand GOOS at a deep level, deeper than just "make a slice, code a test, code the initial behavior then fill out from there." I sense that there is more to GOOS than this -- assumptions that are more or less tacit that make GOOS a good fit for Mocks rather than Mocks simply being one technology the authors have decided to employ.
Clearly, what I was doing in my post to the GOOS message board was an attempt to draw on common background, to phrase GOOS ideas in terminology I have seen before. This is good. The first step towards communication is trying to find what is in common, like two modems negotiating a baud rate. But unlike a modem, whose limitations are inherent in the hardware, I can work my way up from my current 300baud state to 56k, and who knows, maybe to T1 some day.
Wednesday, January 4, 2012
Starting TDD
Last night and this morning, I've been doing some reading in "Growing Object Oriented Software, Guided by Tests," by Steve Freeman and Nat Pryce (hereafter referred to as GOOS). I really like it and I think it's clearing up some of the conceptual problems I had as a newbie with Test-Driven Development (TDD). I'll just list a few here, and maybe in future entries discuss how I'm going to try to do thpoints oute new plot library implementation using TDD. I'm hoping this could be a series on what I'm learning by doing TDD, or, if nothing else, a cheap way to document the design I'm working on :)
First off, I should say that, as with most methodologies, there isn't one normative approach to doing TDD, so I'll try to say "GOOS" when what I'm discussing comes from the book, and "TDD" when it appears to be generic. In the foreword to the book, Kent Beck says, in effect, "this is a good book. It's not how I would do it, but I learned from it." At first, I thought this was a case of what you might call "dissing with faint praise." Perhaps this is because TDDers are so often dogmatic: "TDD IS GOOD FOR YOU. DO IT NOW! BECAUSE I SAID SO," or "you can't call yourself a professional if you don't use it." Then if someone raises objections, the answer is too often "well, there are some hopeless ideologues you just can't convince." Good grief! Why not just say "it made me a more effective professional," a statement that should be perfectly capable of making other professional take notice without all the acrimony? But in any case, I think it is healthy that there are different views of TDD because differences help to drive out what is meaningful. I admit it -- I'm a bit of a Hegelian at heart.
What is missing in the discussions I've seen of TDD, and what the GOOS book tries to answer, is the rationale behind doing it. I'm not looking for "it's X% more effective." I want to understand why it is more effective -- what makes it work? This is a fundamental difference, and it's not academic either. The fact is, when you take on a new project, you have to take on an approach to that project. Saying "I'll use TDD" doesn't get you far, any more than saying "I'll use OO." TDD needs to be applied. Understanding how to apply TDD comes from understanding what TDD is good for, how it works, what sort of things in the project setup to look for. This is not so easy -- the TDD guys are right to argue that you can't give canned answers. But that's precisely why it is important to have a philosophy of TDD as well as a methodology.
OK, so to the beginnings of clearing up my conceptual misunderstandings with TDD. In the following points, I'll list the misconception first, then discuss why it's a misconception.
First off, I should say that, as with most methodologies, there isn't one normative approach to doing TDD, so I'll try to say "GOOS" when what I'm discussing comes from the book, and "TDD" when it appears to be generic. In the foreword to the book, Kent Beck says, in effect, "this is a good book. It's not how I would do it, but I learned from it." At first, I thought this was a case of what you might call "dissing with faint praise." Perhaps this is because TDDers are so often dogmatic: "TDD IS GOOD FOR YOU. DO IT NOW! BECAUSE I SAID SO," or "you can't call yourself a professional if you don't use it." Then if someone raises objections, the answer is too often "well, there are some hopeless ideologues you just can't convince." Good grief! Why not just say "it made me a more effective professional," a statement that should be perfectly capable of making other professional take notice without all the acrimony? But in any case, I think it is healthy that there are different views of TDD because differences help to drive out what is meaningful. I admit it -- I'm a bit of a Hegelian at heart.
What is missing in the discussions I've seen of TDD, and what the GOOS book tries to answer, is the rationale behind doing it. I'm not looking for "it's X% more effective." I want to understand why it is more effective -- what makes it work? This is a fundamental difference, and it's not academic either. The fact is, when you take on a new project, you have to take on an approach to that project. Saying "I'll use TDD" doesn't get you far, any more than saying "I'll use OO." TDD needs to be applied. Understanding how to apply TDD comes from understanding what TDD is good for, how it works, what sort of things in the project setup to look for. This is not so easy -- the TDD guys are right to argue that you can't give canned answers. But that's precisely why it is important to have a philosophy of TDD as well as a methodology.
OK, so to the beginnings of clearing up my conceptual misunderstandings with TDD. In the following points, I'll list the misconception first, then discuss why it's a misconception.
- TDD = no design up front. If GOOS is right, this is nonsense. There's plenty of design up front. There are meetings with stakeholders, rough designs drawn on whiteboards, state diagrams, CRC cards, discussions of what constitutes the first "slice," discussions of what test technologies to use, and likely more. It was surprising to me, actually, to see how many times the authors of GOOS used the phrase "after discussions" or some similar idea. It's irrelevant to me whether design is documented in some fancy 150k tool or on a napkin. The point is, someone has thought about it. Someone has worked through a preliminary version of what needs to happen -- there is a direction. Perhaps anti-design-up-front TDDers prefer to call this "planning." Fine. I don't care.
- TDD = writing unit tests. I am sure I have read TDD folks saying something to the effect that "unit tests are all you need." That's such a bizarre statement, I can't believe I would have randomly made it up. In fact, having worked on some decent-sized systems, "unit test only" is one of the biggest problems I had with TDD. How on earth can you say that's sufficient to unit test a system with a million lines of code? Rubbish! All the interesting problems are integration problems; unit testing is trivial by comparison. The same goes for working with external libraries or applications. GOOS takes the sensible perspective that tests need to exist at multiple levels, with "end-to-end" tests at the highest level, just as the conventional development methodologies I've used would indicate, though done via a different approach. Sanity! Thank goodness for that!
- TDD = start by writing test cases. One of my problems with TDD has been where to start. In the past when I've tried TDD, I thought "OK, so I need to write tests first. What do I start with?" Then, typically, I picked an object I thought I understood (usually a low-level object) and wrote tests for it. Then, after exhaustively testing all the operators, assignment possibilities etc. I'd end up with a nice unit test which needed extensively rewritten when I tried to fit the object in with other objects. GOOS is quite clear on this point: doing this is a bad idea. More than that, it's fundamentally a misconception of what TDD is supposed to accomplish. In the GOOS way of thinking, tests are intended to drive the design of the system by pushing development from areas of knowledge into areas where knowledge is insufficient. This implies you must start from knowledge. But at the beginning where does the knowledge come from? It must come from constraints like client requirements, available technologies and preliminary design team discussions, and in the beginning, that means you have only high-level details. So "beginning" means gathering requirements and priorities, holding preliminary design discussions, and working out a plan which includes identifying a first "slice" of capability from which the application can grow. Only then can you start with the first test.
- TDD = bottom-up. Given that the first word of their title is "growing," you might think that GOOS advocates starting at the bottom with a seed of code, and adding more bits of code until the project is done. What this fails to take into account, however, are the different levels at which TDD operates. As mentioned in point 3), GOOS begins with a slice of capability. This is at the application level, and the first test is an end-to-end test which I think is intended to test the life of the application and its interfaces, though in minimal way -- it might just instantiate an object, connect to a database, retrieve one thing, and shut down, for example. Adding capability involves driving down into the support classes by adding tests that require those supports, then making those tests pass. Thus, if I am understanding this correctly, GOOS is inherently top-down, setting up the application framework first then pushing down to lower levels.
- TDD = getting good code coverage. Code coverage is certainly likely to be a benefit of a TDD approach, but it's not the point of TDD. Rather, TDD is a model of programming that proceeds by placing constraints on the solution space, and increasing those constraints until all requirements are satisfied. This is really the fundamental difference between a TDD approach (or in principle any test-first approach), and a conventional test-last approach. In a test-last approach, there are, of course, constraints; they are simply implicit and embedded in the code. Testing is done at the end to ensure that the implicit constraints meet the requirements. In a test-first approach, constraints are explicit, and are themselves developed as part of the design as the software grows. A potential problem with test-first is that you might spend a lot of time formally expressing and revising those constraints in test cases. A potential problem with test-last is that implicit constraints might be hard to test. And yes, I'm calling these "potential" problems. I'm in no position yet to say whether the effort to do TDD is justified, or gets better results than well-designed test-last code.
Tuesday, June 7, 2011
Courting Absurdity
I have jury duty next week. This has led to me spending time thinking about the model of justice we use. Let's suppose you want to figure out the true circumstances of, say, a robbery. The police have a suspect and evidence. The case comes to trial. This is the big moment where guilt or innocence will be judged -- likely a life-changing decision for the accused. To make this life-changing decision, you bring in 12 individuals from the general population with no particular experience of law or robbery. You disallow them access to any external resources they could use to educate themselves. You forbid them from considering anything not mentioned during the trial. You pass them through a screening process which serves in principle to ensure impartiality, but in practice to eliminate individuals the attorneys think they won't be able to sway, that is, those intelligent enough to see through rhetorical tricks and those more likely to make decisions based on fact and logic rather than emotion. Then you start the trial.
On one side, you have the prosecution team, representing the office of an elected official with a vested interest in looking tough on crime and successful in putting offenders away. On the other side is a defence team, paid to ensure the best outcome for their client, regardless of justice. In the middle is a judge, often also elected and tied in to party politics and thus with a vested interest in taking a certain stance towards crime. Although the jury is charged with determining guilt or innocence, they have no opportunity to ask questions of witnesses, no opportunity to witness the scene of the crime, and no opportunity to examine evidence or background information that might well be relevant but which is considered "inadmissible" based on an arcane set of rules interpreted by the judge. Every attempt is made to control the jury's view of the case.
When the arguments have been completed, the jury retires to consider a verdict, but not before receiving instructions from the judge, who thus has the opportunity to frame the case as he/she sees fit and may even direct a guilty or not guilty verdict. Once the jury does retire, it must decide the case based on one of three standards: "on the preponderance of the evidence"; "on clear and convincing evidence"; or "beyond all reasonable doubt." All of these standards are subjective, especially when the evidence itself is only available through the lenses of prosecuting and defence attorneys. Although in principle the jury is to weigh only the evidence presented in the courtroom, in fact each jury member will bring in their own pre-formed biases and judgements, as well as subjective impressions of the defendent and the attorneys. The very principle that jurors be "peers" of the defendent reduces their ability to dispassionately decide on the evidence.
The system is clearly not about justice or truth. The purpose of the lower courts is not, in any case, either of these. Rather, the court is there to apply the law, regardless of whether that law is just, or whether the determinations made through the court are true.
Jury systems differ significantly from country to country, with some countries not having such a system at all, and others limiting jury trials to only the most serious offences. The history of jury trials in English law goes back a long way. In the beginning (10th century), "jurors" were not picked from among the people. Instead, they were picked from among the minor nobles, who likely had some experience with settling disputes. They did not sit passively as facts were recited by attorneys. Rather, they were responsible for going out and determining what happened themselves, then reporting back. Then, as now, they were required to swear to investigate without bias, and indeed the purpose of the jury seems to have been to stop those in power from arbitrarily getting their own way. While the composition of the jury changed over time, the idea that jurors ought to investigate the case themselves persisted until the 17th century. Even if we decide jury trials are worthwhile, then, there is no reason that we must have the system we do now.
Historically, jury trials functioned in an environment without full-fledged police forces with trained detectives and forensic labs. Coming from this background, it makes sense that jury trials focused on witnesses. Witnesses, however, are unreliable even when not directly biased. Properly applied forensics does not have this problem. Forensic evidence does, however, need to be carefully considered both for what it does and does not show. Forensics-focused TV programs give a false impression -- their goal, after all, is an interesting plot, not accuracy. But where else in life does the average person encounter forensics? A jury taken from society in general is in no position to evaluate expert scientific testimony, especially when both sides can produce experts supporting their position. Judges are likely in no position to do this either.
On one side, you have the prosecution team, representing the office of an elected official with a vested interest in looking tough on crime and successful in putting offenders away. On the other side is a defence team, paid to ensure the best outcome for their client, regardless of justice. In the middle is a judge, often also elected and tied in to party politics and thus with a vested interest in taking a certain stance towards crime. Although the jury is charged with determining guilt or innocence, they have no opportunity to ask questions of witnesses, no opportunity to witness the scene of the crime, and no opportunity to examine evidence or background information that might well be relevant but which is considered "inadmissible" based on an arcane set of rules interpreted by the judge. Every attempt is made to control the jury's view of the case.
When the arguments have been completed, the jury retires to consider a verdict, but not before receiving instructions from the judge, who thus has the opportunity to frame the case as he/she sees fit and may even direct a guilty or not guilty verdict. Once the jury does retire, it must decide the case based on one of three standards: "on the preponderance of the evidence"; "on clear and convincing evidence"; or "beyond all reasonable doubt." All of these standards are subjective, especially when the evidence itself is only available through the lenses of prosecuting and defence attorneys. Although in principle the jury is to weigh only the evidence presented in the courtroom, in fact each jury member will bring in their own pre-formed biases and judgements, as well as subjective impressions of the defendent and the attorneys. The very principle that jurors be "peers" of the defendent reduces their ability to dispassionately decide on the evidence.
The system is clearly not about justice or truth. The purpose of the lower courts is not, in any case, either of these. Rather, the court is there to apply the law, regardless of whether that law is just, or whether the determinations made through the court are true.
Jury systems differ significantly from country to country, with some countries not having such a system at all, and others limiting jury trials to only the most serious offences. The history of jury trials in English law goes back a long way. In the beginning (10th century), "jurors" were not picked from among the people. Instead, they were picked from among the minor nobles, who likely had some experience with settling disputes. They did not sit passively as facts were recited by attorneys. Rather, they were responsible for going out and determining what happened themselves, then reporting back. Then, as now, they were required to swear to investigate without bias, and indeed the purpose of the jury seems to have been to stop those in power from arbitrarily getting their own way. While the composition of the jury changed over time, the idea that jurors ought to investigate the case themselves persisted until the 17th century. Even if we decide jury trials are worthwhile, then, there is no reason that we must have the system we do now.
Historically, jury trials functioned in an environment without full-fledged police forces with trained detectives and forensic labs. Coming from this background, it makes sense that jury trials focused on witnesses. Witnesses, however, are unreliable even when not directly biased. Properly applied forensics does not have this problem. Forensic evidence does, however, need to be carefully considered both for what it does and does not show. Forensics-focused TV programs give a false impression -- their goal, after all, is an interesting plot, not accuracy. But where else in life does the average person encounter forensics? A jury taken from society in general is in no position to evaluate expert scientific testimony, especially when both sides can produce experts supporting their position. Judges are likely in no position to do this either.
Saturday, May 29, 2010
Rrrrrrrrrrrrr
Over three months since my last post, and I'm feeling that sense of inner conflict that says "need to post to express things; can't post because there's too much to express." Since my last post, my life has felt very busy. It's not that I've been working excessively. The class I took required a lot of work, then right at the end of the semester when I was working hard on the semester project, I landed in a research project with an imminent deadline. Fortunately, the research project was enjoyable, and re-emphasized to me how good R is for statistical analysis. Somewhere around 40 lines of R was enough to:
1) Import a csv data matrix
2) Fix missing values
3) Cull out features that don't agree with a target feature
4) Iteratively perform full hierarchical cluster analyses, including calculating quality statistics and graphing results
That's a lot for 40 lines to do! I also used R for the semester project, combining mixture models and boosting, and had a similarly good experience there. So this post has become dedicated to the praise of R in the process of writing it. R, I'm glad you exist.
Using open source projects like R and OpenSceneGraph (which I'm currently looking at for work) has made me want to get involved in one. With what time, I don't know, but I'm going to put that as an objective. Surely there's something I could do!
1) Import a csv data matrix
2) Fix missing values
3) Cull out features that don't agree with a target feature
4) Iteratively perform full hierarchical cluster analyses, including calculating quality statistics and graphing results
That's a lot for 40 lines to do! I also used R for the semester project, combining mixture models and boosting, and had a similarly good experience there. So this post has become dedicated to the praise of R in the process of writing it. R, I'm glad you exist.
Using open source projects like R and OpenSceneGraph (which I'm currently looking at for work) has made me want to get involved in one. With what time, I don't know, but I'm going to put that as an objective. Surely there's something I could do!
Sunday, December 27, 2009
Subscribe to:
Posts (Atom)