One programmer's unit test is another's integration test

The question of what units you are working with is one that will at one time or other have plagued anyone who studied a science or a branch of physical engineering. Teachers go to great lengths to make sure students remember to specify their units. It is not enough to say that the answer is 42. Forty-two what? 42 metres? 42 …

[amanfromMars]

Virtually Real Computers are Quantum Computers Imagined 42BTrue?

What an excellent article, Kevlin. A Real Gem.

"A unit test is a test of behaviour (normally functional) whose success or failure is wholly determined by the correctness of the test and the correctness of the unit under test."

I would agree with that and it is further enhanced and endorsed/enriched with a Third Party, Positive Reinforcement.... however, it does not necessarily require such Enrichment/HyperRadioProActivity to continue its Progression if the correctness of the test and the correctness of the unit under test, is not hindered from further Testing/Progression.

It may be that the unit test/test of behaviour is Unique and therefore Third Party Action may be hindered by a requirement for them to consider their Reaction/ProAction in that it may also have to be a Unique and Innovative Element which would in Effect, Create a new Component.

Of Course, in some Intelligently Designed Programs, such Creation is exactly what the Programmer is Testing for in what is in Essence a Continual BetaTest of Systems...... to Prove a FailSafe Defaulted System which is New and Unique.

In such a case, .."In other words, the unit in question can be isolated at a level below the whole system and its interactions with the environment." .... will not be applicable, as it will actually sit above the whole system, immune to interactions by virtue of the fact that the system below, which would be the status quo system being unit tested/tested by the unit test, has chosen not to show any action at all, either in reaction [negative and obstructive] or proaction [positive and supportive] It will therefore be free to BetaTest further steps in the Program/Programming, which would prove the Enrichment Algorithm Driving the Program is FailSafe Defaulted in Third Party Positive Reinforcement Theory for No Reaction is equally Valid as a Driver in the absence of a Positive and Supportive ProAction should an Element or Component or even a whole new Operating System be Created for BetaTesting.

Such is Magical Mystery Turing in Advanced Artificial Intelligence Concepts for Global Operating Devices.

[Nick]

Measurement Follies

'Teachers go to great lengths to make sure students remember to specify their units. It is not enough to say that the answer is 42. Forty-two what? 42 metres? 42 electronvolts? 42 furlongs per fortnight? Without a clear understanding of what units are involved, certain results and claims can be meaningless, misleading or simply expensive."

Not to mention embarassing

http://www.youtube.com/watch?v=WXGbwIkvh38

[Brian Miller]

Missing forest for pine needles

Ages ago, I learned programming on non-OO languages like this:

#1: Design.

#2: Implement the design.

#3: Test that the code works and matches the design.

#4: Repeat the above as necessary.

It isn't about test first or implement first, it is about DESIGNING first. Time and time again I have seen projects with no design. None. Nothing. A barren wasteland of buzzwords and hype masking OO spaghetti. New languages with obfuscated syntax being "updated" to provide basic functionality in C.

Where are the articles about good design? I don't see them. It has been stupid methodology vs stupid methodology, neither of them having a thing to do with design. Hello, the reality check bounced because the account is overdrawn.

Unit testing is old stuff. Really old stuff. And everybody is acting like its the most amazing discovery on the planet. It is a rehash of what I learned over 20 years ago. And hasn't anything improved since then? NO! Not a thing! A new paint job on how to do basic work.

Everybody bemoans that processors are advancing but the software is stagnant. WAKE UP PEOPLE! Books on good design have been around before 1986. And how many people read them? Passing few! The knowledge of good software design is not in some new language, it is in books!

Software stagnates because nobody bothers to read the material that has been out there for decades. Everything has to be done by a cheesy IDE. All of the multi-threaded problems have been solved for many years. But why is software still single-threaded? Laziness, pure laziness! "Ooo, its too hard!" I hear. "It takes special skills to do that!" What, you mean like reading and designing?

And now everything has spiraled down to nitpicking definitions. Why? Where are those who would say, "If I have seen further it is by standing on ye shoulders of Giants." But nobody bothers to stand on the shoulders of giants. That's the whole problem, right there.

[Adam]

Nice article Kevin,

I would suggest though that the most important point about unit testing is that it formally defines the business rule that is in the original programmers head at the time it is implemented. I have worked on projects with very little documentation and no unit tests which have had few bugs. The bugs were avoided because the original programmer understood their design and made sure not to break any rules. But as the program was enhanced, and other developers who were not intimately involved in the original design started working with the different units, problems start emerging.

I don't see design and unit testing as different entities. I see unit test definition as a design step. If you don't want someone setting a property to a negative value, don't just commit to yourself that you will never do it, write a 5 second unit test that fails in that scenario.

Customer requirements are seldom static easily defined problems. Most of the time, customers themselves do not understand every nuance about what they want, and having good unit test coverage will give you more confidence to remodel your application and data structures when you understand the problem domain better.

The question of scope is to me the most complex side of unit testing, and I believe this to be because there is no simple answer. There is usually no limit to additional test cases that can be created, but there is usually a point of diminishing returns. The real key is knowing what is necessary and when to stop.

[Anonymous Coward]

Yes, unit testing is older than dirt

"The intent of the module or unit test is to find discrepancies between the module's logic and interfaces and its module external specification (the description of the module's function, inputs, outputs, and external effects). The step of compiling the module should also be considered as part of the module test since the complier detects most syntax errors and a few semantic or logic errors."

Glenford J. Myers, "Software Reliability" - 1976

[Simon Parmenter]

Verification and Validation

"A unit test is a test of behaviour (normally functional) whose success or failure is wholly determined by the correctness of the test and the correctness of the unit under test."

This says that the unit test is a test of verification - build the product right. Cannot not a unit test be a validation test - build the right product?

I agree with Brian Miller's comment. When I began learning software engineering nearly 2 decades ago I seem to remember there were five significant basic activities that could be discerned that a person or group of people went through during the production of a software product.

1. Formulation of Requirements from the analysis of answers to questions asked of people/users

2. Formulation of Specifications from the analysis of requirements

3. Design(s) from the analysis of the specifications

4. Design implementation from the analysis of the design(s)

5. Testing - validation and verification - according to the analysis of requirements, specs, and design.

These are have always been interleaved and bi-directionally communicating activities.

I have seen very little evidence of these activities during my career in software.

Questions.

Is an assertion in code a test? A test of validation? A test of verification?

[Jonathan]

testing times

reading the comments i get the impression thatthey have been written by 2 distinct sets of people.

1. brian miller and those agreeing with him who, i'm guessing have worked on large-scale mainframe applications in large companies

2. windows/unix developers.

i've worked in both environments, in the same sort of firms, and really there is very little comparison between the design and planning that goes into each type of system.

any mainframe system that i have worked on:

a) the requirements are gathered

b) the specifications are drawn-up

c) the design is drawn up

all 3 are checked and the tests designed to see that all the requirements are met and all specified behaviour is implemented.

- unit tests (what is a unit test? - please.. it took me 2 weeks in the industry to figure that one out), link(/component) tests, integration tests and overall system tests.

any wintel/unix system i have been near:

a) the requirements are gathered

b) the specifications are drawn-up

c) ... the programmer just gets on and does what they think should be done with no checking back to the requirements or the spec with testing sort of bolted on the end in an ad-hoc manner.

the results have normally been:

- mainframe code with little or no errors by the time it hits production and a clear traceable path between a specific requirement, a specific section in a design doc and test references so that the few errors occuring are easily fixed and retested

- windows/unix code that bounces back and forth between test and development with, each time, the developer insisting they have tested it and it works, fixes for one bug interfering with other code and whole mess of nobody knowing who has changed what or why they did so.

proper design and proper planning...