Testing at Microsoft

I was a contract employee at Microsoft, an SDET I. While I was there I learned quite a lot about testing, and I really believe it made me a much better programmer. Although my time there was short lived, I managed to retain some of my testing knowledge, and I hope to share that with you now. So I’ll quickly and briefly cover the basics of testing. I’m not talking about unit testing either. I’ll break down the types of tests you have into different categories. There may be more.

  • BVT (Base verification tests)
  • –Positive test cases
  • FVT (Functional verification tests)
  • –Positive Test Cases (Tester tests for expected failures )
  • –Negative Test Cases (We catch unexpected failures during testing (hopefully) )

BVT (Base Verification Tests)
These are the basic tests performed to show that the operation is basically working. Sometimes you just want to run the BVT tests because they’re faster and shorter. Example:

It is short and to the point. TEST is a simple macro that checks if the return value of the function is equal to the second parameter. It checks to see if 2*2/1 == 5. Nothing more. It is fast, but it doesn’t check everything. This test is also called a positive test case. A positive test case consists of passing in valid data, and expecting to get a valid result.

FVT (Functional Verification Tests)
The FVTs are a much more in depth testing stage. It consists of a much more thorough testing of the functions, in both quantity, and quality. FVTs consists of the BVT tests as well. So they may be duplicated, but this isn’t necessary. In some cases, the FVTs consists of modifications to the BVTs.
From the example above:

As you can see, we now have a lot more test cases and they’re broken up into positive and negative test cases. We may want then to be in separate functions too, but I didn’t do that here. Our positive test cases have a lot more cases and are simple modifications to the BVT.

The FVT also consists of our negative cases, where, if the function fails, (gives an invalid result the user can’t use) it needs to fail in an expected and deterministic way. Under no circumstances should it crash. When it crashes, this is an “unexpected failure,” and is usually caught very soon, since your tests will also crash. This is the most common unexpected failure. I can’t really think of any other right now.

Also, all of the negative test cases will fail in this scenario because nothing in the function is being thrown. And our tests are expecting it to throw the correct exceptions. When testing, it’s important to test common boundary values. In the example above, good things to test are 0, -1, 1, INF, -INF, NaNs, and other such things. When writing code, it’s important to check for the same boundary conditions immediately at the top of the function.

One final note before I leave. These are NOT unit tests. Unit tests sometimes overlap the BVTs, but they are written by the developer who wrote the code (in this case the Calc function). So they are not written by the tester (or SDET).

Leave a Reply

You must be logged in to post a comment.