Posts tonen met het label Unit test. Alle posts tonen
Posts tonen met het label Unit test. Alle posts tonen

dinsdag 6 oktober 2009

Custom Check In policy with code coverage

I created a check in policy with a code coverage check. To archive this I've used the next process:
1) after testing write to a file (own extension) the test that has been done
2) in the validation part of the custom check in policy I open that particular file and read the corresponding code coverage percentage.

The code: in each test class:

..
private static TestContext _currentContext;
..
[ClassInitialize()]
public static void MyClassInitialize(TestContext testContext)
{
_currentContext = testContext;
}

[ClassCleanup]
public static void MyClassCleanUp()
{
File.WriteAllText(@"\.TU", _currentContext.TestDir );
}
..



So. That's step one. Now the custom check in. I used the example found in MSDN.

I only added some extra methods and references and modified some texts (they're in Dutch, but you can easily modify them. Some explanation: our business objects contain '.BL' in the filename (that's a check). A business object is composed of 2 files e.g.
'Task.BL.cs' (sometimes manually modified and named 'Task.BL.altered.cs') and 'Task.BL.custom.cs' (and out of scope: also a 'Task.resx') . The output-unitfile is created directly aside this files: 'Task.TU'. In this file the last test for this Business Object is specified.

First check is the existence of the '.TU' file.
Second check is the timestamp of the '.TU' compared to the business object file(s).
Third check is the existence of the data.coverage file.
Fourth check is the most important one: is a code coverage of 80% reached.
Last check is the existence of the resource file.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.TeamFoundation.VersionControl.Client;
using System.Windows.Forms;
using System.IO;
using System.Xml;
using System.Data;
using Microsoft.VisualStudio.CodeCoverage;

///
/// http://msdn.microsoft.com/en-us/library/bb668980.aspx
///

[Serializable]
public class AssociatedCheckInPolicy : PolicyBase
{
public AssociatedCheckInPolicy()
: base()
{
}

public override string Description
{
get
{
return "AssociatedCheckInPolicy";
}
}

// This is a string that is stored with the policy definition on the source
// control server. If a user does not have the policy plug-in installed, this string
// is displayed. You can use this to explain to the user how they should
// install the policy plug-in.
public override string InstallationInstructions
{
get { return "To install this policy, read InstallInstructions.txt."; }
}

// This string identifies the type of policy. It is displayed in the
// policy list when you add a new policy to a Team Project.
public override string Type
{
get { return "AssociatedCheckInPolicy"; }
}

// This string is a description of the type of policy. It is displayed
// when you select the policy in the Add Check-in Policy dialog box.
public override string TypeDescription
{
get { return "Voordat deze check-in policy een business object doorlaat, moeten alle tests geslaagd zijn en een code coverage opleveren van minimaal 80%"; }
}

// This method is called by the policy framework when you create
// a new check-in policy or edit an existing check-in policy.
// You can use this to display a UI specific to this policy type
// allowing the user to change the parameters of the policy.
public override bool Edit(IPolicyEditArgs args)
{
// Do not need any custom configuration
return true;
}

private string Hulp { get; set; }

// This method performs the actual policy evaluation.
// It is called by the policy framework at various points in time
// when policy should be evaluated. In this example, the method
// is invoked when various asyc events occur that may have
// invalidated the current list of failures.
public override PolicyFailure[] Evaluate()
{
for (int i = 0; i < PendingCheckin.PendingChanges.CheckedPendingChanges.Length; i++)
{
string className = PendingCheckin.PendingChanges.CheckedPendingChanges[i].LocalItem;
// kijk of .BL erin voorkomt. (alleen voor BL-classes)
if (className.Contains(".BL."))
{
// kijk of de bijbehorende unittests uitgevoerd zijn.
if (File.Exists(className))
{
string testUnitFile = className.Replace(".custom", "").Replace(".altered", "").Replace(".cs", "");
testUnitFile = testUnitFile.Substring(0, testUnitFile.LastIndexOf(".")) + ".TU";
if (File.Exists(testUnitFile))
{
// kijk of de timestamp niet gewijzigd is.
PolicyFailure[] ret = new PolicyFailure[1];
string testdir = File.ReadAllText(testUnitFile);

if (File.GetLastWriteTime(testUnitFile) < File.GetLastWriteTime(className))
{
ret[0] = new PolicyFailure("De code is nieuwer dan de tests.", this);
return ret;
}

PolicyFailure pf = GenerateCodeCoverage(testUnitFile, testdir, @"c:\alurecs\");
if (pf == null)
{
// extra check: bestaat de resx?
string resxFile = testUnitFile.Replace(".TU", ".resx");
if (!File.Exists(resxFile))
{
Hulp = "Maak een resourcefile aan (generator)";
ret[0] = new PolicyFailure("Resource file niet aangetroffen", this);
return ret;
}
return new PolicyFailure[0];
}
ret[0] = pf;
return ret;
}
else
{
PolicyFailure[] ret = new PolicyFailure[1];
Hulp = "Voer de tests uit in de testclass (nieuwste template wel gebruikt?)";
ret[0] = new PolicyFailure("Geen unittest file gevonden", this);
return ret;
}


}
}
}

return new PolicyFailure[0];
}


/// THIS IS THE IMPORTANT PART!
private PolicyFailure GenerateCodeCoverage(string testUnitFile, string testdir, string destdir)
{
string codecoveragedir = Path.Combine(Path.Combine(testdir, "In"), Environment.MachineName);

string unitFile = testUnitFile;
string className = unitFile.Substring(unitFile.LastIndexOf(@"\") + 1).Replace(".TU", "");

// lees trx
//
//
//

string file = testdir + ".trx";
XmlDocument doc = new XmlDocument();
if (File.Exists(file))
{
doc.Load(file);
// //TestRun/ResultSummary/Counters
XmlNode xni = doc.SelectNodes(".")[0].ChildNodes[1].ChildNodes[1].ChildNodes[0];
string passed = xni.Attributes["passed"].Value;
string executed = xni.Attributes["executed"].Value;
if (passed == executed)
{
// code coverage?
string coverageFile = Path.Combine(codecoveragedir, "data.coverage");
if (File.Exists(coverageFile))
{
string binDir = Path.Combine(testdir, "Out");
CoverageInfoManager.ExePath = binDir;
CoverageInfoManager.SymPath = binDir;
CoverageInfo coverageInfo = CoverageInfoManager.CreateInfoFromFile(coverageFile);
CoverageDS data = coverageInfo.BuildDataSet(null);
DataRow dr = data.Class.Select(String.Format("ClassName='{0}'", className))[0];
int bcc = data.Class.Columns.IndexOf(data.Class.BlocksCoveredColumn);
uint bc = (uint)dr[bcc];

int bncc = data.Class.Columns.IndexOf(data.Class.BlocksNotCoveredColumn);
uint bnc = (uint)dr[bncc];

if (Convert.ToDouble(bc) / (Convert.ToDouble(bc + bnc)) < .8)
{
// error!
Hulp = "De 80% drempel is niet gehaald, maak meer tests.";
return new PolicyFailure("80% drempel niet gehaald", this);
}
}
else
{
if (File.Exists(unitFile))
File.Delete(unitFile);
Hulp = "Run de tests uit in de testclass, code coverage is niet gevonden";
return new PolicyFailure("Geen code coverage gebruikt!", this);
}
}
}
else
{
if (File.Exists(unitFile)) File.Delete(unitFile);
Hulp = "Voer de tests uit in de testclass";
return new PolicyFailure("Voer testen uit!", this);
}
return null;
}


// This method is called if the user double-clicks on
// a policy failure in the UI. In this case a message telling the user
// to supply some comments is displayed.
public override void Activate(PolicyFailure failure)
{
MessageBox.Show("1) Maak een testclass (laatste versie template)"
+ Environment.NewLine + "2) Voer alle tests uit"
+ Environment.NewLine + "3) Code coverage > 80%"
, "Oplossen van probleem");
}

// This method is called if the user presses F1 when a policy failure
// is active in the UI. In this example, a message box is displayed.
public override void DisplayHelp(PolicyFailure failure)
{
MessageBox.Show(Hulp, "Prompt Policy Help");
}
}

maandag 28 september 2009

Even more external data in testclasses

Now I created a base class for testing purposes:

It contains the next code:

private
static
DataTable ReadXls(string filename, string sheet) {


using (OleDbConnection con = new
OleDbConnection(String.Format(CultureInfo.InvariantCulture, "Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};Extended Properties=\"Excel 8.0;IMEX=1\"", filename))) {

con.Open();


try

{


using (DataSet myDataSet = new
DataSet())

{

myDataSet.Locale = CultureInfo.InvariantCulture;


using (OleDbDataAdapter myCommand = new
OleDbDataAdapter(String.Format(CultureInfo.InvariantCulture, " SELECT * FROM [{0}$]", sheet), con))

{

myCommand.Fill(myDataSet);

}

con.Close();


return myDataSet.Tables[0];

}

}


catch

{


throw;

}


finally

{

con.Close();

}

}

}

It opens an excel sheet and reads data from it. Every sheet is a database table and the columns should have a caption to address it in the code.


DataTable tabel = ReadXls(Bestand, typeof(T).Name);


foreach (DataRow dr in tabel.Rows)

{

T obj = (T)typeof(T).GetConstructor(new
Type[] { }).Invoke(new
object[] { });


foreach (DataColumn dc in tabel.Columns)

{


string s = dr[dc.ColumnName].ToString();


PropertyInfo pi = obj.GetType().GetProperty(dc.ColumnName);


if (s == "NULL" || string.IsNullOrEmpty(s) ) s = null;

obj.SetValue(dc.ColumnName, s);

                ...

The code above is not complete, I wish you good luck to complete it.

vrijdag 10 juli 2009

Disable Unit tests

To prevent executing a unit test method, add the [Ignore] attribute.

IgnoreAttribute Class (Microsoft.VisualStudio.TestTools.UnitTesting)

woensdag 17 juni 2009

How to: Disable and Enable Unit Tests Visual Studio

You can disable a test to make it temporarily unavailable when you run a group of tests that contains it. You can re-enable it at any time. Why would you disable a test? Consider the following possibilities:

Your product build contains a bug that causes a crash. This bug is located in code that one of your tests exercises. This test is one of many that are run in sequence. Because of the crash that this test indirectly triggers, your test run cannot move past that test in the sequence. The solution is to file a bug on the code that crashes, and disable the test until the bug is fixed. Then, the test sequence can progress beyond that one test.

A specific test in a test run uncovers a bug, which you file. To keep from repeatedly finding the same known failure every time that you run the test selection, disable the test.

Note:
Remember to re-enable a test when its associated bug is fixed.


If you run a test list that contains disabled tests, only the enabled tests within that test list, if any, will be run. If a test list does not contain any enabled tests, the test list itself is disabled.

Using the Ignore Attribute with Unit Tests

Using the Ignore Attribute describes how to mark a unit test method with the IgnoreAttribute attribute. Using this attribute changes the availability of the test, but it differs in the following ways from the other way to disable and enable tests:

As described in Disabling Tests and Enabling Tests, you can use the Test List Editor to disable and enable tests of any type. But the Ignore attribute can be applied only to a test method. This means that you can apply the Ignore attribute only to unit tests.

The flag for enabling and disabling tests is stored in the test metadata file.

The Ignore attribute for a unit test resides in the source file of the unit test, together with the other attributes and source code of the test. This means that, if you disable a unit test in the Test List Editor, and later run the test by using the MSTest.exe command-line utility, the test still runs. But if you mark a unit test with the Ignore attribute, compile the assembly, and then run the test by using MSTest.exe, the test does not run. It does not run because the Ignore attribute has become part of the test.

These two approaches are used in different scenarios:

Ignore attribute. You are a developer who authors both production code and test code. You are in the middle of making a large set of changes, and you know that one of your unit tests will be broken for a while. Therefore, when you check in the test, you mark it with the Ignore attribute to indicate that you do not expect it to work.

Disable and enable. You work in a build lab and you do not have access to the source code of the tests you are running. In this case, you can use only the enable and disable capabilities of the Test List Editor.

Disabling Tests
To disable tests
To disable one test, in the Test List Editor, right-click an enabled test and then click Disable. To disable multiple tests, highlight the enabled tests you want to disable, by pressing CTRL or SHIFT when clicking the tests' rows. Next, right-click the selected rows and then click Disable.

- or -

Select the test in the Test List Editor or the Test View window and press F4 to display the Properties window. Then, set the value of the Test Enabled property to False.

Enabling Tests
To enable tests
To enable one test, in the Test List Editor, right-click a disabled test and then click Enable. To enable multiple tests, highlight the disabled tests you want to enable by pressing CTRL or SHIFT when clicking the tests' rows. Next, right-click the selected rows and then click Enable.

- or -

Select the test in the Test List Editor or the Test View window and press F4 to display the Properties window. Then, set the value of the Test Enabled property to True.

Using the Ignore Attribute
To use the Ignore attribute
Open the unit test to edit its source code. To do this, in the Test List Editor, double-click the unit test.

The source code file that contains the unit test opens.

Add the IgnoreAttribute attribute to the unit test.

The test is disabled. In the Properties window, the Test Enabled property is automatically set to False. To learn more about the scope of this action, see the previous section "Using the Ignore Attribute with Unit Tests."

To re-enable the test, delete or comment out the Ignore attribute.


Reference: msdn