Our team has been using JasmineJs for unit testing our client side JavaScript code. We now have over 3,000 tests! In order to run these tests on multiple browsers we use Karma, the "Spectacular Test Runner for JavaScript" created by people from Google to run all our tests on multiple browsers. We are also using Microsoft's TFS Build (which is leaps and bounds improved over the old XAML build system) to run our tests in Gated Check ins, create build artifacts (dlls, installers, Azure packages), deploy to test environments and all number of fun things.

The default output results in the build from Karma has not been very helpful, just saying 1 of 1 passed or 0 of 1. We'd then have to open up a .trx file and look through the output to find which test failed.

I had some time, so I decided to see if I could get all the tests results showing and more useful like our C# test results. Here's how I go it working.

TLDR;

1. Setup the Karma Trx Reporter.
2. Add Publish Results Build Step using  ../TestResults/**/karma-test-results.trx as the path.
3. If remote Vm for Jasmine tests, copy the .trx file to the Results directory.

The first thing was to get Karma to create a .trx file that the Test Results could read. The hatchteam's Karma trx reporter NPM project. The steps are well laid out on the GitHub page. I named it 'karma-test-results.trx' in the karma.config.js I then ran Karma locally and saw that that trx file is created. Opening that in Visual Studio shows all the test results.

After I checked in the changes, the build output was still empty. I then tweeted Donovan Brown from Microsoft to get another hint. He was kind enough to respond "during the build the .trx is created? If so use the publish test results task. Select VSTest and point to trx." I then added the publish test results build step.

The build still did not find the test results giving this warning "No test result files were found using search pattern 'C:\BuildAgent\_work\1\s\**\karma-test-results.trx'". You might not have this same problem, if you are running the Jasmine tests directly on the build controller. The answer from "sarin" on StackOverflow indicates that it might just work (he's using gulp, we have C# code). We have a separate VM setup just to run the Jasmine tests. I needed to figure out how to get them from the agent to the build controller and reporting in the build.

The solution was to copy the trx file into the Results directory before asserting the results. When the test runs, it creates a folder in c:\users\AppData\Local\VSEQT\QTAgent\{guid}\{agent-machine-name}\. There is a Deployment directory and a Results directory that is deleted when the tests are completed. The build controller takes what is in the Results directory to report the results to the build. 

We are using C# with custom code to start the Karma process and get the results. I'll leave the KarmaProcess.cs to the reader as I don't own all of the code. One hint is that we have batch files to install and upgrade through npm, then execute karma with "karma start UnitTests\karma.conf.js". We then execute that .bat with ProcessStartInfo.

Edit: I added some steps on how to use Grunt instead of C# in this StackOverflow answer.

x
 
1
[TestMethod, Timeout(360000), TestCategory("JSUnit")]
2
public void RunUnitTests()
3
{
4
    // Output the machine this test is being ran on
5
    this.TestContext.WriteLine(String.Format("Test Machine: {0}", Environment.MachineName));
6
7
    var projectDir = this.ResolveUserPortalProjectFolder();
8
    var unitTestsDir = Path.Combine(projectDir, "UnitTests");
9
    var configFile = new FileInfo(Path.Combine(unitTestsDir, @"karma.conf.js"));
10
    var coverageDir = Path.Combine(unitTestsDir, @"Coverage");
11
    var nodeModulesFolder = Path.Combine(projectDir, "node_modules");
12
    var process = new KarmaProcess(configFile, nodeModulesFolder, "unit", coverageDir, unitTestsDir, this.TestContext);
13
    var result = process.RunOnce(TimeSpan.FromMinutes(6));
14
    this.CopyTrxToResults(projectDir);
15
    Assert.AreEqual(0, result, "UserPortal JavaScript unit tests failed");
16
}
17
18
/// <summary>
19
/// Copy the karma trx file to the TestResults folder so it gets to the Build Controller.
20
/// </summary>
21
/// <param name="projectDir"></param>
22
private void CopyTrxToResults(string projectDir)
23
{
24
    this.TestContext.WriteLine("Copy Trx to Results");
25
    var testResultsDir = Path.Combine(projectDir, "..\\", "Results");
26
    var resultsFileName = Path.Combine(projectDir, "karma-test-results.trx");
27
    var destination = Path.Combine(testResultsDir, "karma-test-results.trx");
28
    if (Directory.Exists(testResultsDir))
29
    {
30
        this.TestContext.WriteLine($"copying from {resultsFileName} to {destination}");
31
        File.Copy(resultsFileName, destination);
32
    }
33
    else
34
    {
35
        this.TestContext.WriteLine($"{testResultsDir} does not exist!");
36
    }
37
38
    this.TestContext.WriteLine($"{resultsFileName} exists? {File.Exists(resultsFileName)}");
39
}
40
41
private string ResolveUserPortalProjectFolder()
42
{
43
  if (IsRunningOnTestAgent)
44
  {
45
    // try a relative path from the current location
46
    var relativePath = Path.Combine(Environment.CurrentDirectory, @"..\..\..\", "ProjectName");
47
    if (Directory.Exists(relativePath))
48
    {
49
      return relativePath;
50
    }
51
52
    // default to test run directory for remote test execution
53
    return this.TestContext.DeploymentDirectory;
54
  }
55
  else
56
    return Path.Combine(this.TestContext.TestRunDirectory, @"..\..\ProjectName\");
57
}

I changed the path to ../TestResults/**/karma-test-results.trx and now the .trx file is found.

The tests results now show up in the build results!