Raking .NET projects in TeamCity

Faced with the unpleasant prospect of assembling yet another stack of xml files for an automated build, I thought I’d try rake instead. A couple of people here at 7digital have used Albacore before, so I started there.

1. Build

Use Albacore‘s msbuild task:

require 'albacore'

desc "Clean and build"
msbuild 'clean_and_build' do |msb|
    msb.properties :configuration => :Release
    msb.targets :Clean, :Build
    msb.verbosity = "quiet"
    msb.solution  = "path/to/ProjectName.sln"
end

2. Run tests

This is also very straight forward with Albacore, but slightly more useful is applying the usual TeamCity test result formatting and reporting.

2.1 Tell your build where the NUnit test launcher is

TeamCity already has an NUnit runner, and the recommended way to reference it is with an environment variable.

Note: The runners are in the <TEAM CITY INSTALLATION DIR>/buildAgent/plugins/dotnetPlugin/bin directory.

2.2 Write the task

Once you have the path to the executable, you’re free to apply any of the available runner options.

Assuming you have added the TEAMCITY_NUNIT_LAUNCHER environment variable then the actual execution is then something like:

asm = 'ProjectName.Unit.Tests.dll'
nunit_launcher = ENV["TEAMCITY_NUNIT_LAUNCHER"]
sh("#{nunit_launcher} v2.0 x86 NUnit-2.5.0 #{asm}")

Beats hundreds of lines of xml I reckon.

References

• Albacore — rake tasks for.NET systems
• TeamCity NUnit launcher
• Setting environment variables for your TeamCity build

.NET Process — avoid deadlock with async reads

If you are working with a child process that writes large amounts of data to its redirected stdout (or stderr), it is advisable to read from it asynchronously.

Why read stdout asynchronously?

A pipe is a connection between two processes in which one process writes data to the pipe and the other reads from the pipe. System.Diagnostics.Process.StandardOutput is an example of a pipe.

A child process may block while it waits for the client end to read from its stdout (or stderr).

When redirected, a process’s stdout may reach its limit, it will then wait for its parent to read some data before it will continue. If the parent process is waiting for all the bytes to be written before it reads anything (synchronous read), then it will wait indefinitely.

The point is: redirected streams have a limited buffer, keep them clear to allow process to complete.

So you may encounter deadlock:

[Deadlock] Pipes have a fixed size (often 4096 bytes) and if a process tries to write to a pipe which is full, the write will block until a process reads some data from the pipe.

If your child process is going to write more data than its buffer can contain, you’ll need to read it asynchronously. This stops a process blocking by ensuring there is space to emit data.

Tips

Example: piping a file to lame stdin (Windows)

Use the type command:

$ type file.mp3 | lame --mp3output 64 - "path/to/output.mp3"

Type reads the source file an emits it to its stdout, we’re then piping that directly to lame. In the preceeding example, lame has been instructed to read from stdin and write directly to a file.

To pipe stdout to another process, use something like:

$ type file.mp3 | lame --mp3output 64 - - | another_process

Or redirect to a file:

$ type file.mp3 | lame --mp3output 64 - - > "path/to/output.mp3"

Get a list of running processes (Windows)

Use the query process command.

References

• Pipeline (Unix)
• Pipes (Windows)
• Buffering in stdio streams
• What is a pipe?
• Avoid blocking reads on both stdout and stderr (MSDN)

.NET Process — working with binary output

Lately we discovered an issue while encoding Mp3 files with Lame. Our client reported encoded files we garbled; playable but watery — and full of pops and clicks.

We found this was due to interpreting the binary output from Lame as text — we had mistakenly employed Process.BeginOutputReadLine and its companion event OutputDataReceived.

Process.OutputDataReceived

By observing a Process using its OutputDataReceived event, clients can make asynchronous reads on a process’s StandardOutput.

Process.StandardOutput is a TextReader: it represents a reader that can read a sequential series of characters, i.e., it interprets its underlying stream as text.

When StandardOutput is being read asynchronously, the Process class monitors it, collecting characters into a string. Once it encounters a line ending, it notifies observers (handlers of its OutputDataReceived event), with the line of text it’s been collecting.

In short, the Process‘s underlying byte stream is converted to lines of text, and clients are notified one line at a time.

In doing so, some bytes are discarded: any bytes that (in the current encoding) represent line endings.

As a result of these missing bytes, our output Mp3s were playable, but sounded terrible.

Solution

Bypass StandardOutput. Use its underlying Stream instead.