Ben Biddington

Whatever it is, it's not about "coding"

Posts Tagged ‘rake

Raking .NET projects in TeamCity

leave a comment »

Faced with the unpleasant prospect of assembling yet another stack of xml files for an automated build, I thought I’d try rake instead. A couple of people here at 7digital have used Albacore before, so I started there.

1. Build

Use Albacore‘s msbuild task:

require 'albacore'

desc "Clean and build"
msbuild 'clean_and_build' do |msb|
    msb.properties :configuration => :Release
    msb.targets :Clean, :Build
    msb.verbosity = "quiet"
    msb.solution  = "path/to/ProjectName.sln"
end

2. Run tests

This is also very straight forward with Albacore, but slightly more useful is applying the usual TeamCity test result formatting and reporting.

2.1 Tell your build where the NUnit test launcher is

TeamCity already has an NUnit runner, and the recommended way to reference it is with an environment variable.

Note: The runners are in the <TEAM CITY INSTALLATION DIR>/buildAgent/plugins/dotnetPlugin/bin directory.

2.2 Write the task

Once you have the path to the executable, you’re free to apply any of the available runner options.

Assuming you have added the TEAMCITY_NUNIT_LAUNCHER environment variable then the actual execution is then something like:

asm = 'ProjectName.Unit.Tests.dll'
nunit_launcher = ENV["TEAMCITY_NUNIT_LAUNCHER"]
sh("#{nunit_launcher} v2.0 x86 NUnit-2.5.0 #{asm}")

Beats hundreds of lines of xml I reckon.

References

Written by benbiddington

18 February, 2010 at 13:37

Posted in development

Tagged with , , , , , ,

TeamCity — rake runner aborts early

with 3 comments

How to run TeamCity rakerunner locally

It is important to get the working directory right, otherwise you’ll get a bunch of require errors, for example:

C:\ruby\bin/ruby.exe C:\BuildAgent\plugins\rake-runner\lib\rb\runner\rakerunner.rb

Fails with error:

C:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require’: no such file to load — teamcity/rakerunner_consts (LoadError)
from C:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require’
from C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:20
no such file to load -- teamcity/rakerunner_consts (LoadError)

from C:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'

from C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:20

rake_ext is including a file with path:

require 'teamcity/rakerunner_consts'

And we know require works relative to the working directory which means our working directory must be set to:

C:\BuildAgent\plugins\rake-runner\lib\rb\patch

Rake abort

TeamCity rakerunner exits with message “Rake aborted!” as soon as it encounters a non-zero exit code from any task. Any subsequent tasks are therefore skipped.

This is not desirable behaviour for us because we have reporting tasks that must run always.

Stack trace showing the overridden members:

C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:158:in 'process_exception'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:95:in 'target_exception_handling'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:266:in 'execute'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:578:in 'invoke_with_call_chain'
C:/ruby/lib/ruby/1.8/monitor.rb:242:in 'synchronize'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:571:in 'invoke_with_call_chain'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:564:in 'standard_invoke_with_call_chain'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:235:in 'invoke'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:90:in 'target_exception_handling'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:234:in 'invoke'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2027:in 'invoke_task'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2005:in 'top_level'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2005:in 'each'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2005:in 'top_level'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:311:in 'standard_exception_handling'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:1999:in 'top_level'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:1977:in 'run'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:311:in 'standard_exception_handling'
C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:1974:in 'run'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:179:in 'run'
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rakerunner.rb:40

Here’s what rake Application.top_level looks like:

# Rake lib
def top_level
    standard_exception_handling do
        if options.show_tasks
            display_tasks_and_comments
        elsif options.show_prereqs
            display_prerequisites
        else
            top_level_tasks.each { |task_name| invoke_task(task_name) }
        end
    end
end

The TeamCity version has overridden standard_exception_handling:

# TeamCity Rakerunner
def standard_exception_handling
    begin
        yield
    rescue Rake::ApplicationAbortedException => app_e
        raise
    rescue Exception => exc
        Rake::TeamCityApplication.process_exception(exc)
    end
end

So the yield there is yielding to the result of executing each top level task. The rake task implementation has an execute method like:

# TeamCity Rakerunner
def execute(*args, &block)
    standard_execute_block = Proc.new do
        standard_execute(*args, &block)
    end

    if application.options.dryrun
        Rake::TeamCityApplication.target_exception_handling(
            name, true, "(dry run)", &standard_execute_block)
    else
        Rake::TeamCityApplication.target_exception_handling(
            name, true, &standard_execute_block)
    end
end

Where Rake::TeamCityApplication.target_exception_handling raises Rake::ApplicationAbortedException. We can disable this behaviour by commenting it out.

Solution

The Quick solution is to comment out the line that raises the exception:

# rake_ext line 158, method TeamCityApplication.process_exception
raise Rake::ApplicationAbortedException, exc

The problem now though, is that a failing task will not be reported automatically.

In this project that’s fine because we’re doing this ourselves in a subsequent step, but for other projects using the rakerunner this may prove confusing.

Written by benbiddington

18 May, 2009 at 12:19

Rake — shell task failure

leave a comment »

For my current project, we’re running cucumber as one of our tasks. Our report generation runs afterwards — which we need to happen regardless of cucumber’s exit code (cucumber exits with code 1 when any tests fail).

Handling shell task failure

We noticed that if cucumber fails, rake exits immediately — without running our subsequent report tasks.

We’re running cucumber with rake’s ruby function — which runs the ruby interpreter in its own process by invoking a shell command.

Pseudo-stacktrace (irrelevant source lines omitted):

Shell::CommandProcessor.system(command, *opts)
Rake::Win32.rake_system(*cmd)
FileUtils.sh(*args, &block)
FileUtils.ruby(*args, &block)

The key to this is the behaviour of rake’s FileUtils.sh. Examining the source shows:

If it is not supplied a block, and the shell command fails, and rake exits with error status.

Aside: For some reason, Kernel.system echoes errors.

Example

This task will cause rake to exit immediately with status 1:

task 'this-one-fails' do
    ruby("xxx")
end

while the following task does not cause rake to terminate  normally, any subsequent task will still run:

path = File.expand_path(File.dirname(__FILE__) + ‘/output/cucumber-manual.txt’)
cuke_path = ‘C:/ruby/lib/ruby/gems/1.8/gems/cucumber-0.3.3/bin/cucumber’
ruby(“#{cuke_path} -h > ‘#{path}'”) do |success, exit_code|
puts “Exited with code: #{exit_code}” if !success
end
task 'this-one-succeeds' do
    ruby("xxx") do |success, exit_code|
        puts "Exited with code: #{exit_code.exitstatus}" unless success
    end
end

Getting the last exit status

The $? variable contains the status of the last child process to terminate. This may be useful if steps subsequent to the failed one require this information.

Written by benbiddington

16 May, 2009 at 16:45

TeamCity build agents and FireWatir

leave a comment »

We have had some issues lately getting our automated acceptance tests running in TeamCity on Firefox.

Our failing tests could open browser windows, but not interact with them — the same behaviour exhibited when the JSSH plugin is missing. And the error reported in TeamCity is like:

Unable to connect to machine : 127.0.0.1 on port 9997.
Make sure that JSSh is properly installed and
Firefox is running with '-jssh' option
(Watir::Exception::UnableToStartJSShException)

This was puzzling because we knew we had installed JSSH.

The problem amounts to ensuring the Firefox JSSH extension is enabled for the correct account.

Build agents, user accounts and desktop (GUI) interaction

Only when Local System Account is selected can you also select the Allow service to interact with the desktop option. The “this account” method does not even allow selection of the checkbox (watch carefully, the checkbox is unchecked when you press apply).

teamcity-buildagent-account-control

Running without GUI does not prevent these types of things happening — they’re just not painted on the screen — so to diagnose issues like these it’s preferable to allow interaction with desktop.

This means running build agents as Local System account.

Solution

Firefox allows adding extensions via command line. So, to enable JSSH for every user on your build agent, download the extension, and then run something like:

$ "C:\Program Files\Mozilla Firefox\firefox -install-global-extension 'path\to\extension'"

You can then run Team City Build Agent Service as Local System user and enable desktop interaction.

References

  • [Windows Local System account] The LocalSystem account is a predefined local account used by the service control manager. This account is not recognized by the security subsystem, so you cannot specify its name in a call to the LookupAccountName function. It has extensive privileges on the local computer, and acts as the computer on the network.
  • [Firefox command line options] Various command line options are be available to perform advanced, troubleshooting or system administration tasks. Installation into the application directory is possible from the command line, intended to be used by administrators on multi-user systems.
  • [How does JSSH work?] FireWatir uses JSSh (TCP/IP JavaScript Shell Server for Mozilla) to drive the FireFox browser. JSSh allows other programs (like telnet) to establish JavaScript shell connections to a running Mozilla process. Once that connection is made, we can send JavaScript commands over the connection, which are executed against the DOM of the page loaded in the browser. JSSh listens on port 9997 by default.

Written by benbiddington

12 May, 2009 at 12:00

TeamCity, Rake and shell commands

leave a comment »

We encountered an interesting problem today. while getting automated acceptance tests running in TeamCity.

Why Rake? :

  • Rakefiles (rake‘s version of Makefiles) are completely defined in standard Ruby syntax. No XML files to edit. No quirky Makefile syntax to worry about (is that a tab or a space?)
  • Users can specify tasks with prerequisites.
  • Rake supports rule patterns to synthesize implicit tasks.
  • Flexible FileLists that act like arrays but know about manipulating file names and paths.
  • A library of prepackaged tasks to make building rakefiles easier.

What are we using it for?

Running cucumber.

Our rake file consists of a single task instructing the ruby interpreter to run cucumber with a set of fixtures — simple.

What were the symptoms?

We found that our build would “fail” as soon as we issued the shell command. The rest of the process works as expected — cucumber runs all the test and we get the results logged.

Finally we tracked this down to unexpected behaviour in the TeamCity rake runner.

What was the cause?

The TeamCity rake runner is designed to capture all output from rake, and redirect it to the appropriate TeamCity log.

The problem has arisen because we are issuing a shell command using FileUtils.sh. Notice it’s calling rake_output_message, which does just this by default:

# Send the message to the default rake output (which is $stderr).
def rake_output_message(message)
    $stderr.puts(message)
end

The TeamCity rake runner has overridden this in an attempt to capture these messages. For some reason this implementation sends an error message to TeamCity.

This means that all shell commands fail when running under the TeamCity rake runner.

Why does rake write to $stderr by default?

Don’t know. And the method name is not very evocative: rake_output_message.

Perhaps it’s intended to be overridden.

EDIT, 07-05-2009: All of these issues have been resolved by upgrading to latest version — even the teamcity-deletes-rake-file wrongness.

References

Written by benbiddington

30 April, 2009 at 11:59