Posts Tagged ‘team-city’
TeamCity — rake runner aborts early
How to run TeamCity rakerunner locally
It is important to get the working directory right, otherwise you’ll get a bunch of require errors, for example:
C:\ruby\bin/ruby.exe C:\BuildAgent\plugins\rake-runner\lib\rb\runner\rakerunner.rb
Fails with error:
no such file to load -- teamcity/rakerunner_consts (LoadError) from C:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require' from C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:20
rake_ext is including a file with path:
require 'teamcity/rakerunner_consts'
And we know require works relative to the working directory which means our working directory must be set to:
C:\BuildAgent\plugins\rake-runner\lib\rb\patch
Rake abort
TeamCity rakerunner exits with message “Rake aborted!” as soon as it encounters a non-zero exit code from any task. Any subsequent tasks are therefore skipped.
This is not desirable behaviour for us because we have reporting tasks that must run always.
Stack trace showing the overridden members:
C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:158:in 'process_exception' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:95:in 'target_exception_handling' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:266:in 'execute' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:578:in 'invoke_with_call_chain' C:/ruby/lib/ruby/1.8/monitor.rb:242:in 'synchronize' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:571:in 'invoke_with_call_chain' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:564:in 'standard_invoke_with_call_chain' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:235:in 'invoke' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:90:in 'target_exception_handling' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:234:in 'invoke' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2027:in 'invoke_task' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2005:in 'top_level' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2005:in 'each' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:2005:in 'top_level' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:311:in 'standard_exception_handling' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:1999:in 'top_level' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:1977:in 'run' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:311:in 'standard_exception_handling' C:/ruby/lib/ruby/gems/1.8/gems/rake-0.8.4/lib/rake.rb:1974:in 'run' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rake_ext.rb:179:in 'run' C:/BuildAgent/plugins/rake-runner/lib/rb/runner/rakerunner.rb:40
Here’s what rake Application.top_level looks like:
# Rake lib
def top_level
standard_exception_handling do
if options.show_tasks
display_tasks_and_comments
elsif options.show_prereqs
display_prerequisites
else
top_level_tasks.each { |task_name| invoke_task(task_name) }
end
end
end
The TeamCity version has overridden standard_exception_handling:
# TeamCity Rakerunner
def standard_exception_handling
begin
yield
rescue Rake::ApplicationAbortedException => app_e
raise
rescue Exception => exc
Rake::TeamCityApplication.process_exception(exc)
end
end
So the yield there is yielding to the result of executing each top level task. The rake task implementation has an execute method like:
# TeamCity Rakerunner
def execute(*args, &block)
standard_execute_block = Proc.new do
standard_execute(*args, &block)
end
if application.options.dryrun
Rake::TeamCityApplication.target_exception_handling(
name, true, "(dry run)", &standard_execute_block)
else
Rake::TeamCityApplication.target_exception_handling(
name, true, &standard_execute_block)
end
end
Where Rake::TeamCityApplication.target_exception_handling raises Rake::ApplicationAbortedException. We can disable this behaviour by commenting it out.
Solution
The Quick solution is to comment out the line that raises the exception:
# rake_ext line 158, method TeamCityApplication.process_exception
raise Rake::ApplicationAbortedException, exc
The problem now though, is that a failing task will not be reported automatically.
In this project that’s fine because we’re doing this ourselves in a subsequent step, but for other projects using the rakerunner this may prove confusing.
TeamCity build agents and FireWatir
We have had some issues lately getting our automated acceptance tests running in TeamCity on Firefox.
Our failing tests could open browser windows, but not interact with them — the same behaviour exhibited when the JSSH plugin is missing. And the error reported in TeamCity is like:
Unable to connect to machine : 127.0.0.1 on port 9997. Make sure that JSSh is properly installed and Firefox is running with '-jssh' option (Watir::Exception::UnableToStartJSShException)
This was puzzling because we knew we had installed JSSH.
The problem amounts to ensuring the Firefox JSSH extension is enabled for the correct account.
Build agents, user accounts and desktop (GUI) interaction
Only when Local System Account is selected can you also select the Allow service to interact with the desktop option. The “this account” method does not even allow selection of the checkbox (watch carefully, the checkbox is unchecked when you press apply).
Running without GUI does not prevent these types of things happening — they’re just not painted on the screen — so to diagnose issues like these it’s preferable to allow interaction with desktop.
This means running build agents as Local System account.
Solution
Firefox allows adding extensions via command line. So, to enable JSSH for every user on your build agent, download the extension, and then run something like:
$ "C:\Program Files\Mozilla Firefox\firefox -install-global-extension 'path\to\extension'"
You can then run Team City Build Agent Service as Local System user and enable desktop interaction.
References
- [Windows Local System account] The LocalSystem account is a predefined local account used by the service control manager. This account is not recognized by the security subsystem, so you cannot specify its name in a call to the LookupAccountName function. It has extensive privileges on the local computer, and acts as the computer on the network.
- [Firefox command line options] Various command line options are be available to perform advanced, troubleshooting or system administration tasks. Installation into the application directory is possible from the command line, intended to be used by administrators on multi-user systems.
- [How does JSSH work?] FireWatir uses JSSh
(TCP/IP JavaScript Shell Server for Mozilla) to drive the FireFox browser. JSSh allows other programs (like telnet) to establish JavaScript shell connections to a running Mozilla process. Once that connection is made, we can send JavaScript commands over the connection, which are executed against the DOM of the page loaded in the browser. JSSh listens on port 9997 by default.
TeamCity, Rake and shell commands
We encountered an interesting problem today. while getting automated acceptance tests running in TeamCity.
Why Rake? :
- Rakefiles (rake‘s version of Makefiles) are completely defined in standard Ruby syntax. No XML files to edit. No quirky Makefile syntax to worry about (is that a tab or a space?)
- Users can specify tasks with prerequisites.
- Rake supports rule patterns to synthesize implicit tasks.
- Flexible FileLists that act like arrays but know about manipulating file names and paths.
- A library of prepackaged tasks to make building rakefiles easier.
What are we using it for?
Running cucumber.
Our rake file consists of a single task instructing the ruby interpreter to run cucumber with a set of fixtures — simple.
What were the symptoms?
We found that our build would “fail” as soon as we issued the shell command. The rest of the process works as expected — cucumber runs all the test and we get the results logged.
Finally we tracked this down to unexpected behaviour in the TeamCity rake runner.
What was the cause?
The TeamCity rake runner is designed to capture all output from rake, and redirect it to the appropriate TeamCity log.
The problem has arisen because we are issuing a shell command using FileUtils.sh. Notice it’s calling rake_output_message, which does just this by default:
# Send the message to the default rake output (which is $stderr).
def rake_output_message(message)
$stderr.puts(message)
end
The TeamCity rake runner has overridden this in an attempt to capture these messages. For some reason this implementation sends an error message to TeamCity.
This means that all shell commands fail when running under the TeamCity rake runner.
Why does rake write to $stderr by default?
Don’t know. And the method name is not very evocative: rake_output_message.
Perhaps it’s intended to be overridden.
EDIT, 07-05-2009: All of these issues have been resolved by upgrading to latest version — even the teamcity-deletes-rake-file wrongness.
