Ben Biddington

Whatever it is, it's not about "coding"

Posts Tagged ‘cucumber

Outside-in development taken too far

with 2 comments

You may feel someone is taking things too far when you receive a set of scenarios like this:

Scenario 1: Warm Teapot

Given a Kettle of Boiling Water
When Boiling Water is poured in to the Teapot
Then it should* warm the Teapot

Scenario 2: Brewing the tea

Given a Kettle of Boiling Water
And 5 Teabags
When 5 Teabags are placed in the Teapot
And Boiling Water is poured in to the Teapot
And the Teapot is left with the above ingredients in it for 5 minutes
Then you should have Brewed Tea

Scenario 3: Pouring the tea

Given a Teapot of Brewed Tea
And a Mug with some Milk in it**
When the Brewed Tea is poured in to the Mug
Then you should have a Perfect Cup of Tea

* “Do or do not. There is no should”
** Debatable. I think we need to triangulate this, or consult the client.

Written by benbiddington

3 August, 2010 at 13:37

Posted in development

Tagged with , , ,

Passing cucumber specs on windows

leave a comment »

I had some trouble getting cucumber test features to pass on Windows due to directory separator differences, something like:
 Failing Scenarios:
-cucumber features/background/failing_background.feature:8 # Scenario: failing background
+cucumber features\background\failing_background.feature:8 # Scenario: failing background
There is an easy solution — run features with windows_mri profile, which results in the CUCUMBER_FORWARD_SLASH_PATHS environment variable being set to true.

Example

cucumber features/background.feature:93 -p windows_mri

Written by benbiddington

20 April, 2010 at 13:37

Posted in development

Tagged with , , ,

Rake — shell task failure

leave a comment »

For my current project, we’re running cucumber as one of our tasks. Our report generation runs afterwards — which we need to happen regardless of cucumber’s exit code (cucumber exits with code 1 when any tests fail).

Handling shell task failure

We noticed that if cucumber fails, rake exits immediately — without running our subsequent report tasks.

We’re running cucumber with rake’s ruby function — which runs the ruby interpreter in its own process by invoking a shell command.

Pseudo-stacktrace (irrelevant source lines omitted):

Shell::CommandProcessor.system(command, *opts)
Rake::Win32.rake_system(*cmd)
FileUtils.sh(*args, &block)
FileUtils.ruby(*args, &block)

The key to this is the behaviour of rake’s FileUtils.sh. Examining the source shows:

If it is not supplied a block, and the shell command fails, and rake exits with error status.

Aside: For some reason, Kernel.system echoes errors.

Example

This task will cause rake to exit immediately with status 1:

task 'this-one-fails' do
    ruby("xxx")
end

while the following task does not cause rake to terminate  normally, any subsequent task will still run:

path = File.expand_path(File.dirname(__FILE__) + ‘/output/cucumber-manual.txt’)
cuke_path = ‘C:/ruby/lib/ruby/gems/1.8/gems/cucumber-0.3.3/bin/cucumber’
ruby(“#{cuke_path} -h > ‘#{path}'”) do |success, exit_code|
puts “Exited with code: #{exit_code}” if !success
end
task 'this-one-succeeds' do
    ruby("xxx") do |success, exit_code|
        puts "Exited with code: #{exit_code.exitstatus}" unless success
    end
end

Getting the last exit status

The $? variable contains the status of the last child process to terminate. This may be useful if steps subsequent to the failed one require this information.

Written by benbiddington

16 May, 2009 at 16:45

Cucumber — ANSI formatting

leave a comment »

When cucumber output is piped to file on disk, it may contain non-printable characters and markup, for example:

[32m6 passed steps

Actually looks like this when output to console:

6 passed steps

This is due to cucumber emitting Ansi formatting.

[ANSI escape sequences] are used to control text formatting and other output options on text terminals.

Most of these escape sequences start with the characters ESC (ASCII decimal 27/hex 0x1B/octal 033) and [ (left bracket). This sequence is called CSI for Control Sequence Introducer(or Control Sequence Initiator). There is a single-character CSI (155/0x9B/0233) as well.

The ESC+[ two-character sequence is more often used than the single-character alternative, for details see C0 and C1 control codes. Devices supporting only ASCII (7-bits), or which implement 8-bit code pages which use the 0x80–0x9F control character range for other purposes will recognize only the two-character sequence. Though some encodings use multiple bytes per character, in this topic all characters are single-byte…

This formatting can be suppressed using the:

--[no-]color

switch.

Written by benbiddington

12 May, 2009 at 13:45

TeamCity build agents and FireWatir

leave a comment »

We have had some issues lately getting our automated acceptance tests running in TeamCity on Firefox.

Our failing tests could open browser windows, but not interact with them — the same behaviour exhibited when the JSSH plugin is missing. And the error reported in TeamCity is like:

Unable to connect to machine : 127.0.0.1 on port 9997.
Make sure that JSSh is properly installed and
Firefox is running with '-jssh' option
(Watir::Exception::UnableToStartJSShException)

This was puzzling because we knew we had installed JSSH.

The problem amounts to ensuring the Firefox JSSH extension is enabled for the correct account.

Build agents, user accounts and desktop (GUI) interaction

Only when Local System Account is selected can you also select the Allow service to interact with the desktop option. The “this account” method does not even allow selection of the checkbox (watch carefully, the checkbox is unchecked when you press apply).

teamcity-buildagent-account-control

Running without GUI does not prevent these types of things happening — they’re just not painted on the screen — so to diagnose issues like these it’s preferable to allow interaction with desktop.

This means running build agents as Local System account.

Solution

Firefox allows adding extensions via command line. So, to enable JSSH for every user on your build agent, download the extension, and then run something like:

$ "C:\Program Files\Mozilla Firefox\firefox -install-global-extension 'path\to\extension'"

You can then run Team City Build Agent Service as Local System user and enable desktop interaction.

References

  • [Windows Local System account] The LocalSystem account is a predefined local account used by the service control manager. This account is not recognized by the security subsystem, so you cannot specify its name in a call to the LookupAccountName function. It has extensive privileges on the local computer, and acts as the computer on the network.
  • [Firefox command line options] Various command line options are be available to perform advanced, troubleshooting or system administration tasks. Installation into the application directory is possible from the command line, intended to be used by administrators on multi-user systems.
  • [How does JSSH work?] FireWatir uses JSSh (TCP/IP JavaScript Shell Server for Mozilla) to drive the FireFox browser. JSSh allows other programs (like telnet) to establish JavaScript shell connections to a running Mozilla process. Once that connection is made, we can send JavaScript commands over the connection, which are executed against the DOM of the page loaded in the browser. JSSh listens on port 9997 by default.

Written by benbiddington

12 May, 2009 at 12:00

TeamCity, Rake and shell commands

leave a comment »

We encountered an interesting problem today. while getting automated acceptance tests running in TeamCity.

Why Rake? :

  • Rakefiles (rake‘s version of Makefiles) are completely defined in standard Ruby syntax. No XML files to edit. No quirky Makefile syntax to worry about (is that a tab or a space?)
  • Users can specify tasks with prerequisites.
  • Rake supports rule patterns to synthesize implicit tasks.
  • Flexible FileLists that act like arrays but know about manipulating file names and paths.
  • A library of prepackaged tasks to make building rakefiles easier.

What are we using it for?

Running cucumber.

Our rake file consists of a single task instructing the ruby interpreter to run cucumber with a set of fixtures — simple.

What were the symptoms?

We found that our build would “fail” as soon as we issued the shell command. The rest of the process works as expected — cucumber runs all the test and we get the results logged.

Finally we tracked this down to unexpected behaviour in the TeamCity rake runner.

What was the cause?

The TeamCity rake runner is designed to capture all output from rake, and redirect it to the appropriate TeamCity log.

The problem has arisen because we are issuing a shell command using FileUtils.sh. Notice it’s calling rake_output_message, which does just this by default:

# Send the message to the default rake output (which is $stderr).
def rake_output_message(message)
    $stderr.puts(message)
end

The TeamCity rake runner has overridden this in an attempt to capture these messages. For some reason this implementation sends an error message to TeamCity.

This means that all shell commands fail when running under the TeamCity rake runner.

Why does rake write to $stderr by default?

Don’t know. And the method name is not very evocative: rake_output_message.

Perhaps it’s intended to be overridden.

EDIT, 07-05-2009: All of these issues have been resolved by upgrading to latest version — even the teamcity-deletes-rake-file wrongness.

References

Written by benbiddington

30 April, 2009 at 11:59