Thoughts on Cucumber for Acceptance testing 20170201-20170202

Just a tumbledown of thoughts

Latest first

20170202

Fitnesse / RobotFrameWork, alternatives?

Criteria:

  • Selenium integration
  • Rest API testing facilitated would be nice
  • Active project (not stale)
  • Java & C# possible
  • Self hosted

Footnote: http://serenity-bdd.info/#/ arrrg uses BDD scripts (Cucumber, JBehave etc + variants)

20170202

Cucumber or something lighter (from a user perspective)?

Consolidating my thinking and with pertinence to a banking application..

  • A large proportion our tests are ultimately data centric:
    • either transactional (input = actions/data, output = validate)
      • financial
      • state change e.g. card stop/start
      • applications
    • state validations (input = state, output = validate)
      • e.g. Cannot transfer to account in state xyz
  • A highly desirable outcome would be for non technical users/test analysts to be able to run their own tests

So assuming we implement a tool/framework of some sort over the top of Selenium for users to create tests:

Cucumber / Gherkin:

  • Too verbose.
    • If a test is only possible by a logged in user, then the fact the user is logged in it doesn’t need to be stated in the feature
    • If we need to test a user can / can’t do things based on logged in, then make that a separate feature
  • Verbosity leads to too many step definitions
    • Users need index of known step definitions, so how documented?
    • Needless level of detail

Which leads me to leaner user centric tools like Fitness (over Fit/Slim) or the Robot Framework.

EOM

20170201

Cucumber?

JeffL, Java dev at MSD was enthusiastic about using Cucumber for ‘testing’

The implication was not for unit testing but for end to end / acceptance testing.

But everything I read (and mostly by the guys who made up cucumber) say DON’T use for other than TDD/BDD requirements gathering / dev process.

I.e. specifically not for end to end etc.

So, that forces me to think about what I want to achieve in no particular order, and why do people gravitate to Cucumber..

I want to:

  • Perform automated testing of an application across 3 platforms:
    • Browsers
    • iOS mobiles
    • Android mobiles
    • Note: Currently browser is the functional superset with mobile apps > 80% functionality implemented

We can safely assume we will use Selenium toolset (+ Java) to drive through to the various platforms.

Given that our functionality is at least 80% common across the 3 platforms, I’ve already had a think about selenium page models that allow our tests to be as platform agnostic as possible.

JAVA per page modelling:

  • Base class e.g. PayAndTransfer
    • abstract properties / methods to set template
    • as many methods as possible are complete
  • Extension classes per platform e.g. PayAndTransferBrowser, PayAndTransferAndroid
    • complete abstracts etc
    • methods overridden only where necessary
  • Factory: PayAndTransferFactory accepting platform enum {Browser, Android, iOS}
    • switch(platform).. return new PayAndTransferAndroid etc

As an aside, I’ve never quite agreed with the view that validations should not be performed inside our page models.  I’d prefer our page models be self validating where practical / possible.  If in my test I say do X, I expect the page model to validate that x actually occurred.  I acknowledge there is a tradeoff with regards to execution time so have yet to completely form my view on this.

So now the big one…how do we construct our tests.

Simplest:

  • TestNG (JUnit) in Java IDE

But ideally, anyone should be able to write our tests, give an adequate provided framework.. so I imagine this is where cucumber gets some traction as a testing tool.

https://www.ibm.com/developerworks/library/a-automating-ria/

But where I struggle with Cucumber is the Feature file interpreters.  I envisage a lot of coding required here if I follow the more simplistic examples provided.

Other solutions:

EOM

 

Updated: February 9, 2017 at 9:37 am