Nowadays Selenium is widely used tool for automating web
application. In real life, products are enterprise application, so application
would be an integration of Silverlight
,Java applets,Raphael java script, canvas objects,Javascripts,Jquery. We could
not expect everything to get identified with selenium and can face challenges
in automating the few of the above mentioned UI with selenium.
Sikuli is one of the easiest tool to learn and can be
used to automate anything using image recognition technique.
Here I am trying to share my experience about automating
an enterprise application with integration of selenium and SIkuli.
I was involved in
automating the BPM product (business process modeling ),I have developed a
framework in selenium to test this product .our functional team has developed
around 1000+ tests using this framework.
Few months back our product development team has
introduced new functionality of browser based modeling in which model can be
developed in browser itself. This functionality was developed using canvas
objects and Raphael javascripts.Automating this functionality was very
important for us. Unfortunately selenium was unable to identify it. I tried all
the options from selenium webdriver but all the efforts were in vain. Then We tried
with different tools like Watir, Watij , AUTOIT and white
to identify (business process modeling )BPM canvas objects but unfortunately
these tools are also failed to get it automated.
While evaluating few other tools for BPM automation, I
come across Sikuli (http://www.sikuli.org/), it’s an intelligent tool for recognizing the Canvas
UI. It’s an open source tool.
It automates the UI using image recognition, and this
image recognition technique was finest fit for us to identify Canvas objects
and other modeler components from browser based modeler written in Raphael Java
scripts.
Sikuli permits us to write the modularize tests in Java.
Using this feature of sikuli, I have integrated
it in our selenium framework written in JAva
Sikulli team has provided neat and precise documentation
about tool from basic to advance level, which guided us wherever we stuck up at
any point while scripting with this tool.
As I mentioned earlier our functional team was expert in writing the test using my existing framework written for selenium java. And I don’t want to expose them to new automation tool for automating one functionality from our huge product.so the main challenge for me was
ü To
integrate the sikuli with existing framework of selenium
ü Developing
the new keywords for sikuli , in a same way that I followed for selenium
keywords,so that for our selenium test writer approach would remain same for
writing the tests.
ü With
a detailed feasibility study of sikuli we observed that sikuli has its
limitation and sikuli nowhere sits near selenium, so I decided to keep Sikuli’s
use restricted to only for the UI which couldn’t get automate with selenium and
remaining stuff with selenium.
How to integrate sikuli with existing framework of selenium
?
Sikuli provides sikuli-script.jar.you can download it
from sikuli site as recommended on the download
page,
Import this jar is your java application, once you have this jar in your java
application.
You can invoke sikuli api’s in your selenium framework
by creating Screen objects instance.
protected static Screen sikuliObject = new Screen();
Following is one of the simple way to call selenium and
sikuli api alternatively as per the need of the test.
in following code ,Line 2 to 6 is of selenium code and line 11 to 13 is of
sikuli code.
public void dragModelElementInCanvas(String modelElement, int x,int y) {
String elementToclick = "//a[contains(.,'"+ modelElement + "')]";
WebElement clickabelModelElement = seleniumDriver.findElement(By.xpath(elementToclick));
if (clickabelModelElement != null) {
Actions action = new Actions(driver);
action.clickAndHold(clickabelModelElement).perform();
sleep();
//If you have been observed that selenium is not able to identify the component (e.g canvas)where you would like to above dragged element.
System.out.println("moving mouse to the location " + x + " : " + y);
sikuliObject.mouseMove(new Pattern(getAbsolutePath()).targetOffset(x, y));
System.out.println("simulating mouse up");
sikuliObject.mouseUp(Button.LEFT);
action.release().perform();
} else
throw new KeywordException(
"Timeout recieved while finding element with Xpath:"
+ clickabelModelElement);
}
Important part here is for testwriter whomsever is using your framework will not have to bother which underline tool you are using to automate the stuff. For e.g in above code selenium testwriter will call the method dragModelElementInCanvas ,where he/she will not realize which tool has been used to achieve the task.
Above is the just the reference code that I have documented here for the one who will be getting started withselenium and sikuli’s integration. In real life we have developed many classes where we are using selenium and sikuli Hand to hand. Post your doubts if you are facing any specific issues while integrating selenium and sikuli.i would like to share my experience as we already overcome many challenges that we faced in integration of two automation tools in our existing framework of selenium.
Happy integration !!
