Monday, July 13, 2015

Tricks to make Protractor on non-AngularJS applications faster and more stable



Proctractor is designed for AngularJS applications, it will wait for web pages to settle down before acting on web elements (so I heard, I’ve never tried Proctrator on AngularJS applications). Protractor can be used to test non-AngularJS applications, all you have to do is to put browser.ignoreSynchronization = true; however, you have to do more to make automation more stable. 

This blog shows some tricks I used to make my automation framework more stable. 

Instability


Any UI action, be it to set text on an input, or click on a button, has to wait for the element to be settled down. If for example, when the page is being refreshed, the test case tries to click a button, you might run into issues such as:
  •  No such element
  • StaleElementReferenceError: stale element reference: element is not attached to the page document
  •  Element is not clickable at point(…)


It is easy to test this out.

Write a simple html containing a button, it’s clicking action will cause the page to be refreshed:
<html decorator="blank" title="Hello Apple">
  <head> 
         <script>
                  function refresh(){                          
                            setTimeout(doRefresh, 300);         
                  }
                 
                  function doRefresh(){               
                            document.location.reload();
                  }
         </script>
  </head>
  <body>         
  <button type="button" id="refresh" onclick="refresh()">Refresh</button> 
</html>
And write a simple automation test to keep clicking the button:
beforeEach(function () {
        browser.ignoreSynchronization = true;
    });


    it('test_clickWhenPageBeingRefreshed', function () {
        browser.get('http://localhost:8080/examples/test.html')
            .then(function() {
                co(function*(){
                    for(var i=0; i<500; i++) {
                        yield element(by.id("refresh")).click().then(function () {
                            console.log("clicked");
                            return true;
                        }, function (err) {
                            console.log("Oops!" + err);
                            return false;
                        })
                    }
                })

            });
    })
})
You should see such errors logged (not frequently though):
Oops!StaleElementReferenceError: stale element reference: element is not attached to the page document
  (Session info: chrome=43.0.2357.124)
  (Driver info: chromedriver=2.14.313457 (3d645c400edf2e2c500566c9aa096063e707c9cf),platform=Windows NT 6.1 SP1 x86_64) (WARNING: The server did not provide any stacktrace information)

Wait until an element is displayed/enabled

browser.wait() can be used to wait for a condition to become true: the condition will be checked frequently, once the condition is met or the condition fails to be met after the specified time, browser.wait() returns. 

Since Protractor has a method element.isDisplayed to check whether an element is visible, my first attempt was to wait for this method to return true in browser.wait(), however, this method might throw out errors when the element is not stable:

it('test_clickWhenPageBeingRefreshed', function () {
    browser.get('http://localhost:8080/examples/test.html')
        .then(function() { 
            co(function*(){
                for(var i=0; i<1000; i++) {
                    var ele= element(by.id("refresh"));
                    var visible=yield browser.driver.wait(function () {
                        return ele.isPresent().then(function(present){
                            return present && ele.isDisplayed().then(
                           function(isDisplayed)
                                     {return isDisplayed;},
                           function(error){
                             console.log("Oops! isDisplayed met an error:"+error);
                             return false;});

                        });

                    }, 100, 
                  "wait for element to be visible")
                  .then(function(visible0){
                        console.log("visibility is:"+visible0);
                        return visible0;
                    }, function(error){
                        console.log("Oops! waiting met an error:"+error);
                    });


                    if(visible) {
                        yield ele.click().then(function () {
                        }, function (error) {
                            console.log("Oops!Clicking met an error:" + error);
                        });

                    }

                }

            })

        });
})


Running this program for a while, you can see element.isDisplayed() throws out these errors:

Oops! waiting met an error:StaleElementReferenceError: stale element reference: element is not attached to the page document
Oops! waiting met an error:NoSuchElementError: No element found using locator: By.id("refresh")
  

The same thing happens to element.isEnabled().

So these two methods element.isDisplayed() and element.isEnabled() can’t be used directly to verify an UI element’s status. The solution is to capture errors from element.isDisplayed() and element.isEnabled() and swallow them.
Util.prototype.isElementDisplayedOrEnabled_ = function (element, identity, testEnable) {
    var uponError = true;
    var ret = false;
    var self=this;
    return co(function* () {
        var i = 0;
        while (uponError && i < self.TRY_TIMES) {
            var p;
            if (testEnable) {
                p = element.isEnabled();
            } else {
                p = element.isDisplayed();
            }

            yield p.then(function (testResult) {
                uponError = false;
                ret = testResult;
                return ret;

            }, function (error) {
                uponError = true;
                console.log("Oops!isElementDisplayedOrEnabled_ when waiting for " + identity + "to be visible/enable, met an error:" + error);
                ret=false;
                i++;
            });

        }
        return ret;

    });

}


isElementDisplayedOrEnabled_  will catch errors and try TRY_TIMES times. 

This method is used inside browser.wait() to wait for an element to become visible or enabled or fail after TRY_TIMES times:


Util.prototype.waitElementUntilVisibleOrEnable_=function(element,  identity, options) {
    if(_.isUndefined(identity)){
        identity="";
    }

    if(_.isUndefined(options)){
        options={};
    }
    options.mustBeTrue=this.isDefined(options.mustBeTrue)?options.mustBeTrue:true;
    options.expecation=this.isDefined(options.expecation)?options.expecation:true;
    options.testEnable=this.isDefined(options.testEnable)?options.testEnable:false;
    options.waitTime=this.isDefined(options.waitTime)?options.waitTime:this.TIMEOUT;

    var self=this;

    var waitFun=function(){
        if(options.expecation){
            return element.isPresent().then(function(present){
                return present && self.isElementDisplayedOrEnabled_(element, identity, options);
            });
        }else{
            return element.isPresent().then(function(present){
                return !present || !self.isElementDisplayedOrEnabled_(element, identity, options);
            });
        }
    }
    return browser.driver.wait(function () {
        return waitFun();
    }, options.waitTime, "wait for element to be visible:"+identity).then(function(){
        console.log("waitElementUntilVisibleOrEnable_("+JSON.stringify(options)+") on "+identity+":"+options.expecation);
        return options.expecation;
    }, function(error){
        console.log("Oops!waitElementUntilVisibleOrEnable_("+JSON.stringify(options)+") on "+identity+" met an error:"+error);
        if(options.mustBeTrue){
            throw new Error("Oops!waitElementUntilVisibleOrEnable_("+JSON.stringify(options)+") on "+identity+" met an error:"+error);
        }else{
            return !options.expecation;
        }
    });
}



options.expecation is to specify whether an UI element is expected to be visible/enable or the reverse. options.mustBeTrue is to define whether the expectation must be met, if true, when the expectation fails, waitElementUntilVisibleOrEnable_ throws out an error (error handling is another complex topic, and I will write another blog to discuss it). 

waitElementUntilVisibleOrEnable_ is used before any actions on UI elements, for example:

Util.prototype.findByID=function(id) {
    var xpath='//*[@id="'+id+'"]';
    return this.findByXPath(xpath);
}

Util.prototype.findByXPath = function (xpath,last) {
    if(last){
        xpath="("+xpath+")[last()]";
    }

    var ele = element(by.xpath(xpath));
    this.waitElementUntilVisibleOrEnable_(ele, xpath);

    return ele;
}
Util.prototype.findByName=function(name) {
    var ele = element(by.name(name));
    this.waitElementUntilVisibleOrEnable_(ele, name);
    return ele;
}




Check post conditions




My system under test is really clunky: sometimes after clicking an UI element, for various reasons, the actions supposed to happen (e.g. forwarding to another page) didn’t happen, and the UI element has to be clicked again. Before the system under test solves this issue, I have to use some trick to handle it. 

The solution is to define a post condition, after clicking on UI element, check if the post condition is met, if not, continue to click on the UI element. 


Util.prototype.clickElement=function(ele,identity, postCondition){
    var self=this;
    var clickedTimes=0;

    return co(function*(){
        var clickSuccessful=yield self.doClick_(ele, identity, postCondition).then(function(ret){return ret;}, function(err){throw err;});

        while(!clickSuccessful && clickedTimes < self.TRY_TIMES ){
            console.log("clickElement:click "+identity+" again, try at "+clickedTimes+" times");
            browser.sleep(self.TINY_TIMEOUT);
            clickedTimes++;
            clickSuccessful=yield self.doClick_(ele, identity, postCondition).then(function(ret){return ret;}, function(err){throw err;});
        }

        if(!clickSuccessful){
            console.log("clickElement:click failed at "+identity);
            throw new Error ("clickElement:click failed at "+identity);
        }

        return true;
    });
}

Util.prototype.doClick_=function(ele,identity,postCondition){
    var self=this;
    return co(function*(){

        var clickable= yield self.waitElementUntilClickable_(ele, identity);
        if(!clickable){
            return false;
        }

        var uponStaleError=true;
        var success=false;
        var i=0;
        while(uponStaleError && i<Util.prototype.TRY_TIMES) {
            yield ele.click().then(function(){
                console.log("doClick_:clicked "+identity);
                uponStaleError=false;
                success=true;
            }, function(error){
                uponStaleError = true;
                i++;
                console.log("Oops!doClick_:click"+identity+" again at "+i+" time, error is:" + error);
                browser.sleep(Util.prototype.TINY_TIMEOUT);
            });
        }

        if(!success ){
            return false;
        }

        if(self.isDefined(postCondition)){

            var visible;
            if(self.isDefined(postCondition.id)) {
                visible=yield self.isElementByIdVisible(postCondition.id,  postCondition.visible).then(function(ret){return ret;}, function(){});
            }else if(self.isDefined(postCondition.xpath)){
                visible=yield self.isElementByXPathVisible(postCondition.xpath, postCondition.visible).then(function(ret){return ret;}, function(){});
            }else if(self.isDefined(postCondition.name)){
                visible=yield self.isElementByNameVisible(postCondition.name, postCondition.visible).then(function(ret){return ret;}, function(){});
            }

            if(visible===postCondition.visible) {
                console.log("doClick_:click successful, " + JSON.stringify(postCondition) + " matches!");
                return true;
            }else{
                console.log("doClick_:click "+identity+" failed because postCondition doesn't match:"+JSON.stringify(postCondition));
                return false;
            }
        }else{
            return success;
        }
    })
}

As an example, this following statement clicks the button “Close Task”,  if the clicking is successful, the page should be forwarded to another one and the “Close Task” button should disappear. It checks if “Close Task” is still visible, if so, it will continue to click the button, until the button disappears or it throws out an error after a number of tries:


util.clickToolbarButton("Close Task",{visible:false, xpath:"(//button[text()='Close Task'])[last()]"})

Wait until cursor is not busy




Unlike human action, with automation, when the cursor is busy (the cursor icon is a spinning circle), Protractor can still drive UI actions. The following test proves this:



//html
<html decorator="blank" title="Hello Apple">
  <head> 
         <script>         
                 
                  function wait(){                    
                             document.body.style.cursor  = 'wait';
                  }                
                 
         </script>
  </head>
  <body>         
  <button type="button" id="refresh" name="visible"  onclick="refresh()">Refresh</button>
  <button type="button" id="wait" name="visible"  onclick="wait()">Wait</button>        
  <input type="text" id="text1"  />
  <button type="button" id="b1" name="visible">B1</button>
</html>

//test case
it('test_whenMouseBusy', function () {
    browser.get('http://localhost:8080/examples/test.html')
        .then(function() {
            co(function*(){
                yield util.clickById("wait");
                yield browser.sleep(5000);
                yield util.setTextById("text1", "abc");
                yield util.clickById("b1");
                browser.pause();
            })

        });
})
  
When the cursor was spinning, setting text on an input and clicking on another button still took place.
It makes automation faster, but it caused some problem for my test cases. On many occasions, I had to wait until the cursor is not busy before doing the next UI action. 

For my system under test, the solution turned to be, not waiting for the cursor’s class to change, but for the body’ class to change:


Util.prototype.waitUntilPageSettled=function() {
    this.switchToMain();

    var self=this;
    return browser.driver.wait(function () {
            return element(by.xpath("//body")).getAttribute("class").then(function(v){
                return v.indexOf("x-masked")<0;
            })
        },
        this.TIMEOUT, "waitUntilPageSettled:wait for page to be settled")
        .then(function(){
            console.log("waitUntilPageSettled:page settled ");
            return self.switchToDetailIFrame();
        }, function(error) {             
            console.log("Oops!waitUntilPageSettled met an error:"+error);
            return self.switchToDetailIFrame();      
         });
}

These tricks help make my automation test cases more stable. And they also help in verifying performance dowgrade. Waiting happens inside waitElementUntilVisibleOrEnable_ and waitUntilPageSettled, by controlling the waiting time, I can check whether a version of the system under test is slower than the previous one.






Posted by at
Labels:

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)