Automated Web Testing with Selenium and The NetBeans Rich Client Platform

I developed tooling for work I do for Scripps which helps me create Selenium based automated user tests more easily. These are not open source at this time, though that may be possible some day. I might use the ones I have created as an example to make some better for OSS since I have learned lessons along the way.

This tooling has been built on the NetBeans Rich Client Platform, but too, will run right in the IDE. I show IDE examples in this post. It helps create Java projects for using Selenium for automated user testing. This ties in nicely with the IDE.

There are different technologies supported in the tooling. There is a JavaScript runner or window which allows one to run JavaScript inside Selenium using runScript and getEval. Too, there is project support for Java and Groovy. I don't show the Groovy support here, but it is exactly the same as the Java examples less you will be using a Groovy file instead of a Java file, so you can derive how it works from that.

This is not the same as the tooling available in the current NetBeans auto update centers. I noticed it when I started this, and I don't see that it addresses things in the same manner which I do. I don't focus on the tests in this case as much as I focus on being able to test chunks of code as you are trying to develop the logic which will exercise your application.

The reason I do this is because I do not just write tests. I create automation APIs which can be used to affect the system. This then allows tests to be created using those APIs. That limits the impact of application changes to the high level test scripts which use the APIs I create. Thus, if a change occurs in the application which can break the tests, then I change one file versus many.

The above may be a topic of another post one day. I will now get on with showing the tooling.

First, and not really shown here, I needed a way to proxy calls to a Selenium session. I created a library which implements certain aspects of Selenium to allow this to happen. This simply gets and sets the Selenium session ID so calls for a Selenium session may come from multiple places. In this case, the user serializes those things. You will see it references in code which follows.

Next, I needed a way to manage Selenium server connections, and too I needed a simple way to kick off a Selenium server in the development environment. The "Selenium Manager" top component handles this for me. Certain aspects are configurable.





You can also start and stop the in IDE Selenium server here. The output uses the NetBeans output window for this.



Once you have configured connections, you can start and use them one at a time


Once a live session is available, the manager allows you to stop it or to copy the session ID using context menus


Next, and most importantly, there is the ability to do productive things.

There is a Selenium JavaScript runner. This is a special window where one can type in JavaScript, execute it, and see the response. Too, one can ask for the current HTML from the page based on the DOM and not the plain HTML sources before any JavaScript logic has applied changes.


Notice the syntax highlighting options in the output below. I suppose I could add the ability to view that as JSON too. This is an example of using the "Grab HTML" button



Notice the part above dealing with ret=ret in the JavaScript image. This is due to the way this logic will be run directly in Selenium and the way it returns the last evaluation instead of having the caller use the return statement; this is a Selenium thing (see Selenium.getEval). That could have been worked around in the JavaScript runner if I had put the logic in a function and called it, but for the purposes of using this window that is a little overhead I don't need.

Above, you may have noticed the use of jQuery. How did it get there? Right, there is a difference in the JavaScript used in your application, and that used in Selenium. You can see my other blog post for more information, but I injected jQuery into the current Selenium session using Selenium.addScript from some Java code.

I will show you how that gets there in a bit.

It should be obvious we are using the started session here, but just to be clear, before I can run this JavaScript, I had to start a Selenium session and drive the state of the browser manually to get to what I wanted to operate against. In this case not really as the session opened on the Google home page.

Above, I mentioned injecting jQuery. Well, if you can inject jQuery, then you can inject your own JavaScript as well. You can take the logic you figure out in the JavaScript runner and move that into your own JavaScript files which you will inject.



Next, you can create a regular Java project in the NetBeans IDE. Then, you can create a standard Java class. You can then click on a line in the Java file and bring up a context menu and ask the tooling to inject the required Selenium proxy logic to allow you to write code using the current session.


This can be in a main method as above in which case it will be a local variable, or it can be at the class level. Repeated uses of the above action will just overwrite the previously injected code unless it is deleted; in that case it will be added back. This is needed if you stop the Selenium session and restart it. In this case you get a new session ID. Once you use the action, your class will look like this


Then you fill in some scratch pad type code to exercise some logic you will later add to an actual API. This may seem counter intuitive, but the point is to be able to drive the state of the application with the browser while you write some finite pieces of the whole which you know work. This greatly reduces the time to get the state in place and start the test etc. The develop test cycle becomes easier to manage.



So, how did our JavaScript and jQuery come to reside in the Selenium session? It was put there with another scratch pad Java class.


The related libraries were added to my Java project automatically for me by the tooling too. This includes the proxy library which I first wrote about.


I'm obviously trying to use a very simple example here. To get real work from these tools I have a full blown AUT API I have designed and developed for the project I'm testing. Too, I have developed the infrastructure needed to use the API including a set of tests and Ant build scripts.

The individual tests I can run directly from the IDE as they are JUnit tests, and their core logic can be developed, along with the APIs, running against a live Selenium session. The Selenium JavaScript injection is done automatically by my base JUnit test along with other preliminary things.

Hopefully at some point in the near future I will create an OSS version of this tooling. I don't have an ETA at this moment. But, I hope as I feel it could be useful for many.

Using Selenium and jQuery for Automated User Testing

I am assuming you are familiar with both jQuery and Selenium. Too, I'll assume you want to use jQuery inside locators or in some way through Selenium to make locators easier using more utilities. So, I'll show you how to do that as well as use your own custom JS files and logic through Selenium just the same by using Selenium.getEval.

First, the below is made possible using the Selenium command or API call addScript. Next, we need a way to take our file or URL based resource and convert it easily into a string which is what addScript is expecting. I use this utility I have:

 public static String inputStream2UTF8(InputStream in) throws IOException {
    String ret = null;
    BufferedInputStream bin = new BufferedInputStream(in);
    InputStreamReader isr = new InputStreamReader(bin, "UTF-8");
    StringBuilder sb = new StringBuilder();
    int iread = -1;
    while ((iread = isr.read()) != -1) {
         sb.append((char) iread);
    }
    ret = sb.toString();
    return ret;
}

Next, I like having a more utilitarian method of injecting these resources into the current Selenium session. I use the following two methods together:

  /**
  * Merges all these different resources into a single input stream, puts them
  * into a single JS memory file, and injects this into Selenium using the given
  * js element tag ID (jsTagID).
  * @param resourceLocator class used to locate resources using cpResourcePaths
  * @param cpResourcePaths resource paths relative to resourceLocator or fully qualified
  * @param filePaths file system paths, these can be relative if used from a
  *                  running directory, but generally should be full
  * @param jsTagID the ID of the script element to inject into Selenium
  * @param se the Selenium instance to inject into
  */
 public static void injectJavaScriptResourcesTogether(Class resourceLocator,
         String[] cpResourcePaths,
         String[] filePaths,
         String jsTagID,
         Selenium se) {
     ArrayList ins = new ArrayList();
     try {

         for(String cpResourcePath : cpResourcePaths){
             InputStream in = resourceLocator.getResourceAsStream(cpResourcePath);
             if (in != null) {
                 ins.add(in);
             }
         }

         for(String filePath : filePaths){
             File f = new File(filePath).getAbsoluteFile().getCanonicalFile();
             InputStream in = new FileInputStream(f);
             ins.add(in);
         }

         SequenceInputStream sin = new SequenceInputStream(Collections.enumeration(ins));

         String js = inputStream2UTF8(sin);

         //don't swallow here...let the caller do that if
         //they need to. API should not eat exceptions generally
         se.addScript(js, jsTagID);

     } catch (Throwable e) {
         if (RuntimeException.class.isInstance(e)) {
             throw RuntimeException.class.cast(e);
         } else {
             throw new RuntimeException(e);
         }
     } finally {
         for (Closeable closer : ins) {
             try {
                 closer.close();
             } catch (Throwable e) {
             }
         }
     }
 }

 public static void injectJavaScriptResource(Class resourceLocator,
         String cpResourcePath,
         String jsTagID,
         Selenium se) {
     ArrayList closeables = new ArrayList();
     try {
         InputStream in = resourceLocator.getResourceAsStream(cpResourcePath);
         if (in != null) {
             closeables.add(in);
         }
         String js = inputStream2UTF8(in);

         //don't swallow here...let the caller do that if
         //they need to. API should not eat exceptions generally
         se.addScript(js, jsTagID);

     } catch (Throwable e) {
         if (RuntimeException.class.isInstance(e)) {
             throw RuntimeException.class.cast(e);
         } else {
             throw new RuntimeException(e);
         }
     } finally {
         for (Closeable closer : closeables) {
             try {
                 closer.close();
             } catch (Throwable e) {
             }
         }
     }
 }

You can see in the logic above that I attempt to locate the resources given to the methods in different ways. That is pretty straight forward, so I will let the code document itself.

Next, I add JS files into my Java project in NetBeans in a Java package. I will access those things as classpath resources per the code above. I use the following code below from some of my other source code:

     public void injectSupportingJavaScript() {
     ArrayList closeables = new ArrayList();
     try {
         String[] resources = new String[]{
             "resources/jquery-1.4.4.min.js",
             "resources/utils.js",
             "resources/nav-utils.js",
             "resources/image-dialog-utils.js",
             "resources/asset-dialog-utils.js",
             "resources/module-utils.js"
         };
     
         try {
             SEUtilities.injectJavaScriptResourcesTogether(getClass(),
                     resources,
                     new String[0],
                     "ff-aut-javascript",
                     se);
         } catch (Throwable e) {
             log.log(Level.WARNING, "Unable to inject required Java Script as a single stream. Will attempt to inject the required .js files individually and continue to run.", e);
             try {
                 for (int i = 0; i < resources.length; i++) {
                     SEUtilities.injectJavaScriptResource(getClass(), resources[i], "ff-aut-javascript-" + i, se);
                 }
             } catch (Throwable e2) {
                 StringBuilder emsg = new StringBuilder();
                 emsg.append("Unable to inject required Java Script as individual streams. ");
                 emsg.append("Will not be able to continue as the locators can not be used in testing. ");
                 emsg.append("Selenium may need to be hacked a bit, or ");
                 emsg.append("the Firefly AUT API .js files need to be further broken up. ");
                 emsg.append("This is because Selenium HUB has issues with too large of requests. ");
                 emsg.append("The API already tries to compensate for this, and this measure has failed ");
                 emsg.append("which usually indicates a .js file injected into Selenium at test time has ");
                 emsg.append("grown too large in size.");
                 log.log(Level.SEVERE, emsg.toString(), e);
             }
         }

     } catch (Throwable e) {
         if (RuntimeException.class.isInstance(e)) {
             throw RuntimeException.class.cast(e);
         } else {
             throw new RuntimeException(e);
         }
     } finally {
         for (Closeable closer : closeables) {
             try {
                 closer.close();
             } catch (Throwable e) {
             }
         }
     }
 }

Notice the part about a single resource versus individual in the logic. I do this because there is an issue with Selenium Grid where it seems to use HTTP GET instead of POST in some cases where it should be using POST; at least this is my assumption per the error messages I received. This provides a decent fall back.

In the above, your custom logic will obviously need to inject your own .js files and those will need to be relative to your own class. Once you do that, you can then create a Selenium locater using either pure jQuery inline or you can use your own JS functions to limit the JS logic you have to place in Java files.

Too, using Selenium.getEval(String) you can execute JavaScript directly in the Selenium session. Sometimes you need to do this if your logic depends on some jQuery event listeners and the standard Selenium calls will not activate that logic correctly.

The one caveat to using these things, due to JavaScript targeting, is always pass in window.document to jQuery or your own custom JavaScript functions. The reason is that Selenium will be running in a separate browser window from your application. window.document will point to the window where your applications DOM resides. If you have any question about that specifically let me know.

Some locater example might be:
"dom=your_js_function_doSomething(window.document);"

For getEval it would simply be:
"your_js_function_doSomething(window.document);"

Remeber, the last evaluation or the return statement will be what Selenium returns to the calling logic. Too, you can use the throw statement in your JS to propagate better messages to the calling Java logic.

Enjoy.