Fortunately there is a solution (or actually two) available to this problem. JsTestDriver is a test execution framework that allows you to keep your browser open in the background while executing tests from a command line or an IDE. It does this by starting an HTTP server providing browsers with a special page, which binds them to jsTestDriver and prepares them to receive tests to be executed.
The immediate benefit is that your tests can be executed blindingly fast -- so fast, in fact, that by default jsTestDriver's Eclipse plugin runs your tests each time you save a file. Even largish test suites can be run in milliseconds (yes, milliseconds, not seconds). Another benefit is that you can bind multiple browsers at the same time and jsTestDriver can send the tests to each of them concurrently, thus ensuring that your code works in all the browsers.
There are some problems, though. The first of these is that the jsTestDriver IDE plugins are really buggy. Even if the command line software works fine, the IDE integration does not. It isn't much better to switch between your IDE and a command line than to switch between your IDE and a browser to execute your tests. Sure, with jsTestDriver you can execute your tests with multiple browsers at once, which is indeed a big bonus. But still, with a lacking support for IDEs, the jsTestDriver just isn't quite there yet.
Another problem with jsTestDriver is that it forces forces you to write your tests in jsTestDriver's own format. It does not provide any kind of abstraction over existing testing libraries, so if you've already written a huge amount of tests with, for example, Jasmine, then you're out of luck.
A third problem is that since jsTestDriver is its own test runner executable, the integration with continuous integration systems can get a bit tedious. It can be done, sure, but expect quite a bit of work until you get there.
First of all, I decided that I want my test runner to be tightly integrated with JUnit. This adds a dependency to Java but also removes a whole bunch of problems. Due to the JUnit integration I can run my tests anywhere I can run JUnit based tests. This means a huge amount of different tools: continuous integration servers, build tools, IDEs and so on. I don't have to worry about writing IDE specific plugins -- just let the existing JUnit integrations to do the work for me.
Second, I realized that I will not be able to create a testing framework that is in any way better than the existing frameworks. Thus I wanted the architecture to be flexible enough to allow adding support for any testing framework.
Third, I wanted the architecture to allow different server implementations. This would allow me to run my tests just once without pre-starting the server, which would be great for CI environments, or I could start a server once and keep pounding on it with tests, thus providing me with a good test execution speed.
And last, I wanted to do all of this by using the PhantomJS headless browser.
Where are we currently?
The very first implementation of PhantomRunner can be found on Github. It is currently usable and can execute Jasmine based tests. It requires the latest PhantomJs browser to function and it does not support pre-starting the server. But it does provide two different server implementations for different use case scenarios as a proof of concept.
Already in its current form PhantomRunner is ready to be used. Right now it requires you to compile it yourself (it's built with Maven) and you need to write a single JUnit test class to bind it to JUnit. Below is an example of how this is done (it's actually PhantomJS's own test class and can be found on Github):
PhantomRunner supports adding arbitrary libraries to be loaded before the tests are run, which allows you to write tests that rely on those libraries. For example you might want to use RequireJs to handle your module loading. This allows you to test code reliably with all its dependencies loaded:
What should still be done?