How to Automate Testing of a Java Swing Application

Automate and scale manual testing with AI ->

Above: the test below running against Apache JMeter, sped up with idle time removed.

You can do Java Swing testing end-to-end without component names by describing each step in plain language. TestDriver matches each description against screenshots of the running app, so:

• you test the release your users download, not a build wired for tests
• you don’t call setName() on components or keep names in sync
• your test reads like the steps you’d give a teammate

TestDriver generates and runs black-box end-to-end tests. It starts your app in a fresh virtual machine, looks at the screen and uses a real mouse and keyboard, so it never needs selectors or access to your code. Each test is a normal Vitest file, and every run is recorded as a replay.

The example downloads Apache JMeter, a large open-source Swing app, adds a Thread Group from a right-click menu and checks its thread count is kept. The full project is on GitHub: testdriverai/example-java-swing-app.

Why not AssertJ Swing

AssertJ Swing, the successor to FEST Swing, drives Swing components with real input events. Its getting-started guide looks components up by a unique name you set in your code:

// AssertJ Swing name lookup (illustrative, not run)
window.textBox("numThreads").deleteText().enterText("5");

TestDriver describes the same field the way a user sees it:

await testdriver.find("the Number of Threads (users) text field showing 1 in the Thread Properties section").click();

The name lookup needs a name in the source for every component you touch. The description only changes when the screen does.

Watch the full replay

The replay shows every step with its screenshot and log.

The test

import { describe, expect, it } from "vitest";
import { TestDriver } from "testdriverai/vitest/hooks";

describe("Apache JMeter (Java Swing)", () => {
  it("adds a Thread Group and sets it to 5 threads", async (context) => {
    const testdriver = TestDriver(context);

    // Install a Java runtime and unpack the JMeter release.
    await testdriver.exec(
      "sh",
      "sudo apt-get update -qq && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -qq default-jre >/dev/null",
      600000,
    );
    await testdriver.exec(
      "sh",
      "curl -fsSL -o /tmp/jmeter.tgz https://dlcdn.apache.org/jmeter/binaries/apache-jmeter-5.6.3.tgz && tar -xzf /tmp/jmeter.tgz -C /tmp",
      300000,
    );

    // Start the replay recording, then launch the app.
    await testdriver.provision.dashcam({ title: "Apache JMeter (Java Swing) Thread Group", webLogs: false });
    await testdriver.exec("sh", "cd /tmp && nohup /tmp/apache-jmeter-5.6.3/bin/jmeter >/tmp/jmeter.log 2>&1 &", 10000);

    // Wait for the main window, then right-click Test Plan.
    const testPlan = await testdriver.find(
      "the Test Plan item at the top of the tree on the left side of the Apache JMeter window",
      { timeout: 90000 },
    );
    await testPlan.rightClick();

    // Add > Threads (Users) > Thread Group.
    await testdriver.find("the Add item at the top of the right-click menu").click();
    await testdriver.find("the Threads (Users) item in the Add submenu").click();
    await testdriver.find("the Thread Group item in the Threads (Users) submenu").click();

    // Change Number of Threads from 1 to 5.
    await testdriver.find("the Number of Threads (users) text field showing 1 in the Thread Properties section").click();
    await testdriver.pressKeys(["ctrl", "a"]);
    await testdriver.type("5");

    // Click away and back to make sure the value is kept.
    await testdriver.find("the Test Plan item at the top of the tree on the left side of the Apache JMeter window").click();
    await testdriver.find("the Thread Group item under Test Plan in the tree on the left").click();

    // Wait for the panel, then check the saved value.
    await testdriver.find("the Number of Threads (users) text field showing 5 in the Thread Properties section", {
      timeout: 15000,
    });
    const saved = await testdriver.assert(
      "the JMeter window shows the Thread Group settings panel with the Number of Threads (users) field set to 5",
    );
    expect(saved).toBeTruthy();

    // Hold the final screen so the end of the replay shows the result.
    await testdriver.wait(3000);
  });
});

How it works

Install and launch the app

The first two exec() calls install a Java runtime and unpack the JMeter release on a fresh Linux sandbox. provision.dashcam() starts the replay recording, and the third exec() launches JMeter in the background.

Describe elements by what users see

Each find() takes a description that names the label, the look and the position, such as “the Thread Group item in the Threads (Users) submenu”. The test never inspects the Swing component tree, so cascading menus and right-click menus work like any other step. See locating elements.

Wait for results, then assert

find() with a timeout keeps looking until the element appears, so the test waits for JMeter’s slow first start without a fixed sleep. assert() then checks a plain-language statement against the screen and returns true or false. See making assertions.

Run it yourself

Clone the example repo, or set up TestDriver in your own project and run the test:

npx testdriverai init
npx vitest run java-swing-app.test.mjs

To test your own Swing app:

• Replace the download line with your own JAR or release archive.
• Start your app with java -jar in the launch exec() call.
• On Windows, pass os: "windows" to TestDriver() and install your .msi or .exe with provision.installer().

FAQ

Do I need to name my Swing components? No. Steps are matched on screenshots, so there are no component names or test IDs to add or keep in sync.

Does it work with custom look and feels or JavaFX? TestDriver only sees the rendered window, so it doesn’t depend on the look and feel or the UI toolkit. This demo uses JMeter’s default look and feel.

Can I run it in CI? Yes. It’s a normal Vitest file and the app runs in TestDriver’s cloud sandbox. See CI/CD.