Clicking a disabled button silently does nothing

[mvysny]

Calling TestBenchElement.click() on a disabled button silently does nothing and simply continue the execution of a test. I'd expect the function to fail, giving a proper explanation that the element is disabled and can not be clicked.

The current behavior causes the test to continue, perhaps failing later on in a strange state. This violates the "fail eagerly" principle.

Steps to reproduce:

@Route
public class MainView extends VerticalLayout {
    public MainView() {
        final Button button = new Button("Im disabled", e -> Notification.show("Hello World!"));
        button.setEnabled(false);
        add(button);
    }
}
public class MainViewIT extends AbstractViewTest {
    @Test
    public void bug() throws Exception {
        $(ButtonElement.class).first().click();
    }
}

TestBench 23.5.15, Vaadin 23.5.15

[mshabarov]

It's questionable that the test should fail in this case.
End-to-end tests simulate a user interaction and nothing also happens in browser when a user click on disabled button.
If you need to test the UI to be in a expected state (disabled), you'll better use another approach.

[mvysny]

A test does what a user would do, true, but it also asserts that an UI is in certain state to trigger an action. click() throwing an exception if the button is disabled performs an automatic assert and will make the test more robust, which is a good thing. The testing framework is primarily a tool for the tester to test the app; the more helpful the testing framework is to the person writing the tests, the better.

In other words, there are two testing scenarios when I am calling click(). I, as a tester,

1. want to test that a disabled button does nothing.
2. want to click a button; if the app is buggy and the button is disabled, that is unexpected and I want to be informed as soon as possible.

The case 1 tests Vaadin the framework itself rather than the app. As Vaadin user, I rely on the fact that disabled button does nothing. I, as a tester, am not interested in testing Vaadin, but rather testing my app.