Get help with testing, discuss unit testing strategies etc.


Post by zombeerose »

I have several tests that I can't get to fail within the UI but I can't get them to pass when running headless in our CI pipeline. They are calling the waitForRowsVisible method with a string query.

Pseudo test:

StartTest(function(t) {
    
t.chain( 'render("my_grid")', 'waitForRowsVisible("my_grid")', ); });

Sample error:
fail 6 - Waited too long for: rows to show for panel with id "my_grid-1011"
Failed assertion waitForRowsVisible at line 9 of /...test.js
Condition was not fullfilled during 270000ms.

function() {
    if (!cmp.rendered) return;
    var result = this.query(selector, view.el.dom);
    if (result.length > 0) {
        return result;
    }
}

My harness:

    var harness = new Siesta.Project.Browser.ExtJS();

harness.configure({
    autoCheckGlobals: true,
    defaultTimeout: 30000,
    expectedGlobals: [
        'Ext'
    ],
    needDone: true,
    // pageUrl: // supplied with the test descriptors in startFromUrl
    pauseBetweenTests: 1000,
    showTestDurationColumn: true,
    speedRun: true,
    transparentEx: false,
    viewDOM: false, 
    waitForTimeout: 30000
});

I'm using Siesta v5.6.1.

Any suggestions? I'm stuck.

Thanks!


Post by mats »

Never used that syntax actually, what if you change to:

StartTest(async t => {
    render("my_grid"):
    await t.waitForRowsVisible("my_grid");
});

Post by zombeerose »

Hi Mats,
Still failed with async. Different error:

"fail 6 - The promise returned from the sub-test [test.js] did not resolve within 180000ms"


Post by mats »

Sounds weird, can you make a video recording of the test execution? Maybe that yields some clues...


Post by zombeerose »

Do you want a recording of the test in the UI, because that works? It always succeeds in the UI and fails when headless.


Post by mats »

Try recording it when running in headless mode?


Post by zombeerose »

I have identified the root of the problem. Indeed, rows are not appearing as the failure reports. It's because event simulation is not occurring as I expect during automation. When executing something like this

'type(">> my_searchfield","example[ENTER]")',

, the ENTER keystroke is not occurring and rows don't get loaded. Reading over the docs, all I find is the "simulation" config but that states that the only option during headless tests is synthetic.

Any suggestions/help so that type actions will be executed accurately in a headless mode?

On a side note, the "screenshot" method proved to be very helpful. It made me wonder if you could add a feature that can generate a screenshot whenever an exception occurs or an assertion fails? The harness would define a folder for saving the screenshots and the images would be based on the test name. For troubleshooting automation, this would be very helpful IMO.

Thank you.


Post by zombeerose »

I've also installed xvfb and experimented with that option but it appears to fall back to synthetic events:

siesta/bin/webdriver https://localhost:5000  --xvfb --simulation=native --browser=chrome  --filter="test"
Launching test suite, OS: Linux, agent: Chrome 107.0.0.0
[WARN] Could not calibrate native events simulation server on 127.0.0.1:43456
Reason:  timeout
Are we running in headless mode ?
Falling back to synthetic events

Post by nickolay »

So this code: type(">> my_searchfield","example[ENTER]") - the ENTER simulation only does not work in headless mode? But works in "headfull"?


Post by zombeerose »

Yes - the ENTER is not being handled. I'm trying to narrow down a test case.


Post Reply