Add skip to flaky MacOS RPC test

[driazati]

Skip for #9824

cc @areusch

[KJlaccHoeUM9l]

Hello @driazati!

Thank you for your comment! Our team also noticed this problem.
It looks like the first time this error occurred when running this action in PR#9483.

Failure occurs in a test that seeks to verify that the auto scheduler can be used with an RPC Session. The error occurs due to checking the tuning log after a tuning session. The test says that at this stage, an error code was found inside the log, which says the following:

Errors happen when compiling code on device
(e.g. OpenCL JIT on the device)

It doesn't look like the problem is in the whole test or in the iOS RPC application.

At the moment, it has not been possible to reproduce the problem locally to find out more details about this failure. If you have any additional information about this or can reproduce this problem yourself, then we would be very grateful for any additional observations.

[driazati]

I wasn't able to reproduce it locally either (and it doesn't seem to fail in CI super often either, maybe 1 out of every 20 recent runs on main) and I don't really know the autotuning/rpc code well enough to guess where the problem might be, but it still comes up from time to time so I think it should be disabled until a proper repro/fix is found. It makes more sense to mark this as a flaky failure with xfail rather than skip though so it will still run, but not report as an unexpected error if it fails.

[KJlaccHoeUM9l]

Hello @driazati!
We have only been able to reproduce this issue in the azure cloud.
To do this, we slightly modified main.yaml as follows:

for ((i=1; i < 100; i++)); do python -m pytest -vrP tests/python/contrib/test_rpc_server_device.py; done

It looks like this line has fallen off:

func = remote.load_module (os.path.split (build_res.filename) [1])

The problem requires further investigation.

[areusch]

thanks @driazati can you create a GH issue for this test or mention it in this PR?

[areusch]

sorry one q: has anyone repro'd this on linux? if it hasn't failed there, i'd lean towards disabling it only on windows/os x for now. the GH actions can be retriggered and don't necessarily block commit.

@KJlaccHoeUM9l could you let us know which os you were using in azure?

[areusch]

oh sorry--ignore. i misread the test decorators.