Checked output with `dune build @runtest`

[NoahTheDuke]

I know that the preferred method of running tests is to directly call the test function, typically through a symlink as per the Tutorial, but my current workflow is to use dune build @ocaml-index @runtest --watch --force for reasons not worth getting into here, and this overall works great.

However, I tried to use captured output (through ~checked_output) in a test and the tests are failing, pointing to a _build subdirectory for the capture stdout:

[RUN]   019e692ac0d5 File > parse_environment > prints nicely
[PASS*] 019e692ac0d5 File > parse_environment > prints nicely
• Checked output: stdout
• Missing file containing the expected output: tests/snapshots/example/019e692ac0d5/stdout
• Path to captured stdout: _build/testo/status/example/019e692ac0d5/stdout
• Path to captured log: _build/testo/status/example/019e692ac0d5/log
• Log (stderr) is empty.

I realize that this is probably the reason the tutorial recommends using a symlink, but I really like the --watch/rebuild-and-rerun dev cycle.

Is it possible to use dune build @runtest along with checked output?

[NoahTheDuke]

After a lot more playing around, it seems that dune executes the test from deep in the _build folder, so I need to change ~expectation_workspace_root in my call to interpret_argv:

let () =
  Testo.interpret_argv ~project_name:"example"
    ~expectation_workspace_root:(Fpath.v "../../../tests/snapshots")
    (fun _env -> Testo.get_registered_tests ())

This works for me, but maybe there could be a flag for running with dune that prepends ../../.. to the default expectation workspace?

[mjambon]

dune executes the test from deep in the _build folder

Yes. I don't like it but this is what's happening.

A problem with the path "../../../tests/snapshots" is that it's only valid if you run the test program from the same depth as dune i.e. it won't work if run from the project root which is a pity. An absolute path to the snapshots would work but I don't know how to do this. The test program would need to obtain the project root so that it can find the source location of the snapshots (must be the source rather than a copy for approve to write the correct files). Is there a dune variable that we could use to pass a --project-root argument to the test command in a dune file? Update: maybe --project-root .. would suffice. What matters is that the project root is not hardcoded in the test program.

project root = base folder used to resolve the relative paths for Testo's administrative files: status workspace and snapshots. Defaults to the current folder at the time of launching the test program.

[NoahTheDuke]

I'll do some research to see how other tools handle this. Maybe this is solved already

[mjambon]

I just added a -C/--chdir option. It was sufficient for me to run the tests with Dune as if we were at the source project root. Check out https://github.com/semgrep/testo/blob/main/tests/dune for an example.

[mjambon]

I made a template for getting started with Testo in a Dune project at https://github.com/semgrep/testo/tree/main/template. It's similar to the example above but simplified and explained.