[Feature Request]: On data-contract tests, get sample failed test results for downstream consumption

[rishabh-oswal-opensc]

We are looking into the option of using the datacontract-cli [python api's] to run tests defined on our data contracts.
We will also migrate from custom data contracts specifications to ODCS and use the datacontract-cli.

One of the main use cases we want to PoC is to fetch the failing test results defined on a contract and consume it downstream to feed into a dashboard for the owners to look into.

Looking at the source code,

• We notice that currently this is not supported. Is this something that the team plans to build or is against this feature for some reason ?
• We also see that the checks are executed using SodaCL under the hood.
  • Soda does provide this feature about collecting failed results here
  • Looking at Soda docs, it seems that it is possible to store failed results for checks executed in python by using CustomSamplers example
• Storing the failed test results can be optional flag provided by the data contract test command which is then carried over to the Soda checks and added.

Would be great to know what the team thinks about this feature request and have any plans on contributing or collaborating on it.

Please let me know if you have any more questions around this. Happy to discuss.

Thanks,
Rishabh

[jochenchrist]

To clarify, do you want to get the results of the checks in a structured format, or do want to fetch the actual failing data rows (samples)?

[rishabh-oswal-opensc]

To clarify, do you want to get the results of the checks in a structured format, or do want to fetch the actual failing data rows (samples)?

The actual failing data rows.

[rishabh-oswal-opensc]

Hello @jochenchrist Do you have a suggestion or an opinion here ?
We can also talk on slack if you prefer. Feel free to DM!
Thanks!

[jochenchrist]

I like the feature idea, but honestly, I don't know if this will be coming in the near future, as this might become pretty complex with the number of data sources we support.

[rishabh-oswal-opensc]

I like the feature idea, but honestly, I don't know if this will be coming in the near future, as this might become pretty complex with the number of data sources we support.

Okay, thanks for your answer @jochenchrist,
We will try to fork / enhance and build it on our side for us.