Skip to content

Evaluation: STT evals (POC done) #533

@kartpop

Description

@kartpop

Is your feature request related to a problem?
We currently don't have STT (Speech-to-Text) evaluation capabilities in our platform. NGOs need a way to evaluate transcription
quality by comparing model outputs against ground truth.

Describe the solution you'd like
Integrate STT evaluation into the platform. We've already experimented with Gemini and OpenAI for transcription and have promising
results.

Additional Context

Next Steps

  • Productionize the experimental code
  • Enable NGOs to run STT evaluations on their audio datasets

Metadata

Metadata

Assignees

Labels

sub-parentChild of parent label for roadmap view

Type

No type

Projects

Status

To Do

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions