want to attach pdf file to a model which has ability to read any file using llm instead of pydantic

[mhhnr]

For example, for the Claude 3.7, if you open the chatbot generally on an Anthropic website, there is an option to attach a file and it goes through it, giving an output. Is there a way where I can attach the file to the Chat interface in the playground and send it so that it can reply back with a structured output? I would like to have an example please

[monali7-d]

Hi @mhhnr , thanks for reaching out and supporting Agno. I’ve shared this with the team, we’re working through all requests one by one and will get back to you soon.
If it’s urgent, please let us know. We appreciate your patience!

[Ayush0054]

hey @mhhnr currently on playground we dont have direct file support to models in playground.
that said ,you can still upload files from playground chat using knowledge_base

here is the example for it https://github.com/agno-agi/agno/blob/main/cookbook/playground/upload_files.py

thanks 🙌

[mhhnr]

Please make it fast, i am totally dependent on your framework for a big enterprise solution that i am building. I want to build documet to json kind of feature asap, i am looking for some other framework. I took lots of efforts to convince your framework from google’s ai framework. Please fix asap

…

On Fri, May 16, 2025 at 6:47 PM Ayush ***@***.***> wrote: hey @mhhnr <https://github.com/mhhnr> currently on playground we dont have direct file support to models in playground. that said ,you can still upload files from playground chat using knowledge_base here is the example for it https://github.com/agno-agi/agno/blob/main/cookbook/playground/upload_files.py thanks 🙌 — Reply to this email directly, view it on GitHub <#3002 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANRGTGI62NCVL7VP2GV2H5D26ZTHDAVCNFSM6AAAAAB37J3BGGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGMJXGU4TIMQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mhhnr]

https://docs.anthropic.com/en/docs/build-with-claude/pdf-support i wish if you can add a support to this please asap. On Fri, May 16, 2025 at 6:51 PM M NITHIN REDDY ***@***.***> wrote:

…

Please make it fast, i am totally dependent on your framework for a big enterprise solution that i am building. I want to build documet to json kind of feature asap, i am looking for some other framework. I took lots of efforts to convince your framework from google’s ai framework. Please fix asap On Fri, May 16, 2025 at 6:47 PM Ayush ***@***.***> wrote: > hey @mhhnr <https://github.com/mhhnr> currently on playground we dont > have direct file support to models in playground. > that said ,you can still upload files from playground chat using > knowledge_base > > here is the example for it > https://github.com/agno-agi/agno/blob/main/cookbook/playground/upload_files.py > > thanks 🙌 > > — > Reply to this email directly, view it on GitHub > <#3002 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ANRGTGI62NCVL7VP2GV2H5D26ZTHDAVCNFSM6AAAAAB37J3BGGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGMJXGU4TIMQ> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[Ayush0054]

Hey @mhhnr,
We do support Anthropic file inputs directly to agents using bytes, local, and URL :
• Bytes input example
• Local file input example
• URL input example

However, this is not yet supported in the Playground — we’ll be shipping it soon once the team gets some bandwidth.

Thanks for your patience! 🙌

[mhhnr]

Can i use that same anthropic model through bedrock marketplace

…

On Mon, May 19, 2025 at 4:06 PM Ayush ***@***.***> wrote: Hey @mhhnr <https://github.com/mhhnr>, We do support Anthropic file inputs directly to agents using bytes, local, and URL : • Bytes input example <https://github.com/agno-agi/agno/blob/main/cookbook/models/anthropic/pdf_input_bytes.py> • Local file input example <https://github.com/agno-agi/agno/blob/main/cookbook/models/anthropic/pdf_input_local.py> • URL input example <https://github.com/agno-agi/agno/blob/main/cookbook/models/anthropic/pdf_input_url.py> However, this is not yet supported in the Playground — we’ll be shipping it soon once the team gets some bandwidth. Thanks for your patience! 🙌 — Reply to this email directly, view it on GitHub <#3002 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANRGTGLAYMFO46DIBBH6Y3L27I2VJAVCNFSM6AAAAAB37J3BGGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGMJZHAZDGNY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ayush0054]

yes https://github.com/agno-agi/agno/blob/main/cookbook/models/aws/bedrock/README.md

[mhhnr]

Thanks so much!

[seenimohamed]

When using Agno with multiple providers (OpenAI, Gemini), file handling behavior is inconsistent.

• Gemini models accept File(content=..., mime_type=..., filename=...) directly and work as expected.
• OpenAI models fail with the same approach, requiring manual base64 encoding + special input_file formatting.
• This breaks the abstraction goal of Agno, since provider-agnostic code becomes provider-specific.

this is how I pass file in gemini models and it works

def parse_gemini_pdf(file_path: str) -> str:
    print("\n--- Gemini PDF Parsing ---")
    agent = Agent(
        model=Gemini(
            id="gemini-2.0-flash-001",
            api_key=GEMINI_API_KEY
        )
    )
    file_path = Path(file_path)

    agent.print_response(
        "Summarize the contents of the attached file.",
        files=[File(content=file_path.read_bytes(), mime_type="application/pdf", filename="file-sample.pdf")]
    )

at the same time in openai, the below code isn't working

def parse_openai_pdf(file_path: str) -> str:
   print("\n--- OpenAI PDF Parsing ---")
   agent = Agent(
       model=OpenAIResponses(
           id="gpt-5-mini",
           api_key=OPENAI_API_KEY
       )
   )
   file_path = Path(file_path)

   agent.print_response(
       "Summarize the contents of the attached file.",
       files=[File(content=file_path.read_bytes(), mime_type="application/pdf", filename="file-sample.pdf")]
   )

Rather I have to modify the input to make it work as below

def working_openai_code(file_path: str) -> str:
    agent = Agent(
        model=OpenAIResponses(
            id="gpt-5-mini",
            api_key=OPENAI_API_KEY,
            reasoning_effort="medium",
            reasoning_summary="detailed"
        ),
    )

    encoded_data = to_base64(file_path)
    base64_data = f"data:{encoded_data['mime_type']};base64,{encoded_data['data']}"
    user_input = [{
        "role": "user",
        "content": [
            {
                "type": "input_text",
                "text": "Summarize the contents of the attached file."
            },
            {
                "type": "input_file",
                "filename": "file-sample.pdf",
                "file_data": base64_data,
            }
        ]
    }]

    agent.print_response(user_input, stream=True)
    

Now comes the hard part, I am trying to use parellel steps to hit openai, gemini simultaneously, to do so, i have to encapsulate the working openai model into function and pass pdf through session_state, too much to handle.

Main objective of using agno is free from rewriting code for each ai model provider, but objective isn't met

Someone help me if I am missing something