feat: refactor rest signer in fileio

[kevinjqliu]

Feature Request / Improvement

Context: #2846 (comment)

Currently S3 signer is tightly coupled with FileIO. This is awkward since the S3 Signer requires interaction with remote service (i.e. REST Catalog) and reuse its auth manager.

We can refactor here and create the signer in REST Catalog and pass to FileIO by overriding its _load_file_io function.

Also see _init_sigv4 for inspiration

[sungwy]

I ran into this while reviewing the AuthManager refactor work on the Java/Catalog side. In the OAuth2AuthManager V2 design doc, @adutra calls out the same issue. The S3 signer is tightly coupled to legacy OAuth2 APIs, even though it really needs to live closer to the REST catalog and reuse its auth state.

As AuthManager evolves, it would be good to align on a common pattern where the signer is created at the catalog level and passed into FileIO, rather than being constructed from FileIO properties.

I think it would be good to have a consistent, pluggable signer model across languages.

[adutra]

+1 on this idea, having a signer that is agnostic of the actual authentication manager is the way to go!

On the practical side though, I wonder if it's possible to pass a signer instance from catalog to FileIO in all languages? In Java at least, FileIO instances are created by reflection.