Add support for lambda in total_entries option

[jcoleman]

We use the total_entries option to avoid doing expensive count queries (instead we run queries that limit the number of records examined, because we don't care about exact numbers above a certain amount). But we still want to retain the optimization that no query is run at all if the first page's query returns few enough records that we know we've read all of them.

Hence add support for the total_entries option being passed as a lambda.

• Updated the total_entries logic to allow passing a lambda that is executed only when necessary.
• Retained the optimization to skip execution of the lambda when the first batch size is smaller than the limit.
• Centralized the handling of total_entries values (both lambdas and direct values) in the total_entries= setter.

Note: this code was authored with the help of Copilot and GPT-4o. That being said, I reviewed the resulting code, and I also applied various stylistic edits/refactors to the resulting code.

[jcoleman]

@mislav Any chance you could take a look at this?