Support environment variable for S3 addressing_style config (reopen)

[flypenguin]

(This is basically a copy-paste of @skeggse's #3190, with an updated use case; I am creating a new request as indicated in this comment. /CC @tim-finnigan)

Describe the feature

Allow configuring boto3 to use the virtual host style (MY-BUCKET.s3.amazonaws.com) for talking to a bucket by providing the AWS_S3_ADDRESSING_STYLE environment variable.

There even is a PR already which does it: PR #3191, since this was proposed already.

Use Case

We have an on-premise S3 system from Huawei, and we intend to back up our CNPG-based PostgreSQL database into it. CNPG utilizes boto3-based barman for that purpose. Unfortunately, for reasons yet unknown, it does not use virtual-host-style access.

Debugging is hard, because CNPG is using barman is using boto3, so we are unable to (a) modify or (b) experiment with the code too much. Other applications written by us are using virtual-host-style access without flaws. For those kind of scenarios, a AWS_S3_ADDRESSING_STYLE environment variable would be extremely helpful.

Proposed Solution

Add AWS_S3_ADDRESSING_STYLE to configprovider.py's DEFAULT_S3_CONFIG_VARS addressing_style entry (see #3191).

Other Information

I would like to add two thoughts:

• First, it seems (as of now) totally unclear when AWS will finally disable path-style access once and for all, so my assumption is we will be dealing this a while longer.
• Second, it is also not clear (at least to me) how that feature implementation could hurt the deprecation process, when it finally arrives. It would be incredibly useful in at least one (albeit rare) scenario, which is currently delaying our go-live.

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

n/a

Environment details (OS name and version, etc.)

MacOS, Linux

[tim-finnigan]

Thanks for reaching out. As mentioned in the previous issue you linked, the default behavior should be to use virtual. That value can be set via configuration as documented here: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html.

[flypenguin]

@tim-finnigan yes, I know that, I have read basically everything about the topic :) . Yet, as also mentioned in this post – for whatever reason, this is not happening – we don't see virtual-host based access. So, for us, this "forcing it" variable would help tremendously.

I totally understand limiting things to the necessary to not "bloat" the configuration surface. Yet I am now in a situation in which we are severly impacted by the lack of this setting – so ... I'd say this is useful. We might disagree here, yet my problem does exist :) .

That is why I explicitly ask you (plural, the boto3 team) to re-consider this feature, arguing that it does not hurt, and we're probably stuck with path based for a while longer.