Skip to content

Commit 851673a

Browse files
docs(io): say what can block when building an S3 client
Two comments said building an S3 client can look up a bucket region. It does not here: Arrow resolves a bucket's region only in S3Options::FromUri, which this code does not use. What can block is the AWS SDK waiting on the EC2 metadata service when no static keys are set (about 4 s per build off EC2); a build from static keys does not wait.
1 parent 460ced6 commit 851673a

2 files changed

Lines changed: 7 additions & 7 deletions

File tree

‎src/iceberg/arrow/s3/arrow_s3_file_io.cc‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -221,9 +221,9 @@ class ArrowS3FileIO final : public FileIO, public SupportsStorageCredentials {
221221

222222
/// \brief Build a delegate for each credential this FileIO can serve.
223223
///
224-
/// Runs without holding `mutex_`: building an S3 client can reach out to discover a
225-
/// bucket region, which would stall every concurrent operation. Reads no
226-
/// mutable member state.
224+
/// Runs without holding `mutex_`: building an S3 client can block (without
225+
/// static keys the AWS SDK may wait on the EC2 metadata service), which would
226+
/// stall every concurrent operation. Reads no mutable member state.
227227
Result<DelegatesByPrefix> BuildDelegates(
228228
const std::vector<StorageCredential>& storage_credentials) const;
229229

‎src/iceberg/resolving_file_io.cc‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,10 @@ Result<std::shared_ptr<FileIO>> ResolvingFileIO::FileIOForPath(
4040
const auto scheme = StringUtils::ToLower(LocationUtil::ParseScheme(location));
4141
ICEBERG_ASSIGN_OR_RAISE(const auto name, FileIORegistry::Resolve(scheme));
4242

43-
// Loads without holding `mutex_`: building a client can reach the network (an
44-
// S3 client may look up its bucket region), which would stall every other
45-
// operation. Forwards all credentials; each implementation applies the
46-
// prefixes it understands.
43+
// Loads without holding `mutex_`: building a client can block (an S3 client
44+
// without static keys may wait on the EC2 metadata service), which would
45+
// stall every other operation. Forwards all credentials; each implementation
46+
// applies the prefixes it understands.
4747
auto load = [&](const std::vector<StorageCredential>& credentials)
4848
-> Result<std::shared_ptr<FileIO>> {
4949
ICEBERG_ASSIGN_OR_RAISE(std::shared_ptr<FileIO> io,

0 commit comments

Comments
 (0)