Access BYOB using pre-signed URLs

W&B uses pre-signed URLs to simplify access to blob storage from your AI workloads or user browsers. For basic information on pre-signed URLs, refer to the cloud provider’s documentation:

• Pre-signed URLs for AWS S3
• Signed URLs for Google Cloud Storage
• Shared Access Signature for Azure Blob Storage

How it works:

1. When needed, AI workloads or user browser clients within your network request pre-signed URLs from W&B.
2. W&B responds to the request by accessing the blob storage to generate the pre-signed URL with the required permissions.
3. W&B returns the pre-signed URL to the client.
4. The client uses the pre-signed URL to read or write to the blob storage.

A pre-signed URL expires after:

• Reading: 1 hour
• Writing: 24 hours, to allow more time to upload large objects in chunks.

Team-level access control

Each pre-signed URL is restricted to specific buckets based on team level access control in the W&B platform. If a user is part of a team which is mapped to a blob storage bucket using secure storage connector, and if that user is part of only that team, then the pre-signed URLs generated for their requests would not have permissions to access blob storage buckets mapped to other teams.

Network restriction

W&B recommends restricting the networks that can use pre-signed URLs to access the blob storage, by using IAM policy based restrictions on the buckets.

In case of AWS, one can use VPC or IP address based network restriction. It ensures that your W&B specific buckets are accessed only from networks where your AI workloads are running, or from gateway IP addresses that map to your user machines if your users access artifacts using the W&B UI.

Audit logs

W&B recommends using W&B audit logs together with blob storage specific audit logs. For blob storage audit logs, refer to the documentation for each cloud provider:

• AWS S3 access logs
• Google Cloud Storage audit logs
• Monitor Azure blob storage.

Admin and security teams can use audit logs to keep track of which user is doing what in the W&B product and take necessary action if they determine that some operations need to be limited for certain users.

Determine the user that requested a pre-signed URL

When W&B returns a pre-signed URL for AWS or GCP blob storage, the X-User header contains the requester’s username. The header is not set for Azure blob storage.