Microsoft Launches Cross-Cloud Data Governance for Seamless AWS S3 Access in Azure Databricks

Microsoft announces the general availability of Cross-Cloud Data Governance with Azure Databricks, enabling seamless access and governance of AWS S3 data directly from Azure Databricks via Unity Catalog. This innovation simplifies security, compliance, and data management across cloud platforms without data migration. Unique :

Cross-Cloud Data Governance Now Live on Azure Databricks

Microsoft just rolled out a game-changer for data pros working across clouds. Azure Databricks now supports cross-cloud data governance with AWS S3 via Unity Catalog. This means you can securely access and govern AWS S3 data directly from Azure Databricks—no more tedious data migration or duplication.

What’s New?

Previously, accessing AWS S3 data from Azure Databricks meant costly ETL processes to move data into Azure Data Lake Storage (ADLS). Now, Unity Catalog lets you set up external S3 locations directly in Azure Databricks. This update simplifies your data workflows and slashes operational overhead.

“This release allows teams to directly configure and query AWS S3 data from Azure Databricks without the need to migrate or duplicate datasets.”

Major Updates and Benefits

Unified Governance: Manage access policies and compliance across both Azure and AWS clouds from a single platform.
Frictionless Data Access: Securely discover and analyze data across clouds in one workspace, reducing complexity.
Enhanced Security: Gain centralized visibility with tagging, lineage, classification, and auditing for all cloud storage.

In short, this update tackles the headache of fragmented governance in hybrid cloud environments. Consistency and security are now easier to achieve.

How It Works

Set up is straightforward. Create a read-only AWS IAM role, configure storage credentials in Azure Databricks’ Catalog Explorer, and define external S3 locations. Then, apply consistent permissions across ADLS and S3 data. Finally, start querying your S3 data directly—no data moves required.

“Unity Catalog on Azure Databricks addresses challenges by providing a unified and open governance solution for all data and AI assets.”

Supported Features Include:

AWS IAM role storage credentials
S3 external locations and tables
S3 external volumes
S3 dbutils.fs access
Delta sharing of S3 data from Unity Catalog

Why This Matters for Tech Teams

Hybrid and multicloud setups are the norm, but governance often lags behind. This release helps teams enforce consistent security policies and audit trails across platforms. It also reduces duplicated efforts and operational costs.

For anyone managing data across Azure and AWS, this is a must-watch update. It streamlines governance while boosting security and compliance.

Get Started and Join the Conversation

Ready to try it out? Follow Microsoft’s step-by-step guide to configure cross-cloud governance in Azure Databricks. Plus, don’t miss the Databricks blog for more details.

Also, mark your calendar for the Data + AI Summit 2025 in San Francisco this June. Microsoft will be there showcasing Azure Databricks innovations. It’s a perfect chance to connect with experts and level up your data game.

Azure continues to be a top choice for Databricks workloads, especially with these powerful new governance features.

Eliminates the need for costly and time-consuming ETL processes between AWS S3 and Azure Data Lake Storage.

Supports AWS IAM role storage credentials for secure and controlled access.

Enables centralized management of access policies and auditing across Azure and AWS storage.

Allows querying of external S3 tables and volumes directly within Azure Databricks workspace.

Facilitates Delta sharing of S3 data through Unity Catalog, enhancing collaboration and data interoperability.

From the New blog articles in Microsoft Community Hub