From Synapse to Fabric: Navigating the Data Platform Transition

Blogs

In May 2023, Microsoft introduced its new data platform called Microsoft Fabric. With the introduction of Microsoft Fabric, the support for Azure Synapse as a standalone solution will gradually be phased out. In this blog post we offer a comprehensive overview of this development and its implications, as well as a comparison with the alternatives.   

Microsoft introduced Fabric as an end-to-end analytics platform. It brings together various data analytics, science, and engineering technologies into a single platform. It integrates existing solutions such as Azure Data Factory, Azure Synapse and Power BI. Additionally, it introduces a unified storage solution called OneLake. What OneDrive is for files, OneLake is for data. OneLake is built on top of the existing Azure Data Lake Storage (ADLS) Gen2, which enables the storage of structured and unstructured data. The default storage type for OneLake will be the delta format. The delta format provides enhanced data reliability for your data lake.  

 

While Fabric is still in its public preview phase, and it is not yet entirely clear what the impact will be, it is important to note that Azure Synapse users will eventually need to migrate to Fabric or explore alternative data platform options within Azure.

With Fabric, Microsoft marks the third generation of data platforms. The first generation consisted of standalone Platform-as-a-Service (PaaS) solutions that required integrating the different products. The second generation brought more unified PaaS solutions, such as Synapse Analytics. Fabric is the third generation, offering a Software-as-a-Service (SaaS) solution designed for accessibility by bundling even more tools in a single platform in a PowerBI-like fashion. 

Consider your options

Although Microsoft promises a relatively smooth migration path from Synapse to Fabric, it’s wise to consider other data platform options. Two competitors available in the Azure environment are Databricks and Snowflake. Both competitors are cloud agnostic, making them available in Azure, AWS and GCP.  

Databricks is a data platform centered around Apache Spark, an open-source processing engine for large-scale data processes. It excels in data engineering and data science solutions but offers much more.  One of their flagship solutions is Data Lakehouse, which is built using the open-source delta format. Notably, there are significant similarities between Microsoft’s new OneLake and Databricks’ existing solution. These similarities make it easier to share data between Databricks and Fabric. As a result, it is expected that one is not forced to choose between the two platforms but can leverage the strengths of both. Snowflake, on the other hand, focuses on data warehousing. Rather than constraining a data lake to enhance the data reliability, it relaxed its data warehousing solution to function more like a data lake.  

Final thoughts

While Databricks and Snowflake have already established themselves over the past few years, the strengths and weaknesses of Microsoft Fabric are still unfolding. However, the core components of Fabric such as Data Factory and Power BI have more than proven themselves, and Fabric is effectively building on their successes. As of now, Microsoft has not announced the general availability date for Fabric. Therefore, in the interim, you have two options:

  1. Build in Synapse Analytics: Build your data solutions on Synapse Analytics and assume a smooth transition to Fabric when it becomes generally available. 
  2. Build in Databricks: Alternatively, consider building your solutions on Databricks and storing data in the Data Lakehouse. This solution removes the need for a platform switch. Moreover, since both data lakes use the delta format, your data is easily accessible for Fabric applications, such as PowerBI.  

In conclusion, Microsoft Fabric represents a new development in the data platform space. Although it is not generally available yet, it is essential to carefully assess your options and make informed decisions based on your specific needs and circumstances.   

And Bright Cape is here to help! Our data engineers have the know-how to advise you and help you transition to the new data platform that is most suitable for you. 

Curious if your data solution is future-proof? Get in touch with us and check out a selection of the durable solutions we have built for our clients here.