If you have ever worked with a Data warehousing solution, you would probably say that the most important part of the job is to ensure proper Data ingestion (Data loading). If you lose any data at this point, then the resulting information (reports) will end up inaccurate, failing to represent the facts on which Business decisions are made.
Microsoft Azure provides several services which you can use to ingest data and one of them is Azure Data Factory.
So What is Azure Data Factory?
Azure Data Factory (ADF) is a Platform-as-a-Service offering from Microsoft. The primary purpose of this service could be to do Extract, Transform and Load (ETL) or Extract, Load and Transform (ELT) and this is done via using a concept pipelines. Two types of pipelines to begin with - data movement pipelines (Extract & Load) and also data transformation pipelines (Transform). And being a PaaS service, ADF automatically scales out based on the demand enforced using these pipelines.
ADF is ideal for working with Structured data as well as Unstructured data. ADF allows you to load raw data from many different sources, both on-premises and in the cloud.
Like many other products from Microsoft, I would call ADF as a collection of several tools packaged together,
- For ease of understanding
- To eradicate unnecessary maintenance work
- To streamline the approach to be taken
Microsoft has detailed information in Docs and you can probably start digging from https://docs.microsoft.com/en-us/azure/data-factory/introduction
If you have the necessary details and would like to get started with Azure Data Factory already - you can start in https://azure.microsoft.com/en-us/services/data-factory/
An idea on the pricing details can be found in https://azure.microsoft.com/en-us/pricing/details/data-factory/data-pipeline/
And if you are a hands-on person, there is a GitHub lab tutorial with all needed details in https://github.com/kromerm/adflab
If you would like to have a poster on your wall reminding of you all about Azure Data Factory - then feel free to go in https://aka.ms/visual/azure-data-factory
And probably good to know is that you can utilize upto 5 free low frequency activites with Azure Data Factory by signing up for a free Azure account. More details in https://azure.microsoft.com/en-us/free/free-account-faq/
And if you are into LEARN from Microsoft docs - I would recommend to go through https://docs.microsoft.com/en-us/learn/modules/explore-azure-synapse-analytics/ to get a better perspective on large scale data analytics
No comments:
Post a Comment