Introduction into E.T.L (Extract, Transform and Load)
This a process related to data warehousing which involves the extracting of data out of the source system/Systems
and placing it into a repository or Target.
Extraction
Extracting the data from source systems (Flat Files or other operational systems) and converted into one consolidated data warehouse format which is ready for transformation.
Transformation
Transforming the data may involve the following tasks:
- Cleaning: One of many very important task in the transforming stage because the Source data would always have data that the target system doesn’t support or understand hence cleaning is required.
In some cases the Source can be from many source inputs so Lookup are important to avoid duplication. - Filtering: Now the Source Data would have so many rows but it’s important to send relevant data to your target and filter out the unnecessary data.
- Business Rules: Calculations or Derivations can be performed Here so we can have Correct and readable data at the target.
and many more.
Loading
After proper transformation and data matches the Business Rules loading the data into a target or repository is the final step in the E.T.L (Extract, Transform and Load)
in my next blog we will look into the basic Loading Data from Source to Target
Thanks
Sohail Izebhijie
Free Open Source BI Platform
- AI-Powered Analytics
- Embedded Analytics
- Single Sign-On (SSO)
- Multi-Tenancy
- Paginated Canned Reporting
- Interactive Dashboard
- White Labeling