As data specialists, data integration projects are some of the most complex environments we have to deal with. In particular when it involves data migration.
There are many tasks that need to be performed, ranging from data profiling, data quality assessments, master data management, testing, to the standard ETL processes. We often wind up reinventing the wheel, especially because most of these projects are one-off throw aways. These make our lives that much harder as we are often pressed for time by the business to get on with the job.
In this session, I would like to show you how a generic ETL framework can help save us time by leveraging off some standard processes that can easily be developed and reutilised.
I will show you how with a few simple SSIS tasks and lines of code we can create an efficient ETL process that we can keep using again and again. Because it's extendable, we can easily add additional components for DQS, MDM and reconciliation testing.