Abstract
This paper demonstrates ETLMR, a novel dimensional Extract–
Transform–Load (ETL) programming framework that uses Map-
Reduce to achieve scalability. ETLMR has built-in native support
of data warehouse (DW) specific constructs such as star schemas,
snowflake schemas, and slowly changing dimensions (SCDs). This
makes it possible to build MapReduce-based dimensional ETL
flows very easily. The ETL process can be configured with only
few lines of code. We will demonstrate the concrete steps in using
ETLMR to load data into a (partly snowflaked) DW schema. This
includes configuration of data sources and targets, dimension processing
schemes, fact processing, and deployment. In addition, we
also present the scalability on large data sets.
| Original language | English |
|---|---|
| Journal | Proceedings of the VLDB Endowment |
| Volume | 5 |
| Issue number | 12 |
| Pages (from-to) | 1882-1885 |
| ISSN | 2150-8097 |
| Publication status | Published - 2012 |
| Externally published | Yes |
| Event | 38th International Conference on Very Large Data Bases - Istanbul, Turkey Duration: 27 Aug 2012 → 31 Aug 2012 Conference number: 38 |
Conference
| Conference | 38th International Conference on Very Large Data Bases |
|---|---|
| Number | 38 |
| Country/Territory | Turkey |
| City | Istanbul |
| Period | 27/08/2012 → 31/08/2012 |