My first Publication Agile-Data-Warehouse-Design-eBook | Page 117

96
Chapter 4
Modeling Multiple Events With Agility
Agile designers
might be tempted to
limit the modeling
scope per release
This can result in
silo data marts
that are unable to
support cross-
process analysis
Deploying an agile data warehouse that will eventually handle the multiple busi-
ness events required for enterprise BI is especially challenging. To meet the agile
principles of early and frequent delivery of valuable working software, agile design-
ers may be tempted to limit their modeling scope per release in terms of business
processes and/or stakeholder departments. Unfortunately this can quickly lead to
the silo data mart anti-pattern, of Figure 4-1.
With a tightly-controlled initial scope, BI users can receive their agile data marts
early and obtain valuable insight from them individually on a department by
department basis. So far so good, but when users want to step up to cross-
department, cross-process analysis they find they cannot make the necessary
comparisons due to incompatible or missing descriptions and measures. Rebuild-
ing each data mart from scratch is unthinkable so data is re-extracted from the
source systems so that each department can look at it “their way”. The cost of this
extra work and the inconsistent or conflicting answers that emanate from these
“multiple versions of the truth” drive BI stakeholders crazy!
Figure 4-1
Silo data marts that
cannot be shared:
a data warehouse
anti-pattern
With too limited a
scope data
warehouse design
incurs heavy
technical debt
Traditional BDUF
does not match agile
BI requirements
Silo data marts are examples of technical debt. Agile software development inten-
tionally takes on technical debt when “just barely good enough” code is released.
This makes good sense when the business value of early working software out-
weighs the interest on the debt: the extra effort involved in refactoring the code in
future iterations. However, for DW/BI projects, the cost of servicing high interest
technical debt: refactoring terabytes of incorrectly represented historical data, can
be ruinously high.
The especially high interest of database technical debt could be argued as a good
reason for taking a traditional, non-agile, approach to data requirements gathering
and data warehouse design itself and postponing agile practices for ETL and BI