Software Engineering Stack Exchange
Software Engineering Stack Exchange

Questions tagged [data-warehouse]

31 questions
50
votes
11 answers

Best practice or design patterns for retrieval of data for reporting and dashboards in a domain-rich application

First, I want to say this seems to be a neglected question/area, so if this question needs improvement, help me make this a great question that can benefit others! I'm looking for advice and help from people who have implemented solutions that solve…
richard
  • 3,325
  • 2
  • 25
  • 39
26
votes
8 answers

Is there a programming paradigm that promotes making dependencies extremely obvious to other programmers?

I work in a Data Warehouse that sources multiple systems via many streams and layers with maze-like dependencies linking various artifacts. Pretty much every day I run into situations like this: I run something, it doesn't work, I go through loads…
Christs_Chin
  • 369
  • 2
  • 4
6
votes
4 answers

How and when to choose between Data Consolidation/Data Integration and Data Synchronisation?

I am having some confusion as to how to decide what to choose between Data Consolidation/Data Integration and Data Synchronisation? I know that: Data Consolidation/Data Integration: refers to the collection and integration of data from multiple…
Jason Donnald
  • 161
  • 2
6
votes
3 answers

Designing a data warehouse: action or status based

I am designing a data warehouse for a sales platform. There exists a group of users that offers and a group of users that demand entities. The production data base holds two sources of information: The current status of each entity and to which…
Milla Well
  • 227
  • 2
  • 6
6
votes
6 answers

What are the disadvantages of using stored procedures, SSIS, and SSRS to implement business applications?

Recently, it was suggested that many projects (where I work) should be implemented using some combination of stored procedures (on SQL Sever / T-SQL), SSIS, and SSRS. In one specific project, SSIS would get a file from a SFTP server. The file…
James
  • 285
  • 1
  • 5
  • 13
5
votes
2 answers

What's the best way to learn the MS Business Intelligence stack?

What's the best/easiest way to get into the MS BI stack on my own (preferably with a [kindle] book)? Specifically, I'd like to learn more about SSIS, SSRS, and SSAS in that order. I've tried looking at books on amazon, but I really have no…
jdylanmc
  • 181
  • 1
  • 7
4
votes
2 answers

How should a data warehouse be maintained for a quickly changing schema

I am currently in a process of maintaining a data warehouse for a quickly growing start up company. There is a lot of reporting demands from the clients, and this is usually handled by a data warehouse we set up. However, unlike bigger more…
Yong Jun Kim
  • 151
  • 4
4
votes
2 answers

Am I using Data Warehousing wrong?

I work for a company that has several products, and several databases and tons of data. To sorta merge all this data what we've been doing moving some of our data over to Amazon Redshift (a data warehouse). We store as much as we can here…
Kyle
  • 363
  • 2
  • 3
  • 7
4
votes
2 answers

implementing dynamic query handler on historical data

Context: I have historical data about property (house) sales collected from various sources in a centralized/cloud data source (assume info collection is handled by a third party) Planning to develop an application to query and retrieve data from…
user2390183
  • 224
  • 1
  • 7
4
votes
3 answers

Can a table be both Fact and Dimension

Ok, I am a newbie and don't really think "dimensionally" yet, I have most of my initial schema roughed out but I keep flipping back and forth on one table. I have a Contract table and it has a quantity column (tonnes), and a net price column, which…
PatFromCanada
  • 162
  • 1
  • 8
3
votes
3 answers

How and when dimensions are filled in a data warehouse schema?

I'm reading Ralph Kimball's books and I'm currently exploring the following data warehouse schema. Are both dimension & fact tables populated during the data warehouse creation/update? How often? What about the DimDate table? Do we populate it with…
user2567
3
votes
2 answers

Enterprise application warehousing and relational database

I have a general question about design pattern for an enterprise application. I read a lot about it but its actually hard to find an answer because most you find it rater about how to design a data warehouse (DW) or about how to design pipeline, ETL…
Steffen Mangold
  • 159
  • 5
3
votes
2 answers

Data warehouse architecture for mutating schema

I am setting up an ETL process and small data warehouse for querying the data in a few different dimensions. One issue is that the schema for the objects can mutate over time - mainly that some fields will be added and some removed. What are some…
Rex M
  • 231
  • 1
  • 8
2
votes
1 answer

Indexes on a SQL Server fact table

If I have a SQL Server fact table with four dimensions (OrderDate, Customer, Product, Region), my understanding is that it's best to create a non-clustered index per foreign key (dim key column in the fact table). Assuming that is correct, is it…
MAK
  • 23
  • 4
2
votes
0 answers

How are values such as monthly cost stored in a data warehouse

Would a value such as monthly cost from source data be represented as a fact? If yes, how would a date dimension table be designed around that fact?
user1450259
  • 21
  • 1
1
2 3