However, in the architecture of staging and transformation dataflows, it's likely that the computed entities are sourced from the staging dataflows. In the previous image, the computed entity gets the data directly from the source. This is helpful when you have a set of transformations that need to be done in multiple entities, which are called common transformations. When you reference an entity from another entity, you can use the computed entity. The same thing can happen inside a dataflow. When you use the result of a dataflow in another dataflow, you're using the concept of the computed entity, which means getting data from an "already-processed-and-stored" entity. Use a computed entity as much as possible The following image shows a multi-layered architecture for dataflows in which their entities are then used in Power BI datasets. The other layers should all continue to work fine. When you want to change something, you just need to change it in the layer in which it's located. Trying to do actions in layers ensures the minimum maintenance required. The staging and transformation dataflows can be two layers of a multi-layered dataflow architecture. The staging dataflow has already done that part, and the data will be ready for the transformation layer.Ī layered architecture is an architecture in which you perform actions in separate layers. The transformation dataflow won't need to wait for a long time to get records coming through a slow connection from the source system. This separation also helps in case the source system connection is slow. The transformation dataflows are likely to work without any problem, because they're sourced only from the staging dataflows. All you need to do in that case is to change the staging dataflows. This separation helps if you're migrating the source system to a new system. When you've separated your transformation dataflows from the staging dataflows, the transformation will be independent from the source. The entities are then shown being transformed along with other dataflows, which are then sent out as queries. Image emphasizing staging dataflows and staging storage, and showing the data being accessed from the data source by the staging dataflow, and entities being stored in either Cadavers or Azure Data Lake Storage. Making the transformation dataflows source-independent.Having an intermediate copy of the data for reconciliation purpose, in case the source system data changes.Reducing the load on data gateways if an on-premises data source is used.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |