Incremental data load in PowerPivot through Power Query
We work with very large databases. Every time we update the information, it reloads all the data in the Power Pivot through Power Query, which takes a long time.
Undoubtedly we can make separate queries with the history and have the results added by DAX, but this forces us to duplicate all the metrics in DAX and parameterize everything.
Our request is whether there could be any way to update the data incrementally???? it would be fantastic.
PD.- Now we are using and we recommend our clients to use PowerUpdate that allows to program the update of the files.
Wyatt McNabb commented
As datasets grow over time, the speed decreases. Simple as that!
Oz du Soleil commented
I faced this with a client who needed to upload weekly data. The start of the year was fine. Around June the imports were noticeably slow because the import was re-importing data all the way back to January--data that had not changed.
By September, the imports were ridiculously slow and would sometimes crash after running for more than an hour. Again ... the problem was the re-importing of months of data that had not changed.
Celia Alves commented
Having Incremental Refresh in Power Query for Excel would be extremely helpful. It makes no sense to have to wait a long time for a data model to refresh/load the entire data history when only the last few days or months have changed. Thank you.