Shopping center pioneer ECE implements data warehouse with Microsoft Fabric

The challenge: Outdated infrastructure makes it difficult to move into the future

Managing and developing real estate is the core business of the family-owned ECE Group. Founded in 1965 by Werner Otto—who was also founder of the eponymous mail-order company, the Otto Group—ECE brought the shopping center concept to Germany. Today, besides managing around 200 shopping centers in 12 countries, ECE develops and operates properties in all asset classes as well as entire city districts. Data is the foundation for detailed management of the business—data on stores and visitor flows in the shopping centers, on the 20,000 tenant partners, and on sales, contracts, invoices, and insolvencies. ECE processes around two terabytes of uncompressed data every day. The breadth of information is high, and the data records that have to be merged from over 40 source systems are correspondingly wide-ranging.

In the past, if ECE wanted to know, say, how many empty stores there were in its shopping centers, how many seats were available in the food courts, or when it was time for maintenance, it had to evaluate the relevant data manually. This took a long time and ran up enormous costs. “That’s why, for around 15 years now, we’ve been relying on a digital solution and our on-premises data warehouse to consolidate data from our various systems and make it available to our customers, usually tenants and investors, as required,” says Jan Hambach, Team Leader IT Analytics at ECE Group Services GmbH & Co. KG. Over the years, however, more and more source systems had to be integrated. “The amount of data that we had to load, manage, and process skyrocketed. We initially needed only around three to four hours for our calculations; ultimately, it wound up taking more than eight,” Hambach says. That’s why the IT department gradually expanded the old environment into a hybrid solution using Power BI, moving reporting to the cloud and keeping the data on-premises. The group’s own data centers were getting on in years, and ECE was faced with a decision: Buy new hardware at great expense or take the plunge into the cloud?

“We wanted to bring everything back together in one place, with data proximity ensuring fast loading times,” Hambach explains. “So, we opted for an approach with software as a service, or SaaS. This allows us to concentrate on content, as we no longer have to operate the infrastructure. And it is also easier for us to integrate cutting-edge issues, such as artificial intelligence and other cloud topics.”

Together with adesso, a long-standing member of the Microsoft Partner Network, ECE conducted a preliminary study to find the right technology for this project, and initially opted for a different solution. But then, in May 2023, Microsoft Fabric came on the market. Hambach and his team changed course, with support from cloud solution architects at Microsoft Unified and the Azure cloud migration and modernization center, and turned ECE into a first mover: “We wanted to build a data warehouse to last the next 15 years—and in the long run, Microsoft Fabric is the right solution for us, even if many functions weren’t yet available at the start. We were one of the first companies to use Fabric.” ECE was already using many Microsoft services, and in the end, the prospect of benefiting from the performance, functions, and integration capability of an overarching technology was the deciding factor. “We were also impressed by the cost model, where we pay only for what we actually use,” Hambach says.

The solution: AI helps with implementation, Fabric with performance

The next step was to migrate a total of five different data blocks to the new solution: the center management data, the company data, sales data, process data, and investment data. The latter has already been fully migrated to the cloud; two more blocks will follow by spring 2025 and the remaining two by summer 2025. “In the migration process, we deliberately decided against a ‘big bang.’ We wanted to learn from the various migration steps and apply that experience directly to the next stages,” says Heiko Gronwald, Head of Data Platform Solutions at adesso SE. “Our aim was to enable users to benefit from the new solution as quickly as possible and obtain their content from the cloud.”

During implementation, one thing quickly became clear: the effort required for the move to the cloud was enormous. Code had to be restructured and transferred—a highly repetitive task. “The developers would copy 2,000 lines of code, go through everything by hand, and adjust the relevant sections,” Hambach says. “We wanted to minimize this manual effort with the help of Azure OpenAI Service.” To this end, the team programmed a website where they enter the code. The AI then outputs an annotated and translated version that can be used to continue working immediately, with the code ready to run in Microsoft Fabric. “By using AI, we can reduce our workload by up to 50%. Optimizing the code solely by hand would have made our project 300 to 400 days longer,” Hambach says. “Automated processing also increases quality, because when you’re transferring things back and forth manually, sometimes mistakes just happen.”

In the new data warehouse based on Microsoft Fabric, Azure Data Factory now collects and delivers the raw data from all source systems. That data is then processed, harmonized, and normalized across various levels to create a uniform format. In the end, depending on the use case, the data is output as reports for tenants, investors, or for business planning.

“The entire transformation took place exclusively in the back end. Our aim was for users to not notice the migration at all,” Gronwald says. Hambach adds: “Now that data and reporting are close together again in Microsoft Fabric, users can benefit from the systems’ outstanding performance. For example, we now generate 50-page investor reports entirely with data from the cloud. The solution is also reliable at peak times, when a lot of different data has to be processed, and many reports are generated.”

ECE’s IT team is currently adapting its operating and development processes to the new conditions. “We can already see how the SaaS approach is reducing operating costs and freeing up capacity for core development tasks,” Hambach says.

Another advantage of the new data warehouse is that the platform is easy to use and can be operated together with and by other teams. “Our colleagues in marketing, for example, face the challenge of obtaining data from external third-party systems that are not connected to each other. So, they consolidate it in Fabric and enrich it with our own data,” Hambach explains. “That gives them evaluations of visitor behavior, the number of clicks on our websites, or visitor flows on-site, and they can then incorporate these findings into our centers’ apps to improve the shopping experience for consumers.”

Hambach sums up by saying: “The migration was not always easy and took a lot of time, but you have to expect that when you boldly go ahead with a solution that is new on the market. With Microsoft Fabric, we are equipped for the future—and the platform also enables us to use new technologies such as AI or Copilot.”

Next, the team wants to focus on the topic of real-time data. “For example, at the moment it’s difficult to measure the effects of ads displayed in shopping centers,” Hambach says. “In the future, we will be able to check whether the flow of visitors changes when we advertise a specific product or business. This will make it easier for us to monitor success, allow us to react quickly, and create an even better buying experience.”