
Communications, Media & Telecom - Design and Development of a Google Cloud-Based Data Platform
Communications, Media & Telecom
By developing a Data Lake in Google Cloud and implementing best practices in software and data engineering, synvert achieved faster data access and more efficient data aggregation for analytics, leading to significant time savings and a higher frequency of regular business reports, enabling timely corporate decisions.
Initial situation
The customer’s data was stored in an Oracle database, but not all teams had access to centrally stored data, e.g., website data was missing, and there were data silos for each department, making cross-departmental analyses and reporting difficult.
The database was overloaded and too complex, making data analysis access slow and requiring reports to be processed in pieces, limiting the frequency of available reports.
Architecture
The data lake architecture is built on Google Cloud, using technologies such as Gitlab, dbt, and Google BigQuery.
Generated benefits
By improving the data warehouse, synvert achieved faster data access and more efficient data aggregation for analyses. This leads to significant time savings and increased frequency of regular business reports, enabling timely corporate decision-making.
Users are able to create their own reports in Power BI, with all departments easily and quickly accessing the same data lake, reducing complexity and time to insights. Most data transformations are already performed in the data lake using BigQuery and dbt, allowing departments to adjust data and business logic company-wide.
Services accomplished by synvert
synvert developed a data lake in Google Cloud, and best practices in software and data engineering were introduced. This included Git repositories, CI/CD, object-oriented programming, linting, and test automation. Data quality tests and modeling according to Data Vault 2.0 were also implemented. Data Mesh enabled users to perform cross-domain data analyses. Additionally, internal consultants were successfully trained in the aforementioned technologies and methods.