PZ Cussons opted for AWS to power their proof of concept solution, as they offer all the necessary services and providers to execute the first set of tasks, as well as additional capabilities to support future data-driven use cases.
To optimise their environments strategy, PZ Cussons required a fully scripted end-to-end process that could be easily repeated. To achieve this, Terraform was chosen to create all the native AWS services, including the creation of all database objects.
To maximise automation, AWS Lambda functions were used to verify flat file imports for row counts, column counts, and data types. Once verified, the ETL environment was initiated using SQS, which orchestrated the instantiation of Matillion and other tasks. We chose to use SES (Simple Email Service) to ingest email attachments.
Apart from the native AWS tooling, PZ Cussons also adopted several partner technologies running on AWS to further enhance their solution:
- Matillion for ETL.
- Snowflake for the data warehouse.
- Tableau Online for reporting and visualisation.