MAG
Stories from synvert customers

We are proud to have assisted MAG in its transformation journey. Our project aimed to collect and analyse various data points throughout the passenger journey, and with our DevOps approach we were able to ensure that the production workloads were up and running in AWS just 12 weeks after starting the project!

MAG already had a mature stack of BI and database products for reporting, but they were overstretched and needed to be replaced by a solution that would create a scalable and flexible data solution to help MAG to reach its goals. The new solution would need the following:

• an extended data warehouse.
• scalable and elastic compute.
• the ability to cope with seasonal spikes in passenger travel.

This would allow real-time data streaming, empowering MAG to become a real-time business, as well as giving the control room and security staff a better view of passenger flow and security performance, now able to make decisions based on real-time data. Data requirements included ingestion from several on-premise systems and external data sources for visualisation on several large screens.

Having become proficient in the Canvas expression language, we did several workshops with stakeholders, editing dashboards in real time. It soon became apparent that we had progressed beyond the initial wireframe, and we were able to adopt innovative approaches to presenting information: through collaboration with stakeholders and end-users, we shared the dashboards, collated feedback, and incorporated changes to further improve them.

• This dashboard shows the status of Airport Security, from the number of people entering and leaving to compliance rates, as well as information about the individual lanes. The shape chosen for the security lanes reflects the physical layout inside Airport Security, so that the operating staff can immediately relate the dashboard to a physical lane.
• This shows trend information, displaying those entering the Airport Security area plotted against a forecast of passenger turn-up and queue wait times in different areas.
• Almost the same as a standard FIDS (Flight Information Display Screen), this has the added advantage of matching the percentage of passengers who entered Airport Security against passenger forecasts for upcoming flights.
• A large percentage of passengers arrive at Stansted Airport via train, so delays in their journeys have a significant effect on the number of people in Airport Security. A single train could be carrying hundreds of air passengers, and multiple trains may arrive at almost the same time after delays and other issues.
• The dashboard not only displays the typical train information table, but also shows train arrival times along a timeline; you can see at a glance how much this has changed from the first iteration with Canvas.

Once we decided to use Elasticsearch as the data storage layer, our next focus was on determining our data ingestion needs: we had to gather information from various sources, including on-premise database systems, files that were frequently uploaded to AWS S3 buckets, and external API data sources. For instance, we needed to obtain National Rail data, which we successfully loaded into Elasticsearch with the STOMP (Streaming Text Oriented Messaging Protocol) interfaces. There were a few initial challenges that we had to overcome:

• To get up-to-date data, we had to increase the frequency of polling for data from databases to more than the default interval of one minute.
• The incoming data from STOMP was gzip compressed.

In just the first six months of the project, MAG had successfully transitioned from a single instance database to a scalable data warehouse. With daily sales from store levels automatically ingested from over 50 different retailers, the system was processing over 90% of all sales automatically.

By the end of the first year, MAG had added a real-time streaming solution to ingest data from the car park and fast-track bookings. By the end of the second year,  multiple real-time dashboards were in place to monitor the state of the customer security queues. Data was being ingested from a variety of sources, including trains, queues, metal detector scans, body scans, x-rays, and more.

Canvas is a brilliant tool to enable very impressive, real-time data visualisation. Despite being in beta, Canvas's flexible plug-in system follows the elastic approach to software, enabling users to expand functionality in numerous ways.

In contrast to many other BI tools, which limit users to basic visualisations like graphs and tables with a few predefined gauges and charts, Canvas provides a refreshing level of flexibility, and its ease of use means that data specialists can create unlimited, visually impressive dashboards.