Azure Data Analytics End to End

Let's start with a story, not a 'once upon a time story', a story for your backlog 🙂

As a solution architect
I need to design and build an Azure data analytics platform end to end
to deliver data insights for my customer.

In February 2021 I delivered a talk as part of the Scottish Summit conference on how you could/should build an end to end data platform solution in Azure to deliver data insights and analytics. This is one of my favourite sessions so thought it worth re-sharing the recording here.

Talk Abstract

The resources on offer in Azure are constantly changing, which means as data professionals we need to constantly change too. Updating knowledge and learning new skills. No longer can we rely on products matured over a decade to deliver all our solution requirements. Today, data platform architectures designed in Azure with best intentions and known good practices can go out of date within months. That said, is there now a set of core components we can utilise in the Microsoft cloud to ingest and deliver insights from our data? When does ETL become ELT? When is IaaS better than PaaS? Do we need to consider scaling up or scaling out? And should we start making cost the primary factor for choosing certain technologies? In this session we'll explore the answers to all these questions and more from an architects viewpoint. Based on real world experience lets think about just how far the breadth of our knowledge now needs to reach when starting from nothing and building a complete Microsoft Azure Data Platform solution.

Recording

Outcome

Below is a view of the outcome from the session and the overall architecture that I talk about the delivery of and build up throughout the talk.

Thanks for watching