Marks 70-534 Architecting Microsoft Azure Solutions Course notes June 2017

To give back to the wider community, here are all my course notes for the 70-534 Architecting Microsoft Azure Solutions certification exam. If you read through all of these bullet points and diagrams and understand the lot, you will know enough to pass the exam. to prove that, this is everything I learnt to pass the exam myself, late July 2017.

Contents:
The 2 portals/models
Virtual Networks
IP Addresses
Network Security Groups (NSG’s)
Route tables and IP Forwarding
VNET Peering
Azure Resource Groups
Azure Compute
Azure App Service
Azure Cloud Services
Azure Service Fabric
Azure Virtual Machines
ARM Templates
Windows Azure PowerShell
Automation
Traffic Manager
Using load-balancing services together in Azure
Azure Media Services
Azure Content Delivery Network (CDN)
Active Directory
Azure Active Directory
Authentication method: OAUTH/OPENID
Authentication method: SAML
When to choose SAML, OAUTH or OPENID
AD FS aka ADFS
Azure AD B2B & B2C
AD Application proxy
Graph API
Azure Redis Cache
Azure Multi-Factor Authentication
Azure Service Bus
Encryption
RBAC (Role Based Access Control)
Azure Data Storage
SQL Database
Azure SQL Data Warehouse
DocumentDB (now Azure Cosmos DB)
High Performance Computing
Azure Scheduler
Azure Machine Learning
Azure Search
Disaster Recovery/Business Continuity
Azure site recovery
Azure monitoring
Azure Functions
IoT Hub
Azure Notification Hub
Azure Stream Analytics
API Management
Architect for availability and reliability 

The 2 portals/models

• Azure Classic Portal, http://manage.windowsazure.com Azure Services Model (ASM)
• Azure New Portal, http://portal.azure.com Azure Resource Model (ARM), can use RBAC
• Use the new portal wherever you can.

Virtual Networks

• Azure Virtual Network, often called a VNET
• Setup and configured by customers
• A VM cannot be created without connecting it to a VNET
• VNETS contain one or more subnets
• A VNET cannot span regions
• IPv6 is supported in Azure
• Starting limit of 50 VNETS per subscription, can be upped to 500 upon request (for ARM).
• Access to subnets controlled by NSG’s (Network Security Groups)
• VNETS can be connected to each other with VNET peering or to your on-premises network using a VPN
• In Azure, the smallest subnet is /29 (8 addresses) and the largest is /8 (16 million address). Common ones are in Bold:

		Addresses	Hosts	Netmask	Amount of a Class C
	/30	4	2	255.255.255.252	1/64
	/29	8	6	255.255.255.248	1/32
	/28	16	14	255.255.255.240	1/16
	/27	32	30	255.255.255.224	1/8
	/26	64	62	255.255.255.192	1/4
	/25	128	126	255.255.255.128	1/2
Common Subnet size	/24	256	254	255.255.255.0	1
	/23	512	510	255.255.254.0	2
	/22	1024	1022	255.255.252.0	4
	/21	2048	2046	255.255.248.0	8
	/20	4096	4094	255.255.240.0	16
	/19	8192	8190	255.255.224.0	32
	/18	16384	16382	255.255.192.0	64
	/17	32768	32766	255.255.128.0	128
Common VNET size	/16	65536	65534	255.255.0.0	256

• Typical VNET topology:
  

IP Addresses

• Public IP’s are used for VM’s, Load Balancers, Application Gateways and VPN Gateways
• Private IP’s are used for VM’s, Load Balancers, Application Gateways
• IP address are dynamic by default, you can set static IP’s.
• Limits:
  • 4096 private IPs per VNET
  • 60 public dynamic IP’s
  • 20 public static IP’s
  • Public Static IP’s cost: first 5 are free, rest cost $0.004/hr

Network Security Groups (NSG’s)

• NSG’s are virtual firewalls and contain a list of rules
• NSG’s are stateful
• NSG’s can be associated with NIC’s (VM’s) or subnets
  
• When associated to a subnet, the rules apply to all the VM’s inside that subnet
• The lowest level at which a network security group is restricted is the region, i.e. all resources in an NSG must be in the same region.
• Rules processed in the order of priority. Rule with lower priority is processed before rules with higher priority numbers.
• Default rules for a NSG
  • Allows all traffic within the network
  • Allows all traffic outbound
  • Allows load balancer traffic to come inbound
  • Denys all other traffic
  • This is shown by these default rules:
    
• To create a new inbound rule on a NSG:
  • Give it a name: e.g. web
  • A priority: e.g. 100
  • Source: Any, CIDR (Classless Inter-Domain Routing) block or Tag
  • Service: e.g. HTTP
  • Protocol: Any, TCP, UDP
  • Action: Allow or Deny
• Endpoint-based ACL’s
  • Endpoint-based ACL’s provides the ability to selectively permit or deny traffic for a virtual machine endpoint.
  • ACL’s only work on VM public port endpoints.
  • You can’t have both an NSG and an endpoint ACL applied to the same VM.
  • Microsoft says It is recommended to use Network Security Groups (NSGs) instead of ACLs whenever possible.

Route tables and IP Forwarding

• To force traffic via a route, e.g. stop web traffic from an Azure VM going straight to the internet, but go via an express route to your datacentre then out to the internet.
• In this example, you would create a new ‘Route table’ and add a route like this. 0.0.0.0/0 is the ‘default route’. All packets for destinations not established in the routing table are sent via the default route.

• Then associate a subnet
• Another reason to use Route table and IP forwarding, would be if you deploy a marketplace firewall virtual appliance (instead of using NSG’s), then you will need to create a route to make all traffic go to via that virtual firewall.

VNET Peering

• VNET Peering enables you to connect two VNETS in the same region through the Azure backbone network. There are no bandwidth restrictions,
• The two VNETS are still managed as separate resources, but VM’s in the peered virtual networks can communicate with each other directly by using private IP addresses.
• The peered virtual networks must have non-overlapping IP address spaces.
• No transitive relationship across peerings. For example, if VNET A is peered with VNET B, and VNET B is peered with VNET C, VNET A is not peered to VNET C.
• You can peer virtual networks that exist in two different subscriptions as long a privileged user of both subscriptions authorises the peering and the subscriptions are associated to the same Active Directory tenant.
• Cost: Ingress and egress traffic is charged at both ends of the peered networks.

Meter	Price
Inbound data transfer	$0.013 per GB
Outbound data transfer	$0.013 per GB

• If you want to connect two VNET’s in different regions use a VNET-to-VNET VPN.

Azure Resource Groups

• Only available in the new portal
• Resource Groups allow you to operate on groups of VM’s at once.
• Allows Access control assigned to the Resource group and billing
• A resource can only be a member of 1 Resource Group
• You can’t nest Resource Groups
• You can link resources into a group, to associate a resource from another group with this group
• No limits to the number of resources in a group
• You can’t rename a resource group

Azure Compute

• Azure at a high level consists of 4 compute services:

Compute Options	Type	Azure model	Audience
Web Apps/App Service	PaaS	ARM	Scalable Web Apps, Mobile Apps, API Apps, and Logic Apps for any device
Cloud Services	PaaS	ASM (legacy)	Highly available, scalable n-tier cloud apps with more control of the OS than App Service
Service Fabric	PaaS	ARM	Microservices architecture, recommended for new development over Cloud Services
Virtual Machines	IaaS	ASM & ARM	Customised Windows and Linux VMs with complete control of the OS

Azure App Service

• These services you can’t RDP to the underlying servers
  
• API Apps – Easily build and consume API’s in the cloud
  • API apps in Azure App Service offer features that make it easier to develop and host APIs in the cloud.
  • Easy migration of existing API into Azure API Apps
  • Set up for easy consumption (CORS) Cross Domain
  • Integrated with Visual Studio
  • Integration with Logic Apps
  • Difference between API Apps and Azure API Management, they are complementary services:
    • API Management is about managing APIs. You put an API Management front end on an API to monitor and throttle usage, manipulate input and output, consolidate several APIs into one endpoint, and so forth. The API’s being managed can be hosted anywhere.
    • API Apps is about hosting APIs. The service includes features that facilitate developing and consuming APIs, but it doesn’t do the kind of monitoring, throttling, manipulating, or consolidating that API Management does. If you don’t need API Management features, you can host APIs in API apps without using API Management.

• Logic Apps – Automate business process across SaaS and on-premises (equivalent AWS service is Simple Workflow Service (SWF))
  • Build workflow logic, based on a trigger
  • E.g. If a tweet mentions this word then do that
  • Requires no coding, can be designed inside the browser
  • Start with some kind of trigger, or date/time
  • Trigger causes some action to happen
  • User can start with pre-existing supplied templates
• Web Apps – Web apps that scale with your business
  • 
  • Is PaaS
  • Shared or dedicated VM’s
  • Supports multiple languages .NET, ASP.NET, Java, Node.js, PHP, Ruby, Python, PowerShell and other scripts
  • Supports CI (Continuous Integration) using MS Visual Studio and other Dev platforms
  • Global auto-scaling, high availability
  • 3 methods for deploying web apps code to Azure:
    • FTP or FTPS
    • Kudu (Open source project), it’s a Git repository clone, Kudu provides version control, package restore, MSBuild, and web hooks for continuous deployment and other automation tasks. The Kudu deployment engine supports 3 different types of deployment sources:
      • Content sync from OneDrive and Dropbox
      • Repository-based continuous deployment with auto-sync from GitHub, Bitbucket, and Visual Studio Team Services
      • Repository-based deployment with manual sync from local Git
    • Web Deploy, built into Visual Studio .NET, Web Deploy differs from Kudu in that application binaries are built before they are deployed to Azure.
  • Application templates – Choose from an extensive list of application templates in the Azure Marketplace that let you use a wizard to install popular open-source software such as WordPress, Joomla, and Drupal.
  • Visual Studio integration – Dedicated tools in Visual Studio streamline the work of creating, deploying, and debugging.
  • Site extensions feature allows developers to essentially write “apps” that can be run on an Azure WebApp to add administrative functionality to it. These extensions can also be published to the Site Extensions Gallery, which allows others to install and use the extension as well.
  • WebJobs– On a Web App, run code as background tasks on demand, continuously, or on a schedule (either Azure Scheduler or CRON).
    • A WebJob is designed to be long-running code or a background task without a user interface
    • Can use the WebJobs SDK
    • The following file types are accepted:
      • .cmd, .bat, .exe (using windows cmd)
      • .ps1 (using powershell)
      • .sh (using bash)
      • .php (using php)
      • .py (using python)
      • .js (using node)
      • .jar (using java)
    • To stop a web job from being hibernated, enable the ‘AlwaysOn’ setting.
• Mobile Apps – Build Mobile apps for any device
  
  • Use the Azure SDK to take advantage of Azure Mobile App Services
  • Use custom code
    • An SDK for .NET and Node.js
    • REST API for other languages including PHP, Java and Python
  • Supports all the popular mobile OS’s and Rest API
    • Windows
    • iOS
    • Android
    • HTML5/JavaScript
    • Xamarin (allows developing iOS, Android and Windows Phone apps using C#)
    • Apache Cordova (allows developing iOS, Android and Windows Phone apps using CSS3, HTML5 and JavaScript)
  • Supports Azure AD FS (Active Directory Federation Services)
  • Push Notifications service supporting all the popular mobile OS’s notifications systems
  • Supports staging environments
  • The same CI feature like Web Apps
  • Offline sync feature:
    • Allows mobile client apps to store and retrieve data locally until internet access is online
    • Offline sync uses the concept of ‘local store’ a data storage location on the mobile device that the App syncs when it has internet access
• App Service Plans
  • All App Services run under a shared App Service plan. An App Service plan represents the collection of physical resources used to host your apps.
  • This sharing saves money when hosting multiple apps in a single App Service plan.
  • App Service plans define:
    • Region (West US, East US, etc.)
    • Scale count (one, two, three instances, etc.)
    • Instance size (Small, Medium, Large)
    • Plan (Free, Shared, Basic, Standard, Premium)
  • App service tiers, check this hasn’t updated https://azure.microsoft.com/en-au/pricing/details/app-service/plans/
    NEED TO REMEMBER THIS AS BEST YOU CAN FOR THE EXAM
    
  • Both apps and plans are contained in a resource group. A resource group serves as the lifecycle boundary for every resource that’s within it.
  • Standard and Premium tiers have Automated Backups. If you build stateless apps though you shouldn’t need backups as all data should be stored off the actual App servers.
  • You can scale up your tier to get more CPU, memory, disk space, and extra features like dedicated virtual machines (VMs), custom domains and certificates, staging slots, auto-scaling, and more. This benefits all the Apps running in your App Service Plan.
  • You can scale out your instances to increase the number of VM instances that run your apps in the Basic, Standard and Premium tiers.
  • Auto-scaling to increase how many instances can be done in Standard and Premium plans, choose a metric and the app can scale based on that.
    
• Deployment Slots
  • Before deploying a new or updated app to production, you can stage it.
  • Test it, and then when ready for go live “flip” it over
  • Doesn’t copy any files, Azure just swaps the Virtual IP addresses of the source and destination slots, thereby swapping the URLs of the slots.
  • If it goes wrong you can just flip it back
  • Each deployment slot is like a full-fledged App Service instance.

• • You can do A/B testing by routing a small % of traffic to the staging slot
  • The staging slot before flipping over can be ‘warmed up’, Azure does this by firing a couple requests into the root of the slot.
  • It also applies certain settings that are specific to the target slot:
    • Publishing endpoints
    • Custom Domain Names
    • SSL certificates and bindings
    • Scale settings
    • WebJobs schedulers
    • Site extensions

Azure Cloud Services

• This is a classic ASM service and new deployments should use the ARM App Service instead.
• You can RDP to the underlying servers on this service
  
• Equivalent on AWS is Elastic Beanstalk
• Like App Service, this is PaaS but runs on VM’s you can see and you can RDP to it and install other software on the service
• All the VM’s in a single application run in a single cloud service
  
• Web Role –
  • Can have a public endpoint
  • IIS running in a VM
  • Accepts HTTP and HTTPS requests
• Worker Role –
  • Computation and data management
  • Background jobs
  • Can accept and perform tasks
  • No IIS
  • No public endpoints
• Web Role and Worker Roles –
  • Communicate using messages and queues
  • Can have multiple instances of each role
  • Can auto-scale and automatically load-balance
• You don’t create the VM’s, you just upload your code and Cloud Service creates it all for you
• Azure looks after the OS updates for you

Azure Service Fabric

• Azure Service Fabric is a distributed systems platform that makes it easy to package, deploy, and manage scalable and reliable microservices.
• The main 2 benefits of the microservices approach are:
  • You can scale different parts of your application depending on the needs of the application.
  • Development teams can be more agile as they roll out changes and thus provide features to customers faster and more frequently.
• You can create clusters for Service Fabric in many environments, including Azure or on premises, on Windows Server, or on Linux.
  
• To use Service Fabric (which are microservices), you need to design your application completely differently. Break your monolithic app into microservices e.g. a login authentication service, a service that processes PDF’s, a service that handles user profile changes.
• Comparing Azure Cloud Services to the Azure Service Fabric micro-services architecture:
  

Azure Virtual Machines

• Is IaaS (Infrastructure As A Service)
  
• You can choose from predefined images from the ‘Azure Marketplace’, Windows, Linux, Oracle Linux
• VM’s use virtual hard disks (VHD’s) which are stored in a storage account blob page storage
• Maximum VM’s per Azure subscription is 10,000 per Region.
• VM sizes:
  • A0 is the cheapest with 1 CPU, 750Mb RAM, 20GB Disk
  • The “S” in the name designates “Premium Storage”
  • Hosting tiers: Basic vs Standard
    • Basic: is for dev workloads, no load-balancing, auto-scaling, or memory intensive VM’s
    • Standard: is more flexible, supports all VM configurations and features

Type	Sizes	Notes	Description
General purpose	A0-7, Av2 D, Dv2, DS, DSv2,	A came first then DS with newer chipset	Balanced CPU-to-memory ratio. Ideal for testing and development, small to medium databases, and low to medium traffic web servers.
Compute optimized	F, FS	F for Fast	High CPU-to-memory ratio. Good for medium traffic web servers, network appliances, batch processes, and application servers.
Memory optimized	G, GS, DS, DSv2	Great for databases as high memory	High memory-to-core ratio. Great for relational database servers, medium to large caches, and in-memory analytics.
Storage optimized	LS	Loadsa Storage	High disk throughput and IO. Ideal for Big Data, SQL, and NoSQL databases.
GPU	NV, NC	Think N for Nvidia (Graphics)	Specialized virtual machines targeted for heavy graphic rendering and video editing. Available with single or multiple GPUs.
High performance compute	H, A8-11	High performance	Our fastest and most powerful CPU virtual machines with optional high-throughput network interfaces (RDMA).

• Single VM instance if all disks use Premium Storage gives Azure guaranteed uptime SLA of 99.9%. Which is 44min a month downtime.
• Fault Domains (FD) is essentially a rack of servers. It consumes subsystems like network, power, cooling etc. So 2 VMs in the same availability set means Azure will provision them in to 2 different racks. The maximum number of fault domains varies by region, but it is between 2 and 3.
• Update Domains (UD) are when Microsoft must patch the underlying Hosts. Default is 5 update domains and maximum is 20.
  
• Availability sets – Two or more identical VM’s running under a load balancer, such that if one fails, the others can pick up the load. Assigning VM’s to an availability set will spread them across Fault domains and Update domains automatically to ensure minimum accidental or required downtime. Availability sets gives an Azure guaranteed uptime SLA of 99.95%. Which is 22min a month downtime.
• Use Managed disks for VM’s in an availability set, as it automatically places the disks in different storage clusters providing better reliability.
• Scale Sets
  • Concept of scaling – scaling up (more CPU’s more RAM) vs scaling out (more VM’s), Scale sets are scaling out
  • Scale Sets are like AWS Auto-scaling groups,
  • VM’s in a scale set must be configured the same,
  • Scale operations are balanced across fault and update domains,
  • you can use any standard ARM deployment methods
  • You can manually scale the group of VM’s out and in using the Portal, Azure PowerShell or Azure CLI
  • You can autoscale based on these metrics:
    • Average % CPU
    • Total bytes Network In and Out
    • Total bytes Disk Read and Write
    • Average Disk Read and Write IOPS
  • You can autoscale with a schedule based autoscale (this is planned scaling) and webhooks with alert systems
  • Azure Load Balancers, and Application Gateway integrate seamlessly with scale sets.
  • You can perform management operations on both scale sets and individual VMs within a scale set.
  • The Azure portal provides a customisable resource usage graph.
  • Individual VMs in the scale set are not allocated public IP addresses as it would just get too expensive
  • A scale set supports up to 1,000 VMs. The limit is 100, If you create and upload your own custom VM images.
  • If 0-100 size scale sets are created, all VM’s go in 1 placement group, for each additional 100 VM’s they go in placement groups, with its own fault domains and upgrade domains.
  • To store data on VM’s in a scale set you can store on:
    • Attached data disks
    • Azure files (SMB shared drives)
    • OS drive
    • Temp drive (local, not backed by Azure Storage)
    • Azure data service (for example, Azure tables, Azure blobs)
    • External data service (for example, remote database)
  • Higher level Azure services such as Batch, Container Service, and Service Fabric also use scale sets as their underlying infrastructure in order to simplify large scale deployments and make it easier to deploy cloud scale architectures.

ARM Templates

• Deploy Azure infrastructure as code, ARM templates are in JSON (JavaScript Object Notation)
• ‘Azure Quickstart Templates’ is an ARM Template library: https://azure.microsoft.com/en-us/resources/templates/
• Also ARM Templates are available on GitHub https://github.com/Azure/azure-quickstart-templates
• On the new portal, you can click on a resource and click ‘Automation script’ to see the ARM template code (JSON), PowerShell, .NET, Ruby or CLI code for that resource:
  
• You can also write ARM templates in Visual Studio and deploy straight to Azure.

Windows Azure PowerShell

• Azure PowerShell provides a set of cmdlets that use the Azure Resource Manager (ARM) model for managing your Azure resources. Even has some options that are not available in the portal.
• Azure PowerShell is only available for Windows, you can download it from https://azure.microsoft.com/en-au/downloads/
• Some example Azure PowerShell commands:
  • Login-AzureRmAccount < Log in to Azure
  • New-AzureRmResourceGroup < Create a resource group
  • New-AzureRmVM < create a new ARM VM
  • New-AzureRmLoadBalancer < Create an Azure Network Load Balancer
  • Remove-AzureRmVM < delete a VM
• All the ‘Rm’ in the above commands show these are ARM commands not classic ASM
• Also available is the Azure CLI, now in 2.0 version which has ARM support. It is available for Windows, Linux and Mac, it is not as good for scripting multiple actions, but is great for developers that prefer other OS’s.

Automation

• Azure Automation
  • Microsoft Azure Automation provides a way for users to automate the manual, long-running, error-prone, and frequently repeated tasks that are commonly performed in a cloud and enterprise environment.
  • In Automation, Windows PowerShell scripts and workflows are known as Runbooks
  • Azure Automation Runbook Gallery puts samples, utilities, and scenario runbooks at your fingertips, so that you can get up and running quickly

Type	Description
Graphical	Based on Windows PowerShell and created and edited completely in graphical editor in Azure portal.
Graphical PowerShell Workflow	Based on Windows PowerShell Workflow and created and edited completely in the graphical editor in Azure portal.
PowerShell	Text runbook based on Windows PowerShell script.
PowerShell Workflow	Text runbook based on Windows PowerShell Workflow.

• • Using ‘Hybrid Runbook Worker’, runbooks can run in your local data centre to manage local resources.
  • Author and manage PowerShell Desired State Configuration (DSC) to automate configuration management and allow you to deploy, manage and enforce configuration of your cloud resources stay the same
• Chef
  • Used for Cloud and infrastructure automation
  • A powerful but can be complex scripting language
  • Chef is open source
  • Available for Azure, AWS, Google Cloud and VMware
  • Chef has Recipes which are pre-defined scripts written in Ruby
  • Chef has Cookbooks which contain Recipes and all the attribute values and file distributions
• Puppet
  • Used for Cloud and infrastructure automation
  • A powerful but can be complex scripting language
  • Puppet is Open source
  • Available for Azure, AWS, Google Cloud and VMware + more
  • Puppet has Manifests and Modules

VPN and ExpressRoute

• 2 types of VPN (Virtual Private Network)
  • Point-to-site (P2S)
    • establish a secure tunnel from your computer to your corporate network or Azure
    • VPN client your end using SSTP (Secure Socket Tunnelling Protocol) which does not require physical hardware VPN device, and VPN gateway the other end.
  • Site-to-site (S2S)
    • VPN can also securely connect the networks of 2 offices together, or your corporate office to Azure
    • VPN gateway devices at both ends using IPsec (IKE v1 and IKE v2)
    • 100Mbs or 200Mbs to Azure
• ExpressRoute
  • Private fibre connection to Azure from your datacentre or your co-location facility, doesn’t travel over the public internet.
  • Can be a MPLS end-point
  • Port speed 500mbs up to 10Gbps
  • Metered or Unlimited Data plans
  • The ‘Premium’ add-on benefits:
    • Increased route limits for peering, from 4,000 to 10,000 routes
    • Global connectivity for services, connect via one region and access all others in the world
    • Increased number of VNET links per ExpressRoute from 10 to a larger limit
  • ExpressRoute has a 99.95% availability guarantee

DNS Basics

• A record – points a domain or subdomain to an IP address
• CNAME – points a domain or subdomain to another hostname
• MX record – specifies which email server(s) to deliver email to
• NS record – specifies the NameServer for the domain

Azure Load Balancer types

• Load Balancer – Transport layer TCP, UDP (Layer 4). It provides network-level distribution of traffic across instances of an application running in the same Azure data centre.
• Application Gateway – Application layer (Layer 7). provides application delivery controller (ADC) as a service, offering various Layer 7 load-balancing capabilities for your application.
• Traffic Manager – DNS level (DNS is an application so is layer 7). Provides global DNS load balancing. It looks at incoming DNS requests and responds with a healthy endpoint, clients then connect to those endpoints directly.

Service	Azure Load Balancer	Application Gateway	Traffic Manager
Technology	Transport level (Layer 4)	Application level (Layer 7)	DNS level (Layer 7)
Application protocols supported	Any	HTTP, HTTPS, and WebSockets	Any (An HTTP endpoint is required for endpoint monitoring)
Internal VNET support	Can be used for both Internet facing and internal (VNET) applications	Can be used for both Internet facing and internal (VNET) applications	Only supports Internet-facing applications
Endpoints	Azure internal IP’s only	Any IP address – internal or external	Any IP address – internal or external
Endpoint Monitoring	Supported via probes	Supported via probes	Supported via HTTP/HTTPS GET
SSL offloading	Not Supported	Supported	Not Supported

Load Balancer

• Layer 4 (TCP, UDP) load balancer
• Internet facing load balancer or internal load balancer
  
• 3 types of probes supported
  • Guest agent probe (on Platform as a Service Virtual Machines only): The load balancer utilises the guest agent inside the VM. The guest agent listens and responds with an HTTP 200 OK response only when the instance is in the ready state.
  • HTTP custom probe: The load balancer will regularly probe your endpoint (every 15 seconds, by default). The instance is considered to be healthy if it responds with a TCP ACK or HTTP 200 within the timeout period (default of 31 seconds).
  • TCP custom probe: This probe relies on successful TCP session establishment to a defined probe port.

Application Gateway

• Layer 7 (application) layer load balancer used for HTTP/HTTPS/WebSocket traffic only
• Application Gateway can be configured as an Internet-facing gateway, an internal-only gateway, or a combination of both.
• It provides a rich set of diagnostics and logging capabilities
• Load balancing modes:
  • Round Robin
  • URL-based content routing, allow different web folders to go to different groups of back-end servers
• Has a Web Application Firewall (WAF) to protect against common web-based attacks
• SSL offload, so you install the SSL certs on the gateway and not on the web server layer, reduces CPU load for the web servers
  

Traffic Manager

• Simular to Route 53 in AWS
• Routes across multiple deployments in different regions
• Only supports Internet-facing applications
• Checks on health of a VM or load balancer
• Remember the traffic doesn’t go through the traffic manager
  
• Configure endpoint monitoring
  • To configure endpoint monitoring, you must specify the following settings on your Traffic Manager profile:
  • Protocol. Choose HTTP or HTTPS. It’s important to note that HTTPS monitoring does not verify whether your SSL certificate is valid–it only checks that the certificate is present.
  • Port. Choose the port used for the request.
  • Path. Give the relative path and the name of the webpage or file that the monitoring accesses. A forward slash (/) is a valid entry for the relative path. This value implies that the file is in the root directory (default).
• There are four traffic routing methods available in Traffic Manager:
  • Priority: when you want to use a primary service endpoint for all traffic, and provide backups in case the primary or the backup endpoints are unavailable.
  • Weighted: when you want to distribute traffic across a set of endpoints, either evenly or according to weights, which you define. The weight is an integer from 1 to 1000. This parameter is optional. If omitted, Traffic Managers uses a default weight of ‘1’. Using a higher weight means more frequent traffic.
  • Performance: when you have endpoints in different geographic locations and you want end users to use the “closest” endpoint in terms of the lowest network latency.
  • Geographic: users are directed to specific endpoints (Azure, External or Nested) based on which geographic location their DNS query originates from. Examples include complying with data sovereignty mandates, localisation of content & user experience and measuring traffic from different regions.
• ‘Nested profiles’ are possible to combine two different load balancing methods. For example, you can have the Geographic routing method be top level, and then Priority routing method below that.

Using load-balancing services together in Azure

• We can use Traffic Manager, Application Gateway and Load Balancer together to form a multi-geo, auto-scaling, load-balanced web site like this:
  

Azure Media Services

• Media Ingest, encoding, format conversion, content protection, media analytics, and both on-demand and live streaming capabilities using an integrated content delivery network.
• Build end to end workflows.
• Converting existing media files:
  
• Live streaming:
  

Azure Content Delivery Network (CDN)

• Hosts static web content closer to users E.g. audio, video, images
• Origin servers are the source, Edge servers are the CDN
• There are three Azure CDN products:
  • Azure CDN Standard from Akamai – Least features
  • Azure CDN Standard from Verizon
  • Azure CDN Premium from Verizon – most features
    Full comparison chart here https://docs.microsoft.com/en-us/azure/cdn/cdn-overview

Active Directory

• AD parts:
  • AD DS – Domain Services, how employees log into Windows
  • AD LDS – Lightweight Directory Services
  • AD CS – Certificate Service, public key infrastructure (PKI)
  • AD FS – Federation Services, for single sign-on often with external SaaS products
  • AD RMS – Rights Management Services, for protecting documents
• Azure AD does not have all the above features AD has.
• There are two options for running AD in Azure:
  • Built in Azure AD
  • Windows VM’s running AD

Azure Active Directory

• Azure AD –
  • Is a subset of the features of regular AD,
  • Is a flat structure mainly used for identity verification,
  • Has 99.9% uptime,
  • Could be labelled IDaaS, (IDentity As A Service)
  • Windows 10 + devices can ‘Azure AD join’
  • Editions available Free, Basic, Premium P1 and Premium P2, feature comparison chart here https://www.microsoft.com/en-cy/cloud-platform/azure-active-directory-features
  • Azure AD does support:
    • Users and groups
    • Computers and devices
    • SAML
  • Azure AD does not support:
    • LDAP
    • Kerberos
• Azure AD Domain Services – is a managed domain service providing domain join, group policy, LDAP, Kerberos/NTLM authentication that is fully compatible with Windows Servers/clients.
• The whole point of AAD DS is that it removes the need for you to build and manage your own domain controllers on the cloud.
  
• Azure AD Connect – will integrate your on-premises directories with Azure Active Directory. This allows you to provide a common identity for your users for Office 365, Azure, and SaaS applications integrated with Azure AD. AADSync and DirSync are older methods and aren’t supported after April 2017.

Authentication method: OAUTH/OPENID

• OAuth is typically used for Internet applications/websites, it works only over HTTP
• OAuth is an open standard for authentication with untrusted clients like a mobile phone
• A trusted client would be a server where there is no way a user can see the back-end source code or modify it.
• Allows developers to build integrations into our backend network without the trust in the client,
• The application requests a token from an identity provider, they do not ever see your password or need to know it.
  
• OpenID was created for federated authentication, that is, letting a third-party authenticate your users for you, by using accounts they already have.

Authentication method: SAML

• Is typically used in Enterprise SSO scenarios and is not just restricted to over HTTP
• Azure Active Directory (Azure AD) supports for Single Sign-On via SAML (Security Assertion Markup Language).
• SAML claims are a way an identity is verified when you don’t have a trust, and usually over the internet you don’t.
• SAML works on a token system. A user authenticates with one system, and that system gives them a token. The user then presents that token to another system, and after that system verifies the token is valid, it accepts the user as authenticated.
  

When to choose SAML, OAUTH or OPENID

• Following are the points which can be useful to consider which one to use among OpenID, OAuth or SAML or any of their combination:
  • If the use case is to develop SSO where at least one partner is enterprise use SAML, otherwise use OpenID.
  • If the use case involves mobile devices for API authorization then use OAuth.
  • If use case requires a centralised identity provider the use SAML.

AD FS aka ADFS

• Active Directory Federation Services
• Federation with Azure AD enables users to authenticate using on-premises credentials and access 3^rd party applications often SaaS apps via single sign-on.
• AD FS can be installed on on-prem servers/VM’s or on Azure VM’s.
• ADFS aims to reduce the complexity around password management and guest account provisioning, and it has taken on additional importance as organisations and employees rely more on Software as a Service (SaaS) and Web applications.
• Once a user logs in with his or her Windows credentials, AD Federation Services authenticates access to all approved third-party systems.
• ADFS uses SAML

Azure AD B2B & B2C

• Azure AD B2B (Business to Business) lets you enable access to your corporate applications from partner-managed identities. You can create cross-company relationships by inviting and authorising users from partner companies to access your resources.
• Azure B2B supports most of the federation features of ADFS, so can be used to replace ADFS running on Windows servers on-prem or on Azure VM’s.
• Azure AD B2B allows you to authenticate corporate users from other companies.
  
• Azure AD B2C (Business to Consumer) is a cloud identity management solution for consumer-facing web and mobile applications, it allows you to authenticate customers using social identities of their choice, but still within the context of Azure AD, supports:
  • Facebook
  • Google
  • Amazon
  • LinkedIn
  • Microsoft
  • Yahoo
  • OpenID
  • Azure AD
  • Windows AD via Azure FS (Federated Services)
  • And more
• So when used, your apps login page could present additional login options like this:
  
• Azure AD B2C is not targeted at other organisation users but consumers so B2C and B2B complement each other.

AD Application proxy

• Azure AD Application Proxy helps you support remote workers by publishing on-premises web applications or applications hosted behind a Remote Desktop Gateway so they can be accessed over the internet.
• Could be a SharePoint server or OWA (Outlook Web Access) hosted on-prem
• This means you don’t need a VPN connection or DMZ hosted web servers.
• You need to install a connector inside your network
• The end users authenticate with the Application proxy and all traffic goes through it.
  
• Setting up Application Proxy is accomplished in two steps:
  1. Enable Application Proxy and configure the connector.
  2. Publish applications – use the quick and easy wizard to get your on-premises apps published and accessible remotely.

Graph API

• Graph API allows developers to build web and mobile apps that integrate with any platform
• REST based API
• Azure Active Directory Graph API provides programmatic access to Azure AD to perform queries and create, read, update, and delete (CRUD) operations on directory data and objects.
• You must present a valid authentication token with each REST request in the Authentication header

Azure Redis Cache

• Redis cache managed by Microsoft and accessible from any application within Azure.
• Place in front of an Azure database to speed it up with Redis in-memory caching
• Available in the following tiers:
  • Basic – Single node, multiple sizes, ideal for development/test and non-critical workloads. The basic tier has no SLA.
  • Standard – A replicated cache in a two-node primary/secondary configuration managed by Microsoft, with a high-availability SLA.
  • Premium – All of the Standard tier features, better performance over Basic and Standard-tier caches, bigger max size (530GB), Redis Cluster to give a high-availability SLA, Redis persistence snapshots and back up to provide disaster recovery, and supports VNet’s to give enhanced security.
  • Full tiers comparison table: https://azure.microsoft.com/en-au/pricing/details/cache/
• Basic and Standard caches are available in sizes up to 53 GB, and Premium caches are available in sizes up to 530 GB with more on request.

Azure Multi-Factor Authentication

• Two-step verification is a method of authentication that requires more than one verification method and adds a critical second layer of security to user sign-ins and transactions.
• You should turn it on for Azure Cloud Admins as best practice
• You can build it into your application
• All based on you having your phone as your second form of authentication
  • Soft token App on phone – uses ‘Microsoft Authenticator App’
  • Text you
  • Call you
• multi-factor authentication for Azure Administrators is free
• Azure Multi-Factor Authentication “full” version for all your users is paid for:
  • Is included in Azure Active Directory Premium (P1 and P2 plans)
  • or, costs via per-user consumption or per-authentication consumption based.

Azure Service Bus

• Application messaging queue, like AWS Simple Queue Service (SQS)
• Service Bus supports two distinct messaging patterns:
  • Service Bus Relay
    • Requires the server and client to both be online at the same time to send and receive messages
    • Supports SOAP, WS-*, and REST,
    • Supports traditional one-way messaging, request/response messaging, and peer-to-peer messaging.
  • Service Bus messaging or Brokered messaging
    • Producers (senders) and consumers (receivers) do not have to be online at the same time.
    • Stores messages in a queue until the consuming party is ready to receive them.
    • Core components are queues, topics, and subscriptions.
    • A ‘namespace’ is a scoping container for all messaging components. Multiple queues and topics can reside within a single namespace, and namespaces often serve as application containers.

• Service Bus comes in Basic, Standard and Premium tiers, comparison table and pricing here: https://azure.microsoft.com/en-gb/pricing/details/service-bus/

Azure Storage Queue

• Storage queues, which are part of the Azure storage infrastructure, feature a simple REST-based Get/Put/Peek interface, providing reliable, persistent messaging within and between services.
• Storage Queues are like the Azure Service Bus, but Storage queues were introduced first, as a dedicated queue storage mechanism built on top of Azure Storage services.
• Service Bus queues were built on top of the broader “messaging” infrastructure designed to be far more integrated.
• How it works:
  • Code pulls a message from the queue with the GetMessage command, by default, this message stays invisible for 30 seconds so others don’t read it
  • When the code has finished its task with the message, it deletes the message from the queue with the DeleteMessage command.
• Maximum message size 64 KB, Maximum queue size 500TB.
• You should choose Azure Store Queues over Azure Service Bus for these 3 reasons, otherwise for messaging queue requirements, always select the more modern Azure Service Bus:
  • Your application must store over 80 GB of messages in a queue, where the messages have a lifetime shorter than 7 days.
  • Your application wants to track progress for processing a message inside of the queue
  • You require server-side logs of all the transactions executed against your queues.

Encryption

• Secure data in transit
  • By using HTTPS with SSL
  • By using a VPN
  • By using ExpressRoute
  • Client-side encryption, if you encrypt it before you send it to Azure
• Secure data at rest
  • Enable the Azure ‘Storage Service Encryption’ (SSE) on your storage account to encrypt Azure Blob Storage and File Storage, 256-bit AES encryption. SSE only encrypts newly created data after the encryption is enabled. All encryption keys are stored, encrypted, and managed by Microsoft. SSE works for the following:
    • Standard Storage: General purpose storage accounts for Blobs and File storage and Blob storage accounts
    • Premium storage
    • All redundancy levels (LRS, ZRS, GRS, RA-GRS)
    • Azure Resource Manager storage accounts (but not classic)
    • All regions.
  • ‘Azure Disk Encryption’ is a different encryption service that only encrypts the virtual disks used by VM’s. For Windows VM’s uses BitLocker, for Linux VM’s uses DM-Crypt. Keys are kept in the ‘Azure Key Vault’.
  • Azure Rights Management (RMS) Is a service that encrypts sensitive files and data, and provides ways to access that encrypted data for authorised and authenticated users. It is not an Azure specific service, it is used across O365, mobile devices, Windows file servers, AD, Exchange and SharePoint servers.
  • Client-side encryption, encryption all outside Azure, encrypt it on the client and then store it on Azure

RBAC (Role Based Access Control)

• Allows fine-grained access control, grant Roles to Users, Groups or Application
• Three Azure RBAC roles that are used very commonly are:
  • Owner has full access to all resources including the right to delegate access to others.
  • Contributor can create and manage all types of Azure resources but can’t grant access to others.
  • Reader can view existing Azure resources.
• 38 in total ‘RBAC built-in roles’ by Azure
• You can create a custom role
• Access is inherited down e.g. Subscription > Resource Group > VM
• RBAC assignment stops at the Resource level, you can’t assign RBAC to a single table or blob
• Remember an Azure subscription can only belong to one Azure AD

Azure Data Storage

• Storage Accounts
  
  • General-purpose Storage Accounts
    • Gives you access to Azure Storage services such as Tables, Queues, Files, Blobs and Azure virtual machine disks under a single account. This type of storage account has two performance tiers:
      • A standard storage performance tier which allows you to store Tables, Queues, Files, Blobs and Azure virtual machine disks.
        Maximum total request rate of 20,000 IOPS.
      • A premium storage performance tier which currently only supports Azure virtual machine disks.
        Maximum total throughput rate of 50 Gbps.
  • Blob Storage Accounts
    • Specialised storage account for storing your unstructured data as blobs (objects) in Azure Storage
    • Blob storage accounts expose the Access Tier attribute which can be specified during account creation and modified later as needed. Two types:
      • A Hot access tier which indicates that the objects in the storage account will be more frequently accessed. This allows you to store data at a lower access cost.
      • A Cool access tier which indicates that the objects in the storage account will be less frequently accessed. This allows you to store data at a lower data storage cost.
• Table Storage
  • NoSQL model, for storing structured but Schemaless data, uses name-value pair
  • Rows are called entities, each entity can have up to 25 properties
  • PartitionKey & RowKey are primary key
    • PartitionKey is like a table name or object type
    • RowKey is the primary key of the row
  • Timestamp is also mandatory
  • There are no indexes
  • Table Storage is built to scale
  • Azure automatically scales
• Blob Storage
  • Blob = Binary Large OBject
  • Is a Container model which you create Blobs inside with tags and metadata
  • Containers don’t actually support folders, it is a flat model, but you can create ‘virtual folders’ by uploading a file like “folder/1.txt” which will store ‘1.txt’ under ‘folder’
  • Three types of Blob storage:
    • Block Blobs
      • Object storage, like AWS S3
      • Try to remember the ‘O’ in ‘blOck’ meaning Object
      • Optimised for streaming and storing objects in the cloud, optimised for reading
      • Maximum size of a block blob object is 4.75 TB
      • Good for images, videos, documents, e.t.c
    • Append Blobs
      • Simular to Block Blobs but optimised for appending data to the end of a file
      • Maximum size for an append blob is 195 GB
      • Good for log files
    • Page Blobs (Azure VHD’s)
      • Provide persistent block storage and are Azure’s equivalent to AWS EBS
      • Azure VM’s VHD’s are stored as Page Blobs
      • Represents a “hard disk”, where any part of the storage can be updated
      • Page blobs are a collection of 512-byte pages optimised for random read and write operations.
      • Maximum size for a page blob is 8 TB
      • Option of ‘Premium Storage’ by providing SSD-based storage with provisioned IOPS and low latency. Pricing is based on a flat monthly fee, regardless of the actual amount of storage that is consumed.
• Azure Queue Storage
  • Reliable, persistent messaging between services
  • Servers pull messages from the queue than delete them
  • Maximum message size is 64KB, Maximum queue size 500TB
  • Expected to be temporary
• File Share Storage
  • Works as a network share using SMB 3.0 so can be shared among several VM’s
  • Like AWS EFS, but provides SMB protocol instead of NFS
  • Also supports REST API
  • Good for storing files like log files or backups
  • Subject to the limits of the Azure Storage account, which is 500TB limit on all storage types.
  • Can create directories inside the share
  • Each file can be up to 1TB max size
  • 5TB limit on a single share
  • Files addressable by URL e.g: https://something.file.core.windows.net/someshare/somedir/somefile.txt
  • Offered as LRS or GRS redundancy.
• Storage Redundancy levels
  • Locally redundant storage (LRS) – three copies of data within a single data centre
  • Zone redundant storage (ZRS) – stores three copies of data across multiple data centres throughout a region but is only applicable to object storage with block blobs.
  • Geographically redundant storage (GRS) – distributes 6 copies of data, 3 copies in 1 region and 3 copies in a 2^nd region. This option protects against natural disaster scenarios that can affect a whole region.
  • Geographically redundant storage with read access (RA-GRS) – is a modification of GRS in that users have immediate read access to redundant copies in case of a disaster.
  • When you create a new storage account, Read-access geo-redundant storage (RA-GRS) is the default.
• Transferring data to and from Azure Storage
  • You can use the AzCopy command-line utility to copy blob, file, and table data within your storage account or between storage accounts.
  • ‘Azure storage explorer’ a GUI version of AzCopy
  • The Azure Import/Export service provides a way to import blob data into or export blob data from your storage account via a hard drive disk mailed to the Azure data centre.
  • The way you access a storage account is the same inside Azure as outside. Your applications connect to the Azure storage account using a public endpoint.
• Granting access to a storage account
  • You can provide access either using:
    • Storage account keys – Your storage account includes both a primary and secondary access key, both of which grant full administrative access to your account, and all resources within it. Exposing either of these keys opens your account to the possibility of malicious or negligent use.
    • Shared access signatures (SAS) – provide a safe alternative to Storage account keys, that allows clients to read, write, and delete data in your storage account according to the permissions you’ve explicitly granted. You can grant access to specific objects, time durations and from specific IP addresses. Access is provided by a SAS token. example of a SAS URI, showing the resource URI and the SAS token:
      

SQL Database

• Azure SQL Database is a relational database-as-a service (DBaaS) using the Microsoft SQL Server Engine.
• Maximum database size on the Premium tier is 4 TB.
• SQL Database offers four service tiers to support lightweight to heavyweight database workloads: Basic, Standard, Premium, and Premium RS. You can adjust performance without downtime.
• Premium RS tier supports same large database sizes and IOPS of Premium tier, but with Reduced Storage copies so it’s far cheaper.
  
• Database performance is measured in DTU’s (Database Transaction Units) which are made up of blended measure of CPU, memory, I/O (data and transaction log I/O).
• Two deployment options in Azure SQL:
  • Single databases
  • Elastic databases – your database automatically scales up and down to meet demand, allowing you to use pools of storage that are charged by the hour. With elastic pools, you don’t need to focus on dialling database performance up and down as demand for resources fluctuates.
    
• For business continuity and global scalability SQL database provides
  • Automatic backups: SQL Database automatically performs full, differential, and transaction log backups.
  • Point-in-time restores: SQL Database supports recovery to any point in time within the automatic backup retention period.
  • Active geo-replication: SQL Database allows you to configure up to four readable secondary databases in either the same or globally distributed Azure data centres.
  • Failover groups: SQL Database allows you to enable geo-replication and failover.
• You can configure SQL Database to store resource metrics and diagnostics into:
  • Azure Storage
  • Azure Event Hub
  • Azure Log Analytics
• Other features include:
  • SQL Database dynamic data masking limits sensitive data exposure by masking it to non-privileged users.
  • Automatic index management
  • ‘Always Encrypted’ encrypts the data at rest using Transparent Data Encryption (TDE) and in transit and stores the keys in the Azure key vault
  • Azure Active Directory integration and multi-factor authentication
• You can manage and develop SQL Database using these tools:
  • The Azure portal
  • SQL Server Management Studio
  • SQL Server Data Tools in Visual Studio
  • Visual Studio Code
• Database Sharding – Divide the database into horizontal partitions or shards. Each shard has the same schema, but holds its own distinct subset of the data. Sharding can improve scalability when storing and accessing large volumes of data.

Azure SQL Data Warehouse

• Azure SQL Data Warehouse is a massively parallel processing (MPP) cloud-based, scale-out, relational database capable of processing massive volumes of data.
• SQL Data Warehouse:
  • Combines the SQL Server relational database with Azure cloud scale-out capabilities.
  • Decouples storage from compute.
  • Enables increasing, decreasing, pausing, or resuming compute.
  • Integrates across the Azure platform.
  • Utilizes SQL Server Transact-SQL (T-SQL) and tools.
  • Complies with various legal and business security requirements such as SOC and ISO.
• It’s performance is measured in DWU’s which measure underlying resources like CPU, memory, IOPS, which are allocated to your SQL Data Warehouse.

DocumentDB (now Azure Cosmos DB)

• • Azure DocumentDB is a NoSQL document database service designed from the ground up to natively support JSON and JavaScript directly inside the database engine.
  • Stores JSON (Javascript Object Notation) which is in text format ‘documents’
  • Largest document size is 2MB.
  • Good for gaming, IoT, social media apps, large apps that are public facing
  • Pricing
    • Tiered model which limits how many requests per month you can make,
    • Performance model which starts cheap and pricing on storage and how many requests
  • Now as of May 2017 is part of Azure Cosmos DB which as many more features like supporting additional API’s like MongoDB, Graph and Table.

High Performance Computing

• On premises, Microsoft provide a ‘Microsoft HPC Pack’ you can run on your own servers
• It can extend hybrid model to run on IaaS VM’s on Azure, spin up lots and lots of VM’s
• PaaS (Azure Batch service)
  
  • provides job scheduling and auto-scaling of compute resources as a platform service
  • can run on demand or on a schedule across a pool of VMs
  • Budget, quotas and limits
  • Choose Linux or Windows to run jobs
  • Use the large VM sizes like A8, A9, A10 and A11 and the newer H-series, these are the compute-intensive instances
  • These VM’s allow access to the super-fast RDMA network on their 2^nd NIC which is under 3ms latency and 32 Gbps throughput.
  • Head node and compute nodes:
    • Head node controls the distribution of jobs, Head node can be hosted on-prem and optionally send to Compute nodes on the cloud (hybrid)
    • Compute nodes work on the tasks

Azure Scheduler

• Schedule and automatically invoke recurring jobs in the cloud
• Scheduler creates, maintains, and invokes scheduled work. Scheduler does not host any workloads or run any code. It only invokes code hosted elsewhere—in Azure, on-premises, or with another provider.
• It invokes via HTTP, HTTPS, a storage queue, a service bus queue, or a service bus topic.
• Azure WebJobs (part of the Web Apps feature in Azure App Service) and other Azure scheduling capabilities use Scheduler in the background.
• By default, if a job fails, Scheduler tries the job again four more times, at 30-second intervals

Azure Machine Learning

• Machine learning is a technique of data science that helps computers learn from large sets of existing data to forecast future behaviours, outcomes, and trends.
• Azure Machine Learning is a cloud service that allows you to create jobs for these predictive analytics
• Data science says that you need relevant, connected, accurate and enough data in order to be able to use it
• The more data you have, the better
• Data experts (people) decide which is the best fields to analyse
  
• The Machine Learning Studio, you can quickly create predictive models by dragging, dropping, and connecting modules.
  
• Big Data is the name given to the branch of computer science devoted to storing and analysing massive amounts of data e.g. Facebook or Twitter main databases, how to analyse billions of emails
• Azure HDinsight service is for analysing big data
• Uses Apache Hadoop cluster in the cloud. Hadoop refers to a type of cluster that has:
  • The Hadoop distributed file system (HDFS)
  • YARN for job scheduling and resource management
  • MapReduce for parallel processing
• Provides a framework for big data operations:
  • Manage
  • Analyse
  • Report

Azure Search

• PaaS – Search as a Service
• Integrate search into your mobile and web apps using REST API or .NET SDK
• Simple query or Apache Lucene search syntax’s available
• Provides features like search suggestions, highlight matches, facets and filters, 56 languages supported, Geosearch, 99.9% SLA
• Steps to implementation:
  • Step 1: Provision service
  • Step 2: Create index
  • Step 3: Index data
  • Step 4: Search

Azure Backup

• Use Azure backup to backup ARM VM’s
• First you will need a ‘Recovery services vault’ which provides Azure VM backups and Site Recovery
• It can backup an Azure VM as well as on-prem VM’s
  
• Application consistent backup for Windows Azure VMs and File-system consistent backup for Linux Azure VMs without the need to shutdown VM.
• Choose a different region than the region of the servers you are backing up
• The default backup policy is Daily backups at 8am and retention for 30 days.
• Can do Full or Incremental backups
• Can do VM level restores or file level restores.
• Limit of 1.7Tb per volume/drive can be backed up
• Overall limit of 500TB in an Azure storage account which the Recovery services vault sits in.

Disaster Recovery/Business Continuity

• An Azure region is an area within a geography containing one or more datacentres.
• Each Azure region is paired with another region within the same geography, together making a regional pair:
  

Geography	Paired regions
North America	North Central US	South Central US
North America	East US	West US
North America	US East 2	US Central
North America	West US 2	West Central US
Europe	North Europe	West Europe
Asia	South East Asia	East Asia
China	East China	North China
Japan	Japan East	Japan West
Brazil	Brazil South (1)	South Central US
Australia	Australia East	Australia Southeast
US Government	US Gov Iowa	US Gov Virginia
India	Central India	South India
Canada	Canada Central	Canada East
UK	UK West	UK South

• Azure recommends that you replicate workloads across regional pairs to benefit from Azure’s isolation and availability policies.
• Sequential updates – Planned Azure system updates are rolled out to paired regions sequentially (not at the same time) to minimize downtime, the effect of bugs, and logical failures in the rare event of a bad update
• No. 2 below is GRS (Geo-Redundant Storage)
• No. 3 below is Azure SQL Standard Geo-Replication, you can configure asynchronous replication of transactions to a paired region.
  
• No. 5 above at least 300 miles of separation between datacentres in a regional pair.

Azure site recovery

• A disaster recovery to the cloud service (DRaaS)
• Automates recovery of an on-prem datacentre using Hyper-V, VMware or physical servers to Azure.
• You can use Site Recovery to migrate on-premises and AWS instances to Azure VMs,
• You can run test failovers to support disaster recovery drills, without affecting production environments and the ongoing replication.
• You can run planned failovers for expected outages with zero-data loss, or unplanned failovers with minimal data loss (depending on replication frequency) for unexpected disasters.
• You can create recovery plans for sequenced failover and recovery of multi-tiered apps.

Azure monitoring

• Application Insights
  • Offers rich detection and diagnostics for issues at the application layer of your service. This is about YOUR code and how well it runs. AppInsights is well-integrated on top of data from Azure Monitoring but can also be used to track desktop and web applications deployed anywhere! You can push custom data from other services to it.
  • Application Insights is designed to monitor PaaS solutions but it’s not limited there
  • Its SDKs allow you to implement and collect telemetry from a vast variety of applications (Node.js, JavaScript, Python, Ruby, .NET, Java etc).
  • If your solution is already deployed in Azure AppService, you can enable AppInsights from the applications settings without the need to reconfigure or redeploy your code.
  • Provides you with real-time analytics, errors and performance counters from the app level
  • All collected data ends up on Azure
  • App Insights is not free, there are Basic and Enterprise tiers and you pay per node and data stored.
    
• Azure Monitor
  • Monitoring the health of your infrastructure and availability, you now can monitor Azure activity and diagnostic logs and create automation tasks as a response to a specific rule.
  • Offers visualization, query, routing, alerting, auto-scale, and automation on data both from the Azure infrastructure (Activity Log) and each individual Azure resource (Diagnostic Logs). It was previously known as “Azure Insights” to confuse matters more 🙂
  • Azure monitor is built into Azure and is free.
• Operations Management Suite (OMS)
  • OMS sits at the highest level on Azure in terms of monitoring.
  • “The one dashboard to rule them all”, OMS can be used to monitor and manage infrastructure on Azure, on-premises and third-party cloud-based infrastructure (AWS and Google Cloud). Azure Monitor and Application Insights data can be routed directly to Log Analytics so you can see metrics and logs for your entire environment in one place.
  • It doesn’t matter if your solution runs on Azure, AWS, Google Cloud or on premises
  • OMS is the accumulation of a number of services and operations previously run as separate products, like the Active Directory Health service (RAP) along with Azure Monitor and Application Insights all wrapped in one.
  • Azure Monitor and Application Insights can feed high level data to OMS to allow operations teams to have a quick check on the health of all monitored applications.
  • OMS is broken into 4 main areas:
    • Insight & Analytics
    • Automation & Control
    • Security & Compliance
    • Protection & Recovery (Backup & DR)
  • OMS has a large number of add-ins to extend the capabilities of the service
  • OMS is not free and you pay for it per VM/node or there are a couple of packages available, E1 and E2.
  • To get the data into OMS, either:
    • make the VM’s write directly to a OMS workspace via the manually installed agent or easier by the Log Analytics VM extension,
    • enable logging on the VM’s to a storage account, and make OMS read from that storage account.

• System Center Operations Manager (SCOM)
  • Starting to appear more legacy now compared to the cloud offering of OMS
  • To make it work with Azure, you need to install the ‘Azure Management Pack for Operations Manager’ This pack allows monitoring of Azure PaaS services like SQL Database and Azure Web Apps.
  • Other services on Azure where you can RDP in and install software like Cloud Services and Virtual Machines you install a SCOM agent.
  • System Center Global Service Monitor part of SCOM is a cloud service that provides a simplified way to monitor the availability of external-web-based applications from multiple locations around the world. Importantly, Global Service Monitor monitors applications from the perspective of the customers who use them. E.g. how long is my site taking to load in London?

Azure Functions

• Is Function as a Service (FaaS), runs code or ‘functions’ on demand in the cloud like AWS Lambda
• Azure Functions lets you develop serverless applications on Microsoft Azure.
• Supports languages,
  • Command line – Bash, Batch,
  • Scripting – PowerShell,
  • Programming – C#, F#, Node.js, PHP, Python,
• Supports packages
  • NuGet – a free and open-source package manager designed for the Microsoft development platform
  • Npm – is a package manager for the JavaScript/Node.js programming language
• Code your functions right in the portal or set up continuous integration and deploy your code through GitHub, Visual Studio Team Services, or other supported development tools.
• Triggers can be:
  • TimerTrigger – Execute cleanup or other batch tasks on a predefined schedule.
  • Data processing – Process Azure Storage blobs when they are added/changed
  • Webhook + API’s:
    • EventHubTrigger – Respond to events delivered to an Azure Event Hub, particularly useful for IoT
    • HTTPTrigger – Trigger the execution of your code by using an HTTP request.
    • Generic webhook – Process webhook HTTP requests from any service that supports webhooks.
    • GitHub webhook – Respond to events that occur in your GitHub repositories.
    • QueueTrigger – Respond to messages as they arrive in an Azure Storage queue.
    • ServiceBusQueueTrigger – Connect your code to other Azure services or on-premises services by listening to message queues.
    • ServiceBusTopicTrigger – Connect your code to other Azure services or on-premises services by subscribing to topics.
• Charged under a Consumption plan by Execution Time and how many Total Executions, you get 1 Million free executions a month, or under put it under a App Service Plan

IoT Hub

• Azure IoT Hub is a fully managed service that enables reliable and secure bidirectional communications between millions of IoT (Internet of Things) devices and a solution back end.
• Provides multiple device-to-cloud and cloud-to-device communication options, including one-way messaging, file transfer, and request-reply method
• Provides built-in declarative message routing to other Azure services.
• Provides a queryable store for device metadata and synchronized state information.
• Enables secure communications and access control using per-device security keys or X.509 certificates.
• Provides extensive monitoring for device connectivity and device identity management events.
• Includes device libraries for the most popular languages and platforms.
• Facilitates two-way communication between IoT devices and Azure. Difference with Azure Event Hub which only allows information or a ticket to come into Azure, IoT Hub is two-way
• IoT Hub supports more protocols than Azure event hub
• Is optimised to support millions of simultaneously connected devices
• IoT hub allows file uploads, whilst the Event Hub doesn’t.
  

Azure Notification Hub

• Azure Notification Hubs provide an easy-to-use, multi-mobile platform, scaled-out push engine.
• You can easily send targeted and personalized push notifications to any mobile platform from any cloud or on-premises backend.
• E.g. Send breaking news notifications to millions with low latency.
• To send a notification to all customers across the iOS, Android, and Windows phone versions of an app, the developer must work with APNS (Apple Push Notification Service), FCM (Firebase Cloud Messaging), and WNS (Windows Notification Service), while batching the sends.

Azure Stream Analytics

• Azure Stream Analytics is a fully managed, cost effective real-time event processing engine that helps to unlock deep insights from live data. Simular to AWS Kinesis Streams.
• Stream Analytics makes it easy to set up real-time analytic computations on data streaming from devices, sensors, web sites, social media, applications, infrastructure systems, and more.
• Monitors and analyses streaming data instead of data stationary sitting on a database.
• Useful for performing real-time analytics for your IoT solutions
• Key parts are the Input (can be an Event hub or IoT Hub), Query and Output
• With a few clicks in the Azure portal, you can author a Stream Analytics job specifying the input source of the streaming data, the output sink for the results of your job, and a data transformation expressed in a SQL-like language.
• Azure Stream Analytics is priced by the number of streaming units required to process the data into the service.
  

API Management

• API Management helps organizations publish APIs to external, partner and internal developers to unlock the potential of their data and services.
• Each API represents a set of operations available to developers.
• Secure your APIs using a key, token and IP filtering and your backend services sits behind the API management gateway
• The target API apps can be hosted outside of Azure, on your own premises or even within AWS, ANYWHERE.
• Rate limit, quotas, and IP restriction policies can also be implemented at the API or individual operation level.
• The system is made up of the following components:
  • The API gateway is the endpoint
  • The publisher portal is the administrative interface where you set up your API program
  • The developer portal serves as the main web presence for developers
• API Management is offered in three tiers: Developer, Standard and Premium.
• API management service supports JSON Web Token (JWT) validation. JSON Web Token (JWT) is an open standard that defines a compact and self-contained way for securely transmitting information between parties as a JSON object.
  

Architect for availability and reliability

• Understand Azure’s SLA’s – Read up on what the key ones are, Single VM SLA for example
• Design for failure always, use multiple instances across subnets and Regions, use Availability sets
• Recovery time Objective (RTO) – maximum amount of time to restore application functionality
• Recovery Point Objective (RPO) – acceptable time window of data lost
• Design with Fault domain and Update domains in mind
• If you design with a single VM, design the app to gracefully handle unexcepted restarts without corruption
• Understand the difference between stateful and stateless apps
• Avoid single points of failure
• Design code to handle errors in communication i.e. retries
• Design loose coupled apps, use Service bus messaging platform and Queues
• Use application monitoring – Azure Application Insights
• Use auto scaling – Azure Scale Sets

End of notes.