Plan, Deliver & Operate Microsoft Teams with Confidence
Microsoft released the Skype Operations Framework (SOF) in the summer of 2016 with much fanfare. The SOF then morphed into the Practical Guidance for Cloud Voice which included other assets and instruction for Microsoft Teams. Overall, the resources have become a guide for implementing and managing a reliable and cost effective Unified Communications (UC) system based on the Microsoft Teams platform.
Useful for deploying Microsoft Teams and migrating to Teams, the practical guides and operations overview are a set of guiding principles to ensure the successful adoption of Microsoft Teams enterprise-wide. Adopting and migrating to an enterprise-wide Unified Communications (UC) system such as Microsoft Teams is mission-critical and strategic for an organization.
Plan > Deliver > Operate
While many IT engineers & network administrators probably hoped for a big ‘Fix-It’ button, the SOF is a remarkable guide and set of best-practices for implementing enterprise UC. Created based on demand from customers and partners interested in a formal methodology for migrating to Skype for Business, the responsible teams at Microsoft should be applauded for their efforts.
Manage Risk, Assess Network Readiness, & Remediate Poor Network Conditions
The Skype Operations Framework helps manage and reduce the risk of adopting Skype for Business across an organization including branch office, satellite and remote workers. While the potential savings from an enterprise-wide Unified Communications system such as Skype for Business are massive, the savings can’t be achieved unless the adoption and deployment risk is minimized. Managing risk, discovering and remediating poor network conditions are the chief goals of Skype Operations Framework.
Whether your testing your existing Skype for Business installation or planning a move to Skype for Business Online, CloudReady can be used to fully test the capacity, latency, packet loss and other network requirements BEFORE network changes are made, or before implementing a move to Azure ExpressRoute. CloudReady enables IT administrators and network engineers to fully model a deployment against the actual Skype clients.
Network Readiness for the Skype Operations Framework
A core component of the Skype Operations Framework is performing assessments to determine network readiness. Factors such as network latency, packet loss, bandwidth and peak traffic loads are critical for ensuring success. The SOF mandates Skype traffic simulation using the Network Assessment Tool or other tools as part of their Plan, Deliver, and Operate guidelines:
- Two stages of network readiness; essentials and advanced with the advanced being required if problems are uncovered or there are known Quality of Service (QoS) issues
- For essentials and preliminary discovery, the SOF recommends traffic simulation for each Customer Edge segment and each Client Segment in any sites where end-users will be running Skype for Business
- Different network topologies should be tested; if you have end-users accessing your UC solutions from WiFi, MPLS or VPN networks, those access points should be tested
Comparing the Network Assessment Tool and CloudReady
While improvements are likely planned for the Network Assessment Tool, CloudReady Skype Sensors are easier and more accurate for network assessment, testing and load simulation. The SOF recommends the tests be re-run anytime network changes are performed.
Key differences between the Network Assessment Tool and CloudReady
The Network Assessment Tool requires manual, per-machine configuration and deployment from different points within the network. Configuration is done by editing .NET .config files for ports, duration and other test criteria. There’s no deployment or setup help.
CloudReady Skype sensors are deployed via the Exoprise Management Client with secure step-by-step setup wizards. Because the CloudReady Skype sensors automate the actual client you can use your standard build machines or existing machines with Skype Clients installed. CloudReady has low overhead so existing client machines can be utilized. Distribution of the Exoprise agent and sensors can be automated.
The Network Assessment Tool doesn’t execute the actual Skype Client when simulating tests. Only the media/RTP stack is tested; there are no tests for SIP login or auto-discover performance across the network.
CloudReady Skype sensors leverage the real Skype for Business client as part of their tests. By using the real Skype for Business clients (or standalone Skype for Business clients), network operators can be assured the real end-user experience is measured.
The Network Assessment Tool is designed to be run manually each and every time for a duration. Re-running the assessment tool must be automated and coordinated across different endpoints and the data has to be manually collected and aggregated.
CloudReady Skype sensors continuously test the different modalities such as IM, presence, publishing and sign-in. Skype sensors are automatically run from the CloudReady Private Site (Win32 service). No additional orchestration or scheduling is required. The data and test results are automatically collected at the CloudReady servers.
By default, the Network Assessment Tool uses a tone-only wav file for simulating audio traffic and messages. The .wav file can be replaced, but requires deployment and distribution to each endpoint involved in the test.
CloudReady Skype Audio senors use a real audio conversation in each direction, from the sensor to the CloudReady receiver, testing each of the different network and edge segments as part of the execution. This enables inbound and outbound metric collection and Audio quality statistics from the underlying Skype for Business Client.
Using the Network Assessment Tool requires separate results be captured, transformed and aggregated for each of the different runs from each collection points. Orchestrating this collection can be time-consuming, error prone and inefficient.
CloudReady Skype sensors automatically capture & upload each execution where the statistics are analyzed against automatically configured alarms. Sensor results can be immediately compared to other sensors and against a cloud-based real-time benchmark.
Deployment and automation of the Network Assessment Tool is not within the scope of the Skype Operations Framework. Its possible to use tools like System Center Configuration Manager (SCCM) to deploy the tool but different tools will be needed to periodically schedule the execution.
CloudReady has multiple tools and technologies in place to speed deployment of its sensors. Private Site deployment can be delegated to different groups within an organization. Once the sites are deployed many different sensors can be deployed and organized with the CloudReady PowerShell Management API. Sensors can be easily configured and moved about different site locations for full Client and Edge Segment tests.
The Network Assessment Tool that is part of the SOF only tests the Skype Media Stack. There is no support for simulating other network loads or stress within the environment. The SOF recommends periodically running the tool at representative times during peek network traffic or similar.
CloudReady includes over 30 sensors that fully test Office 365 and other cloud and network-based workloads. By leveraging additional sensors like the Bandwidth and VoIP sensors customers can simulate any network traffic alongside the actual Skype for Business Client. No waiting around for peak network traffic. Generate repeatable baselines.
The Network Assessment Tool doesn’t support testing Azure or Office 365 ExpressRoute as the SOF recommends.
CloudReady Azure VoIP Sensors can be rapidly deployed against Azure VMs to test Azure ExpressRoute functionality and network capacity. CloudReady supports many peer-to-peer and Client/Edge testing sensors for end-to-end measurements and monitoring.
Getting Started With CloudReady
Ease of use is a core feature of the entire CloudReady platform. Everything is wizard-driven and capable of automation; agent deployment can be delegated to different teams in different time-zones for modeling and assessing enterprise-wide network conditions across branch offices and access points.
- Determine your machines. As per the Skype Operations Framework, multiple machines in different network and client edge scenarios should be chosen. For advanced network readiness and further testing, customers should chose networks in different branch offices, against WiFi access points, and even ExpressRoute tests.
- For the Skype sensors you will need real Skype accounts, VoIP and Bandwidth sensors don’t require accounts and operate in peer-to-peer mode for testing edge and Azure scenarios.
- Deploy your Exoprise CloudReady sites via the various deployment methodologies.
- Deploy and configure your Skype Sensors using our PowerShell Management API
Detecting Network Impairment With CloudReady Skype Sensors
The Skype Operations Framework details the recommended optimal network characteristics for a Skype for Business deployment. Here are some of the definitions for metrics that are critical and captured by the Network Assessment Tool as well as the CloudReady Skype Sensors.
Latency is the time it takes to get an IP packet from point A to Point B on the network. As opposed to other network test tools which might just simply ping a network interface with ICMP, the CloudReady sensors gather the latency statistic as seen by the real Skype for Business Clients. This is the same metric that is captured and measured in the historical Call Quality Reports. The difference with the CloudReady Skype sensors is that this metric captured proactively before an end-user experiences poor latency between network segments.
Recommended Network Latency
Microsoft as part of the SOF recommends that latency be kept under 50ms one-way between the customer Skype clients and Microsoft’s network edge (their datacenters). From the customers edge (gateway) they recommend latency be kept under 30ms. All of these metrics are available as part of the CloudReady Skype Sensors as well as the Azure VoIP and Network Path Performance metrics for each sensor run.
Packet Loss is the percentage of packets that are lost in a given window of time. Packet loss directly affects the Skype for Business Audio, Video and screensharing modalities. Small or individual packet loss won’t have much of an impact but back-to-back losses will cause audio and video cut-out. Packet loss can easily be tested in various ways with CloudReady; packet loss is captured from the actual Skype Clients as well as from the simulated VoIP and bandwidth sensors which can be pointed to Azure, AWS or deployed in a peer-to-peer setup.
Recommended Minimum Packet Loss
Microsoft recommends packet loss be kept to less then 1% during any 15 second interval and consecutive packet loss (burst) be kept to less then 10% during any 200 millisecond interval. These metrics are captured by the CloudReady Sensors as well as assisting in pin-pointing the network device that is responsible for the packet loss when possible through the Network Path Performance measurement that is captured with each run.
Inter-packet Arrival Jitter
Jitter is the average change in delay between successive packets. When there is excessive jitter in a network that exceeds the buffering capability of the client then end-users will notice cut-out of audio, video or screen-sharing modalities. CloudReady Sensors capture actual jitter metrics as reported by the Skype client. CloudReady VoIP sensors also measure jitter between multiple network segments or various cloud deployments.
Recommended Network Jitter Targets
Microsoft recommends network jitter be kept below 30ms for the entire path between the Skype client and Microsoft’s edge network. Jitter should be kept to below 15ms from the customers gateway to Microsoft’s edge.
Mean Opinion Score (MOS)
The MOS metric is a well-known standard for representing overall call quality for the duration of the call. MOS scores are measured on a scale of 1 to 5 with 5 being near perfect audio quality and 1 being real bad. CloudReady captures the inbound and outbound MOS scores and values that contribute to MOS degrade as part of each sensor run. The MOS metric is critical in determining overall network suitability for Skype for Business and helpful in comparing MOS metrics for network operators since its an easily quantified, known metric.
Recommended MOS Score
MOS scores should consistently be kept at or around 4 for a successful Skype for Business implementation.
Exoprise CloudReady Enterprise Use Cases
Using CloudReady for pre-migration network readiness and assessments like the Skype Operations Framework are just one of the uses for the CloudReady Skype Monitoring. The SOF goes into great detail about when and why customers should re-evaluate their network during Skype for Business delivery and operations.
Microsoft recommends that every time there is a network or capacity change, network readiness should be re-examined and tested. Lofty goals, for sure, but it does make sense that for significant network changes after deployment, measuring network conditions against the network baseline is critical. Beyond this, CloudReady for Skype Monitoring is used by customers for:
- Proactive monitoring of Skype for Business – know before the big conference call, not after. Even with enhanced Skype call quality data it still takes forever to aggregate and its historical. What good is a Skype Call Quality Dashboard for last weeks calls?
- Baselines. The importance of Skype network baselines cannot be overstated. Exoprise CloudReady Skype senors run and execute synthetic Skype sessions over and over again. They are repeatable and will generate accurate baselines of your network quality as opposed to being subject to the vagaries of production users and their schedules.
- Utilizing CloudReady Skype sensors before and after network changes means potentially not impacting your production users. You don’t have to wait for changes to be reflected in weeks old dashboards that may have affected important calls.
- Model your network and simulate Skype calls across branch office, SD-WAN and MPLS scenarios. How can you do it without using the real Skype for Business client?