Azure Applied AI Services

May 25, 2021
Azure Applied AI Services Diagram

Each year, Microsoft Build is like Christmas for developers, especially those focused on the Microsoft stack. It's when they make the most exciting announcements, release the best new software, and start to solidify the roadmap of innovation for the next year.

While we don't get to enjoy the conference in person this year, we do get the same great collection of announcements. This year's highlight for engineers and developers interested in machine learning is the Azure Applied AI Services.

Microsoft has been Democratizing AI since 2016. Applied AI Services takes this ideal another step forward. Microsoft has already made machine learning accessible to developers through their outstanding Cognitive Services, and this offering will make certain scenarios even more accessible.

As you learn about Applied AI Services, I encourage you to consider how your team can embed these into your existing applications, both external and internal facing. Let AI help your business solve its most significant challenges and bring a new level of innovation to your products.

These Applied AI Services include business logic to solve common business problems in a customizable way for your individual needs. If you've felt like AI and Machine Learning are beyond your company’s reach or "only for big companies," these services are the answer. They allow you to begin to bring machine learning to your business, elevating your products and offerings.

Where do Applied AI Services fit?

Azure Applied AI Services fit very nicely with Microsoft's existing ML offerings. We can divide Microsoft's offerings into three categories, each building upon the last. The first is Azure Machine Learning; this offering includes the capability to build and train custom machine learning models, support for hosting them, and ML Ops practices for deployment.

The second offering is Azure Cognitive Services. Cognitive Services feature pre-trained models ready to use as-is or customizable—using your own data to help solve a custom scenario, using common models.

The third and new category is Azure Applied AI Services. These services apply business logic to common cognitive service model(s) to solve common business scenarios. Each level of service requires less time and lower cost to integrate into your software and bring to market and provides a solution for virtually any machine learning problem within your business domain.

I'm excited at the possibilities each of these Applied AI Services present, and I would encourage a close examination relative to your business. Each offering has widespread applications in most business verticals. Often, we find that an outside viewpoint can help you understand where and how AI fits within your business domain. Once AI becomes this accessible for your use-cases, you must act quickly; if you don't, one of your competitors will.

The Services

Each of the Azure Applied AI Services Focuses on a Specific Area

Jump ahead to a specific service:

Azure Video Analyzer [Video]

Azure Video Analyzer is an Azure Applied AI Service to extract actionable insights from videos. This service allows you to capture, record, and analyze videos. Video Analyzer will enable you to gain rich insights from stored and streaming videos. Below is an example of Azure Video Analyzer applied to the problem of Turnaround Management by ZeroG in their DeepTurnaround product:

Azure Video Analyzer for Aircraft Turnaround

Turnaround is the time an aircraft spends between parking and starting pushback for its next departure. During this time, the aircraft needs to be parked, fueled, loaded with cargo, connected to power, and cleaned. The management of turnaround is pivotal to maintaining a smooth-running airline and directly impacts the flight's profitability.

In this example, Video Analyzer tracks each step during aircraft turnaround and understands when each step in the process has started and stopped. This information can be used to automate the process, identify potential problems as they occur, and collect metrics to evaluate the entire fleet’s turnaround performance.

This Applied AI service could be used in any vertical where someone is making a visual inspection and then making a decision based on that visual information—even managing sophisticated workflows as seen above. Many companies already collect visual data through cameras or videos they could leverage to improve efficiency, spot problems as they occur, or review video in large quantities.

What video is your company collecting that could provide actionable insights for your business? What decision performed through visual inspection could be automated or made more accurate through Applied AI?

Contact Us to discuss how to take advantage of Azure Video Analyzer for your business.

Azure Metrics Advisor [Decision]

Azure Metrics Advisor is an Azure Applied AI Service for detecting anomalies in a variety of scenarios. This service simplifies data ingestion and augmentation, lowering entry barriers to performing anomaly detection in a scenario. Azure Metrics Advisor also adds the ability to monitor and diagnose issues with root-cause analysis and incident alerting.

Metrics Advisor Workflow

My favorite feature of Metrics Advisor is the customizability. Each metric collected allows you to give feedback on collected points. Based on this feedback, the smart detection will adjust to better match your needs without requiring custom ML model building. Simply mark a point as “an anomaly” or “not an anomaly," and Azure Metrics Advisor will do the rest.

Metrics Advisor also allows you to provide feedback for multiple continuous points when the anomaly can only be detected within the context of surrounding points in a time series.

Metrics Advisor allows the user to identify points where trends have changed. By identifying the inflection point of the time series data, the model will automatically understand how to better identify where anomalies have occurred. Finally, metrics advisor allows you to identify seasonality. Marking the seasonality of your time series further helps Metrics Advisor identify anomalies.

When an incident does occur, the tooling in Azure Metrics Advisor allows your team to quickly do root-cause analysis, giving them sophisticated views into the data through interactive graphs and trees:

Diagnostic Tools

The incident hub provides diagnostic information that lets you perform in-depth analysis and offers automatic suggestions for likely causes of the incident with root cause advice. The incident tree enables you to diagnose current incidents and older incidents and see the relationship between them. Each incident allows anomaly drill down, showing all dimensionality data related to the incident and providing visualizations.

This Applied AI service could be used in any vertical where identifying and responding to anomalies can improve workflow or prevent product defects. Often companies have many data points that are being collected but not analyzed or that they're discarding completely. This data combined with Metrics Advisor could be leveraged to improve efficiency, spot problems as they occur, or prevent damage to equipment.

What data is your company collecting you could use to identify problems before your customers? Does your team have reactive problem-solving that could become proactive, saving money, improving products, or improving end-user experience through Applied AI?

Contact Us to discuss how to take advantage of Azure Metrics Advisor for your business.

Azure Bot Service [Conversation]

Azure Bot Service is an Azure Applied AI Service for enterprise-grade bot development through conversational AI experiences. This service integrates AI and Natural language capabilities into a framework designed for open, extensible bot development. These bots are designed to interact naturally with users and meet users on the platforms they already use.

Bot Framework Architecture

Like all Azure Applied AI Services, the Bot Framework is built on top of cognitive services. It leverages Speech, QnA, Language Understanding, and Vision to provide a natural interactive assistant. The Bot Framework is designed to assume your brand and personality, so it feels like an extension of your business.

The Bot Framework can integrate with a repository of knowledge using QnA and integrate customizable skills to integrate off-the-shelf functionality. Newly released is the addition of bot framework components, allowing your company to publish components other bots can incorporate and allowing you to integrate other companies' components from a directory.

Bot Framework Composer adds an end-to-end development experience, allowing for visual editing of conversation flows. It provides tooling to help author natural language understanding and QnA-based components. It features a powerful system for language generation, templating, and debugging. Composer allows you to accelerate development from months to days and focus on the business logic of your bot instead of the need to learn a new framework.

This Applied AI service is an excellent fit for any vertical. Interactive bots and user agents are great for both internal and customer-facing deployments. Bots are often used to augment existing service agents, answer common and repetitive questions, or solve straightforward problems. Bots can free up your employees to focus on complicated and challenging interactions that require problem-solving. Intelligent bots can also improve interactions--moving away from call trees and toward natural, comfortable interactions.

Can your company save money by automating interactions with bots? Can a bot improve customer or employee satisfaction by providing a natural exchange? Can your bot become an extension of your brand, providing a pleasing experience?

Contact Us to discuss how to take advantage of Azure Bot Service for your business.

Azure Form Recognizer [Documents]

Azure Form Recognizer is an Azure Applied AI Service for creating structured data from documents. This service extracts text and layout information from documents along with their bounding boxes. Form Recognizer does this through prebuilt models or custom models trained with your data.

Extracted table layout information

Azure Form Recognizer uses prebuilt models to help solve common business scenarios. This release features improved model accuracy for each prebuilt model along with reclassifying Form Recognizer as an Azure Applied AI Service. These models include models for Invoices, Sales Receipts, Business Cards, and ID Cards. In addition, Form Recognizer will now include handwritten classification and take into account reading order.

The custom model building capability uses transfer learning to build on top of an existing model. This allows your custom model to be created with as few as five examples of your custom documents. The training can occur without labeling, using unsupervised learning to understand the document's layout and discover fields. In cases where the unsupervised learning can’t find the labels for your document, a labeling tool is provided to help your team quickly label data to build their custom model.

This Applied AI service could be used in any workflow that involves paper documents. Most companies have at least some paper documents they receive, and that they spend time and money consuming and adding to their systems. Even worse, these documents often end up not being processed at all, getting discarded or filed, leaving errors undiscovered, or creating misunderstandings between your company and your customers.

What documents can your company consume using form recognizer to provide workflow improvements for your business? How can you add value for your customers by improving the speed and accuracy of the ingestion of their documents?

Contact Us to discuss how to take advantage of Azure Form Recognizer for your business.

Azure Cognitive Search [Search]

Azure Cognitive Search joins the family of Azure Applied AI Services, providing AI-powered cloud search for your products and services. Azure Search has been a part of the Azure stack of services for years and, in recent years, has acquired ML capabilities. These capabilities add cognitive services into the ingestion pipeline for Azure search, enriching data with off-the-shelf and custom machine learning.

Azure Cognitive Search's off-the-shelf ML enrichment capabilities

Azure Cognitive Search includes ML Capabilities for language identification and translation and OCR capabilities for better image comprehension. Cognitive search also has language comprehension that provides key phrase extraction, location, people, and organization detection. All of this is layered on top of a world-class search engine you can customize to your business needs, including auto-complete, geospatial search, and faceting capabilities. These features are all made available as a fully managed PAAS service allowing your team to focus on providing business value, not supporting complex infrastructure.

Recently, semantic search capabilities were added to Cognitive Search, adding semantic ranking to search results to allow natural language queries to help rank more relevant answers to user questions. For the end-user, this means instead of simply getting the best match based on words, they will receive the most relevant result to their search intent.

This release of Azure Cognitive Search includes new capabilities to ingest data from new sources such as SharePoint Online Indexer, Azure Files Indexer, and using PowerQuery Connectors. These capabilities enable a set of new business cases to easily meet your data as-is and make it highly searchable. In addition, this release sees improvements in Text Analytics Skills, adding additional capabilities including improved health-term recognition  for medical field applications, sentiment, and opinion mining.

This Applied AI Service could be used in any business where documents, data, and files need to be quickly and efficiently searched by users. This tool can make a mountain of data easy to search, even if that data is in a format that is traditionally difficult to index, such as images or office documents. Many products can see dramatic speed and efficiency improvements using Azure Cognitive search. This service can make your application faster and significantly improve workflows saving time, money and improving your feature set.

What workflow or product does your company have that world-class ML-powered search capabilities could improve? How can you leapfrog your competition with semantic search results that always seem to find just what the user is seeking?

Contact Us to discuss how to take advantage of Azure Cognitive Search for your business.

Azure Immersive Reader [Accessibility]

Azure Immersive Reader is an Azure Applied AI Service for helping users read and comprehend documents and books. This service provides a turn-key solution to help users with accessibility issues and learning to read. Immersive Reader assists readers with features like reading aloud, translating languages, and focusing attention by highlighting and using color.

Azure Immersive Reader targets users of all ages with a diverse range of abilities to boost reading comprehension with proven literacy-enhancing features. Immersive Reader already provides users with the ability to translate text into more than 60 different languages, simple API call-based integration, and continuous optimization. It highlights words and uses color to isolate content and improve readability.  It can display pictures for common words, highlight parts of speech, and even read content aloud.

Azure Immersive reader has capabilities to highlight parts of speech such as verbs, nouns, pronouns, and adjectives using both color and labeling. Reader includes language auto-detection capabilities making it even easier for developers to integrate immersive Reader into their application. Finally, Immersive reader is easy to integrate using iframe's for UI Frame creation.

This Applied AI service could be used in any vertical where someone is presenting text content. The Immersive Reader tool can be used in an educational setting to assist literacy or could be used to make your content more accessible to users, including those who speak a different language. Many companies are attempting to address accessibility and translation on their own through countless hours of expensive development, but with Immersive Reader, your team can take off-the-shelf ML and add worldwide accessibility in a fraction of the time.

What application is your company developing that could include additional markets with translation into other languages? What product or service could be more inclusive to those with visual impairments, learning disabilities, or literacy issues?

Contact Us to discuss how to take advantage of Azure Immersive Reader for your business.
Cameron Vetter
My name is Cameron Vetter and I'm the Principal Architect at the Octavian Technology Group.
Microsoft MVP. Cloud. Enterprise Architecture. Machine Learning. Mixed Reality.
Transformative and Impactful,
Put Us to Work for You
Contact Us