Why has Machine learning and Artificial Intelligence been so successful?
There are various reasons for the success of machine learning. Let’s call them as:
Data
Democratization
Devices
Abundant Data Democratized model and Relatively fast and
Learning Algorithms cheap cloud/compute
Data – We live in an environment where data grows rapidly. Social Media and E-commerce platforms are generating data in huge quantity. This results in abundance of data.
Democratization – We do have pre-built algorithms available online, this makes our job easy. Our work is to identify the right algorithm for the data available.
Cloud/Compute – Availability of cloud storage and compute for less cost is one of the important reason for the rapid growth of data.
In this blog, I would like to explain the process of leveraging AWS Machine learning services for language processing.
Automation with Amazon Transcribe, Amazon Translate and Amazon Comprehend
Amazon Transcribe - Amazon Transcribe is an automatic speech recognition (ASR) service that makes it easy for developers to add speech-to-text capability to their applications. Using the Amazon Transcribe API, you can analyse audio files stored in Amazon S3 and have the service return a text file of the transcribed speech. You can also send a live audio stream to Amazon Transcribe and receive a stream of transcripts in real time.
Amazon Transcribe can be used for lots of common applications, including the transcription of customer service calls and generating subtitles on audio and video content. The service can transcribe audio files stored in common formats, like WAV and MP3, with time stamps for every word so that you can easily locate the audio in the original source by searching for the text. Amazon Transcribe is continually learning and improving to keep pace with the evolution of language.
The Key Features are,
Easy-To-Read Transcriptions – It uses deep learning to add punctuation and formatting so the output can be used without further editing.
Timestamp Generation – Returns timestamp for each text so backtracking here is made simple.
Custom Vocabulary – You can add new words to the base vocabulary specific to your use case, such as product name, domain-specific terms or a person name.
Recognize Multiple Speaker – ability to identify when speaker changes like telephone calls, meetings and television shows.
Streaming Transcription – supports streaming transcription in following languages
British English (en-GB)
US English (en - US)
French (fr-FR)
Canadian French (fr-CA)
US Spanish (es-US)
Amazon Translate - Amazon Translate is a neural machine translation service that delivers fast, high-quality, and affordable language translation. Neural machine translation is a form of language translation automation that uses deep learning models to deliver more accurate and more natural sounding translation than traditional statistical and rule-based translation algorithms. Amazon Translate allows you to localize content - such as websites and applications - for international users, and to easily translate large volumes of text efficiently.
The Key Features are,
Robust – These engines always learn from new and expanded dataset makes it highly accurate.
Easy-to-Integrate – Simple API calls can be used to integrate Amazon translate with your existing application or workflows.
Customizable – Custom Terminology feature allows you to customize the translation based on your business needs. This will reduce the cost and result in faster translations.
Scalable – Amazon Translate is capable of scaling automatically based on the volume of data uploaded.
Amazon Comprehend - Amazon Comprehend is a natural language processing (NLP) service that uses machine learning to find insights and relationships in text. No machine learning experience required. There might be hidden insights sitting in your unstructured data such as customer emails, product reviews, social media etc… Amazon Comprehend uses machine learning to help you uncover the insights and relationships in your unstructured data. The service identifies the language of the text; extracts key phrases, places, people, brands, or events; understands how positive or negative the text is; analyses text using tokenization and parts of speech; and automatically organizes a collection of text files by topic. You can also use AutoML capabilities in Amazon Comprehend to build a custom set of entities or text classification models that are tailored uniquely to your organization’s needs.
The Key aspects are,
Insights – Provides meaning full and hidden insights from customer product reviews, social media feeds, news article and other sources.
Intelligence – Consolidate data based on the topics automatically.
Training Model – Seamless integration with machine learning models, for example classifying documents based on the category. You simply provide labels and comprehend takes care of the rest.
Supporting General and Industry Specific Text - Powered by state-of-the-art machine learning models, Amazon Comprehend can discover insights from unstructured text like social media posts, emails, and web pages.
Our Solution
Process Flow
Customer’s media files are uploaded to s3 bucket and AWS lambda function will be triggered that will expose the data to transcribe which will in turn fire the transcribe job.
Wait for the job to be completed and get the transcript.
Once you get the transcript, you can detect entity, analyse sentiments and detect syntax using Amazon Comprehend.
Once the insights are identified, you can visualise using Amazon Athena and Amazon QuickSight.
Our Use Case
In our case, the customer was left with huge amount of customer’s call centre recordings in audio format and wanted to transcribe the data and perform text analytics to derive meaningful insights which Amazon Transcribe and Amazon Comprehend made the job easier. First, we stored the files in S3 Bucket. Once the data became available, the lambda function was triggered in order for Amazon Transcribe to access the data and transcribe all the audios into text. Then Amazon Comprehend was called to read the data and determine the Entity, Topics and analyse sentiments. The data was also available for visualisation using Amazon QuickSight. The solution helped the customer to find key indicators such as,
Location and Region based issues
Time range where maximum number of issues are reported
Often reported issues
Employee Performance
Customer Sentiments
Trend over period
We are also recommending to the customer to extend this use case by building a chatbot that will leverage advanced Machine Learning and NLP techniques that will help to address the end customer queries faster with high accuracy and without human interaction.
To know more about ML AI enabled predictive analytics click the link https://www.oneglobesystems.com/ml-ai-enabled-predictive-analytics