Language Processing and Discovering Insights in text using ML and AI leveraging AWS:

Why has Machine learning and Artificial Intelligence been so successful?

There are various reasons for the success of machine learning. Let’s call them as:

• Data

• Democratization

• Devices

        Abundant Data                        Democratized model and        Relatively fast and

                                                            Learning Algorithms                cheap cloud/compute

Data – We live in an environment where data grows rapidly. Social Media and E-commerce platforms are generating data in huge quantity. This results in abundance of data.

Democratization – We do have pre-built algorithms available online, this makes our job easy. Our work is to identify the right algorithm for the data available.

Cloud/Compute – Availability of cloud storage and compute for less cost is one of the important reason for the rapid growth of data.

In this blog, I would like to explain the process of leveraging AWS Machine learning services for language processing.

Automation with Amazon Transcribe, Amazon Translate and Amazon Comprehend

Amazon Transcribe - Amazon Transcribe is an automatic speech recognition (ASR) service that makes it easy for developers to add speech-to-text capability to their applications. Using the Amazon Transcribe API, you can analyse audio files stored in Amazon S3 and have the service return a text file of the transcribed speech. You can also send a live audio stream to Amazon Transcribe and receive a stream of transcripts in real time.

Amazon Transcribe can be used for lots of common applications, including the transcription of customer service calls and generating subtitles on audio and video content. The service can transcribe audio files stored in common formats, like WAV and MP3, with time stamps for every word so that you can easily locate the audio in the original source by searching for the text. Amazon Transcribe is continually learning and improving to keep pace with the evolution of language.

The Key Features are,

• Easy-To-Read Transcriptions – It uses deep learning to add punctuation and formatting so the output can be used without further editing.

• Timestamp Generation – Returns timestamp for each text so backtracking here is made simple.

• Custom Vocabulary – You can add new words to the base vocabulary specific to your use case, such as product name, domain-specific terms or a person name.

• Recognize Multiple Speaker – ability to identify when speaker changes like telephone calls, meetings and television shows.

• Streaming Transcription – supports streaming transcription in following languages

• British English (en-GB)

• US English (en - US)

• French (fr-FR)

• Canadian French (fr-CA)

• US Spanish (es-US)

Amazon Translate - Amazon Translate is a neural machine translation service that delivers fast, high-quality, and affordable language translation. Neural machine translation is a form of language translation automation that uses deep learning models to deliver more accurate and more natural sounding translation than traditional statistical and rule-based translation algorithms. Amazon Translate allows you to localize content - such as websites and applications - for international users, and to easily translate large volumes of text efficiently.

The Key Features are,

• Robust – These engines always learn from new and expanded dataset makes it highly accurate.

• Easy-to-Integrate – Simple API calls can be used to integrate Amazon translate with your existing application or workflows.

• Customizable – Custom Terminology feature allows you to customize the translation based on your business needs. This will reduce the cost and result in faster translations.

• Scalable – Amazon Translate is capable of scaling automatically based on the volume of data uploaded.

Amazon Comprehend - Amazon Comprehend is a natural language processing (NLP) service that uses machine learning to find insights and relationships in text. No machine learning experience required. There might be hidden insights sitting in your unstructured data such as customer emails, product reviews, social media etc… Amazon Comprehend uses machine learning to help you uncover the insights and relationships in your unstructured data. The service identifies the language of the text; extracts key phrases, places, people, brands, or events; understands how positive or negative the text is; analyses text using tokenization and parts of speech; and automatically organizes a collection of text files by topic. You can also use AutoML capabilities in Amazon Comprehend to build a custom set of entities or text classification models that are tailored uniquely to your organization’s needs.

The Key aspects are,

• Insights – Provides meaning full and hidden insights from customer product reviews, social media feeds, news article and other sources.

• Intelligence – Consolidate data based on the topics automatically.

• Training Model – Seamless integration with machine learning models, for example classifying documents based on the category. You simply provide labels and comprehend takes care of the rest.

• Supporting General and Industry Specific Text - Powered by state-of-the-art machine learning models, Amazon Comprehend can discover insights from unstructured text like social media posts, emails, and web pages.

Our Solution