AWS SageMaker Data Wrangler: New Data Prep for Machine Learning

Amazon Web Services (AWS) has introduced a new offering today, Amazon SageMaker Data Wrangler, designed to streamline the data preparation process for machine learning model training. Alongside this, the company is also releasing SageMaker Feature Store, accessible within SageMaker Studio, a service that simplifies the process of labeling, arranging, locating, and distributing machine learning features.

AWS is also unveiling SageMaker Pipelines, a new service that integrates seamlessly with the existing platform and delivers a CI/CD solution for machine learning, enabling the creation and automation of workflows, alongside a comprehensive audit history for model elements such as training data and settings.

During his keynote presentation at the AWS re:Invent conference, AWS CEO Andy Jassy highlighted that preparing data continues to be a significant hurdle in the field of machine learning. Initially, users must develop and execute queries and code to extract data from their storage systems, followed by writing additional queries to transform the data and integrate necessary features. This preparatory work diverts focus from the core task of model development and concentrates on the underlying infrastructure.

Data Wrangler provides access to more than 300 pre-established data transformation functions, assisting users in modifying column data types or filling in absent data using average or median calculations. The service also incorporates integrated visualization capabilities to aid in identifying potential inaccuracies, along with tools to detect and resolve data inconsistencies prior to model deployment.

These processes can be preserved within a notebook or as a script for team collaboration and reproducibility, and can also be incorporated into SageMaker Pipelines to further automate the overall workflow.

It is important to recognize that several emerging companies are addressing this same challenge. The preparation of data for machine learning is, in fact, a frequently encountered issue within the industry. However, many organizations still rely on internally developed tools, making this a suitable area for a fully managed service.