Comet, a leader in AI development, today announced a strategic partnership with Amazon Web Services (AWS) to bring its platform to Amazon SageMaker AI’s newly launched capability, Amazon SageMaker partner AI apps. The collaboration enables AI developers using Amazon SageMaker AI to access Comet’s end-to-end model evaluation platform, which includes experiment tracking, large language model (LLM) evaluation and production monitoring.

About Amazon SageMaker partner AI apps

Amazon SageMaker AI is a fully managed service that combines a broad set of tools to enable high-performance, cost-effective machine learning and generative AI development for any use case. With SageMaker AI, developers can build, train and deploy ML models at scale using tools and capabilities like notebooks, debuggers, profilers, pipelines, MLOps and more – all in one integrated development environment.

The freshly unveiled Amazon SageMaker partner AI apps capability enables SageMaker AI customers to seamlessly discover and use industry-leading third-party generative AI and ML development applications that run privately and fully integrated with SageMaker AI. Amazon SageMaker partner AI apps provides this set of applications as fully managed capabilities that meet the security standards and data handling measures expected by AWS customers. Working closely with partners, Amazon does all the heavy lifting of deploying, upgrading, managing, and scaling the apps so customers can focus on bringing new AI solutions to market faster.

Comet and Amazon SageMaker AI: A “Better Together” Solution for ML and GenAI Projects

Through the Comet integration on Amazon SageMaker partner AI apps, ML and GenAI teams can access a comprehensive platform for building, evaluating and managing AI projects. By bringing Comet’s evaluation tools, experiment tracking, and production monitoring to Amazon SageMaker AI, customers gain a unified solution for end-to-end model management, optimizing projects across the entire ML lifecycle.

Within SageMaker Studio, Notebooks and Pipelines, teams can leverage Comet’s robust tracking and monitoring capabilities to oversee model lifecycles from training through production, bringing transparency and reproducibility to ML workflows. For GenAI applications, Comet’s tools allow automated tracking and evaluation of LLM performance, efficiently optimizing applications pre and post-deployment.

Additionally, SageMaker customers who purchase Comet’s integration will have full access to Opik, Comet’s leading end-to-end LLM evaluation platform.

The full Comet environment – including Opik – appears directly inside the SageMaker Studio and is deployed, managed and supported by AWS, providing a holistic experience and seamless integration.

Each Comet deployment through Amazon SageMaker partner AI apps is securely isolated and provisioned automatically, ensuring customer data privacy. Customers can seamlessly integrate Comet’s advanced tools without altering their existing SageMaker AI workflows, supported by Amazon SageMaker partner AI apps’ identity and session management, and AWS’ secure infrastructure.

The partnership enhances SageMaker AI’s ML and GenAI development tools by introducing advanced experiment tracking and LLM evaluations via Comet, offering a high-value solution to customers seeking in-depth model evaluation tools. It combines Comet’s experiment management and model monitoring capabilities with SageMaker’s infrastructure purpose-built tools for each step of the ML and FM lifecycle.

“Hundreds of thousands of customers use Amazon SageMaker AI to build, train, and deploy ML and foundation models. They also leverage specialized applications from AWS partners for the unique needs of their business and industry,” said Ankur Mehrotra, Director and General Manager of Amazon SageMaker AI. “Amazon SageMaker partner AI apps enables customers to easily discover, deploy, and use generative artificial intelligence (AI) and machine learning (ML) development applications from leading app providers like Comet privately and securely. We are excited to have Comet as one of our first launch partners. We believe Comet’s capabilities for experiment tracking and model production monitoring will help accelerate our customers’ journey in building generative AI and ML models.”

“The AI/ML community at Natwest Group are long-standing users of both SageMaker AI and Comet, which help us quickly develop new solutions for our customers and colleagues, ranging from detecting fraud more quickly and efficiently, to creating a deep understanding of the reason and nature of our customer conversations,” said Greig Cowan, Head of AI and Data Science Innovation at Natwest Group, a Comet customer. “The announcement of Comet being available as a SageMaker partner AI app will simplify our technology estate and developer workflow, allowing our data scientists and ML engineers a better experience for experiment tracking, cataloging and registering models, and monitoring those models in production. These improvements and enablers will ultimately translate to better customer outcomes and experiences.”

To learn more about the project and partnership, visit www.comet.com/site/sagemaker.

About Comet

Comet provides an end-to-end model evaluation platform for AI developers, with best in class LLM evaluation, experiment tracking, and production monitoring. Comet’s platform is trusted by over 150 enterprise customers including Netflix, Cepsa, Etsy, Uber and Zappos. Individuals and academic teams use Comet’s platform to advance research in their fields of study. Founded in 2017, Comet is headquartered in New York, NY with a remote workforce in 14 countries on four continents. Comet is free to individuals and academic teams. Startup, team, and enterprise licensing is also available. To learn more, visit www.comet.com.

About AWS

Since 2006, Amazon Web Services has been the world’s most comprehensive and broadly adopted cloud. AWS has been continually expanding its services to support virtually any workload, and it now has more than 240 fully featured services for compute, storage, databases, networking, analytics, machine learning and artificial intelligence (AI), Internet of Things (IoT), mobile, security, hybrid, media, and application development, deployment, and management from 108 Availability Zones within 34 geographic regions, with announced plans for 18 more Availability Zones and six more AWS Regions in Mexico, New Zealand, the Kingdom of Saudi Arabia, Taiwan, Thailand, and the AWS European Sovereign Cloud. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—trust AWS to power their infrastructure, become more agile, and lower costs. To learn more about AWS, visit aws.amazon.com.