Scale AI Review

• Annotation across text, image, video, and 3D sensor fusion including LiDAR, covering document processing, transcription, full motion video, and geospatial data
• A project, task, and batch structure where every task inherits one taxonomy, keeping instructions consistent across a whole dataset
• Taxonomies combining annotation classes, global attributes about a full task, per-annotation attributes, and link attributes that record relationships between two annotations
• Nucleus for dataset management, letting teams visualize data at scale, curate segments, and refine existing annotations
• Reinforcement learning environments that reproduce web applications, desktop virtual machines, and live tool servers for Slack, HubSpot, and Linear

Scale also runs Outlier, a separate platform where freelance specialists take on AI training projects: writing difficult prompts with correct answers, building grading rubrics, and rating or ranking model responses. That work feeds model training and evaluation rather than the labeling pipelines described above. For a company that would otherwise hire annotators, build its own pipelines, and maintain evaluation tooling in house, one contract replaces three separate cost centers.

AI Features

• Reinforcement learning from human feedback, which applies human preferences to model outputs after pre-training
• Red teaming that uses prompt injection to surface vulnerabilities across misinformation, unqualified advice, bias, privacy leakage, cyberattack assistance, and dangerous substances
• Model comparison that scores two candidates side by side on factual accuracy, hallucination rate, context recall, and latency
• Retrieval-grounded answers in Donovan that cite the source documents they draw from inside connected knowledge bases
• Agent orchestration for long-running asynchronous and multi-agent workflows, handled by the platform's Agentex and AgentOps layers
• A learning loop that turns production usage and human review into training signal, plus Dialect, a decision layer that encodes how an organization makes judgments
• Natural language dataset search and automatic tagging, which let teams find and prioritize data slices without writing queries

Evaluation scoring, retrieval, and agent orchestration are all genuine AI features. The learning loop is harder to classify, because Scale describes agents that improve without manual retraining, yet the mechanism depends on human feedback that people capture and structure. That makes it a well-engineered human process wrapped in automation rather than a model teaching itself. The same holds for annotation quality: machine assistance speeds the work, but vetted specialists reviewing edge cases produce the guarantee.

Integrations

• Enterprise content sources including Confluence, SharePoint, and S3, structured through purpose-built pipelines
• Cloud storage for attachments across Amazon S3, Google Cloud Storage, and Azure Blob Storage, each through its own service credentials
• Deployment on AWS, Azure, and Google Cloud, so data stays inside the customer's environment
• Model providers spanning OpenAI, Google, Meta, Mistral, and Cohere, with switching that does not require rebuilding the application
• External data in Donovan through a prebuilt connector to the GDELT global events database
• Simulated tool servers mirroring Slack, HubSpot, and Linear inside the training environments

Scale offers a public REST API with resource-oriented URLs and JSON responses, official Python and JavaScript clients, callbacks on task completion, and separate sandbox and live modes selected by the API key. It handles one object per request and does not support bulk updates, and a separate Python client logs interactions and trace spans.

Data Security and Compliance

Customers keep ownership of their data, their business logic, and any custom solution built for them, while Scale retains the underlying platform and technical frameworks. The company states plainly that no vendor lock-in applies and that agent code belongs to the enterprise. Certifications include SOC 2 Type II and ISO 27001, with FedRAMP High authorization and a Department of Defense IL4 provisional authorization for public sector work. The infrastructure supports GDPR and CCPA obligations along with customer-specific audit requirements, and onshore data processing is available as an option. Agents that reach production carry a full audit trail and source-cited outputs, which matters for any organization that must reconstruct why a system produced a given answer.

Language: Interface and Customer Support

Scale publishes its documentation in English across the main site and the separate GenAI Platform portal, and the portal includes a multilingual in-page assistant that answers questions about the platform. For sovereign and public sector engagements, Scale places multilingual teams on the ground, including Arabic-speaking staff, so that capability transfers to the local organization rather than staying with the vendor.

AI Language: The Tool Itself

For the moment, the GenAI Platform covers 80 languages, which puts it well ahead of most tools in this category. The platform runs third-party models rather than one proprietary model, so the quality a team gets in any given language depends partly on which provider it picks for that workload. On the data side, the expert network that writes prompts, builds rubrics, and rates answers works across a wide range of languages, and that is what makes evaluation and fine-tuning possible outside English.

Mobile Access

Scale offers no iOS or Android app. Everything runs in the browser, through one dashboard for the data platform and a separate application for the GenAI Platform. Neither adapts to a phone screen in any way the documentation describes, so a small screen gives you the desktop interface as is. This matters less here than it would for a tool you open between meetings. Reviewing labeled data, reading evaluation results, and configuring agents are all desk work, and nobody manages an annotation pipeline from a phone. Donovan runs on classified and air-gapped networks, which rules out ordinary mobile access by design.

Support, Onboarding, and Account Management

• Support runs during business hours in the Pacific time zone, Monday through Friday, and pauses for major public holidays
• Pro and Nucleus customers contact a named Engagement Manager or Field Engineer, who triages and routes the issue internally
• Larger engagements get a dedicated engineering and operations team, and physical AI programs add in-house robotics researchers who help write data specifications
• Public sector and sovereign programs include forward deployed engineers plus upskilling and enablement so the customer's own staff can operate the system
• Documentation spans a quick start, end-to-end guides, an API reference, code recipes, and release notes

The documentation covers common use cases well, while advanced customization still requires a conversation with the support team. A team with no technical staff will lean heavily on the assigned engineer. Scale builds its engagements around exactly that, so people teach you the platform instead of the product doing it on its own.

Ease of Use and UX

Initial setup goes quickly, and the interface is well organized for the amount it holds.

What takes time comes later. Knowledge bases, assistants, and models each offer so many options that new users lose track of how the pieces connect, and reviewers want an interactive walkthrough with sample data and guided prompts to make that first hour easier. Retrieval controls stay coarse too. When an answer skips the most relevant source, nothing on screen tells you whether your data, your settings, or your prompt caused it. One team spent considerable time getting custom labeling workflows to handle its own industry vocabulary correctly. A team sees something working on the first day. Making it work consistently is the part that takes weeks.

• Scale publishes no pricing and no plan tiers on its product pages, and every path ends at a demo request or a sales contact
• Scale Pro is the high-volume data platform, which Scale sells with dedicated Engagement Managers, customized project setup, and quality guarantees backed by service level agreements
• The GenAI Platform and Donovan are separate products rather than tiers of one plan
• Data labeling costs scale with volume and rise further when ambiguous cases need human review

Scale pricing cannot be compared against alternatives before a sales conversation, and one reviewer specifically asked for more transparent pricing as a product improvement. A team should estimate how much of its data needs human judgment before it models the total cost. The structure rewards buyers with predictable, high volumes. Teams running small or irregular workloads carry the overhead of a sales-led relationship without the volume that justifies it.

Mayo Clinic works with Scale to develop and deploy AI applications for clinical care, running on the Scale Generative AI Platform. The obstacle was industry-wide before it was specific: a 2025 study found that 77% of health systems cite immature AI tools as a significant barrier to adoption. Inside the clinic, clinicians spent long stretches reviewing patient health records before the initial consultation, and critical safety events such as wrong-site surgeries or serious falls sat hidden inside routine reporting noise that staff worked through by hand.

The collaboration prioritizes three areas: cutting record review time before consultations, detecting safety events automatically, and reducing administrative tasks so Mayo staff operate at the top of their license while the clinic improves round-the-clock care. Scale reports that since launch, doctors spend an average of eleven more minutes with each patient while reliably maintaining an expert standard of care. Data remains inside Mayo Clinic's secure, HIPAA-compliant environment.

"Ultimately, this work is about helping patients by using innovation to deliver more timely, coordinated and compassionate care," said Matthew R. Callstrom, M.D., Ph.D., medical director, AI program at Mayo Clinic. "This collaboration combines each organization's expertise to advance care delivery and patient experience."

You can read the full case here.

Aspect	Scale	LangSmith	Langdock
Primary job	Training data, evaluation, and enterprise agent infrastructure	Observing, evaluating, and deploying agents teams build themselves	AI adoption across a whole workforce
Human workforce included	Yes, a managed expert network	No, though your own domain experts can review outputs and annotate traces	No
Who operates it	Scale engineers alongside your team	Your engineering team, with Fleet opening no-code agents to non-technical staff	Any employee
Deployment options	Customer VPC on three clouds, plus classified and air-gapped networks	Hosted service with data residency in three regions, your own cloud, or self-hosted	European Union hosting, dedicated, own cloud, or on-premises
How you start	Sales-led demo, no published pricing	Self-serve signup, with published pricing and a demo option	Seven-day trial with no card

These three tools solve different problems, and picking between them starts with what you are building:

Choose Scale if you need labeled data, model evaluation, or agents running under audit in a regulated or classified setting, and you have engineers to work with a vendor team.

Choose LangSmith if your developers already build agents and the missing piece is visibility: tracing full conversations, monitoring cost, latency, and errors in production, and running evaluators against real traces before shipping a change. Its Fleet layer also lets non-technical staff describe an agent in plain language, which narrows the gap with tools built for general business users.

Choose Langdock if nobody on staff writes code and the goal is getting a whole team using AI safely, with guided onboarding and native integrations doing the heavy lifting.