NAV
cURL JavaScript Python Ruby

Introduction

API Endpoint

https://api.scale.ai/v1/

Welcome to the Scale! You can use our API to access Scale endpoints, which can create, access, and cancel human tasks.

Computer Vision

Contact Sales to discuss these products:

Natural Language

Anyone can use our API! Sign up here to get started using Scale. You can also contact us for volume pricing or any inquiries.

You can also join our Slack channel here!

Getting Started with Scale

As part of Scale’s mission to democratize access to intelligent data, we’re committed to making high quality, human-powered training data accessible for anyone in the AI development community. That’s why we provide two ways for customers to work with us: On-Demand and Enterprise engagements. The On-Demand model means no commitment requirements, no platform fees, and you pay-as-you-go. Furthermore, with On-Demand, you can just sign up and send in tasks via our developer-friendly API. After a combination of human work and review, smart tools, and statistical confidence checks and machine learning checks, we’ll return scalable, accurate training data. It’s a great way to get started and experience Scale’s ease-of-use and quality.

Ready to get started? Follow the 4 steps below and you’ll be well on your way to building a robust training dataset for your next AI application.

Sign Up

If you haven’t already, create a Scale account. When prompted, input your credit card information to activate your account. You’ll only be charged for completed tasks under our On-Demand rates (your first 5 tasks are free), which are based on the task type, volume of data and number of annotations with each task.

Note: On-demand tasks are completed best-effort for quality and response urgency. For quality & turnaround SLAs, chat with us about our Enterprise engagements.

Design Your Task

To maximize the quality of your task, we recommend writing clear and concise instructions that include:

Tip: To allow you to collaborate on instructions with your team and embed formatting and images, create your instructions in a shared document via Google Docs and embed it into the API call via markdown:

  1. Create a Google Doc with your instructions
  2. Click File → Publish To Web → Embed (in the window that appears)
  3. Copy the iFrame tag that’s provided: <iframe src="[YOUR_CUSTOM_DOC_LINK]"></iframe>
  4. Paste the iFrame tag in the instruction parameter.

Send Your Task

Before submitting an API call, retrieve your Team API Keys and authenticate access to our API:

Explore our docs to find the task type that’s the best fit for your use case and the specific parameters you must define for each task object. When you’re ready, write your API call following our docs (we support cURL, JavaScript, Python, and Ruby), and submit it via Terminal or a 3rd-party software like Postman.

Note: Our On-Demand service limits to 5000 tasks per month. If you’d like to send more tasks, chat with us about our Enterprise engagements.

Review Your Response

Once you submit your task, you will receive a unique task_id, which you can use to track the task via your Dashboard. Here’s what you can do on each page:

Overview

View a summary of your submissions over time and the status for each task type.

Tasks

View a more granular list of your submitted tasks, filterable by status, project, and batch. If you click a specific task, you can view the original API call. For completed tasks, there are additional options that will allow you Audit the task, Resend Callback, and download the JSON response. You can also mass export the JSON responses for all your tasks via the Export JSON link.

Tip: Use our Teams feature to allow multiple users to have access to the same set of tasks.

Quality Assurance

Audit a random sample of tasks for quality and view stats for tasks you’ve reviewed. When you audit a task, we give you access to our annotation application where you can see the original source data with completed labels. You can Approve or Reject the task, and provide feedback if the task was incorrect.

If you any questions, feel free to join our Slack channel here or ping us on our web chat. If you’re interested in learning more about our Enterprise engagements, get in touch with us.

Client Libraries / SDKs

To install the client library, use the following command:

pip install --upgrade scaleapi

npm install scaleapi --save

gem install scaleapi

Currently we have client libraries available for the following languages:

We also have unofficial SDKs in the following languages:

Let us know if you want (or are interested in writing) a library for a language not represented here!

Authentication

To authorize, use this code:

# With curl, you can just pass the correct header with each request
curl "api_endpoint_here" \
  -u "{{ApiKey}}:"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

require 'scale'

scale = Scale.new(api_key: '{{ApiKey}}')
=> #<Scale:0x007fcc1292fe88 @api_key="{{ApiKey}}", @callback_auth_key=nil, @default_request_params={:callback_url=>nil}, @logging=false>

You must replace {{ApiKey}} with your personal API key. If you sign up or log in, your API key will be automatically filled in the docs. Your test API key {{ApiKey}} is included in all the examples on this page, so you can test any example right away. Only you can see this value.

Scale uses API keys to allow access to the API. You can find your API keys on your dashboard, which you can access by logging in or signing up.

Scale expects for the API key to be included in all API requests to the server via HTTP Basic Auth. Provide your API key as the basic auth username value. You do not need to provide a password. You can do this using the -u flag:

-u "{{ApiKey}}:"

Test and Live Modes

To make the API as explorable as possible, accounts have test mode and live mode API keys. There is no “switch” for changing between modes, just use the appropriate key to perform a live or test API requests.

Requests made with test mode credentials are not completed by a human, and therefore have incorrect test responses. Requests made with live mode credentials are always completed by a human, and will incur a charge past your first 5 API requests.

Callback Authentication

If you’d like to authenticate our callbacks, we set a scale-callback-auth HTTP header on each of our callbacks. The value will be equal to your Live Callback Auth Key shown on your dashboard. If this header is not set, or it is set incorrectly, the callback is not from Scale.

Task Object

The task object represents a single task that you create with Scale and is completed by a worker.

Attributes

{
  "task_id": "576ba74eec471ff9b01557cc",
  "completed_at": "2016-06-23T09:10:02.798Z",
  "created_at": "2016-06-23T09:09:34.752Z",
  "callback_url": "http://www.example.com/callback",
  "type": "categorization",
  "status": "completed",
  "instruction": "Would you say this item is big or small?",
  "params": {
    "attachment_type": "text",
    "attachment": "car",
    "categories": [
      "big",
      "small"
    ]
  },
  "callback_succeeded": true,
  "response": {
    "category": "big"
  },
  "metadata": {}
}
Attribute Type Description
task_id string The task_id is the unique identifier for the task.
type string The type of the task. Currently, we support categorization, transcription, audiotranscription, comparison, annotation, and datacollection.
instruction string A markdown-enabled string explaining the instructions for the task. You can use markdown to show example images, give structure to your instructions, and more. HTML tags are unsupported.
params object An object with the parameters of the task based on the type. For categorization, for example, this will include attachment_type, attachment, and categories.
response object An object corresponding to the response once the task is completed. For categorization, it will have the attribute category, corresponding to the chosen category.
callback_url string A string of the URL that should be POSTed once the task is completed for the response data. See the Callback section for more details.
status string The status of the task, one of pending, completed, canceled or error.
created_at timestamp A string of the UTC timestamp of when the task was created.
completed_at timestamp A string of the UTC timestamp of when the task was completed. This will only be filled in after it is completed.
callback_succeeded boolean A boolean stating whether or not the callback succeeded. If the callback returns with a 2xx status code, the value will be true. If the callback fails to return a 2xx status code through all retries, then the value will be false.
customer_review_status string The status of the QA’d task, one of pending, accepted, fixed or rejected.
metadata object, default {} A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

Metadata

Tasks objects have a metadata parameter. You can use this parameter to attach key-value data to tasks.

Metadata is useful for storing additional, structured information on an object. As an example, you could store a video’s unique identifier in your system on its content moderation categorization task. You can also use it to denote the end use case for the task, as “content moderation” or “data categorization” for example. Metadata is not used by Scale (e.g., to affect how the task is done).

Attachments

Callback example for bad attachment URL:

{
  "error": "One or more attachments could not be downloaded.",
  "attachments": [
    {
      "statusCode": 500,
      "url": "https://your-s3-bucket.s3.amazonaws.com/attachment-1.png"
    },
    {
      "statusCode": 500,
      "url": "https://your-s3-bucket.s3.amazonaws.com/attachment-2.png"
    }
  ]
}

Tasks often require a file associated with them that Scale calls an attachment. For example, an annotation task requires an image file to show to a Scaler. These attachments are not limited to images and may also be audio, video, or pdf files, or a website, or even plain text. For all attachment types except plain text, the attachment must be specified via a URL. See the specific task documentation below for some examples.

Currently we only support fetching URLs whose protocol is HTTP or HTTPS. Make sure that the URL is a direct link to the file you wish to use as an attachment and not a document showing a preview of the file. For certain storage providers we may be able to rewrite URLs that are previews of the file you are trying to attach to the correct direct download URL. HTTPS is preferred over HTTP, as HTTPS guarantees data integrity, preventing against corruption.

Scale will attempt to fetch the attachment via the URL you provide. If we do not receive an HTTP 200 response when attempting to fetch your attachment(s), we will send your callback_url an error with the bad HTTP codes we received, and the task status will be error.

For audio and video attachments we support the containers and formats listed on this Chromium project page; for image attachments we support JPEG, PNG, BMP, GIF, and TIFF file formats up to a maximum of 10000 pixels in width or height.

Create Computer Vision Tasks

2D Box Annotation

curl "https://api.scale.ai/v1/task/annotation" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Draw a box around each **car** and **pedestrian**." \
  -d attachment_type=image \
  -d attachment="http://i.imgur.com/XOJbalC.jpg" \
  -d objects_to_annotate="car" \
  -d objects_to_annotate="pedestrian" \
  -d with_labels=true \
  -d min_width="30" \
  -d min_height="30"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_annotation_task(
    callback_url='http://www.example.com/callback',
    instruction='Draw a box around each **car** and **pedestrian**',
    attachment_type='image',
    attachment='http://i.imgur.com/XOJbalC.jpg',
    objects_to_annotate=['car', 'pedestrian'],
    with_labels=True,
    min_width='30',
    min_height='30'
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createAnnotationTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Draw a box around each **car** and **pedestrian**',
  'attachment_type': 'image',
  'attachment': 'http://i.imgur.com/XOJbalC.jpg',
  'objects_to_annotate': ['car', 'pedestrian'],
  'with_labels': true,
  'min_width': '30',
  'min_height': '30'
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_annotation_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Draw a box around each **car** and **pedestrian**',
  attachment_type: 'image',
  attachment: 'http://i.imgur.com/XOJbalC.jpg',
  objects_to_annotate: ['car', 'pedestrian'],
  with_labels: true,
  min_width: '30',
  min_height: '30'
})
=> #<Scale::Api::Tasks::ImageRecognition:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="annotation", @instruction="Draw a box around each **car** and **pedestrian**", @params={"with_labels"=>true, "objects_to_annotate"=>["car", "pedestrian"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/v4cBreD.jpg"}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "5774cc78b01249ab09f089dd",
  "created_at": "2016-9-03T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "annotation",
  "status": "pending",
  "instruction": "Draw a box around each **car** and **pedestrian**",
  "params": {
    "with_labels": true,
    "min_width": 30,
    "min_height": 30,
    "objects_to_annotate": [
      "car",
      "pedestrian"
    ],
    "attachment_type": "image",
    "attachment": "http://i.imgur.com/XOJbalC.jpg"
  },
  "metadata": {}
}

This endpoint creates an annotation task. In this task, one of our Scalers view the given image and draw bounding boxes around the specified objects, returning the positions and sizes of these boxes.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

You can optionally provide additional markdown-enabled instructions via the instruction parameter.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

It is recommended, but not required, for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.

You may also provide min_width and min_height parameters, which will tell Scalers to only annotate objects whose bounding boxes are of dimension at least min_width x min_height.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.ai/v1/task/annotation

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
objects_to_annotate [string] An array of strings describing which objects you’d like bounding boxes to be drawn around. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 7 objects. If you need more objects, try using nested annotation labels.
attachment string A URL to the image you’d like to be annotated with bounding boxes.
with_labels (optional, default true) boolean Specifies whether you’d like labels for each bounding box in the response. Each label will be a member of the objects_to_annotate array.
min_height (optional) integer, default 0 The minimum height in pixels of the bounding boxes you’d like to be made.
min_width (optional) integer, default 0 The minimum width in pixels of the bounding boxes you’d like to be made.
instruction (optional) string A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type (optional, default image) string Describes what type of file the attachment is. We currently only support image for the annotation endpoint.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
layers (optional) object A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

Callback Format

Example callback body sent on completion

{
  "response": {
    "annotations": [
      {
        "left": 2,
        "top": 4,
        "width": 3,
        "height": 5,
        "label": "pedestrian"
      },
      {
        "left": 7,
        "top": 5,
        "width": 14,
        "height": 5,
        "label": "car"
      },
      { ... },
      { ... }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have either an error field or an annotations field.

If the annotation was completed successfully, the annotations field will contain an array of annotations. Each annotation will have the following values:

Visual representations of the task responses are accessible via the Scale dashboard. In these, the border is drawn inside the bounding box. The example response body (at right) corresponds to the image below.

If the attachment was invalid, the error will be detailed in the error field.

Point Annotation

curl "https://api.scale.ai/v1/task/pointannotation" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Draw a point on every **headlight** and **brakelight** of a car in the image." \
  -d attachment_type=image \
  -d attachment="http://i.imgur.com/XOJbalC.jpg" \
  -d objects_to_annotate="headlight" \
  -d objects_to_annotate="brakelight" \
  -d with_labels=true

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_pointannotation_task(
    callback_url='http://www.example.com/callback',
    instruction='Draw a point on every **headlight** and **brakelight** of a car in the image.',
    attachment_type='image',
    attachment='http://i.imgur.com/XOJbalC.jpg',
    objects_to_annotate=['headlight', 'brakelight'],
    with_labels=True
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createPointannotationTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Draw a point on every **headlight** and **brakelight** of a car in the image.',
  'attachment_type': 'image',
  'attachment': 'http://i.imgur.com/XOJbalC.jpg',
  'objects_to_annotate': ['headlight', 'brakelight'],
  'with_labels': true
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_pointannotation_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Draw a point on every **headlight** and **brakelight** of a car in the image.',
  attachment_type: 'image',
  attachment: 'http://i.imgur.com/XOJbalC.jpg',
  objects_to_annotate: ['headlight', 'brakelight'],
  with_labels: true
})
=> #<Scale::Api::Tasks::Pointannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="pointannotation", @instruction="Draw a point on every **headlight** and **brakelight** of a car in the image.", @params={"with_labels"=>true, "objects_to_annotate"=>["headlight", "brakelight"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"},@response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "5774cc78b01249ab09f089dd",
  "created_at": "2016-9-03T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "pointannotation",
  "status": "pending",
  "instruction": "Draw a point on every **headlight** and **brakelight** of a car in the image.",
  "params": {
    "with_labels": true,
    "objects_to_annotate": [
      "headlight",
      "brakelight"
    ],
    "attachment_type": "image",
    "attachment": "http://i.imgur.com/XOJbalC.jpg"
  },
  "metadata": {}
}

This endpoint creates a pointannotation task. In this task, one of our Scalers view the given image and draw points at the specified locations, returning the locations of these points.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated. You may include at most 6 objects in the objects_to_annotate parameter. If you need more objects, try using nested annotation labels.

You can optionally provide additional markdown-enabled instructions via the instruction parameter.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.ai/v1/task/pointannotation

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
objects_to_annotate [string] An array of strings describing which objects you’d like points to be drawn on. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 6 objects. If you need more objects, try using nested annotation labels.
attachment string A URL to the image you’d like to be annotated with points.
with_labels (optional, default true) boolean Specifies whether you’d like labels for each point in the response. Each label will be a member of the objects_to_annotate array.
instruction (optional) string A markdown-enabled string explaining how to draw the points. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type (optional, default image) string Describes what type of file the attachment is. We currently only support image for the point annotation endpoint.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
layers (optional) object A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

Callback Format

Example callback body sent on completion

{
  "response": {
    "annotations": [
      {
        "label": "headlight",
        "x": 123,
        "y": 10
      },
      {
        "label": "headlight",
        "x": 140,
        "y": 49
      },
      {
        "label": "brakelight",
        "x": 67,
        "y": 34
      }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // task inlined for convenience
    ...
  }
}

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain either an annotations field or an error field.

The annotations field will contain an array of point annotations. Each annotation will have the following values:

If the attachment was invalid, the error will be detailed in the error field.

Line Annotation

curl "https://api.scale.ai/v1/task/lineannotation" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Annotate lines over all of the **lane lines** in the street-level image." \
  -d attachment_type=image \
  -d attachment="http://i.imgur.com/XOJbalC.jpg" \
  -d objects_to_annotate="solid line" \
  -d objects_to_annotate="dashed line" \
  -d with_labels=true

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_lineannotation_task(
    callback_url='http://www.example.com/callback',
    instruction='Annotate lines over all of the **lane lines** in the street-level image.',
    attachment_type='image',
    attachment='http://i.imgur.com/XOJbalC.jpg',
    objects_to_annotate=['solid line', 'dashed line'],
    with_labels=True
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createLineannotationTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Annotate lines over all of the **lane lines** in the street-level image.',
  'attachment_type': 'image',
  'attachment': 'http://i.imgur.com/XOJbalC.jpg',
  'objects_to_annotate': ['solid line', 'dashed line'],
  'with_labels': true
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_lineannotation_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Annotate lines over all of the **lane lines** in the street-level image.',
  attachment_type: 'image',
  attachment: 'http://i.imgur.com/XOJbalC.jpg',
  objects_to_annotate: ['solid line', 'dashed line'],
  with_labels: true
})
=> #<Scale::Api::Tasks::Lineannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="lineannotation", @instruction="Annotate lines over all of the **lane lines** in the street-level image.", @params={"with_labels"=>true, "objects_to_annotate"=>["solid line", "dashed line"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "5774cc78b01249ab09f089dd",
  "created_at": "2016-9-03T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "lineannotation",
  "status": "pending",
  "instruction": "Annotate lines over all of the **lane lines** in the street-level image.",
  "params": {
    "with_labels": true,
    "objects_to_annotate": [
      "solid line",
      "dashed line"
    ],
    "attachment_type": "image",
    "attachment": "http://i.imgur.com/XOJbalC.jpg"
  },
  "metadata": {}
}

This endpoint creates a lineannotation task. In this task, one of our Scalers view the given image and draw segmented lines along each object, returning the vertices of these segmented lines.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

You can optionally provide additional markdown-enabled instructions via the instruction parameter.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

If you’d prefer splines as opposed to segmented lines, then you may specify the splines flag as true.

You can also optionally set min_vertices and/or max_vertices which specify the minimum and maximum number of vertices that you would want on a line. Note the range is inclusive of both bounds. For example, if min_vertices is 2 and max_vertices is 2, then Scale will only return line segments.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.ai/v1/task/lineannotation

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
objects_to_annotate [string] An array of strings describing which objects you’d like segmented lines to be drawn along. Each string should be singular and self-descriptive (ex: “lane line”, “crop line”). You may include at most 3 objects. If you need more objects, try using nested annotation labels.
attachment string A URL to the image you’d like to be annotated with segmented lines.
with_labels (optional, default true) boolean Specifies whether you’d like labels for each segmented line in the response. Each label will be a member of the objects_to_annotate array.
instruction (optional) string A markdown-enabled string explaining how to draw the segmented lines. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type (optional, default image) string Describes what type of file the attachment is. We currently only support image for the line annotation endpoint.
splines (optional, default false) boolean Specifies whether or not you’d like your lines drawn as cardinal splines instead of segmented lines
min_vertices (optional, default 1) number An optional parameter defining the minimum number of vertices in a valid line annotation for your request.
max_vertices (optional, default null) number An optional parameter defining the maximum number of vertices in a valid line annotation for your request. Must be at least min_vertices.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
layers (optional) object A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

Callback Format

Example callback body sent on completion

{
  "response": {
    "annotations": [
      {
        "label": "solid line",
        "vertices": [
            {
                "x": 123,
                "y": 10
            },
            {
                "x": 140,
                "y": 49
            },
            {
                "x": 67,
                "y": 34
            }
        ]
      },
      { ... },
      { ... }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain either an annotations field or an error field.

The annotations field will contain an array of annotations. Each annotation will have the following values:

If the attachment was invalid, the error will be detailed in the error field.

Cuboid Annotation

curl "https://api.scale.ai/v1/task/cuboidannotation" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Draw a box around each **car** and **pedestrian**." \
  -d attachment_type=image \
  -d attachment="http://i.imgur.com/XOJbalC.jpg" \
  -d objects_to_annotate="car" \
  -d objects_to_annotate="pedestrian" \
  -d with_labels=true \
  -d min_width="30" \
  -d min_height="30"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_cuboidannotation_task(
    callback_url='http://www.example.com/callback',
    instruction='Draw a cuboid around each car or truck.',
    attachment_type='image',
    attachment='http://i.imgur.com/v4cBreD.jpg',
    objects_to_annotate=['headlight', 'brakelight'],
    with_labels=True
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createCuboidannotationTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Draw a cuboid around each car or truck.',
  'attachment_type': 'image',
  'attachment': 'http://i.imgur.com/v4cBreD.jpg',
  'objects_to_annotate': ['car', 'truck'],
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_cuboidannotation_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Draw a cuboid around each car or truck.',
  attachment_type: 'image',
  attachment: 'http://i.imgur.com/v4cBreD.jpg',
  objects_to_annotate: ['car', 'truck'],
})
=> #<Scale::Api::Tasks::Cuboidannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="cuboidannotation", @instruction="Draw a cuboid around each car or truck", @params={"with_labels"=>true, "objects_to_annotate"=>["car", "truck"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/v4cBreD.jpg"}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "5774cc78b01249ab09f089dd",
  "created_at": "2016-9-03T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "cuboidannotation",
  "status": "pending",
  "instruction": "Draw a cuboid around each car or truck.",
  "params": {
    "objects_to_annotate": [
      "car",
      "truck"
    ],
    "attachment_type": "image",
    "attachment": "http://i.imgur.com/XOJbalC.jpg"
  },
  "metadata": {}
}

This endpoint creates a cuboidannotation task. Given a 2D image, and camera intrinsics and extrinsics as optional parameters, Scale will annotate the image with perspective cuboids and return the vertices in the image of these cuboids. If camera intrinsics and extrinsics are provided as well, Scale will return scale-invariant 3D coordinates with respect to the camera, i.e. assuming the camera is at the origin.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the callbacks section. The attachment is a URL to an image you’d like to be annotated. objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

You can optionally provide additional markdown-enabled instructions via the instruction parameter.

It is strongly recommended for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.

You may also provide min_width and min_height parameters, which will tell Scalers to only annotate objects whose cuboids are of dimension at least min_width x min_height.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.ai/v1/task/cuboidannotation

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
objects_to_annotate Array<string> An array of strings describing which objects you’d like cuboids to be drawn around. Each string should be singular and self-descriptive (ex: “car”, “street sign”, “pedestrian”). You may include at most 6 objects. If you need more objects, try using nested annotation labels.
attachment string A URL to the image you’d like to be annotated with perspective cuboids.
instruction (optional) string A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more.
camera_intrinsics (optional) object Object defining camera intrinsics, in format {fx: 1300, fy: 1300, cx: 600, cy: 300, scalefactor: 2.75, skew: 0} (skew defaults to 0, scalefactor defaults to 1). scalefactor is used if the image sent is of different dimensions from the original photo (if the attachment is half the original, set scalefactor to 2) to correct the focal lengths, offsets. Use in conjunction with camera_rotation_quaternion to get perspective-corrected cuboids and 3d points.
camera_rotation_quaternion (optional) object Object defining the rotation of the camera in relation to the world. Expressed as a quaternion, in format {w: 0.5, x: 0.5, y: 0.5, z: 0.5}. Use in conjunction with camera_intrinsics to get perspective-corrected cuboids and 3d points. Note that the z-axis of the camera frame represents the camera’s optical axis.
min_height (optional, default 0) integer The minimum height in pixels of the bounding cuboids you’d like to be made.
min_width (optional, default 0) integer The minimum width in pixels of the bounding cuboids you’d like to be made.
min_annotations (optional, default 0) integer The minimum number of cuboids that will be returned. Should only be used when one is certain that the image contains at least this many annotatable cuboids.
max_annotations (optional) default no limit The maximum number of cuboids that can be annotated in the image. A value of 0 (default) means no limit.
attachment_type (optional, default image) string Describes what type of file the attachment is. We currently only support image for the annotation endpoint.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
layers (optional) object A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

Response & Callback Format

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain only an annotations field.

The annotations field will contain an array of Cuboid objects. We define the Vertex and Cuboid objects below:

Definition: Vertex objects

Vertex objects are represented as dictionaries with the following structure:

Key Description
x The distance, in pixels, between the vertex and the left border of the image.
y The distance, in pixels, between the vertex and the top border of the image.
type This will be “vertex” for all points
description An plaintext description of this vertex

This is the list of all possible descriptions for a vertex:

Definition: Edge objects

Edge objects are represented as dictionaries with the following structure:

Key Description
x1 The distance, in pixels, between the first vertex and the left border of the image.
y1 The distance, in pixels, between the first vertex and the top border of the image.
x2 The distance, in pixels, between the second vertex and the left border of the image.
y2 The distance, in pixels, between the second vertex and the top border of the image.
type This will be “edge” for all edges
description An plaintext description of this edge

This is the list of all possible descriptions for an edge:

Definition: Cuboid objects

Points on cuboid are returned in this order for both points_2d and points_3d:
       3-------2
      /|      /|
     / |     / |
    0-------1  |
    |  7----|--6
    | /     | /
    4-------5

Cuboid objects are represented as dictionaries with the following structure:

Key Description
label string belonging to objects_to_annotate which describes what kind of object the cuboid annotates
vertices A list of Vertex objects defining all visible vertices of the cuboid.
edges A list of Edge objects defining the edges of the cuboid.
points_2d (optional) If camera_rotation_quaternion, camera_intrinsics were provided, gives 2d points of all 8 vertices of the cuboid after perspective correction.
points_3d (optional) If camera_rotation_quaternion, camera_intrinsics were provided, gives 3d coordinates (arbitrarily scaled, relative to the camera location) of all 8 vertices of the cuboid after perspective correction.

Example callback body sent on completion

{
  "response": {
    "annotations": [
      {
        "label": "car",
        "vertices": [
          {
            "description": "face-topleft",
            "y": 270,
            "x": 293,
            "type": "vertex"
          },
          {
            "description": "face-bottomleft",
            "y": 437,
            "x": 293,
            "type": "vertex"
          },
          {
            "description": "face-topright",
            "y": 270,
            "x": 471,
            "type": "vertex"
          },
          {
            "description": "face-bottomright",
            "y": 437,
            "x": 471,
            "type": "vertex"
          },
          {
            "description": "side-topcorner",
            "y": 286,
            "x": 607,
            "type": "vertex"
          },
          {
            "description": "side-bottomcorner",
            "y": 373,
            "x": 607,
            "type": "vertex"
          }
        ],
        "edges": [
          {
            "description": "face-top",
            "x1": 293,
            "y1": 270,
            "x2": 471,
            "y2": 270,
            "type": "edge"
          },
          {
            "description": "face-right",
            "x1": 471,
            "y1": 270,
            "x2": 471,
            "y2": 437,
            "type": "edge"
          },
          {
            "description": "face-bottom",
            "x1": 471,
            "y1": 437,
            "x2": 293,
            "y2": 437,
            "type": "edge"
          },
          {
            "description": "face-left",
            "x1": 293,
            "y1": 437,
            "x2": 293,
            "y2": 270,
            "type": "edge"
          },
          {
            "description": "side-top",
            "x1": 471,
            "y1": 270,
            "x2": 607,
            "y2": 286,
            "type": "edge"
          },
          {
            "description": "side-bottom",
            "x1": 471,
            "y1": 437,
            "x2": 607,
            "y2": 373,
            "type": "edge"
          }
        ]
      }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

Polygon Annotation

curl "https://api.scale.ai/v1/task/polygonannotation" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Draw a tight polygon around every **car** in the image." \
  -d attachment_type=image \
  -d attachment="http://i.imgur.com/XOJbalC.jpg" \
  -d objects_to_annotate="car" \
  -d objects_to_annotate="truck" \
  -d with_labels=true

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_polygonannotation_task(
    callback_url='http://www.example.com/callback',
    instruction='Draw a tight polygon around every **car** in the image.',
    attachment_type='image',
    attachment='http://i.imgur.com/XOJbalC.jpg',
    objects_to_annotate=['car', 'truck'],
    with_labels=True
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createPolygonannotationTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Draw a tight polygon around every **car** in the image.',
  'attachment_type': 'image',
  'attachment': 'http://i.imgur.com/XOJbalC.jpg',
  'objects_to_annotate': ['car', 'truck'],
  'with_labels': true
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_polygonannotation_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Draw a tight polygon around every **car** in the image.',
  attachment_type: 'image',
  attachment: 'http://i.imgur.com/XOJbalC.jpg',
  objects_to_annotate: ['car', 'truck'],
  with_labels: true
})
=> #<Scale::Api::Tasks::Polygonannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="polygonannotation", @instruction="Draw a tight polygon around every **car** in the image.", @params={"with_labels"=>true, "objects_to_annotate"=>["car", "truck"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "5774cc78b01249ab09f089dd",
  "created_at": "2016-9-03T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "polygonannotation",
  "status": "pending",
  "instruction": "Draw a tight polygon around every **car** in the image.",
  "params": {
    "with_labels": true,
    "objects_to_annotate": [
      "car",
      "truck"
    ],
    "attachment_type": "image",
    "attachment": "http://i.imgur.com/XOJbalC.jpg"
  },
  "metadata": {}
}

This endpoint creates a polygonannotation task. In this task, one of our Scalers view the given image and draw bounding polygons around the specified objects, returning the vertices of these polygons.

The required parameters for this task are callback_url, attachment, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.

objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

You can optionally provide additional markdown-enabled instructions via the instruction parameter.

Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate list. If your application does not require the particular label, you can set the with_labels parameter to false.

You can also optionally set min_vertices and/or max_vertices which specify the minimum and maximum number of vertices that you’d like on a polygon. Note the range is inclusive of both bounds. For example, if min_vertices is 3 and max_vertices is 5, then Scale will only return triangles, quadrilaterals, and pentagons.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.ai/v1/task/polygonannotation

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
objects_to_annotate [string] An array of strings describing which objects you’d like bounding polygons to be drawn around. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 6 objects. If you need more objects, try using nested annotation labels.
attachment string A URL to the image you’d like to be annotated with bounding polygons.
with_labels (optional, default true) boolean Specifies whether you’d like labels for each bounding polygon in the response. Each label will be a member of the objects_to_annotate array.
instruction (optional) string A markdown-enabled string explaining how to draw the bounding polygons. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type (optional, default image) string Describes what type of file the attachment is. We currently only support image for the polygon annotation endpoint.
min_vertices (optional, default 1) number An optional parameter defining the minimum number of vertices in a valid polygon annotation for your request.
max_vertices (optional, default null) number An optional parameter defining the maximum number of vertices in a valid polygon annotation for your request. Must be at least min_vertices.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
layers (optional) object A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail.

Callback Format

Example callback body sent on completion

{
  "response": {
    "annotations": [
      {
        "label": "car",
        "vertices": [
            {
                "x": 123,
                "y": 10
            },
            {
                "x": 140,
                "y": 49
            },
            {
                "x": 67,
                "y": 34
            }
        ]
      },
      { ... },
      { ... }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // task inlined for convenience
    ...
  }
}

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain either an annotations field or an error field.

The annotations field will contain an array of annotations. Each annotation will have the following values:

If the attachment was invalid, the error will be detailed in the error field.

Semantic Segmentation

curl "https://api.scale.ai/v1/task/segmentannotation" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Please segment the image using the given labels." \
  -d attachment_type=image \
  -d attachment="http://i.imgur.com/XOJbalC.jpg" \
  -d labels="background" \
  -d labels="road" \
  -d labels="vegetation" \
  -d labels="lane marking" \
  -d instance_labels="vehicle" \
  -d instance_labels="pedestrian" \
  -d allow_unlabeled=false

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_segmentannotation_task(
    callback_url='http://www.example.com/callback',
    instruction='Please segment the image using the given labels.',
    attachment_type='image',
    attachment='http://i.imgur.com/XOJbalC.jpg',
    labels=['background', 'road', 'vegetation', 'lane marking'],
    instance_labels=['vehicle', 'pedestrian'],
    allow_unlabeled=False
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createSegmentannotationTask({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Please segment the image using the given labels.',
  attachment_type: 'image',
  attachment: 'http://i.imgur.com/XOJbalC.jpg',
  labels: ['background', 'road', 'vegetation', 'lane marking'],
  instance_labels: ['vehicle', 'pedestrian'],
  allow_unlabeled: false
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_segmentannotation_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Please segment the image using the given labels.',
  attachment_type: 'image',
  attachment: 'http://i.imgur.com/XOJbalC.jpg',
  labels: ['background', 'road', 'vegetation', 'lane marking'],
  instance_labels: ['vehicle', 'pedestrian'],
  allow_unlabeled: false
})
=> #<Scale::Api::Tasks::Segmentannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="segmentannotation", @instruction="Please segment the image using the given labels.", @params={"allow_unlabeled"=>false, "labels"=>['background', 'road', 'vegetation', 'lane marking'], "instance_labels"=>['vehicle', 'pedestrian'], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "5774cc78b01249ab09f089dd",
  "created_at": "2016-9-03T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "segmentannotation",
  "status": "pending",
  "instruction": "Please segment the image using the given labels.",
  "params": {
    "allow_unlabeled": false,
    "labels": [
      "background",
      "road",
      "vegetation",
      "lane marking"
    ],
    "instance_labels": [
      "vehicle",
      "pedestrian"
    ],
    "attachment_type": "image",
    "attachment": "http://i.imgur.com/XOJbalC.jpg"
  },
  "metadata": {}
}

This endpoint creates a segmentannotation task. In this task, one of our Scalers view the given image and classify every pixel of the image according to the labels provided. You will get a full semantic, pixel-wise, dense segmentation of the image.

We also support instance-aware semantic segmentations, also called panoptic segmentation, via the instance_labels parameter.

The required parameters for this task are callback_url, attachment, and labels. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be segmented.

labels is an array of strings describing the different types of objects you’d like to be used to segment the image.

You can optionally provide additional markdown-enabled instructions via the instruction parameter.

You can also optionally set allow_unlabeled to true, which will allow the existence of unlabeled pixels in the segmentation response - otherwise, all pixels in the image will be classified (in which case it’s important that there are labels for everything in the image, to avoid misclassification).

The response you will receive will be a series of images where each pixel’s value corresponds to the label, either via a numerical index or a color mapping. You will also get separate masks for each label for convenience.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.ai/v1/task/segmentannotation

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
labels Array<string> An array of strings describing the different types of objects you’d like to be used to segment the image. Each string should be singular and descriptive (ex: car, background, pole). You may include at most 50 labels.
instance_labels (optional, default []) Array<string> An array of strings describing the different types of objects you’d like to be segmented on a per-instance basis. In the case of instance_labels, the objects will get different masks / colors for each instance. For example, if you defined car as an instance_label, each individual car would get a separate mask in the image, allowing you to distinguish between them. Instance labels cannot overlap with standard labels. You may include at most 10 instance labels. If you need more instance labels, try using nested annotation labels.
attachment string A URL to the image you’d like to be segmented.
allow_unlabeled (optional, default false) boolean Specifies whether you’ll allow unlabeled pixels in the segmentation. If left with the default value of false, all pixels in the image will be classified using the labels you provided.
instruction (optional) string A markdown-enabled string explaining how carry out the semantic segmentation. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type (optional, default image) string Describes what type of file the attachment is. We currently only support image for the semantic segmentation endpoint.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

Callback Format

Example callback body sent on completion

{
  "task": {
    // task inlined for convenience
    ...
  },
  "response": {
    "annotations": {
      "unlabeled": null,
      "labeled": {
        "lane marking": "https://scaleapi-attachments.s3.amazonaws.com/3f184900-6809-11e7-bb22-c346fd2b0658",
        "vehicle": [
          "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40a5f60",
          "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40a5120"
        ],
        "pedestrian": [
          "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40a5550",
          "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40b5230"
        ],
        "road": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b756bd10-8119-11e7-ac39-7d56f40a5f60",
        "vegetation": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b756bd11-8119-11e7-ac39-7d56f40a5f60",
        "background": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7573240-8119-11e7-ac39-7d56f40a5f60"
      },
      "combined": {
        "image": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7573241-8119-11e7-ac39-7d56f40a5f60",
        "indexedImage": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/ba9a9d70-8119-11e7-ac39-7d56f40a5f60"
      }
    },
    "labelMapping": {
      "lane marking": {
        "color": "#9e3fff",
        "index": 6
      },
      "vehicle": [
          {
            "color": "#8000ff",
            "index": 5
          },
          {
            "color": "#ff0080",
            "index": 4
          }
      ],
      "pedestrian": [
          {
            "color": "#80ffff",
            "index": 7
          },
          {
            "color": "#ff8888",
            "index": 8
          }
      ],
      "background": {
        "color": "#00ffff",
        "index": 3
      },
      "road": {
        "color": "#80ff00",
        "index": 2
      },
      "vegetation": {
        "color": "#ff0000",
        "index": 1
      }
    },
    "instances": {}
  },
  "task_id": "598e0da3e8068e06002d9407"
}

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain a labelMapping and an annotations field.

labelMapping is a dictionary where the keys are each label name, and the value is an object with the index and color used to represent each label in the response images. In the case of instance_labels, the value will be an array of such objects with index and color keys, representing each instance.

annotations is an object that contains URLs of PNG images describing the segmentation result in different ways:

Video Annotation

curl "https://api.scale.ai/v1/task/videoboxannotation" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Please track each object" \
  -d attachment_type=image \
  -d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png" \
  -d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png" \
  -d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png" \
  -d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png" \
  -d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png" \
  -d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png" \
  -d objects_to_annotate="car" \
  -d objects_to_annotate="pedestrian" \
  -d objects_to_annotate="cyclist" \
  -d annotation_attributes[location][conditions][label_condition][label][]='cyclist' \
  -d annotation_attributes[location][description]='Where is the cyclist?' \
  -d annotation_attributes[location][choices][]='Sidewalk' \
  -d annotation_attributes[location][choices][]='Bike lane' \
  -d annotation_attributes[location][choices][]='Shared lane' \
  -d annotation_attributes[location][choices][]='Road' \
  -d annotation_attributes[location][choices][]='Other' \
  -d with_labels=true \
  -d min_width="30" \
  -d min_height="30"

import requests
import json

payload = {
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Please track each object',
  'attachment_type': 'image',
  'attachments': [
    'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png',
    'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png',
    'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png',
    'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png',
    'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png',
    'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png',
  ],
  'objects_to_annotate': [ 'car', 'pedestrian', 'cyclist' ],
  'annotation_attributes': {
    'location': {
      'conditions': {
        'label_condition': {
          'label': [
            'cyclist'
          ]
        }
      },
      'description': 'Where is the cyclist?',
      'choices': [
        'Sidewalk',
        'Bike lane',
        'Shared lane',
        'Road',
        'Other'
      ]
    }
  },
  'with_labels': True,
  'min_width': '30',
  'min_height': '30',
}

headers = {"Content-Type": "application/json"}

task_request = requests.post("https://api.scale.ai/v1/task/videoboxannotation",
  json=payload,
  headers=headers,
  auth=('YOUR_SCALEAPI_KEY', ''))

print task_request.json()

// please see the Python example

# please see the Python example

The above command returns an object structured like this:

{
    "callback_url": "http://www.example.com/callback",
    "created_at": "2018-03-02T23:45:17.821Z",
    "instruction": "Please track each object",
    "is_test": false,
    "metadata": {},
    "params": {
        "annotation_attributes": {
            "location": {
                "choices": [
                    "Sidewalk",
                    "Bike lane",
                    "Shared lane",
                    "Road",
                    "Other"
                ],
                "conditions": {
                    "label_condition": {
                        "label": [
                            "cyclist"
                        ]
                    }
                },
                "description": "Where is the cyclist?"
            }
        },
        "attachment_type": "image",
        "attachments": [
            "https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png",
            "https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png",
            "https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png",
            "https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png",
            "https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png",
            "https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png"
        ],
        "min_height": 30,
        "min_width": 30,
        "objects_to_annotate": [
            "car",
            "pedestrian",
            "cyclist"
        ],
        "with_labels": true
    },
    "status": "pending",
    "task_id": "5a99e20de50d4979ce6d291e",
    "type": "videoboxannotation"
}

This endpoint creates a videoboxannotation task. Given a series of images sampled from a video, (which we will now refer to as “frames”), Scale will annotate each frame with bounding boxes.

The required parameters for this task are callback_url, attachments, and objects_to_annotate. The callback_url is the URL which will be POSTed on task completion, and is described in more detail in the callbacks section. The attachments are a list of URLs to the frames you’d like to be annotated. objects_to_annotate is an array of strings describing the different types of objects you’d like annotated.

You can optionally provide additional markdown-enabled instructions via the instruction parameter.

It is strongly recommended for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.

You may also provide min_width and min_height parameters, which will tell Scalers to only annotate objects whose boxes are of dimension at least min_width x min_height.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

HTTP Request

POST https://api.scale.ai/v1/task/videoboxannotation

Parameters

Parameter Type Description
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
objects_to_annotate Array<string> An array of strings describing which objects you’d like boxes to be drawn around. Each string should be singular and self-descriptive (ex: “car”, “street sign”, “pedestrian”). You may include at most 6 objects.
instruction (optional) string A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more.
min_height (optional, default 0) integer The minimum height in pixels of the bounding boxes you’d like to be made.
min_width (optional, default 0) integer The minimum width in pixels of the bounding boxes you’d like to be made.
attachment_type (optional, default image) string Describes what type of file the attachment is. We currently support image and video. In the case of video, we support all the formats that can be decoded by FFMPEG.
attachments Array<string> An array of URLs to the frames you’d like to be annotated with bounding boxes. This is required if attachment_type is image and must be omitted if attachment_type is video.
attachment string URL of the video file you’d like to be annotated with bounding boxes. This is required if attachment_type is video and must be omitted if attachment_type is image.
frame_rate (optional, default 1) integer Number of frames to capture in one second. This is ignored if attachment_type is image.
start_time (optional, default 0) integer Start time in seconds. This is ignored if attachment_type is image.
duration_time (optional, defaults to video length) integer Duration of the video in seconds. This is ignored if attachment_type is image.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
annotation_attributes (optional, default {}) object See the Annotation Attributes for more details about annotation attributes.

Response & Callback Format

Example callback body sent on completion

{
  "response": {
    "annotations": {
        "url": "https://scaleapi-results.s3.amazonaws.com/5e5dc4a0-1d9b-11e8-bce8-41122de76216"
    }
  },
  "task_id": "",
  "task": { ... }
}

The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain a link to the annotations file containing a list of annotations per frame. Each annotation object contains its unique id to be used for tracking an object as well as its position, label, and attributes.

Definition: Annotation objects

Example annotations file JSON

[
  {
    "annotations": [
      {
        "uuid": "26201bd9-72ab-4b63-ab7d-df4e671161b9",
        "type": "box",
        "label": "cyclist",
        "top": 144,
        "left": 845,
        "height": 168,
        "width": 72,
        "attributes": {
          "location": "Sidewalk"
        }
      },
      {
        "uuid": "4fa43f3a-81cf-492f-8b6e-3b7b2c824d05",
        "type": "box",
        "label": "car",
        "top": 190,
        "left": 330,
        "height": 76,
        "width": 109
      }
    ]
  },
  {
    "annotations": [
      {
        "uuid": "26201bd9-72ab-4b63-ab7d-df4e671161b9",
        "type": "box",
        "label": "cyclist",
        "top": 146,
        "left": 830,
        "height": 154,
        "width": 62,
        "attributes": {
          "location": "Sidewalk"
        }
      },
      {
        "uuid": "4fa43f3a-81cf-492f-8b6e-3b7b2c824d05",
        "type": "box",
        "label": "car",
        "top": 192,
        "left": 289,
        "height": 90,
        "width": 129
      }
    ]
  }
]

Annotation objects are represented as dictionaries with the following structure:

Key Type Description
uuid string A computer-generated identifier for this annotation’s path that can be used to track it across different frames
label string The label of this annotation. Across all frames, any annotation objects with the same uuid will have the same label.
attributes object See the Annotation Attributes section for more details about the attributes response field.
type string This will be “box” for all annotations.
left number The distance, in pixels, between the vertex and the left border of the image.
top number The distance, in pixels, between the vertex and the top border of the image.
width number The width, in pixels, of the bounding box.
height number The height, in pixels, of the bounding box.

The annotations file will be stored on S3 and be a JSON encoded list of annotation objects per frame. The index of each annotation list will correspond to the frame defined in the original response.

The order of the annotations lists will correspond directly to the order of the frame attachments that you sent in your original request. That is, the first annotation list will correspond to the first frame in the attachments array, etc.

Once the JSON file is reviewed and uploaded, we will send a response to your callback URL.

Computer Vision Layers

All of these input formats have the same formatting as our responses, so passing one Scale tasks’ data to another Scale task is really easy.

// This example uses express with body-parser
var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

app.post('/polygon_task_callback_handler', function(req, res) {
  // Validate callback auth key
  ...

  var polygons = req.body.response.annotations;
  var attachment = req.body.task.params.attachment;

  client.createAnnotationTask({
    'callback_url': 'http://www.example.com/annotation_task_callback_handler',
    'instruction': 'Draw a box around each **car** and **pedestrian**',
    'attachment_type': 'image',
    'attachment': attachment,
    'objects_to_annotate': ['car', 'pedestrian'],
    'with_labels': true,
    'min_width': '30',
    'min_height': '30',
    'layers': {
      'polygons': polygons
    }
  }, (err, task) => {
      // do something with task
  });
});

# This example uses Flask
import scaleapi
from flask import request, Flask

app = Flask(__name__)

client = scaleapi.ScaleClient('{{ApiKey}}')

@app.route('/polygon_task_callback_handler', methods=['POST'])
def create_annotation_task():
  # Validate callback auth key
  ...

  polygons = request.json['response']['annotations']
  attachment = request.json['task']['params']['attachment']

  task = client.create_annotation_task(
      callback_url='http://www.example.com/annotation_task_callback_handler',
      instruction='Draw a box around each **car** and **pedestrian**',
      attachment_type='image',
      attachment=attachment,
      objects_to_annotate=['car', 'pedestrian'],
      with_labels=True,
      min_width='30',
      min_height='30',
      layers={'polygons': polygons}
  )

  # do something with the task

# This example uses Rails
require 'scale'

class ExampleController < ActionController::Base
  def initialize
    @scale = Scale.new(api_key: '{{ApiKey}}')
  end

  # This should be mapped to a POST route
  def polygon_task_callback_handler
    # Validate callback auth key
    ...

    polygons = params['response']['annotations']
    attachment = params['task']['params']['attachment']

    task = @scale.create_annotation_task({
      callback_url: 'http://www.example.com/annotation_task_callback_handler',
      instruction: 'Draw a box around each **car** and **pedestrian**',
      attachment_type: 'image',
      attachment: attachment,
      objects_to_annotate: ['car', 'pedestrian'],
      with_labels: true,
      min_width: '30',
      min_height: '30',
      layers: {polygons: polygons}
    })

    # do something with the task
  end
end

“Layers” can be used to specify existing (read-only) boxes, lines, polygons, and/or cuboids to be pre-drawn on an image.

For instance, you could specify boxes around the cars of an image, for a task that requires drawing polygons around the currently boxed cars. Or you could create a task for drawing boxes around all cars which you hadn’t already recognized.

To specify layers, you can pass an optional layers parameter in the request for annotation, lineannotation, polygonannotation, or pointannotation tasks. They can also be used in categorization tasks if the attachment_type is image.

The layers parameter can contains fields for boxes, lines, polygons, and cuboids, which are arrays of the corresponding elements. Each of these elements are specified in the same format as the responses for their respective endpoints.

2D Boxes

Example layers param with boxes

{
    "lines": [
        ...
    ],
    "polygons": [
        ...
    ],
    "boxes": [
        {
            "label": "car",
            "height": 97,
            "width": 147,
            "top": 229,
            "left": 300
        }
    ],
    "cuboids": [
        ...
    ]
}

Each of the boxes is specified with an object containing left, top, width, and height keys, and an optional label.

Polygons

Example layers param with polygons

{
    ...
    "polygons": [
        {
            "vertices": [
                {
                    "y": 145,
                    "x": 356
                },
                {
                    "y": 103,
                    "x": 502
                },
                {
                    "y": 264,
                    "x": 482
                }
            ],
            "label": "building"
        }
    ]
}

polygons are objects with an optional label field, and a vertices field which contains a list of objects with x and y attributes.

Lines & Splines

Example layers param with lines

{
    ...
    "lines": [
        {
            "vertices": [
                {
                    "y": 323,
                    "x": 414
                },
                {
                    "y": 164,
                    "x": 616
                },
                {
                    "y": 334,
                    "x": 776
                }
            ],
            "label" : "crosswalk",
            "spline": true
        }
    ]
}

lines are objects with an optional label field, and a vertices field which contains a list of objects with x and y attributes, similar to polygons. Additionally, line objects have an optional spline flag, which determines whether the curve used is linear (if the value is false or the param is not passed) or cardinal (if the value is true).

3D Cuboids

Example layers param with cuboids

{
    ...
    "cuboids": [
        {
            "vertices" : [
                {
                    "description" : "face-topleft",
                    "y" : 219.0,
                    "x" : 137.0
                },
                {
                    "description" : "face-bottomleft",
                    "y" : 318.0,
                    "x" : 137.0
                },
                {
                    "description" : "face-topright",
                    "y" : 219.0,
                    "x" : 245.0
                },
                {
                    "description" : "face-bottomright",
                    "y" : 318.0,
                    "x" : 245.0
                },
                {
                    "description" : "side-topcorner",
                    "y" : 165.0,
                    "x" : 316.0
                },
                {
                    "description" : "side-bottomcorner",
                    "y" : 264.0,
                    "x" : 316.0
                }
            ],
            "label" : "car"
        }
    ]
}

cuboids are described in a similar way as polygons or lines, but the vertices also need a description, using the following values to identify them:

Nested Annotation Labels

curl "https://api.scale.ai/v1/task/annotation" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json" \
  -X POST \
  -d '
{
        "callback_url": "http://www.example.com/callback",
    "attachment": "https://i.imgur.com/VDPoOZE.jpg",
    "attachment_type": "image",
    "metadata": {
        "some_metadata": "some metadata"
    },
    "instruction": "Annotate the cars and pedestrians",
    "min_width": 30,
    "min_height": 30,
    "objects_to_annotate": [
      {
        "choice": "Vehicle",
        "subchoices": ["Car", "Truck", "Train", "Motorcycle"]
      },
      {
        "choice": "Pedestrian",
        "subchoices": ["Animal", "Adult", "Child"]
      }
    ],
    "annotation_attributes": {
        "occlusion": {
            "description": "What percent of the object is occluded?",
            "choices": ["0%", "25%", "50%", "75%"],
            "conditions": {
                "label_condition": {
                    "label": ["Car", "Truck", "Motorcycle"]
                }
            }
        }
    }
}'

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_cuboidannotation_task(
    callback_url="http://www.example.com/callback",
    attachment="https://i.imgur.com/VDPoOZE.jpg",
    attachment_type="image",
    metadata={
        "some_metadata": "some metadata"
    },
    instruction="Annotate the cars and pedestrians",
    min_width=30,
    min_height=30,
    objects_to_annotate=[
        {
            "choice": "Vehicle",
            "subchoices": ["Car", "Truck", "Train", "Motorcycle"]
        },
        {
            "choice": "Pedestrian",
            "subchoices": ["Animal", "Adult", "Child"]
        }
    ],
    annotation_attributes={
        "occlusion": {
            "description": "What percent of the object is occluded?",
            "choices": ["0%", "25%", "50%", "75%"],
            "conditions": {
                "label_condition": {
                    "label": ["Car", "Truck", "Motorcycle"]
                }
            }
        }
    }
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createCuboidannotationTask({
  callback_url: 'http://www.example.com/callback',
  attachment: 'https://i.imgur.com/VDPoOZE.jpg',
  attachment_type: 'image',
  metadata: {
    some_metadata: 'some metadata'
  },
  instruction: 'Annotate the cars and pedestrians',
  min_width: 30,
  min_height: 30,
  objects_to_annotate: [
    {
      choice: 'Vehicle',
      subchoices: ['Car', 'Truck', 'Train', 'Motorcycle']
    },
    {
      choice: 'Pedestrian',
      subchoices: ['Animal', 'Adult', 'Child']
    }
  ],
  annotation_attributes: {
    occlusion: {
      description: 'What percent of the object is occluded?',
      choices: ['0%', '25%', '50%', '75%'],
      conditions: {
        label_condition: {
          label: ['Car', 'Truck', 'Motorcycle']
        }
      }
    }
  }
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_cuboidannotation_task({
  callback_url: "http://www.example.com/callback",
  attachment: "https://i.imgur.com/VDPoOZE.jpg",
  attachment_type: "image",
  metadata: {
    some_metadata: "some metadata"
  },
  instruction: "Annotate the cars and pedestrians",
  min_width: 30,
  min_height: 30,
  objects_to_annotate: [
    {
      choice: "Vehicle",
      subchoices: ["Car", "Truck", "Train", "Motorcycle"]
    },
    {
      choice: "Pedestrian",
      subchoices: ["Animal", "Adult", "Child"]
    }
  ],
  annotation_attributes: {
    occlusion: {
      description: "What percent of the object is occluded?",
      choices: ["0%", "25%", "50%", "75%"],
      conditions: {
        label_condition: {
          label: ["Car", "Truck", "Motorcycle"]
        }
      }
    }
  }
})

The above command returns an object structured like this:

{
  "task_id": "5774cc78b01249ab09f089dd",
  "created_at": "2016-9-03T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "annotation",
  "status": "pending",
  "instruction": "Annotate the cars and pedestrians",
    "params": {
    "attachment": "https://i.imgur.com/VDPoOZE.jpg",
    "attachment_type": "image",
    "objects_to_annotate": [
      {
        "choice": "Vehicle",
        "subchoices": ["Car", "Truck", "Train", "Motorcycle"]
      },
      {
        "choice": "Pedestrian",
        "subchoices": ["Animal", "Adult", "Child"]
      }
    ],
    "with_labels": false,
    "min_width": 30,
    "min_height": 30,
    "examples": [],
    "annotation_attributes": {
      "occlusion": {
        "description": "What percent of the object is occluded?",
        "choices": ["0%", "25%", "50%", "75%"],
        "conditions": {
          "label_condition": {
            "label": ["Car", "Truck", "Motorcycle"]
          }
        }
      }
    }
  },
  "metadata": {}
}

There are often annotation tasks that have too many label choices for a worker to efficiently sort through them all at once. In those cases, a solution is to use nested labels, where labels may have subcategories within them. A simple example illustrated in the example API call on the right is when labeling several different kinds of pedestrians and vehicles. While there may be a large amount of total labels, using subchoices a worker can first categorize an object as pedestrian or vehicle, and based on that choice select the specific type of pedestrian or vehicle.

Nested labels may be specified both for the object labels (the objects_to_annotate array parameter), as well as in the choices array of a categorical annotation attribute. In both cases, you would specify a nested label by using an object instead of a string.

For example, for an objects_to_annotate array of ["Vehicle", "Pedestrian"], you could instead add a nested label by passing an array like ["Vehicle", {"choice": "Pedestrian", "subchoices": ["Animal", "Adult", "Child"]}]. Then, if a worker selected “Pedestrian” for an annotation, they would be further prompted to choose one of the corresponding subchoices for that annotation.

These hierarchies can themselves be nested, so that the subchoices array of a nested label may itself specify further nested labels.

Response Format

Nested labels are only intended for grouping a large set of labels together. Thus, the response will be the same as with unnested labels, where only the final “leaf” label that the worker selected is reported. In the example above, if a worker selected “Pedestrian” and then “Animal” for an annotation, that annotation’s label would be “Animal”.

An example response with nested labels is below:

{
  "response": {
    "annotations": [
      {
        "left": 123,
        "top": 10,
        "width": 121,
        "height": 39,
        "label": "Motorcycle",
        "attributes": {
          "occlusion": "0%"
        }
      },
      {
        "left": 82,
        "top": 56,
        "width": 64,
        "height": 30,
        "label": "Animal"
      },
      { ... },
      { ... }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

Conditional attributes

Nested labels can be used when specifying conditional attributes as well.

As with the response object for a nested label, the condition should specify the “leaf” labels that it is conditional on.

For example, if in the above example you wanted an “occlusion” attribute for some mix of the “leaf” labels, you could specify the label condition for the attribute as e.g. { "label": ["Car", "Truck", "Motorcycle", "Adult"] }. If you wanted the attribute to be conditional on all the subchoices of a given label (e.g., all “Vehicle"s), you would simply specify all of those subchoices in the array, e.g. { "label": ["Car", "Truck", "Train", "Motorcycle"] }.

Annotation Attributes

In many cases, it is useful to have more human-judged metadata on top of each annotation for a 2D image annotation task, for example, measuring the occlusion-level of all vehicles in an image.

To achieve this, we support annotation_attributes, an object representing additional attributes that you’d like to capture per image annotation.

You may use annotation_attributes to define categorical attributes, numerical attributes, angle attributes, or text attributes for each annotation. You define the type of the attribute using the type property of the attribute, which is either category, number, angle, or text. It defaults to category.

The format for annotation_attributes is an object whose key-value pairs all specify attributes of each annotation that you want to capture. The schema differs slightly based on the type of the attribute:

Categorical Attributes

Example definition of a categorical attribute.

{
  ...
  "annotation_attributes": {
    "example_category": {
      "type": "category",
      "description": "What is the answer to this example question?",
      "choices": [
        "Answer",
        "Other"
      ]
    }
  }
}

Each key-value defining a categorical attribute should have the following structure:

Numerical Attributes

Example definition of a numerical attribute.

{
  ...
  "annotation_attributes": {
    "example_number": {
      "type": "number",
      "description": "What is the answer to this example question?",
      "min": 0,
      "max": 100
    }
  }
}

Each key-value defining a numerical attribute should have the following structure:

Angle Attributes

Example definition of a angle attribute.

{
  ...
  "annotation_attributes": {
    "example_angle": {
      "type": "angle",
      "description": "What is the angle of this object?",
    }
  }
}

An angle attribute allows you to receive some angular information of the attribute, such as the heading yaw angle or heading pitch angle. This will return a number from 0 to 360, which is the value of the angle in degrees.

Each key-value defining an angle attribute should have the following structure:

Text Attributes

Example definition of a text attribute.

{
  ...
  "annotation_attributes": {
    "example_text": {
      "type": "text",
      "description": "What does this sign say?",
    }
  }
}

A text attribute allows you to receive freeform text, such as a transcription of a road sign or a car’s license plate. This will return a string.

Each key-value defining a text attribute should have the following structure:

Conditionality

The conditions JSON describes the set of conditions under which the attribute should be collected. Each of the conditions described in the JSON are AND’d together, meaning that adding more conditions is more restrictive.

The conditions JSON can have up to two key-values: label_condition and attribute_conditions.

Definition: Condition

Example Condition objects

{ "label": "vehicle" } // condition for the label being equal to vehicle
{ "label": ["car", "truck"] } // condition for the label to be equal to either car or truck
{ "is_parked": "Yes" } // condition for the is_parked attribute to be equal to be Yes

A Condition is an object defined by a single key and a value. The key describes the semantic string that you want to check, whether it is 'label' for the label_condition, or the attribute identifier for attribute_conditions.

The value is either a string or an array of strings. In the case of a string, the label or attribute will be checked for strict equality with the string. In the case of an array of strings, the label or attribute will be checked for membership within the array.

Condition Parameters

The label_condition key-value will describe the label or labels that you want to collect this attribute for. The value is a Condition object, which must have key equal to 'label'.

The attribute_conditions key-value will describe under what conditions with the other attributes you’d like to collect this attribute for. The value should a Condition object or a list of Condition objects.

attribute_conditions are automatically verified for circular dependencies, and tasks will be rejected if any circular dependencies are found.

Request Format

Example task payload for an annotation task

{
  "callback_url": "http://www.example.com/callback",
  "instruction": "Draw boxes around the vehicles in the image.",
  "attachment_type": "image",
  "attachment": "http://i.imgur.com/v4cBreD.jpg",
  "objects_to_annotate": [
    "car",
    "bus"
  ],
  "with_labels": true,
  "annotation_attributes": {
    "parked": {
      "description": "Is the car currently parked?",
      "choices": [
        "Yes",
        "No"
      ]
    },
    "heading": {
      "description": "Which direction is the car heading",
      "choices": [
        "left",
        "right",
        "back",
        "front"
      ],
      "conditions": {
        "label_condition": {
          "label": "car"
        },
        "attribute_conditions": [
          {
            "parked": "No"
          }
        ]
      }
    }
  }
}

To create a task with attributes, simply add the annotation_attributes parameter to your task creation request using the format described above.

Response Format

An example response with attributes is below:

{
  "response": {
    "annotations": [
      {
        "left": 123,
        "top": 10,
        "width": 121,
        "height": 39,
        "label": "car",
        "attributes": {
          "parked": "Yes",
          "heading": "left"
        }
      },
      {
        "left": 82,
        "top": 56,
        "width": 64,
        "height": 30,
        "label": "car",
        "attributes": {
          "parked": "No",
          "heading": "front"
        }
      },
      { ... },
      { ... }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

In addition to the standard attributes for the annotation response, if you specified annotation_attributes in the request, each annotation object will contain an attributes JSON. It will contain keys for each of the attributes you defined in the annotation_attributes schema, and the values will be the chosen categorical value or numerical value by the Scaler.

Create Categorization Task

Multiple

curl "https://api.scale.ai/v1/task/categorization" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Categorize this movie trailer" \
  -d attachment_type=website \
  -d attachment="https://www.youtube.com/embed/6ZfuNTqbHE8" \
  -d taxonomies[watch][type]=category \
  -d taxonomies[watch][description]="Are you going to watch the full movie?" \
  -d taxonomies[watch][choices][]="yes" \
  -d taxonomies[watch][choices][]="no" \
  -d taxonomies[genre][type]=category \
  -d taxonomies[genre][description]="What genre does it belong to?" \
  -d taxonomies[genre][choices][]="Action" \
  -d taxonomies[genre][choices][]="Adventure" \
  -d taxonomies[genre][choices][]="Comedy" \
  -d taxonomies[genre][choices][]="Crime" \
  -d taxonomies[genre][choices][]="Drama" \
  -d taxonomies[genre][allow_multiple]=true

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_categorization_task(
    callback_url='http://www.example.com/callback',
    instruction='Categorize this movie trailer',
    attachment_type='website',
    attachment='https://www.youtube.com/embed/6ZfuNTqbHE8',
    taxonomies={
      'watch': {
        'type': 'category',
        'description': 'Are you going to watch the full movie?',
        'choices': ['yes', 'no']
      },
      'genre': {
        'type': 'category',
        'description': 'What genre does it belong to?',
        'choices': ['Action', 'Adventure', 'Comedy', 'Crime', 'Drama'],
        'allow_multiple': True
      }
    }
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createCategorizationTask(
  {
    callback_url: 'http://www.example.com/callback',
    instruction: 'Categorize this movie trailer',
    attachment_type: 'website',
    attachment: 'https://www.youtube.com/embed/6ZfuNTqbHE8',
    taxonomies: {
      watch: {
        type: 'category',
        description: 'Are you going to watch the full movie?',
        choices: ['yes', 'no']
      },
      genre: {
        type: 'category',
        description: 'What genre does it belong to?',
        choices: ['Action', 'Adventure', 'Comedy', 'Crime', 'Drama'],
        allow_multiple: true
      }
    }
  },
  (err, task) => {
    // do something with task
  }
);

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_categorization_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Categorize this movie trailer',
  attachment_type: 'website',
  attachment: 'https://www.youtube.com/embed/6ZfuNTqbHE8',
  taxonomies: {
    watch: {
      type: 'category',
      description: 'Are you going to watch the full movie?',
      choices: ['yes', 'no']
    },
    genre: {
      type: 'category',
      description: 'What genre does it belong to?',
      choices: ['Action', 'Adventure', 'Comedy', 'Crime', 'Drama'],
      allow_multiple: true
    }
  }
})

The above command returns an object structured like this:

{
  "task_id": "5c3f9c25744e7d005052e319",
  "created_at": "2019-01-16T21:03:33.166Z",
  "callback_url": "http://www.example.com/callback",
  "type": "categorization",
  "status": "pending",
  "instruction": "Categorize this movie trailer",
  "params": {
    "attachment_type": "website",
    "attachment": "https://www.youtube.com/embed/6ZfuNTqbHE8",
    "taxonomies": {
      "watch": {
        "type": "category",
        "description": "Are you going to watch the full movie?",
        "choices": [
          "yes",
          "no"
        ]
      },
      "genre": {
        "type": "category",
        "description": "What genre does it belong to?",
        "choices": [
          "Action",
          "Adventure",
          "Comedy",
          "Crime",
          "Drama"
        ],
        "allow_multiple": true
      }
    }
  },
  "is_test": false,
  "metadata": {},
  "processed_attachments": []
}

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and select the correct answer for every taxonomy/question. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabled instruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally taxonomies. The format for taxonomies is an object whose key-value pairs specify taxonomies/questions for the attachment.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment and taxonomies will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.ai/v1/task/categorization

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
instruction string A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type string One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.
attachment string The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.
taxonomies object An object whose key-value pairs specify taxonomies/questions for the attachment.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
layers (optional) object A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

Categorical Taxonomies

Example definition of a categorical taxonomy.

{
  ...
  "taxonomies": {
    "genre": {
      "type": "category",
      "description": "What genre does it belong to?",
      "choices": ["Action", "Adventure", "Comedy", "Crime", "Drama"],
      "allow_multiple": true
    }
  }
}

Each key-value defining a categorical taxonomy should have the following structure:

Categorical Taxonomies: nested choices

There are often categorization tasks that have too many choices for a worker to efficiently sort through them all at once. In those cases, a solution is to use nested choices, where choices may have subchoices within them. A simple example illustrated in the example API call on the right is when selecting a movie genre. While there may be a large amount of total choices, using subchoices a worker can first categorize a movie as adventure, and based on that choice select the specific type of adventure.

For example, for a choices array of ["Action", "Adventure"], you could instead add a nested choice by passing an array like ["Action", { "choice": "Adventure", "subchoices": ["Anime", "Superhero"] }]. Then, if a worker selected Adventure, they would be further prompted to choose one of the corresponding subchoices.

These hierarchies can themselves be nested, so that the subchoices array may itself specify further nested choices.

Numerical Taxonomies

Example definition of a numerical taxonomy.

{
  ...
  "taxonomies": {
    "genre": {
      "type": "number",
      "description": "Rate the movie",
      "min": 0,
      "max": 10,
      "step": 1
    }
  }
}

Each key-value defining a numerical taxonomy should have the following structure:

Callback Format

The callback body will look like:

{
  "response": {
    "taxonomies": {
      "genre": ["Action", "Adventure", "Comedy", "Crime", "Drama"],
      "watch": ["yes"]
    }
  },
  "task_id": "576ba74eec471ff9b01557cc",
  "task": {
    // populated task for convenience
    ...
  }
}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a taxonomies field.

For categorical taxonomies, the value will be an array of choices, each one being one of the original choices.

For numerical taxonomies, the value will be a number.

Nested choices are only intended for grouping a large set of choices together. Thus, the response will be the same as with plain choices, where only the final “leaf” choice that the worker selected is reported. In the example above, if a worker selected Adventure and then Superhero, the choice would be Superhero.

Single (deprecated)

curl "https://api.scale.ai/v1/task/categorization" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Is this company public or private?" \
  -d attachment_type=website \
  -d attachment="http://www.google.com/" \
  -d categories=public \
  -d categories=private

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_categorization_task(
    callback_url='http://www.example.com/callback',
    instruction='Is this company public or private?',
    attachment_type='website',
    attachment='http://www.google.com/',
    categories=['public', 'private']
)


var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createCategorizationTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Is this company public or private?',
  'attachment_type': 'website',
  'attachment': 'http://www.google.com/',
  'categories': ['public', 'private']
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_categorization_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Is this company public or private?',
  attachment_type: 'website',
  attachment: 'https://www.google.com',
  categories: ['public', 'private']
})
=> #<Scale::Api::Tasks::Categorization:0x007fcc11819bf8 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"allow_multiple"=>false, "categories"=>["public", "private"], "attachment"=>"https://www.google.com", "attachment_type"=>"website"}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "576ba74eec471ff9b01557cc",
  "created_at": "2016-06-23T09:09:34.752Z",
  "callback_url": "http://www.example.com/callback",
  "type": "categorization",
  "status": "pending",
  "instruction": "Is this company public or private?",
  "params": {
    "attachment_type": "website",
    "attachment": "http://www.google.com/",
    "categories": [
      "public",
      "private"
    ]
  },
  "metadata": {}
}

This endpoint creates a categorization task. In this task, one of our workers will view the attachment and choose a category for it according to the instruction. You may allow multiple categories to be chosen by setting allow_multiple to true. Example use cases are spam detection, copyright detection, product categorization, etc.

This task involves a markdown-enabled instruction about how to make the categorization, an attachment of what you’d like to categorize, an attachment_type, and finally a list of categories.

There is an optional category_ids parameter, which you can use to impose an id system over the categories. The format of this parameter should be a dictionary, where the keys are the ids (as strings), and then the values are the category values provided in categories. An example is:

"category_ids": {
  "123": "Blue Cross",
  "124": "Red Cross"
}

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment, categories, and category_ids (optional) will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.ai/v1/task/categorization

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
instruction string A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type string One of text, image, video, audio, website, or pdf. Describes what type of file the attachment is.
attachment string The attachment to be categorized. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.
categories [string] An array of strings for the categories which you’d like the object to be sorted between.
category_ids (optional) dictionary An optional dictionary where the keys are the optional ids, and the values are the category values provided in categories.
allow_multiple (optional) boolean Default is false. Determines whether you allow multiple categories to be chosen for the attachment
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
layers (optional) object A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail.

Callback Format

If allow_multiple is false, the callback body will look like:

{
  "response": {
    "category": "one_of_the_categories"
  },
  "task_id": "576ba74eec471ff9b01557cc",
  "task": {
    // populated task for convenience
    ...
  }
}

If allow_multiple is true, the callback body will look like:

{
  "response": {
    "category": ["some_of", "the_categories"]
  },
  "task_id": "576ba74eec471ff9b01557cc",
  "task": {
    // populated task for convenience
    ...
  }
}

Example if category_ids is provided and allow_multiple is false

{
  "response": {
    "category": "Blue Cross",
    "category_id": "123"
  },
  "task_id": "576ba74eec471ff9b01557cc",
  "task": {
    // populated task for convenience
    ...
  }
}

Example if category_ids is provided and allow_multiple is true


{
  "response": {
    "category": ["Blue Cross", "Red Cross"],
    "category_id": ["123", "124"]
  },
  "task_id": "576ba74eec471ff9b01557cc",
  "task": {
    // populated task for convenience
    ...
  }
}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a category field and potentially a category_id field.

If allow_multiple is false, then the value will be a string equal to one of the original categories.

If allow_multiple is true, the value will be an array of categories, each one being one of the original categories.

If category_ids was provided, there will be another field category_id corresponding to the given id of the chosen category/categories.

Create Comparison Task

curl "https://api.scale.ai/v1/task/comparison" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Do the objects in these images have the same pattern?" \
  -d attachment_type=image \
  -d attachments="http://i.ebayimg.com/00/\$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG" \
  -d attachments="http://images.wisegeek.com/checkered-tablecloth.jpg" \
  -d choices="yes" \
  -d choices="no"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_comparison_task(
    callback_url='http://www.example.com/callback',
    instruction='Do the objects in these images have the same pattern?',
    attachment_type='image',
    attachments=[
        'http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG',
        'http://images.wisegeek.com/checkered-tablecloth.jpg'
    ],
    choices=['yes', 'no']
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createComparisonTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Do the objects in these images have the same pattern?',
  'attachment_type': 'image',
  'attachments': [
    'http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG',
    'http://images.wisegeek.com/checkered-tablecloth.jpg'
  ],
  'choices': ['yes', 'no']
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_comparison_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Do the objects in these images have the same pattern?',
  attachments: [
    'http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG',
    'http://images.wisegeek.com/checkered-tablecloth.jpg'
  ],
  attachment_type: 'image',
  choices: ['yes', 'no']
})
=> #<Scale::Api::Tasks::Comparison:0x007fcc109636e0 @task_id="58a6378aaa9d139b20a42532", @type="comparison", @instruction="Do the objects in these images have the same pattern?", @params={"choices"=>["yes", "no"], "attachment_type"=>"image", "attachments"=>["http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG", "http://images.wisegeek.com/checkered-tablecloth.jpg"]}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:42 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "5774cc78b02487c424f089dd",
  "created_at": "2016-06-30T07:38:32.368Z",
  "callback_url": "http://www.example.com/callback",
  "type": "comparison",
  "status": "pending",
  "instruction": "Do the objects in these images have the same pattern?",
  "params": {
    "choices": [
      "yes",
      "no"
    ],
    "attachment_type": "image",
    "attachments": [
      "http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG",
      "http://images.wisegeek.com/checkered-tablecloth.jpg"
    ]
  },
  "metadata": {}
}

This endpoint creates a comparison task. In this task, one of our workers view the given attachments and do any comparison requested.

This task involves a markdown-enabled instruction, an array of attachments, and an attachment_type.

At least of the fields or choices parameters must specified for the data to be returned. choices is an array of strings from which the worker to choose, and fields is useful for free-text response.

fields is a dictionary where the keys are the keys you’d like the results to be returned under, and values are the descriptions you’d like to show the human worker.

If successful, Scale will immediately return the generated task object, of which you should store the task_id.

The parameters attachment_type, attachments, choices, and fields will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.ai/v1/task/comparison

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
instruction string A markdown-enabled string explaining how to compare the attachments. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type string One of text, image, video, audio, website, or pdf. Describes what type of file the attachments are.
attachments array An array of attachments to compare. If attachment_type is text, then each attachment should be plaintext. Otherwise, they should be URLs pointing to the attachments.
fields (optional) dictionary A dictionary corresponding to the fields to be recorded. Keys are the keys you’d like the fields to be returned under, and values are descriptions to be shown to human workers.
choices (optional) [string] An array of strings for the choices to be given to the worker. One of choices or fields must be specified.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

Callback Format

Example callback body:

{
  "response": {
    "choice": "yes",
    "fields": {
      "difference": "The patterns are identical."
    }
  },
  "task_id": "576ba74eec471ff9b01557cc",
  "task": {
    // populated task for convenience
    ...
  }
}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a fields field and/or choice field depending on the original request.

If your original call provided choices, choice will be one of the original choices.

If your original call provided fields, fields will have keys corresponding to the keys you provided in the parameters, with values the transcribed value.

Create OCR Transcription Task

curl "https://api.scale.ai/v1/task/transcription" \
  -u "{{ApiKey}}:" \
  -d callback_url="http://www.example.com/callback" \
  -d instruction="Transcribe the given fields." \
  -d attachment_type=website \
  -d attachment="http://news.ycombinator.com/" \
  -d fields[title]="Title of Webpage" \
  -d fields[top_result]="Title of the top result"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')

client.create_transcription_task(
    callback_url='http://www.example.com/callback',
    instruction='Transcribe the given fields.',
    attachment_type='website',
    attachment='http://news.ycombinator.com/',
    fields={
        'title': 'Title of Webpage',
        'top_result': 'Title of the top result'
    }
)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

client.createTranscriptionTask({
  'callback_url': 'http://www.example.com/callback',
  'instruction': 'Transcribe the given fields.',
  'attachment_type': 'website',
  'attachment': 'http://news.ycombinator.com/',
  'fields': {
    'title': 'Title of Webpage',
    'top_result': 'Title of the top result'
  }
}, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.create_transcription_task({
  callback_url: 'http://www.example.com/callback',
  instruction: 'Transcribe the given fields.',
  attachment_type: 'website',
  attachment: 'http://news.ycombinator.com/',
  fields: {
    title: 'Title of Webpage',
    top_result: 'Title of the top result'
  }
})

=> #<Scale::Api::Tasks::Transcription:0x007fcc1098d828 @task_id="58a6361eaa9d139b20a4252d", @type="transcription", @instruction="Transcribe the given fields.", @params={"fields"=>{"top_result"=>"Title of the top result", "title"=>"Title of Webpage"}, "attachment"=>"http://news.ycombinator.com/", "attachment_type"=>"website"}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:30:38 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>

The above command returns an object structured like this:

{
  "task_id": "576de9dc1ea5f917d56fc2a0",
  "created_at": "2016-06-25T02:18:04.248Z",
  "callback_url": "http://www.example.com/callback",
  "type": "transcription",
  "status": "pending",
  "instruction": "Transcribe the given fields.",
  "params": {
    "fields": {
      "title": "Title of Webpage",
      "top_result": "Title of the top result"
    },
    "attachment": "http://news.ycombinator.com/",
    "attachment_type": "website"
  },
  "metadata": {}
}

This endpoint creates a transcription task. In this task, one of our workers will read an attachment and arbitrarily transcribe any information you’d like. Example use cases could be transcribing information from PDFs, manually scraping a web page for information, etc.

This task involves a markdown-enabled instruction about how to transcribe the attachment, an attachment of what you’d like to transcribe, an attachment_type, fields, and repeatable_fields.

fields is a dictionary which describes items you’d like transcribed for the attachment. Examples are phone numbers, names, etc. repeatable_fields is a dictionary which describes items which you’d like transcribed for the attachment which appear in the attachment multiple times. Examples are the row-by-row items of an invoice or purchase order.

In addition, we now support choices for repeatable_fields where you can specify an array of choices for the repeatable field.

At least one of fields or repeatable_fields is required. Both fields and repeatable_fields are dictionaries where the keys are the identifiers you’d like the results to be returned using, and values are plaintext descriptions you’d like to show the Scaler as they complete the task.

If successful, Scale will immediately return the generated task object, of which you should at least store the task_id.

The parameters attachment_type, attachment, fields, and repeatable_fields will be stored in the params object of the constructed task object.

HTTP Request

POST https://api.scale.ai/v1/task/transcription

Parameters

Parameter Type Description
project (optional) string The name of the project to associate this task with. See Projects section for more details.
batch (optional) string The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details.
callback_url string The full url (including the scheme http:// or https://) of the callback when the task is completed. See the Callback section for more details about callbacks.
instruction string A markdown-enabled string explaining how to transcribe the attachment. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type string One of image, pdf, or website. Describes what type of file the attachment is.
attachment string The attachment to be transcribed. If attachment_type is text, then it should be plaintext. Otherwise, it should be a URL pointing to the attachment.
fields (optional if using repeatable_fields) object A dictionary corresponding to the fields to be transcribed. Keys are the identifiers you’d like the fields to be returned using, and values are descriptions to be shown to the Scalers as they complete the task.
repeatable_fields (optional if using fields) object If your task requires a transcription of items which might be repeated within the attachment, such as rows of an invoice, then this dictionary describes those fields. With the addition of choices, there are now two acceptable formats. The first acceptable format is the same format as fields to generate repeatable text fields. The second acceptable format is a dictionary with keys of description (identifiers) and choices (the list of choices for that input). In addition, for repeatable_fields with the same format as fields, our internal task model will represent each repeatable field as a dictionary object with key description to the original value.
metadata (optional, default {}) object A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.

Callback Format

Example callback response sent on completion with fields

{
  "response": {
    "fields": {
      "title": "Some Title",
      "top_result": "The Top Result or Something"
    }
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

Example callback response sent on completion with repeatable_fields

{
  "response": {
    "repeatable_fields": [
      {
        "description": "Espresso",
        "amount": "10.00"
      },
      {
        "description": "Ice Cream",
        "amount": "9.99"
      }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

The response object, which is part of the callback POST request and permanently stored as part of the task object, will have a fields field.

fields will have keys corresponding to the keys you provided in the parameters, with values equal to the transcribed value.

If you requested repeatable_fields, repeatable_fields in the response will be an array of such dictionaries, with keys corresponding to the keys you provided in the repeatable_fields parameter, and values corresponding to the transcribed value. Each element of the array will correspond to one transcribed value in the attachment.

Projects

Projects are a way of organizing similar tasks, so that one can share parameters among tasks, or control what workers are allowed to work on certain classes of tasks.

The parameters associated with a project will be inherited by tasks created under that project. Currently the only project parameter supported is instruction; instructions defined in a project will be concatenated to the end of the instructions of tasks created under the project.

To use a project for a given task, add a project parameter when creating the task that references the project name; e.g., to associate a task with project my_project, add project: my_project to the task creation request. Note that projects created using your test API key can only be used for creating test tasks; likewise with the live API key and live tasks.

Project Creation

API Endpoint

https://api.scale.ai/v1/projects

Example API call

curl "https://api.scale.ai/v1/projects" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json" \
  -X POST \
  -d '
{
    "type": "annotation",
    "name": "kitten_labeling",
    "params": {
        "instruction": "please label the kittens in this image"
    }
}'

import requests
import json
payload = {
    'type': 'annotation',
    'name': 'kitten_labeling',
    'params': {
        'instruction': 'please label the kittens in this image'
    },
}

headers = {"Content-Type": "application/json"}

create_project_request = requests.post("https://api.scale.ai/v1/projects",
  json=payload,
  headers=headers,
  auth=('{{ApiKey}}', ''))

print create_project_request.json()

const request = require('request');

const payload = {
    type: 'annotation',
    name: 'kitten_labeling',
    params: {
        instruction: 'please label the kittens in this image'
    }
};

request.post({
    url: 'https://api.scale.ai/v1/projects',
    json: true,
    auth: {user: '{{ApiKey}}', pass: ''},
    body: payload,
}, (err, response, body) => {
  console.log(body);
});

require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/projects")

req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
req.body = {
    type: 'annotation',
    name: 'kitten_labeling',
    params: {
        instruction: 'please label the kittens in this image'
    }
}.to_json

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "type": "annotation",
    "name": "kitten_labeling",
    "param_history": [
        {
            "instruction": "please label the kittens in the image",
            "version": 0,
            "created_at": "2018-04-20T07:38:32.368Z"
        }
    ],
    "created_at": "2018-04-20T07:38:32.368Z"
}

HTTP Request

POST https://api.scale.ai/v1/projects

Parameters

Parameter Type Description
type string The task type of all the tasks belonging to this project
name string Name identifying this project. Must be unique among all your projects. When creating tasks for this project, you should add a project: <name> parameter to the task creation request to associate the task with this project.
params (optional) object Default parameters for a task created under this project. Currently only supports params.instruction, which get appended to the instructions of any task created under this project.

Retrieval

API Endpoint

https://api.scale.ai/v1/projects/:projectName

Example API call

curl "https://api.scale.ai/v1/projects/kitten_labeling" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json" \

import requests
import json

headers = {"Content-Type": "application/json"}

get_project_request = requests.get("https://api.scale.ai/v1/projects/kitten_labeling",
  headers=headers,
  auth=('{{ApiKey}}', ''))

print get_project_request.json()

const request = require('request');

const payload = {
    instruction: 'please label the kittens and the cats in this image'
};

request.get({
    url: 'https://api.scale.ai/v1/projects/kitten_labeling',
    json: true,
    auth: {user: '{{ApiKey}}', pass: ''},
}, (err, response, body) => {
  console.log(body);
});

require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/projects/kitten_labeling")

req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "type": "annotation",
    "name": "kitten_labeling",
    "param_history": [
        {
            "instruction": "please label the kittens in the image",
            "version": 0,
            "created_at": "2018-04-20T07:38:32.368Z"
        }
    ],
    "created_at": "2018-04-20T07:38:32.368Z"
}

HTTP Request

GET https://api.scale.ai/v1/projects/:projectName returns the information for a project with name :projectName.

GET https://api.scale.ai/v1/projects returns information for all projects owned by a user.

List Tasks

To retrieve a list of tasks that belong to a particular project, use the project option in the List Tasks Endpoint.

Parameter Updating

API Endpoint

https://api.scale.ai/v1/projects/:projectName/setParams

Example API call

curl "https://api.scale.ai/v1/projects/kitten_labeling/setParams" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json" \
  -X POST \
  -d '
{
    "instruction": "please label the kittens and the cats in this image"
}'

import requests
import json
payload = {
    'instruction': 'please label the kittens and the cats in this image'
}

headers = {"Content-Type": "application/json"}

set_params_request = requests.post("https://api.scale.ai/v1/projects/kitten_labeling/setParams",
  json=payload,
  headers=headers,
  auth=('{{ApiKey}}', ''))

print set_params_request.json()

const request = require('request');

const payload = {
    instruction: 'please label the kittens and the cats in this image'
};

request.post({
    url: 'https://api.scale.ai/v1/projects/kitten_labeling/setParams',
    json: true,
    auth: {user: '{{ApiKey}}', pass: ''},
    body: payload
}, (err, response, body) => {
  console.log(body);
});


require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/projects/kitten_labeling/setParams")

req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
req.body = {instruction: 'please label the kittens and the cats in this image'}.to_json

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "type": "annotation",
    "name": "kitten_labeling",
    "param_history": [
        {
            "instruction": "please label the kittens in the image",
            "version": 0,
            "created_at": "2018-04-20T07:38:32.368Z"
        },
        {
            "instruction": "please label the kittens and the cats in the image",
            "version": 1,
            "created_at": "2018-04-20T10:38:32.368Z"
        }
    ],
    "created_at": "2018-04-20T07:38:32.368Z"
}

HTTP Request

POST https://api.scale.ai/v1/projects/:projectName/setParams

Projects keep a history of the parameters that they were set with. Tasks created under a project inherit the latest params of the project (the last entry in param_history), one can also specify project_param_version when creating a task to inherit from an older set of params (or use -1 to skip parameter inheritance altogether.)

Currently we only support params.instruction, which get appended to the instructions of any task created under this project.

Batches

Batches are a way to logically group together tasks under the same project.

Note that the API key you use to create batches/projects/tasks with must all match: test projects can only be used with test batches, and test tasks; likewise with live projects, live batches, and live tasks.

To use a batch for a given task, add a batch parameter when creating the task that references the batch name; e.g., to associate a task with batch my_batch, add batch: my_batch to the task creation request. Note that batchs created using your test API key can only be used for creating test tasks; likewise with the live API key and live tasks.

Workflow

  1. Batch is created (status staging)
  2. Tasks are added to batch (must specify what batch to add to in task creation request), at this point in time tasks cannot be worked on
  3. Batch is finalized (status inprogress)
  4. Tasks are worked on and completed (new tasks cannot be added to this batch)
  5. Final task is completed, batch is completed (status completed) and callback is fired

Batch Creation

API Endpoint

https://api.scale.ai/v1/batches

Example API call

curl "https://api.scale.ai/v1/batches" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json" \
  -X POST \
  -d '
{
    "project": "kitten_labeling",
    "name": "kitten_labeling_2018-07",
    "callback": "http://mycallback.com/"
}'

import requests
import json
payload = {
    'project': 'kitten_labeling',
    'name': 'kitten_labeling_2018-07',
    'callback': 'http://mycallback.com',
}

headers = {"Content-Type": "application/json"}

create_batch_request = requests.post("https://api.scale.ai/v1/batches",
  json=payload,
  headers=headers,
  auth=('{{ApiKey}}', ''))

print create_batch_request.json()

const request = require('request');

const payload = {
    project: 'kitten_labeling',
    name: 'kitten_labeling_2018-07',
    callback: 'http://mycallback.com',
};

request.post({
    url: 'https://api.scale.ai/v1/batches',
    json: true,
    auth: {user: '{{ApiKey}}', pass: ''},
    body: payload,
}, (err, response, body) => {
  console.log(body);
});

require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/batches")

req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
req.body = {
    project: 'kitten_labeling',
    name: 'kitten_labeling_2018-07',
    callback: 'http://mycallback.com',
}.to_json

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "project": "kitten_labeling",
    "name": "kitten_labeling_2018-07",
    "status": "staging",
    "callback": "http://mycallback.com",
    "created_at": "2018-08-01T09:09:10.108Z"
}

Create a batch under a project, so that you can associate new tasks with this batch.

HTTP Request

POST https://api.scale.ai/v1/batches

Parameters

Parameter Type Description
project string The name of the project this batch (and its tasks) belong to
name string Name identifying this batch. Must be unique among all batches belonging to a customer.
callback string Email to notify, or URL to POST to when a batch is complete.

Callback will be POSTed with the format below:

{
    "batch": "kitten_labeling_2018-07", // the name of the batch that was just completed
    "status": "completed", // the status of the batch
    "stats": { // counts of the tasks of this batch, grouped by their status
        "completed": 20,
        "canceled": 3,
        "error": 5,
    }
}

Retrieval

Example API call

curl "https://api.scale.ai/v1/batches/kitten_labeling" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json"

import requests
import json

headers = {"Content-Type": "application/json"}

batch_info_request = requests.get("https://api.scale.ai/v1/batches/kitten_labeling",
  headers=headers,
  auth=('{{ApiKey}}', ''))

print batch_info_request.json()

const request = require('request');

request.post({
    url: 'https://api.scale.ai/v1/batches/kitten_labeling',
    json: true,
    auth: {user: '{{ApiKey}}', pass: ''},
}, (err, response, body) => {
  console.log(body);
});

require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/batches/kitten_labeling")

req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "project": "kitten_labeling",
    "name": "kitten_labeling_2018-07",
    "status": "staging",
    "callback": "http://mycallback.com",
    "created_at": "2018-08-01T09:09:10.108Z"
}

Get information about a batch with name :batchName.

HTTP Endpoint

GET https://api.scale.ai/v1/batches/:batchName

Batch Status

Example API call

curl "https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json"

import requests
import json

headers = {"Content-Type": "application/json"}

status_request = requests.get("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status",
  headers=headers,
  auth=('{{ApiKey}}', ''))

print status_request.json()

const request = require('request');

request.post({
    url: 'https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status',
    json: true,
    auth: {user: '{{ApiKey}}', pass: ''},
}, (err, response, body) => {
  console.log(body);
});

require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status")

req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "status": "staging",
    "pending": 20
}

This endpoint returns the status of a batch with name :batchName, as well as the counts of its tasks grouped by task status.

HTTP Request

GET https://api.scale.ai/v1/batches/:batchName/status

List All Batches

This is a paged endpoint for all of your batches. Batches will be returned in descending order based on created_at. Pagination is based off limit and offset parameters, which determine the page size and how many results to skip.

Example API Call

curl "https://api.scale.ai/v1/batches?project=kitten_labeling" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json" \

import requests
import json

headers = {"Content-Type": "application/json"}

batch_list_request = requests.get("https://api.scale.ai/v1/batches?project=kitten_labeling",
  headers=headers,
  auth=('{{ApiKey}}', ''))

print batch_list_request.json()

const request = require('request');

request.get({
    url: 'https://api.scale.ai/v1/batches?project=kitten_labeling',
    json: true,
    auth: {user: '{{ApiKey}}', pass: ''},
    body: payload,
}, (err, response, body) => {
  console.log(body);
});

require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/batches?project=kitten_labeling")

req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "docs": [
        {
            "project": "kitten_labeling",
            "name": "kitten_labeling_2018-07",
            "status": "staging",
            "callback": "http://mycallback.com",
            "created_at": "2018-08-01T09:09:10.108Z"
        }
    ],
    "total": 1,
    "limit": 100,
    "offset": 0,
    "has_more": false
}

HTTP Request

GET https://api.scale.ai/v1/batches

Parameters (all optional)

Parameter Type Description
project string Project name to filter batches by
status string Status to filter batches by (staging or in_progress or completed)
status string Status to filter batches by (staging or in_progress or completed)
start_time ISO 8601 date The minimum value of created_at for batches to be returned
end_time ISO 8601 date The maximum value of created_at for batches to be returned
limit integer Number between 1–100, number of results per page (default 100)
offset integer Number of results to skip for showing following pages (default 0)

List Tasks

To retrieve a list of tasks that belong to a particular batch, see the batch option in the List Tasks Endpoint.

Finalize Batch

Example API Call

curl "https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize" \
  -u "{{ApiKey}}:" \
  -H "Content-Type: application/json" \
  -X POST

import requests
import json

headers = {"Content-Type": "application/json"}

finalize_request = requests.post("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize"
  headers=headers,
  auth=('{{ApiKey}}', ''))

print finalize_request.json()

const request = require('request');

request.get({
    url: 'https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize',
    json: true
    auth: {user: '{{ApiKey}}', pass: ''},
    body: payload,
}, (err, response, body) => {
  console.log(body);
});

require "net/http"
require "uri"

uri = URI.parse("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize")

req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')

response = http.request('request')
print response.body

The above command returns an object structured like this:

{
    "project": "kitten_labeling",
    "name": "kitten_labeling_2018-07",
    "status": "in_progress",
    "callback": "http://mycallback.com",
    "created_at": "2018-08-01T09:09:10.108Z"
}

Finalizes a batch with name :batchName so its tasks can be worked on.

HTTP Request

POST https://api.scale.ai/v1/batches/:batchName/finalize

Callbacks

The callback_url will be POSTed with application/json data of the following object form:

{
  "task": {
    "task_id": "576c41bf13e36b0600b02b34",
    "completed_at": "2016-06-23T21:54:44.904Z",
    "response": {
      "category": "red"
    },
    "created_at": "2016-06-23T20:08:31.573Z",
    "callback_url": "http://www.example.com/callback",
    "type": "categorization",
    "status": "completed",
    "instruction": "Is this object red or blue?",
    "params": {
      "attachment_type": "text",
      "attachment": "tomato",
      "categories": [
        "red",
        "blue"
      ]
    },
    "metadata": {}
  },
  "response": {
    "category": "red"
  },
  "task_id": "576c41bf13e36b0600b02b34"
}

On your tasks, you will be required to supply a callback_url, a fully qualified URL that we will POST with the results of the task when completed. The data will be served as a JSON body (application/json). Alternately, you can set a default callback URL in your profile, which will be used for tasks that do not specify one.

Additionally, in order to simplify testing and add support for email automation pipelines, you may provide an email address as the callback_url. In this case, each completed task will result in an email sent with the body as the task’s JSON payload.

You should respond to the POST request with a 2xx status code. If we do not receive a 2xx status code, we will continue to retry up to 20 times over the course of the next 24 hours.

If we receive a 2xx status code, the task will be populated with a true value for the callback_succeeded parameter. Otherwise, if we do not receive a 2xx status code on any retry, the task will be populated with a false value for the callback_succeeded parameter.

Getting Started

We have sample callback server implementations, which you can deploy to Heroku in seconds, in the following languages:

If you’re just testing and want to try a few requests, the easiest way to get started is to use a RequestBin and send requests using your https://someHash.x.pipedream.net URL as the callback_url. You can also use ngrok to expose a local server to the internet for fast prototyping.

Feel free to chat us on Slack or email us if you have any trouble.

Authentication

If you’d like to authenticate our callbacks, we set a scale-callback-auth HTTP header on each of our callbacks. The value will be equal to your Live Callback Auth Key shown on your dashboard. If this header is not set, or it is set incorrectly, the callback is not from Scale.

POST Data

Attribute Type Description
task_id string The task_id is the unique identifier for the task.
status string The status of the task when it was completed. Normally completed, but can also be error in the case that a task failed processing.
response object The response object of the completed request. For categorization, it will contain a category attribute of the assigned category.
task object The full task object for reference and convenience.

Task Endpoints

Retrieve a Task

curl "https://api.scale.ai/v1/task/{task_id}" \
  -u "{{ApiKey}}:"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')
task_id = 'YOUR_TASK_ID'

task = client.fetch_task(task_id)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

var task_id = 'YOUR_TASK_ID';

client.fetchTask(task_id, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

scale.tasks.find("58a63795aa9d139b20a42535")
=> #<Scale::Api::Tasks::Categorization:0x007fcc10978ab8 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"attachment_type"=>"website", "attachment"=>"https://www.google.com", "categories"=>["public", "private"], "allow_multiple"=>false}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}, @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>>

The above command returns an object structured like this:

{
  "task_id": "576ba74eec471ff9b01557cc",
  "completed_at": "2016-06-23T09:10:02.798Z",
  "created_at": "2016-06-23T09:09:34.752Z",
  "callback_url": "http://www.example.com/callback",
  "type": "categorization",
  "status": "completed",
  "instruction": "Would you say this item is big or small?",
  "params": {
    "attachment_type": "text",
    "attachment": "car",
    "categories": [
      "big",
      "small"
    ]
  },
  "response": {
    "category": "big"
  },
  "metadata": {}
}

This endpoint retrieves a specific task.

HTTP Request

GET https://api.scale.ai/v1/task/{TASK_ID}

URL Parameters

Parameter Description
task_id The task_id of the task to retrieve

Returns

Returns a task if a valid identifier was provided, and returns a 404 error otherwise.

Cancel a task

curl -X POST "https://api.scale.ai/v1/task/{task_id}/cancel" \
  -u "{{ApiKey}}:"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')
task_id = 'YOUR_TASK_ID'

task = client.cancel_task(task_id)

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

var task_id = 'YOUR_TASK_ID';

client.cancelTask(task_id, (err, task) => {
    // do something with task
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

canceled_task = scale.tasks.cancel("YOUR_TASK_ID") # Returns the appropriate Task object with status set to canceled
=> #<Scale::Api::Tasks::Categorization:0x007fcc1108b288 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"attachment_type"=>"website", "attachment"=>"https://www.google.com", "categories"=>["public", "private"], "allow_multiple"=>false}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="canceled", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}, @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>>

canceled_task.cancelled?
=> true

The above command returns an object structured like this:

{
  "task_id": "576ba74eec471ff9b01557cc",
  "created_at": "2016-06-23T09:09:34.752Z",
  "callback_url": "http://www.example.com/callback",
  "type": "categorization",
  "status": "canceled",
  "instruction": "Would you say this item is big or small?",
    "params": {
    "attachment_type": "text",
    "attachment": "car",
    "categories": [
      "big",
      "small"
    ]
  },
  "metadata": {}
}

This endpoint cancels a task so that it will not be completed.

You may only cancel pending tasks, and the endpoint will return a 500 error code if you attempt to cancel a completed task.

HTTP Request

POST https://api.scale.ai/v1/task/{TASK_ID}/cancel

URL Parameters

Parameter Description
task_id The task_id of the task to cancel

Returns

Returns the canceled task if a valid identifier for a pending task was provided, and returns a 404 error or 500 error otherwise.

List All Tasks

curl "https://api.scale.ai/v1/tasks" \
  -u "{{ApiKey}}:"

import scaleapi

client = scaleapi.ScaleClient('{{ApiKey}}')
task_id = 'YOUR_TASK_ID'

# specify URL params as kwargs to tasks()
tasklist = client.tasks()

var scaleapi = require('scaleapi');

var client = scaleapi.ScaleClient('{{ApiKey}}');

// specify URL parameters as properties in the params object
var params = {};
client.tasks(params, (err, tasklist) => {
    // do something with tasklist
});

require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')

# You can pass params like start_time or end_time as keys to filter results
scale.tasks.where
=> #<Scale::Api::TaskList:0x007fcc11822a28 @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>, @docs=[#<Scale::Api::Tasks::Categorization:0x007fcc118227f8 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"attachment_type"=>"website", "attachment"=>"https://www.google.com", "categories"=>["public", "private"], "allow_multiple"=>false}, @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}, @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>>], @limit=1, @offset=0, @has_more=true, @params={:start_time=>nil, :end_time=>nil, :limit=>1, :offset=>0, :status=>nil, :type=>nil}>
# Scale::Api::TaskList implements Enumerable, so it is Array-like.

The above command returns an object structured like this:

{
  "docs": [
    {
      "task_id": "576b998b4628d1bfaed7d3a4",
      "created_at": "2016-06-23T08:10:51.032Z",
      "callback_url": "http://www.example.com/callback",
      "type": "categorization",
      "status": "completed",
      "instruction": "Is this object big or small?",
      "params": {
        "attachment_type": "text",
        "attachment": "ant",
        "categories": [
          "big",
          "small"
        ]
      },
      "completed_at": "2016-06-23T19:36:23.084Z",
      "response": {
        "category": "small"
      },
      "metadata": {}
    },
    {
      "task_id": "576ba301eed30241b0e9bbf7",
      "created_at": "2016-06-23T08:51:13.903Z",
      "callback_url": "http://www.example.com/callback",
      "type": "categorization",
      "status": "completed",
      "instruction": "Is this object big or small?",
      "params": {
        "attachment_type": "text",
        "attachment": "T-Rex",
        "categories": [
          "big",
          "small"
        ]
      },
      "completed_at": "2016-06-23T09:09:10.108Z",
      "response": {
        "category": "big"
      },
      "metadata": {}
    }
  ],
  "total": 2,
  "limit": 100,
  "offset": 0,
  "has_more": false
}

This is a paged endpoint retrieves a list of your tasks. The tasks will be returned in descending order based on created_at time. The pagination is based on the limit and offset parameters, which determine the page size and how many results to skip.

HTTP Request

GET https://api.scale.ai/v1/tasks

URL Parameters

Parameter Type Description Required
start_time ISO 8601 Date The minimum value of created_at for tasks to be returned optional
end_time ISO 8601 Date The maximum value of created_at for tasks to be returned optional
status string The status of the task - can be: completed, pending, or canceled optional
type string The type of the task - can be: transcription, categorization, comparison, annotation, datacollection, audiotranscription, or any other task type. optional
project string The name of the project that the returned tasks must belong to. optional
batch string The name of the batch that the returned tasks must belong to. optional
limit integer A number between 1 and 100, the maximum number of results to display per page optional, default 100
offset integer The number of results to skip, for showing the next page optional, default 0

Returns

Returns a list of your tasks.

Errors

// example task.response with failed attachment
{
    "error": "One or more attachments could not be downloaded.", // reason for error
    "attachments": [
        {
            "statusCode": 403, // HTTP code received when fetching attachment
            "url": "http://example.com/kitten.png", // attachment URL
        }
    ]
}

The Scale uses the following HTTP codes:

Error Code Meaning
200 OK – Everything worked as expected.
400 Bad Request – The request was unacceptable, often due to missing a required parameter.
401 Unauthorized – No valid API key provided.
402 Not enabled – Please contact [email protected] before creating this type of task.
404 Not Found – The requested resource doesn’t exist.
429 Too Many Requests – Too many requests hit the API too quickly.
500 Internal Server Error – We had a problem with our server. Try again later.

In the event of one or more task attachments having invalid data, the task callback will be invoked with an error response detailing the problems, and the task’s status will be set to error. Additional detail about the error and failed attachments will be stored in the task response.

cURL JavaScript Python Ruby