Introduction
API Endpoint
https://api.scale.ai/v1/
Welcome to the Scale! You can use our API to access Scale endpoints, which can create, access, and cancel human tasks.
Computer Vision
Contact Sales to discuss these products:
Other Products
Anyone can use our API! Sign up here to get started using Scale. You can also contact us for volume pricing or any inquiries.
You can also join our Slack channel here!
Getting Started with Scale
As part of Scale’s mission to democratize access to intelligent data, we’re committed to making high quality, human-powered training data accessible for anyone in the AI development community. That’s why we provide two ways for customers to work with us: On-Demand and Enterprise engagements. The On-Demand model means no commitment requirements, no platform fees, and you pay-as-you-go. Furthermore, with On-Demand, you can just sign up and send in tasks via our developer-friendly API. After a combination of human work and review, smart tools, and statistical confidence checks and machine learning checks, we’ll return scalable, accurate training data. It’s a great way to get started and experience Scale’s ease-of-use and quality.
Ready to get started? Follow the 4 steps below and you’ll be well on your way to building a robust training dataset for your next AI application.
Sign Up
If you haven’t already, create a Scale account. When prompted, input your credit card information to activate your account. You’ll only be charged for completed tasks under our On-Demand rates (your first 5 tasks are free), which are based on the task type, volume of data, number of annotations with each task, and the response urgency.
Note: On-demand tasks are completed best-effort for quality and response urgency. For quality & turnaround SLAs, chat with us about our Enterprise engagements.
Design Your Task
To maximize the quality of your task, we recommend writing clear and concise instructions that include:
- Step-by-step directions
- Rules and example images (i.e. minimum label size, when to not label an object)
- Label classes/fields with definitions and example images
- Screenshots of correct and incorrect tasks for comparison
Tip: To allow you to collaborate on instructions with your team and embed formatting and images, create your instructions in a shared document via Google Docs and embed it into the API call via markdown:
- Create a Google Doc with your instructions
- Click File → Publish To Web → Embed (in the window that appears)
- Copy the iFrame tag that’s provided:
<iframe src="[YOUR_CUSTOM_DOC_LINK]"></iframe>
- Paste the iFrame tag in the instruction parameter.
Send Your Task
Before submitting an API call, retrieve your Team API Keys and authenticate access to our API:
Explore our docs to find the task type that’s the best fit for your use case and the specific parameters you must define for each task object. When you’re ready, write your API call following our docs (we support cURL, JavaScript, Python, and Ruby), and submit it via Terminal or a 3rd-party software like Postman.
Note: Our On-Demand service limits to 5000 tasks per month. If you’d like to send more tasks, chat with us about our Enterprise engagements.
Review Your Response
Once you submit your task, you will receive a unique task_id
, which you can use to track the task via your Dashboard. Here’s what you can do on each page:
Overview
View a summary of your submissions over time and the status for each task type.
Tasks
View a more granular list of your submitted tasks, filterable by status, project, and batch. If you click a specific task, you can view the original API call. For completed tasks, there are additional options that will allow you Audit the task, Resend Callback, and download the JSON response. You can also mass export the JSON responses for all your tasks via the Export JSON link.
Tip: Use our Teams feature to allow multiple users to have access to the same set of tasks.
Quality Assurance
Audit a random sample of tasks for quality and view stats for tasks you’ve reviewed. When you audit a task, we give you access to our annotation application where you can see the original source data with completed labels. You can Approve or Reject the task, and provide feedback if the task was incorrect.
If you any questions, feel free to join our Slack channel here or ping us on our web chat. If you’re interested in learning more about our Enterprise engagements, get in touch with us.
Client Libraries / SDKs
To install the client library, use the following command:
pip install --upgrade scaleapi
npm install scaleapi --save
gem install scaleapi
Currently we have client libraries available for the following languages:
We also have unofficial SDKs in the following languages:
- Android SDK (thanks to vanshg!)
Let us know if you want (or are interested in writing) a library for a language not represented here!
Authentication
To authorize, use this code:
# With curl, you can just pass the correct header with each request
curl "api_endpoint_here" \
-u "{{ApiKey}}:"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
=> #<Scale:0x007fcc1292fe88 @api_key="{{ApiKey}}", @callback_auth_key=nil, @default_request_params={:callback_url=>nil}, @logging=false>
You must replace
{{ApiKey}}
with your personal API key. If you sign up or log in, your API key will be automatically filled in the docs. Your test API key{{ApiKey}}
is included in all the examples on this page, so you can test any example right away. Only you can see this value.
Scale uses API keys to allow access to the API. You can find your API keys on your dashboard, which you can access by logging in or signing up.
Scale expects for the API key to be included in all API requests to the server via HTTP Basic Auth. Provide your API key as the basic auth username value. You do not need to provide a password. You can do this using the -u
flag:
-u "{{ApiKey}}:"
Test and Live Modes
To make the API as explorable as possible, accounts have test mode and live mode API keys. There is no “switch” for changing between modes, just use the appropriate key to perform a live or test API requests.
Requests made with test mode credentials are not completed by a human, and therefore have incorrect test responses. Requests made with live mode credentials are always completed by a human, and will incur a charge past your first 5 API requests.
Callback Authentication
If you’d like to authenticate our callbacks, we set a scale-callback-auth
HTTP header on each of our callbacks. The value will be equal to your Live Callback Auth Key
shown on your dashboard. If this header is not set, or it is set incorrectly, the callback is not from Scale.
Task Object
The task object represents a single task that you create with Scale and is completed by a worker.
Attributes
{
"task_id": "576ba74eec471ff9b01557cc",
"completed_at": "2016-06-23T09:10:02.798Z",
"created_at": "2016-06-23T09:09:34.752Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "completed",
"instruction": "Would you say this item is big or small?",
"urgency": "express",
"params": {
"attachment_type": "text",
"attachment": "car",
"categories": [
"big",
"small"
]
},
"callback_succeeded": true,
"response": {
"category": "big"
},
"metadata": {}
}
Attribute | Type | Description |
---|---|---|
task_id |
string | The task_id is the unique identifier for the task. |
type |
string | The type of the task. Currently, we support categorization , transcription , audiotranscription , comparison , annotation , and datacollection . |
instruction |
string | A markdown-enabled string explaining the instructions for the task. You can use markdown to show example images, give structure to your instructions, and more. HTML tags are unsupported. |
params |
object | An object with the parameters of the task based on the type. For categorization , for example, this will include attachment_type , attachment , and categories . |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
response |
object | An object corresponding to the response once the task is completed. For categorization , it will have the attribute category , corresponding to the chosen category. |
callback_url |
string | A string of the URL that should be POSTed once the task is completed for the response data. See the Callback section for more details. |
status |
string | The status of the task, one of pending , completed , canceled or error . |
created_at |
timestamp | A string of the UTC timestamp of when the task was created. |
completed_at |
timestamp | A string of the UTC timestamp of when the task was completed. This will only be filled in after it is completed. |
callback_succeeded |
boolean | A boolean stating whether or not the callback succeeded. If the callback returns with a 2xx status code, the value will be true . If the callback fails to return a 2xx status code through all retries, then the value will be false . |
metadata |
object, default {} |
A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
Metadata
Tasks objects have a metadata parameter. You can use this parameter to attach key-value data to tasks.
Metadata is useful for storing additional, structured information on an object. As an example, you could store a video’s unique identifier in your system on its content moderation categorization
task. You can also use it to denote the end use case for the task, as “content moderation” or “data categorization” for example. Metadata is not used by Scale (e.g., to affect how the task is done).
Urgency
One of express
or standard
. We aim to complete express tasks within several hours, and standard tasks within a few days. Completion times are best-effort for non-enterprise users. If you have more than 1000 requests per month or require a task completion time SLA, chat with us about Enterprise Plans!
Attachments
Callback example for bad attachment URL:
{
"error": "One or more attachments could not be downloaded.",
"attachments": [
{
"statusCode": 500,
"url": "https://your-s3-bucket.s3.amazonaws.com/attachment-1.png"
},
{
"statusCode": 500,
"url": "https://your-s3-bucket.s3.amazonaws.com/attachment-2.png"
}
]
}
Tasks often require a file associated with them that Scale calls an attachment. For example, an annotation task requires an image file to show to a Scaler. These attachments are not limited to images and may also be audio, video, or pdf files, or a website, or even plain text. For all attachment types except plain text, the attachment must be specified via a URL. See the specific task documentation below for some examples.
Currently we only support fetching URLs whose protocol is HTTP or HTTPS. Make sure that the URL is a direct link to the file you wish to use as an attachment and not a document showing a preview of the file. For certain storage providers we may be able to rewrite URLs that are previews of the file you are trying to attach to the correct direct download URL. HTTPS is preferred over HTTP, as HTTPS guarantees data integrity, preventing against corruption.
Scale will attempt to fetch the attachment via the URL you provide. If we do not receive an HTTP 200 response when attempting to fetch your attachment(s), we will send your callback_url
an error with the bad HTTP codes we received, and the task status will be error
.
For audio and video attachments we support the containers and formats listed on this Chromium project page; for image attachments we support JPEG, PNG, BMP, GIF, and TIFF file formats.
Create Computer Vision Tasks
2D Box Annotation
curl "https://api.scale.ai/v1/task/annotation" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Draw a box around each **car** and **pedestrian**." \
-d attachment_type=image \
-d attachment="http://i.imgur.com/XOJbalC.jpg" \
-d objects_to_annotate="car" \
-d objects_to_annotate="pedestrian" \
-d with_labels=true \
-d min_width="30" \
-d min_height="30"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_annotation_task(
callback_url='http://www.example.com/callback',
instruction='Draw a box around each **car** and **pedestrian**',
attachment_type='image',
attachment='http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate=['car', 'pedestrian'],
with_labels=True,
min_width='30',
min_height='30'
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createAnnotationTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Draw a box around each **car** and **pedestrian**',
'attachment_type': 'image',
'attachment': 'http://i.imgur.com/XOJbalC.jpg',
'objects_to_annotate': ['car', 'pedestrian'],
'with_labels': true,
'min_width': '30',
'min_height': '30'
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_annotation_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Draw a box around each **car** and **pedestrian**',
attachment_type: 'image',
attachment: 'http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate: ['car', 'pedestrian'],
with_labels: true,
min_width: '30',
min_height: '30'
})
=> #<Scale::Api::Tasks::ImageRecognition:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="annotation", @instruction="Draw a box around each **car** and **pedestrian**", @params={"with_labels"=>true, "objects_to_annotate"=>["car", "pedestrian"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/v4cBreD.jpg"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "5774cc78b01249ab09f089dd",
"created_at": "2016-9-03T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "annotation",
"status": "pending",
"instruction": "Draw a box around each **car** and **pedestrian**",
"urgency": "standard",
"params": {
"with_labels": true,
"min_width": 30,
"min_height": 30,
"objects_to_annotate": [
"car",
"pedestrian"
],
"attachment_type": "image",
"attachment": "http://i.imgur.com/XOJbalC.jpg"
},
"metadata": {}
}
This endpoint creates an annotation
task. In this task, one of our Scalers view the given image and draw bounding boxes around the specified objects, returning the positions and sizes of these boxes.
The required parameters for this task are callback_url
, attachment
, and objects_to_annotate
. The callback_url
is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment
is a URL to an image you’d like to be annotated.
objects_to_annotate
is an array of strings describing the different types of objects you’d like annotated.
You can optionally provide additional markdown-enabled instructions via the instruction
parameter.
Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate
list. If your application does not require the particular label, you can set the with_labels
parameter to false
.
It is recommended, but not required, for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.
You may also provide min_width
and min_height
parameters, which will tell Scalers to only annotate objects whose bounding boxes are of dimension at least min_width
x min_height
.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
HTTP Request
POST https://api.scale.ai/v1/task/annotation
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
objects_to_annotate |
[string] | An array of strings describing which objects you’d like bounding boxes to be drawn around. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 7 objects. If you need more objects, try using nested annotation labels. |
attachment |
string | A URL to the image you’d like to be annotated with bounding boxes. |
with_labels (optional, default true ) |
boolean | Specifies whether you’d like labels for each bounding box in the response. Each label will be a member of the objects_to_annotate array. |
min_height (optional) |
integer, default 0 | The minimum height in pixels of the bounding boxes you’d like to be made. |
min_width (optional) |
integer, default 0 | The minimum width in pixels of the bounding boxes you’d like to be made. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
instruction (optional) |
string | A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type (optional, default image ) |
string | Describes what type of file the attachment is. We currently only support image for the annotation endpoint. |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
layers (optional) |
object | A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail. |
Callback Format
Example callback body sent on completion
{
"response": {
"annotations": [
{
"left": 2,
"top": 4,
"width": 3,
"height": 5,
"label": "pedestrian"
},
{
"left": 7,
"top": 5,
"width": 14,
"height": 5,
"label": "car"
},
{ ... },
{ ... }
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// populated task for convenience
...
}
}
The response
object, which is part of the callback POST request and permanently stored as part of the task object, will have either an error
field or an annotations
field.
If the annotation was completed successfully, the annotations
field will contain an array of annotations. Each annotation will have the following values:
left
: The distance, in pixels, between the left border of the bounding box and the left border of the image.top
: The distance, in pixels, between the top border of the bounding box and the top border of the image.width
: The width, in pixels, of the bounding box.height
: The height, in pixels, of the bounding box.label
(ifwith_labels
istrue
): The label for the bounding box, which will be one of the specifiedtask.params.objects_to_annotate
.
Visual representations of the task responses are accessible via the Scale dashboard. In these, the border is drawn inside the bounding box. The example response body (at right) corresponds to the image below.
If the attachment was invalid, the error will be detailed in the error
field.
Point Annotation
curl "https://api.scale.ai/v1/task/pointannotation" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Draw a point on every **headlight** and **brakelight** of a car in the image." \
-d attachment_type=image \
-d attachment="http://i.imgur.com/XOJbalC.jpg" \
-d objects_to_annotate="headlight" \
-d objects_to_annotate="brakelight" \
-d with_labels=true
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_pointannotation_task(
callback_url='http://www.example.com/callback',
instruction='Draw a point on every **headlight** and **brakelight** of a car in the image.',
attachment_type='image',
attachment='http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate=['headlight', 'brakelight'],
with_labels=True
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createPointannotationTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Draw a point on every **headlight** and **brakelight** of a car in the image.',
'attachment_type': 'image',
'attachment': 'http://i.imgur.com/XOJbalC.jpg',
'objects_to_annotate': ['headlight', 'brakelight'],
'with_labels': true
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_pointannotation_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Draw a point on every **headlight** and **brakelight** of a car in the image.',
attachment_type: 'image',
attachment: 'http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate: ['headlight', 'brakelight'],
with_labels: true
})
=> #<Scale::Api::Tasks::Pointannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="pointannotation", @instruction="Draw a point on every **headlight** and **brakelight** of a car in the image.", @params={"with_labels"=>true, "objects_to_annotate"=>["headlight", "brakelight"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "5774cc78b01249ab09f089dd",
"created_at": "2016-9-03T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "pointannotation",
"status": "pending",
"instruction": "Draw a point on every **headlight** and **brakelight** of a car in the image.",
"urgency": "standard",
"params": {
"with_labels": true,
"objects_to_annotate": [
"headlight",
"brakelight"
],
"attachment_type": "image",
"attachment": "http://i.imgur.com/XOJbalC.jpg"
},
"metadata": {}
}
This endpoint creates a pointannotation
task. In this task, one of our Scalers view the given image and draw points at the specified locations, returning the locations of these points.
The required parameters for this task are callback_url
, attachment
, and objects_to_annotate
. The callback_url
is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.
objects_to_annotate
is an array of strings describing the different types of objects you’d like annotated. You may include at most 6 objects in the objects_to_annotate
parameter. If you need more objects, try using nested annotation labels.
You can optionally provide additional markdown-enabled instructions via the instruction
parameter.
Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate
list. If your application does not require the particular label, you can set the with_labels
parameter to false
.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
HTTP Request
POST https://api.scale.ai/v1/task/pointannotation
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
objects_to_annotate |
[string] | An array of strings describing which objects you’d like points to be drawn on. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 6 objects. If you need more objects, try using nested annotation labels. |
attachment |
string | A URL to the image you’d like to be annotated with points. |
with_labels (optional, default true ) |
boolean | Specifies whether you’d like labels for each point in the response. Each label will be a member of the objects_to_annotate array. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
instruction (optional) |
string | A markdown-enabled string explaining how to draw the points. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type (optional, default image ) |
string | Describes what type of file the attachment is. We currently only support image for the point annotation endpoint. |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
layers (optional) |
object | A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail. |
Callback Format
Example callback body sent on completion
{
"response": {
"annotations": [
{
"label": "headlight",
"x": 123,
"y": 10
},
{
"label": "headlight",
"x": 140,
"y": 49
},
{
"label": "brakelight",
"x": 67,
"y": 34
}
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// task inlined for convenience
...
}
}
The response
field, which is part of the callback POST request and permanently stored as part of the task object, will contain either an annotations
field or an error
field.
The annotations
field will contain an array of point annotations. Each annotation will have the following values:
x
: The distance, in pixels, between the point and the left border of the image.y
: The distance, in pixels, between the point and the top border of the image.label
(ifwith_labels
istrue
): The label for the point, which will be one of the specifiedtask.params.objects_to_annotate
.
If the attachment was invalid, the error will be detailed in the error
field.
Line Annotation
curl "https://api.scale.ai/v1/task/lineannotation" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Annotate lines over all of the **lane lines** in the street-level image." \
-d attachment_type=image \
-d attachment="http://i.imgur.com/XOJbalC.jpg" \
-d objects_to_annotate="solid line" \
-d objects_to_annotate="dashed line" \
-d with_labels=true
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_lineannotation_task(
callback_url='http://www.example.com/callback',
instruction='Annotate lines over all of the **lane lines** in the street-level image.',
attachment_type='image',
attachment='http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate=['solid line', 'dashed line'],
with_labels=True
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createLineannotationTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Annotate lines over all of the **lane lines** in the street-level image.',
'attachment_type': 'image',
'attachment': 'http://i.imgur.com/XOJbalC.jpg',
'objects_to_annotate': ['solid line', 'dashed line'],
'with_labels': true
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_lineannotation_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Annotate lines over all of the **lane lines** in the street-level image.',
attachment_type: 'image',
attachment: 'http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate: ['solid line', 'dashed line'],
with_labels: true
})
=> #<Scale::Api::Tasks::Lineannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="lineannotation", @instruction="Annotate lines over all of the **lane lines** in the street-level image.", @params={"with_labels"=>true, "objects_to_annotate"=>["solid line", "dashed line"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "5774cc78b01249ab09f089dd",
"created_at": "2016-9-03T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "lineannotation",
"status": "pending",
"instruction": "Annotate lines over all of the **lane lines** in the street-level image.",
"urgency": "standard",
"params": {
"with_labels": true,
"objects_to_annotate": [
"solid line",
"dashed line"
],
"attachment_type": "image",
"attachment": "http://i.imgur.com/XOJbalC.jpg"
},
"metadata": {}
}
This endpoint creates a lineannotation
task. In this task, one of our Scalers view the given image and draw segmented lines along each object, returning the vertices of these segmented lines.
The required parameters for this task are callback_url
, attachment
, and objects_to_annotate
. The callback_url
is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.
objects_to_annotate
is an array of strings describing the different types of objects you’d like annotated.
You can optionally provide additional markdown-enabled instructions via the instruction
parameter.
Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate
list. If your application does not require the particular label, you can set the with_labels
parameter to false
.
If you’d prefer splines as opposed to segmented lines, then you may specify the splines
flag as true.
You can also optionally set min_vertices
and/or max_vertices
which specify the minimum and maximum number of vertices that you would want on a line. Note the range is inclusive of both bounds. For example, if min_vertices
is 2 and max_vertices
is 2, then Scale will only return line segments.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
HTTP Request
POST https://api.scale.ai/v1/task/lineannotation
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
objects_to_annotate |
[string] | An array of strings describing which objects you’d like segmented lines to be drawn along. Each string should be singular and self-descriptive (ex: “lane line”, “crop line”). You may include at most 3 objects. If you need more objects, try using nested annotation labels. |
attachment |
string | A URL to the image you’d like to be annotated with segmented lines. |
with_labels (optional, default true ) |
boolean | Specifies whether you’d like labels for each segmented line in the response. Each label will be a member of the objects_to_annotate array. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
instruction (optional) |
string | A markdown-enabled string explaining how to draw the segmented lines. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type (optional, default image ) |
string | Describes what type of file the attachment is. We currently only support image for the line annotation endpoint. |
splines (optional, default false ) |
boolean | Specifies whether or not you’d like your lines drawn as cardinal splines instead of segmented lines |
min_vertices (optional, default 1) |
number | An optional parameter defining the minimum number of vertices in a valid line annotation for your request. |
max_vertices (optional, default null ) |
number | An optional parameter defining the maximum number of vertices in a valid line annotation for your request. Must be at least min_vertices . |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
layers (optional) |
object | A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail. |
Callback Format
Example callback body sent on completion
{
"response": {
"annotations": [
{
"label": "solid line",
"vertices": [
{
"x": 123,
"y": 10
},
{
"x": 140,
"y": 49
},
{
"x": 67,
"y": 34
}
]
},
{ ... },
{ ... }
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// populated task for convenience
...
}
}
The response
field, which is part of the callback POST request and permanently stored as part of the task object, will contain either an annotations
field or an error
field.
The annotations
field will contain an array of annotations. Each annotation will have the following values:
vertices
: An array describing the vertices of the segmented line, listed in order from one end to another. Each vertex will be described by an object with the following structure:x
: The distance, in pixels, between the vertex and the left border of the image.y
: The distance, in pixels, between the vertex and the top border of the image.
label
(ifwith_labels
istrue
): The label for the line, which will be one of the specifiedtask.params.objects_to_annotate
.
If the attachment was invalid, the error will be detailed in the error
field.
Cuboid Annotation
curl "https://api.scale.ai/v1/task/cuboidannotation" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Draw a box around each **car** and **pedestrian**." \
-d attachment_type=image \
-d attachment="http://i.imgur.com/XOJbalC.jpg" \
-d objects_to_annotate="car" \
-d objects_to_annotate="pedestrian" \
-d with_labels=true \
-d min_width="30" \
-d min_height="30"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_cuboidannotation_task(
callback_url='http://www.example.com/callback',
instruction='Draw a cuboid around each car or truck.',
attachment_type='image',
attachment='http://i.imgur.com/v4cBreD.jpg',
objects_to_annotate=['headlight', 'brakelight'],
with_labels=True
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createCuboidannotationTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Draw a cuboid around each car or truck.',
'attachment_type': 'image',
'attachment': 'http://i.imgur.com/v4cBreD.jpg',
'objects_to_annotate': ['car', 'truck'],
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_cuboidannotation_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Draw a cuboid around each car or truck.',
attachment_type: 'image',
attachment: 'http://i.imgur.com/v4cBreD.jpg',
objects_to_annotate: ['car', 'truck'],
})
=> #<Scale::Api::Tasks::Cuboidannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="cuboidannotation", @instruction="Draw a cuboid around each car or truck", @params={"with_labels"=>true, "objects_to_annotate"=>["car", "truck"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/v4cBreD.jpg"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "5774cc78b01249ab09f089dd",
"created_at": "2016-9-03T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "cuboidannotation",
"status": "pending",
"instruction": "Draw a cuboid around each car or truck.",
"urgency": "standard",
"params": {
"objects_to_annotate": [
"car",
"truck"
],
"attachment_type": "image",
"attachment": "http://i.imgur.com/XOJbalC.jpg"
},
"metadata": {}
}
This endpoint creates a cuboidannotation
task. Given a 2D image, and camera intrinsics and extrinsics as optional parameters, Scale will annotate the image with perspective cuboids and return the vertices in the image of these cuboids. If camera intrinsics and extrinsics are provided as well, Scale will return scale-invariant 3D coordinates with respect to the camera, i.e. assuming the camera is at the origin.
The required parameters for this task are callback_url
, attachment
, and objects_to_annotate
. The callback_url
is the URL which will be POSTed on task completion, and is described in more detail in the callbacks section. The attachment
is a URL to an image you’d like to be annotated. objects_to_annotate
is an array of strings describing the different types of objects you’d like annotated.
You can optionally provide additional markdown-enabled instructions via the instruction
parameter.
It is strongly recommended for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.
You may also provide min_width
and min_height
parameters, which will tell Scalers to only annotate objects whose cuboids are of dimension at least min_width
x min_height
.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
HTTP Request
POST https://api.scale.ai/v1/task/cuboidannotation
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string |
The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
objects_to_annotate |
Array<string> |
An array of strings describing which objects you’d like cuboids to be drawn around. Each string should be singular and self-descriptive (ex: “car”, “street sign”, “pedestrian”). You may include at most 6 objects. If you need more objects, try using nested annotation labels. |
attachment |
string |
A URL to the image you’d like to be annotated with perspective cuboids. |
instruction (optional) |
string |
A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more. |
camera_intrinsics (optional) |
object |
Object defining camera intrinsics, in format {fx: 1300, fy: 1300, cx: 600, cy: 300, scalefactor: 2.75, skew: 0} (skew defaults to 0, scalefactor defaults to 1). scalefactor is used if the image sent is of different dimensions from the original photo (if the attachment is half the original, set scalefactor to 2) to correct the focal lengths, offsets. Use in conjunction with camera_rotation_quaternion to get perspective-corrected cuboids and 3d points. |
camera_rotation_quaternion (optional) |
object |
Object defining the rotation of the camera in relation to the world. Expressed as a quaternion, in format {w: 0.5, x: 0.5, y: 0.5, z: 0.5} . Use in conjunction with camera_intrinsics to get perspective-corrected cuboids and 3d points. Note that the z-axis of the camera frame represents the camera’s optical axis. |
min_height (optional, default 0 ) |
integer |
The minimum height in pixels of the bounding cuboids you’d like to be made. |
min_width (optional, default 0 ) |
integer |
The minimum width in pixels of the bounding cuboids you’d like to be made. |
min_annotations (optional, default 0 ) |
integer |
The minimum number of cuboids that will be returned. Should only be used when one is certain that the image contains at least this many annotatable cuboids. |
max_annotations (optional) |
default no limit | The maximum number of cuboids that can be annotated in the image. A value of 0 (default ) means no limit. |
attachment_type (optional, default image ) |
string |
Describes what type of file the attachment is. We currently only support image for the annotation endpoint. |
metadata (optional, default {} ) |
object |
A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
layers (optional) |
object |
A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
Response & Callback Format
The response
field, which is part of the callback POST request and permanently stored as part of the task object, will contain only an annotations
field.
The annotations field will contain an array of Cuboid
objects. We define the Vertex
and Cuboid
objects below:
Definition: Vertex
objects
Vertex
objects are represented as dictionaries with the following structure:
Key | Description |
---|---|
x |
The distance, in pixels, between the vertex and the left border of the image. |
y |
The distance, in pixels, between the vertex and the top border of the image. |
type |
This will be “vertex” for all points |
description |
An plaintext description of this vertex |
This is the list of all possible descriptions for a vertex:
face-topleft
face-bottomleft
face-topright
face-bottomright
side-topcorner
side-bottomcorner
Definition: Edge
objects
Edge
objects are represented as dictionaries with the following structure:
Key | Description |
---|---|
x1 |
The distance, in pixels, between the first vertex and the left border of the image. |
y1 |
The distance, in pixels, between the first vertex and the top border of the image. |
x2 |
The distance, in pixels, between the second vertex and the left border of the image. |
y2 |
The distance, in pixels, between the second vertex and the top border of the image. |
type |
This will be “edge” for all edges |
description |
An plaintext description of this edge |
This is the list of all possible descriptions for an edge:
face-top
face-bottom
face-left
face-right
side-top
side-bottom
Definition: Cuboid
objects
Points on cuboid are returned in this order for both points_2d and points_3d:
3-------2
/| /|
/ | / |
0-------1 |
| 7----|--6
| / | /
4-------5
Cuboid
objects are represented as dictionaries with the following structure:
Key | Description |
---|---|
label |
string belonging to objects_to_annotate which describes what kind of object the cuboid annotates |
vertices |
A list of Vertex objects defining all visible vertices of the cuboid. |
edges |
A list of Edge objects defining the edges of the cuboid. |
points_2d (optional) |
If camera_rotation_quaternion , camera_intrinsics were provided, gives 2d points of all 8 vertices of the cuboid after perspective correction. |
points_3d (optional) |
If camera_rotation_quaternion , camera_intrinsics were provided, gives 3d coordinates (arbitrarily scaled, relative to the camera location) of all 8 vertices of the cuboid after perspective correction. |
Example callback body sent on completion
{
"response": {
"annotations": [
{
"label": "car",
"vertices": [
{
"description": "face-topleft",
"y": 270,
"x": 293,
"type": "vertex"
},
{
"description": "face-bottomleft",
"y": 437,
"x": 293,
"type": "vertex"
},
{
"description": "face-topright",
"y": 270,
"x": 471,
"type": "vertex"
},
{
"description": "face-bottomright",
"y": 437,
"x": 471,
"type": "vertex"
},
{
"description": "side-topcorner",
"y": 286,
"x": 607,
"type": "vertex"
},
{
"description": "side-bottomcorner",
"y": 373,
"x": 607,
"type": "vertex"
}
],
"edges": [
{
"description": "face-top",
"x1": 293,
"y1": 270,
"x2": 471,
"y2": 270,
"type": "edge"
},
{
"description": "face-right",
"x1": 471,
"y1": 270,
"x2": 471,
"y2": 437,
"type": "edge"
},
{
"description": "face-bottom",
"x1": 471,
"y1": 437,
"x2": 293,
"y2": 437,
"type": "edge"
},
{
"description": "face-left",
"x1": 293,
"y1": 437,
"x2": 293,
"y2": 270,
"type": "edge"
},
{
"description": "side-top",
"x1": 471,
"y1": 270,
"x2": 607,
"y2": 286,
"type": "edge"
},
{
"description": "side-bottom",
"x1": 471,
"y1": 437,
"x2": 607,
"y2": 373,
"type": "edge"
}
]
}
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// populated task for convenience
...
}
}
Polygon Annotation
curl "https://api.scale.ai/v1/task/polygonannotation" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Draw a tight polygon around every **car** in the image." \
-d attachment_type=image \
-d attachment="http://i.imgur.com/XOJbalC.jpg" \
-d objects_to_annotate="car" \
-d objects_to_annotate="truck" \
-d with_labels=true
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_polygonannotation_task(
callback_url='http://www.example.com/callback',
instruction='Draw a tight polygon around every **car** in the image.',
attachment_type='image',
attachment='http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate=['car', 'truck'],
with_labels=True
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createPolygonannotationTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Draw a tight polygon around every **car** in the image.',
'attachment_type': 'image',
'attachment': 'http://i.imgur.com/XOJbalC.jpg',
'objects_to_annotate': ['car', 'truck'],
'with_labels': true
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_polygonannotation_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Draw a tight polygon around every **car** in the image.',
attachment_type: 'image',
attachment: 'http://i.imgur.com/XOJbalC.jpg',
objects_to_annotate: ['car', 'truck'],
with_labels: true
})
=> #<Scale::Api::Tasks::Polygonannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="polygonannotation", @instruction="Draw a tight polygon around every **car** in the image.", @params={"with_labels"=>true, "objects_to_annotate"=>["car", "truck"], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "5774cc78b01249ab09f089dd",
"created_at": "2016-9-03T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "polygonannotation",
"status": "pending",
"instruction": "Draw a tight polygon around every **car** in the image.",
"urgency": "standard",
"params": {
"with_labels": true,
"objects_to_annotate": [
"car",
"truck"
],
"attachment_type": "image",
"attachment": "http://i.imgur.com/XOJbalC.jpg"
},
"metadata": {}
}
This endpoint creates a polygonannotation
task. In this task, one of our Scalers view the given image and draw bounding polygons around the specified objects, returning the vertices of these polygons.
The required parameters for this task are callback_url
, attachment
, and objects_to_annotate
. The callback_url
is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment is a URL to an image you’d like to be annotated.
objects_to_annotate
is an array of strings describing the different types of objects you’d like annotated.
You can optionally provide additional markdown-enabled instructions via the instruction
parameter.
Each annotation will be annotated with a label specifying what type of object it is, with the label coming from the provided objects_to_annotate
list. If your application does not require the particular label, you can set the with_labels
parameter to false
.
You can also optionally set min_vertices
and/or max_vertices
which specify the minimum and maximum number of vertices that you’d like on a polygon. Note the range is inclusive of both bounds. For example, if min_vertices
is 3 and max_vertices
is 5, then Scale will only return triangles, quadrilaterals, and pentagons.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
HTTP Request
POST https://api.scale.ai/v1/task/polygonannotation
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
objects_to_annotate |
[string] | An array of strings describing which objects you’d like bounding polygons to be drawn around. Each string should be singular and self-descriptive (ex: “cat”, “street sign”, “potato”). You may include at most 6 objects. If you need more objects, try using nested annotation labels. |
attachment |
string | A URL to the image you’d like to be annotated with bounding polygons. |
with_labels (optional, default true ) |
boolean | Specifies whether you’d like labels for each bounding polygon in the response. Each label will be a member of the objects_to_annotate array. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
instruction (optional) |
string | A markdown-enabled string explaining how to draw the bounding polygons. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type (optional, default image ) |
string | Describes what type of file the attachment is. We currently only support image for the polygon annotation endpoint. |
min_vertices (optional, default 1) |
number | An optional parameter defining the minimum number of vertices in a valid polygon annotation for your request. |
max_vertices (optional, default null ) |
number | An optional parameter defining the maximum number of vertices in a valid polygon annotation for your request. Must be at least min_vertices . |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
layers (optional) |
object | A set of existing read-only objects to be pre-drawn on the image. See the Layers section for more detail. |
Callback Format
Example callback body sent on completion
{
"response": {
"annotations": [
{
"label": "car",
"vertices": [
{
"x": 123,
"y": 10
},
{
"x": 140,
"y": 49
},
{
"x": 67,
"y": 34
}
]
},
{ ... },
{ ... }
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// task inlined for convenience
...
}
}
The response
field, which is part of the callback POST request and permanently stored as part of the task object, will contain either an annotations
field or an error
field.
The annotations
field will contain an array of annotations. Each annotation will have the following values:
vertices
: An array describing the vertices of the polygon, listed in order from one end to another. Each vertex will be described by an object with the following structure:x
: The distance, in pixels, between the vertex and the left border of the image.y
: The distance, in pixels, between the vertex and the top border of the image.
label
(ifwith_labels
istrue
): The label for the line, which will be one of the specifiedtask.params.objects_to_annotate
.
If the attachment was invalid, the error will be detailed in the error
field.
Semantic Segmentation
curl "https://api.scale.ai/v1/task/segmentannotation" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Please segment the image using the given labels." \
-d attachment_type=image \
-d attachment="http://i.imgur.com/XOJbalC.jpg" \
-d labels="background" \
-d labels="road" \
-d labels="vegetation" \
-d labels="lane marking" \
-d instance_labels="vehicle" \
-d instance_labels="pedestrian" \
-d allow_unlabeled=false
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_segmentannotation_task(
callback_url='http://www.example.com/callback',
instruction='Please segment the image using the given labels.',
attachment_type='image',
attachment='http://i.imgur.com/XOJbalC.jpg',
labels=['background', 'road', 'vegetation', 'lane marking'],
instance_labels=['vehicle', 'pedestrian'],
allow_unlabeled=False
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createSegmentannotationTask({
callback_url: 'http://www.example.com/callback',
instruction: 'Please segment the image using the given labels.',
attachment_type: 'image',
attachment: 'http://i.imgur.com/XOJbalC.jpg',
labels: ['background', 'road', 'vegetation', 'lane marking'],
instance_labels: ['vehicle', 'pedestrian'],
allow_unlabeled: false
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_segmentannotation_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Please segment the image using the given labels.',
attachment_type: 'image',
attachment: 'http://i.imgur.com/XOJbalC.jpg',
labels: ['background', 'road', 'vegetation', 'lane marking'],
instance_labels: ['vehicle', 'pedestrian'],
allow_unlabeled: false
})
=> #<Scale::Api::Tasks::Segmentannotation:0x007fcc11092f10 @task_id="58a6363baa9d139b20a4252f", @type="segmentannotation", @instruction="Please segment the image using the given labels.", @params={"allow_unlabeled"=>false, "labels"=>['background', 'road', 'vegetation', 'lane marking'], "instance_labels"=>['vehicle', 'pedestrian'], "attachment_type"=>"image", "attachment"=>"http://i.imgur.com/XOJbalC.jpg"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:31:07 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "5774cc78b01249ab09f089dd",
"created_at": "2016-9-03T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "segmentannotation",
"status": "pending",
"instruction": "Please segment the image using the given labels.",
"urgency": "standard",
"params": {
"allow_unlabeled": false,
"labels": [
"background",
"road",
"vegetation",
"lane marking"
],
"instance_labels": [
"vehicle",
"pedestrian"
],
"attachment_type": "image",
"attachment": "http://i.imgur.com/XOJbalC.jpg"
},
"metadata": {}
}
This endpoint creates a segmentannotation
task. In this task, one of our Scalers view the given image and classify every pixel of the image according to the labels provided. You will get a full semantic, pixel-wise, dense segmentation of the image.
We also support instance-aware semantic segmentations, also called panoptic segmentation, via the instance_labels
parameter.
The required parameters for this task are callback_url
, attachment
, and labels
. The callback_url
is the URL which will be POSTed on task completion, and is described in more detail in the Callbacks section. The attachment
is a URL to an image you’d like to be segmented.
labels
is an array of strings describing the different types of objects you’d like to be used to segment the image.
You can optionally provide additional markdown-enabled instructions via the instruction
parameter.
You can also optionally set allow_unlabeled
to true, which will allow the existence of unlabeled pixels in the segmentation response - otherwise, all pixels in the image will be classified (in which case it’s important that there are labels for everything in the image, to avoid misclassification).
The response you will receive will be a series of images where each pixel’s value corresponds to the label, either via a numerical index or a color mapping. You will also get separate masks for each label for convenience.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
HTTP Request
POST https://api.scale.ai/v1/task/segmentannotation
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
labels |
Array<string> |
An array of strings describing the different types of objects you’d like to be used to segment the image. Each string should be singular and descriptive (ex: car , background , pole ). You may include at most 50 labels. |
instance_labels (optional, default []) |
Array<string> |
An array of strings describing the different types of objects you’d like to be segmented on a per-instance basis. In the case of instance_labels , the objects will get different masks / colors for each instance. For example, if you defined car as an instance_label , each individual car would get a separate mask in the image, allowing you to distinguish between them. You may include at most 10 instance labels. If you need more instance labels, try using nested annotation labels. |
attachment
| string | A URL to the image you’d like to be segmented.
allow_unlabeled
(optional, default false
) | boolean | Specifies whether you’ll allow unlabeled pixels in the segmentation. If left with the default value of false
, all pixels in the image will be classified using the labels you provided.
urgency
(optional, default standard
) | string | A string describing the urgency of the response. One of express
or standard
. See the Urgency section for more details.
instruction
(optional) | string | A markdown-enabled string explaining how carry out the semantic segmentation. You can use markdown to show example images, give structure to your instructions, and more.
attachment_type
(optional, default image
) | string | Describes what type of file the attachment is. We currently only support image
for the semantic segmentation endpoint.
metadata
(optional, default {}
) | object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format.
Callback Format
Example callback body sent on completion
{
"task": {
// task inlined for convenience
...
},
"response": {
"annotations": {
"unlabeled": null,
"labeled": {
"lane marking": "https://scaleapi-attachments.s3.amazonaws.com/3f184900-6809-11e7-bb22-c346fd2b0658",
"vehicle": [
"https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40a5f60",
"https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40a5120"
],
"pedestrian": [
"https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40a5550",
"https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7566ef0-8119-11e7-ac39-7d56f40b5230"
],
"road": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b756bd10-8119-11e7-ac39-7d56f40a5f60",
"vegetation": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b756bd11-8119-11e7-ac39-7d56f40a5f60",
"background": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7573240-8119-11e7-ac39-7d56f40a5f60"
},
"combined": {
"image": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/b7573241-8119-11e7-ac39-7d56f40a5f60",
"indexedImage": "https://scaleapi-attachments.s3-ap-northeast-1.amazonaws.com/ba9a9d70-8119-11e7-ac39-7d56f40a5f60"
}
},
"labelMapping": {
"lane marking": {
"color": "#9e3fff",
"index": 6
},
"vehicle": [
{
"color": "#8000ff",
"index": 5
},
{
"color": "#ff0080",
"index": 4
}
],
"pedestrian": [
{
"color": "#80ffff",
"index": 7
},
{
"color": "#ff8888",
"index": 8
}
],
"background": {
"color": "#00ffff",
"index": 3
},
"road": {
"color": "#80ff00",
"index": 2
},
"vegetation": {
"color": "#ff0000",
"index": 1
}
},
"instances": {}
},
"task_id": "598e0da3e8068e06002d9407"
}
The response
field, which is part of the callback POST request and permanently stored as part of the task
object, will contain a labelMapping
and an annotations
field.
labelMapping
is a dictionary where the keys are each label name, and the value is an object with the index
and color
used to represent each label in the response images. In the case of instance_labels
, the value will be an array of such objects with index
and color
keys, representing each instance.
annotations
is an object that contains URLs of PNG images describing the segmentation result in different ways:
combined
: an object containing the URLs of two PNG images describing the segmentation response:image
: a 4-channel PNG image URL where each non-transparent colored pixel represents a pixel annotated with a given label, and each transparent pixel represents an unlabeled one. Thecolor
values in thelabelMapping
object are used.indexedImage
: a 1-channel PNG image URL where each positive integer value represents a pixel annotated with a given label, and each 0 value represents an unlabeled pixel. Theindex
values in thelabelMapping
object are used.
labeled
: a dictionary where the keys are each label name, and the values are either:- a 4-channel PNG image URL where each non-transparent colored pixel (using the label’s
color
) represents a pixel annotated with that label in the segmentation response, or null
, if there is no pixel in the image annotated with that label in the segmentation response- In the case of an
instance_label
, an array of 4-channel PNG image URLs where each non-transparent colored pixel (using the label’scolor
) represents a pixel annotated with that label in the segmentation response. Each element of the array will represent a different instance of the label.
- a 4-channel PNG image URL where each non-transparent colored pixel (using the label’s
unlabeled
:- a 4-channel PNG image URL where each non-transparent black (#000000) pixel represents a pixel left unlabeled in the segmentation response, or
null
, if there is no pixel left unlabeled in the segmentation response
Video Annotation
curl "https://api.scale.ai/v1/task/videoboxannotation" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Please track each object" \
-d attachment_type=image \
-d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png" \
-d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png" \
-d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png" \
-d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png" \
-d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png" \
-d attachments="https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png" \
-d objects_to_annotate="car" \
-d objects_to_annotate="pedestrian" \
-d objects_to_annotate="cyclist" \
-d annotation_attributes[location][conditions][label_condition][label][]='cyclist' \
-d annotation_attributes[location][description]='Where is the cyclist?' \
-d annotation_attributes[location][choices][]='Sidewalk' \
-d annotation_attributes[location][choices][]='Bike lane' \
-d annotation_attributes[location][choices][]='Shared lane' \
-d annotation_attributes[location][choices][]='Road' \
-d annotation_attributes[location][choices][]='Other' \
-d with_labels=true \
-d min_width="30" \
-d min_height="30"
import requests
import json
payload = {
'callback_url': 'http://www.example.com/callback',
'instruction': 'Please track each object',
'attachment_type': 'image',
'attachments': [
'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png',
'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png',
'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png',
'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png',
'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png',
'https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png',
],
'objects_to_annotate': [ 'car', 'pedestrian', 'cyclist' ],
'annotation_attributes': {
'location': {
'conditions': {
'label_condition': {
'label': [
'cyclist'
]
}
},
'description': 'Where is the cyclist?',
'choices': [
'Sidewalk',
'Bike lane',
'Shared lane',
'Road',
'Other'
]
}
},
'with_labels': True,
'min_width': '30',
'min_height': '30',
}
headers = {"Content-Type": "application/json"}
task_request = requests.post("https://api.scale.ai/v1/task/videoboxannotation",
json=payload,
headers=headers,
auth=('YOUR_SCALEAPI_KEY', ''))
print task_request.json()
// please see the Python example
# please see the Python example
The above command returns an object structured like this:
{
"callback_url": "http://www.example.com/callback",
"created_at": "2018-03-02T23:45:17.821Z",
"instruction": "Please track each object",
"is_test": false,
"metadata": {},
"params": {
"annotation_attributes": {
"location": {
"choices": [
"Sidewalk",
"Bike lane",
"Shared lane",
"Road",
"Other"
],
"conditions": {
"label_condition": {
"label": [
"cyclist"
]
}
},
"description": "Where is the cyclist?"
}
},
"attachment_type": "image",
"attachments": [
"https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000000.png",
"https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000001.png",
"https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000002.png",
"https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000003.png",
"https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000004.png",
"https://s3-us-west-2.amazonaws.com/scaleapi-lidar-images/2011_09_26_drive_0051_sync/image_02/data/0000000005.png"
],
"min_height": 30,
"min_width": 30,
"objects_to_annotate": [
"car",
"pedestrian",
"cyclist"
],
"with_labels": true
},
"status": "pending",
"task_id": "5a99e20de50d4979ce6d291e",
"type": "videoboxannotation",
"urgency": "day"
}
This endpoint creates a videoboxannotation
task. Given a series of images sampled from a video, (which we will now refer to as “frames”), Scale will annotate each frame with bounding boxes.
The required parameters for this task are callback_url
, attachments
, and objects_to_annotate
. The callback_url
is the URL which will be POSTed on task completion, and is described in more detail in the callbacks section. The attachments
are a list of URLs to the frames you’d like to be annotated. objects_to_annotate
is an array of strings describing the different types of objects you’d like annotated.
You can optionally provide additional markdown-enabled instructions via the instruction parameter.
It is strongly recommended for you to flesh out your Markdown instructions with many examples of tasks being done correctly and incorrectly.
You may also provide min_width
and min_height
parameters, which will tell Scalers to only annotate objects whose boxes are of dimension at least min_width
x min_height
.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
HTTP Request
POST https://api.scale.ai/v1/task/videoboxannotation
Parameters
Parameter | Type | Description |
---|---|---|
callback_url |
string |
The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
objects_to_annotate |
Array<string> |
An array of strings describing which objects you’d like boxes to be drawn around. Each string should be singular and self-descriptive (ex: “car”, “street sign”, “pedestrian”). You may include at most 6 objects. |
instruction (optional) |
string |
A markdown-enabled string explaining how to draw the bounding boxes. You can use markdown to show example images, give structure to your instructions, and more. |
min_height (optional, default 0 ) |
integer |
The minimum height in pixels of the bounding boxes you’d like to be made. |
min_width (optional, default 0 ) |
integer |
The minimum width in pixels of the bounding boxes you’d like to be made. |
attachment_type (optional, default image ) |
string |
Describes what type of file the attachment is. We currently support image and video . In the case of video, we support all the formats that can be decoded by FFMPEG. |
attachments |
Array<string> |
An array of URLs to the frames you’d like to be annotated with bounding boxes. This is required if attachment_type is image and must be omitted if attachment_type is video . |
attachment |
string |
URL of the video file you’d like to be annotated with bounding boxes. This is required if attachment_type is video and must be omitted if attachment_type is image . |
frame_rate (optional, default 1 ) |
integer |
Number of frames to capture in one second. This is ignored if attachment_type is image. |
start_time (optional, default 0 ) |
integer |
Start time in seconds. This is ignored if attachment_type is image . |
duration_time (optional, defaults to video length) |
integer |
Duration of the video in seconds. This is ignored if attachment_type is image . |
metadata (optional, default {} ) |
object |
A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
urgency (optional, default day ) |
string |
A string describing the urgency of the response. One of immediate , day , or week , where immediate is a best effort six-hour response time (turnaround time is typically 6 to 24 hours). |
annotation_attributes (optional, default {} ) |
object |
See the Annotation Attributes for more details about annotation attributes. |
Response & Callback Format
Example callback body sent on completion
{
"response": {
"annotations": {
"url": "https://scaleapi-results.s3.amazonaws.com/5e5dc4a0-1d9b-11e8-bce8-41122de76216"
}
},
"task_id": "",
"task": { ... }
}
The response field, which is part of the callback POST request and permanently stored as part of the task object, will contain a link to the annotations file
containing a list of annotations per frame. Each annotation object contains its unique id to be used for tracking an object as well as its position, label, and attributes.
Definition: Annotation
objects
Example
annotations file
JSON
[
{
"annotations": [
{
"uuid": "26201bd9-72ab-4b63-ab7d-df4e671161b9",
"type": "box",
"label": "cyclist",
"top": 144,
"left": 845,
"height": 168,
"width": 72,
"attributes": {
"location": "Sidewalk"
}
},
{
"uuid": "4fa43f3a-81cf-492f-8b6e-3b7b2c824d05",
"type": "box",
"label": "car",
"top": 190,
"left": 330,
"height": 76,
"width": 109
}
]
},
{
"annotations": [
{
"uuid": "26201bd9-72ab-4b63-ab7d-df4e671161b9",
"type": "box",
"label": "cyclist",
"top": 146,
"left": 830,
"height": 154,
"width": 62,
"attributes": {
"location": "Sidewalk"
}
},
{
"uuid": "4fa43f3a-81cf-492f-8b6e-3b7b2c824d05",
"type": "box",
"label": "car",
"top": 192,
"left": 289,
"height": 90,
"width": 129
}
]
}
]
Annotation
objects are represented as dictionaries with the following structure:
Key | Type | Description |
---|---|---|
uuid |
string |
A computer-generated identifier for this annotation’s path that can be used to track it across different frames |
label |
string |
The label of this annotation. Across all frames, any annotation objects with the same uuid will have the same label. |
attributes |
object |
See the Annotation Attributes section for more details about the attributes response field. |
type |
string |
This will be “box” for all annotations. |
left |
number | The distance, in pixels, between the vertex and the left border of the image. |
top |
number | The distance, in pixels, between the vertex and the top border of the image. |
width |
number | The width, in pixels, of the bounding box. |
height |
number | The height, in pixels, of the bounding box. |
The annotations file
will be stored on S3 and be a JSON encoded list of annotation objects per frame. The index of each annotation list will correspond to the frame defined in the original response.
The order of the annotations lists will correspond directly to the order of the frame attachments that you sent in your original request. That is, the first annotation list will correspond to the first frame in the attachments array, etc.
Once the JSON file is reviewed and uploaded, we will send a response to your callback URL.
Computer Vision Layers
All of these input formats have the same formatting as our responses, so passing one Scale tasks’ data to another Scale task is really easy.
// This example uses express with body-parser
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
app.post('/polygon_task_callback_handler', function(req, res) {
// Validate callback auth key
...
var polygons = req.body.response.annotations;
var attachment = req.body.task.params.attachment;
client.createAnnotationTask({
'callback_url': 'http://www.example.com/annotation_task_callback_handler',
'instruction': 'Draw a box around each **car** and **pedestrian**',
'attachment_type': 'image',
'attachment': attachment,
'objects_to_annotate': ['car', 'pedestrian'],
'with_labels': true,
'min_width': '30',
'min_height': '30',
'layers': {
'polygons': polygons
}
}, (err, task) => {
// do something with task
});
});
# This example uses Flask
import scaleapi
from flask import request, Flask
app = Flask(__name__)
client = scaleapi.ScaleClient('{{ApiKey}}')
@app.route('/polygon_task_callback_handler', methods=['POST'])
def create_annotation_task():
# Validate callback auth key
...
polygons = request.json['response']['annotations']
attachment = request.json['task']['params']['attachment']
task = client.create_annotation_task(
callback_url='http://www.example.com/annotation_task_callback_handler',
instruction='Draw a box around each **car** and **pedestrian**',
attachment_type='image',
attachment=attachment,
objects_to_annotate=['car', 'pedestrian'],
with_labels=True,
min_width='30',
min_height='30',
layers={'polygons': polygons}
)
# do something with the task
# This example uses Rails
require 'scale'
class ExampleController < ActionController::Base
def initialize
@scale = Scale.new(api_key: '{{ApiKey}}')
end
# This should be mapped to a POST route
def polygon_task_callback_handler
# Validate callback auth key
...
polygons = params['response']['annotations']
attachment = params['task']['params']['attachment']
task = @scale.create_annotation_task({
callback_url: 'http://www.example.com/annotation_task_callback_handler',
instruction: 'Draw a box around each **car** and **pedestrian**',
attachment_type: 'image',
attachment: attachment,
objects_to_annotate: ['car', 'pedestrian'],
with_labels: true,
min_width: '30',
min_height: '30',
layers: {polygons: polygons}
})
# do something with the task
end
end
“Layers” can be used to specify existing (read-only) boxes, lines, polygons, and/or cuboids to be pre-drawn on an image.
For instance, you could specify boxes around the cars of an image, for a task that requires drawing polygons around the currently boxed cars. Or you could create a task for drawing boxes around all cars which you hadn’t already recognized.
To specify layers, you can pass an optional layers
parameter in the request for annotation
, lineannotation
, polygonannotation
, or pointannotation
tasks. They can also be used in categorization
tasks if the attachment_type
is image
.
The layers
parameter can contains fields for boxes
, lines
, polygons
, and cuboids
, which are arrays of the corresponding elements. Each of these elements are specified in the same format as the responses for their respective endpoints.
2D Boxes
Example layers param with boxes
{
"lines": [
...
],
"polygons": [
...
],
"boxes": [
{
"label": "car",
"height": 97,
"width": 147,
"top": 229,
"left": 300
}
],
"cuboids": [
...
]
}
Each of the boxes
is specified with an object containing left
, top
, width
, and height
keys, and an optional label
.
Polygons
Example layers param with polygons
{
...
"polygons": [
{
"vertices": [
{
"y": 145,
"x": 356
},
{
"y": 103,
"x": 502
},
{
"y": 264,
"x": 482
}
],
"label": "building"
}
]
}
polygons
are objects with an optional label
field, and a vertices
field which contains a list of objects with x
and y
attributes.
Lines & Splines
Example layers param with lines
{
...
"lines": [
{
"vertices": [
{
"y": 323,
"x": 414
},
{
"y": 164,
"x": 616
},
{
"y": 334,
"x": 776
}
],
"label" : "crosswalk",
"spline": true
}
]
}
lines
are objects with an optional label
field, and a vertices
field which contains a list of objects with x
and y
attributes, similar to polygons. Additionally, line objects have an optional spline
flag, which determines whether the curve used is linear (if the value is false
or the param is not passed) or cardinal (if the value is true
).
3D Cuboids
Example layers param with cuboids
{
...
"cuboids": [
{
"vertices" : [
{
"description" : "face-topleft",
"y" : 219.0,
"x" : 137.0
},
{
"description" : "face-bottomleft",
"y" : 318.0,
"x" : 137.0
},
{
"description" : "face-topright",
"y" : 219.0,
"x" : 245.0
},
{
"description" : "face-bottomright",
"y" : 318.0,
"x" : 245.0
},
{
"description" : "side-topcorner",
"y" : 165.0,
"x" : 316.0
},
{
"description" : "side-bottomcorner",
"y" : 264.0,
"x" : 316.0
}
],
"label" : "car"
}
]
}
cuboids
are described in a similar way as polygons or lines, but the vertices
also need a description
, using the following values to identify them:
- face-topleft
- face-bottomleft
- face-topright
- face-bottomright
- side-topcorner
- side-bottomcorner
Nested Annotation Labels
curl "https://api.scale.ai/v1/task/annotation" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json" \
-X POST \
-d '
{
"urgency": "standard",
"callback_url": "http://www.example.com/callback",
"attachment": "https://i.imgur.com/VDPoOZE.jpg",
"attachment_type": "image",
"metadata": {
"some_metadata": "some metadata"
},
"instruction": "Annotate the cars and pedestrians",
"min_width": 30,
"min_height": 30,
"objects_to_annotate": [
{
"choice": "Vehicle",
"subchoices": ["Car", "Truck", "Train", "Motorcycle"]
},
{
"choice": "Pedestrian",
"subchoices": ["Animal", "Adult", "Child"]
}
],
"annotation_attributes": {
"occlusion": {
"description": "What percent of the object is occluded?",
"choices": ["0%", "25%", "50%", "75%"],
"conditions": {
"label_condition": {
"label": ["Car", "Truck", "Motorcycle"]
}
}
}
}
}'
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_cuboidannotation_task(
urgency="standard",
callback_url="http://www.example.com/callback",
attachment="https://i.imgur.com/VDPoOZE.jpg",
attachment_type="image",
metadata={
"some_metadata": "some metadata"
},
instruction="Annotate the cars and pedestrians",
min_width=30,
min_height=30,
objects_to_annotate=[
{
"choice": "Vehicle",
"subchoices": ["Car", "Truck", "Train", "Motorcycle"]
},
{
"choice": "Pedestrian",
"subchoices": ["Animal", "Adult", "Child"]
}
],
annotation_attributes={
"occlusion": {
"description": "What percent of the object is occluded?",
"choices": ["0%", "25%", "50%", "75%"],
"conditions": {
"label_condition": {
"label": ["Car", "Truck", "Motorcycle"]
}
}
}
}
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createCuboidannotationTask({
urgency: 'standard',
callback_url: 'http://www.example.com/callback',
attachment: 'https://i.imgur.com/VDPoOZE.jpg',
attachment_type: 'image',
metadata: {
some_metadata: 'some metadata'
},
instruction: 'Annotate the cars and pedestrians',
min_width: 30,
min_height: 30,
objects_to_annotate: [
{
choice: 'Vehicle',
subchoices: ['Car', 'Truck', 'Train', 'Motorcycle']
},
{
choice: 'Pedestrian',
subchoices: ['Animal', 'Adult', 'Child']
}
],
annotation_attributes: {
occlusion: {
description: 'What percent of the object is occluded?',
choices: ['0%', '25%', '50%', '75%'],
conditions: {
label_condition: {
label: ['Car', 'Truck', 'Motorcycle']
}
}
}
}
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_cuboidannotation_task({
urgency: "standard",
callback_url: "http://www.example.com/callback",
attachment: "https://i.imgur.com/VDPoOZE.jpg",
attachment_type: "image",
metadata: {
some_metadata: "some metadata"
},
instruction: "Annotate the cars and pedestrians",
min_width: 30,
min_height: 30,
objects_to_annotate: [
{
choice: "Vehicle",
subchoices: ["Car", "Truck", "Train", "Motorcycle"]
},
{
choice: "Pedestrian",
subchoices: ["Animal", "Adult", "Child"]
}
],
annotation_attributes: {
occlusion: {
description: "What percent of the object is occluded?",
choices: ["0%", "25%", "50%", "75%"],
conditions: {
label_condition: {
label: ["Car", "Truck", "Motorcycle"]
}
}
}
}
})
The above command returns an object structured like this:
{
"task_id": "5774cc78b01249ab09f089dd",
"created_at": "2016-9-03T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "annotation",
"status": "pending",
"instruction": "Annotate the cars and pedestrians",
"urgency": "standard",
"params": {
"attachment": "https://i.imgur.com/VDPoOZE.jpg",
"attachment_type": "image",
"objects_to_annotate": [
{
"choice": "Vehicle",
"subchoices": ["Car", "Truck", "Train", "Motorcycle"]
},
{
"choice": "Pedestrian",
"subchoices": ["Animal", "Adult", "Child"]
}
],
"with_labels": false,
"min_width": 30,
"min_height": 30,
"examples": [],
"annotation_attributes": {
"occlusion": {
"description": "What percent of the object is occluded?",
"choices": ["0%", "25%", "50%", "75%"],
"conditions": {
"label_condition": {
"label": ["Car", "Truck", "Motorcycle"]
}
}
}
}
},
"metadata": {}
}
There are often annotation tasks that have too many label choices for a worker to efficiently sort through them all at once. In those cases, a solution is to use nested labels, where labels may have subcategories within them. A simple example illustrated in the example API call on the right is when labeling several different kinds of pedestrians and vehicles. While there may be a large amount of total labels, using subchoices a worker can first categorize an object as pedestrian or vehicle, and based on that choice select the specific type of pedestrian or vehicle.
Nested labels may be specified both for the object labels (the objects_to_annotate
array parameter), as well as in the choices
array of a
categorical annotation attribute. In both cases, you would specify a nested label by using an object instead of a string.
For example, for an objects_to_annotate
array of ["Vehicle", "Pedestrian"]
, you could instead add a nested label by passing an array like
["Vehicle", {"choice": "Pedestrian", "subchoices": ["Animal", "Adult", "Child"]}]
. Then, if a worker selected “Pedestrian” for an annotation,
they would be further prompted to choose one of the corresponding subchoices for that annotation.
These hierarchies can themselves be nested, so that the subchoices
array of a nested label may itself specify further nested labels.
Response Format
Nested labels are only intended for grouping a large set of labels together. Thus, the response will be the same as with unnested labels, where only the final “leaf” label that the worker selected is reported. In the example above, if a worker selected “Pedestrian” and then “Animal” for an annotation, that annotation’s label would be “Animal”.
An example response with nested labels is below:
{
"response": {
"annotations": [
{
"left": 123,
"top": 10,
"width": 121,
"height": 39,
"label": "Motorcycle",
"attributes": {
"occlusion": "0%"
}
},
{
"left": 82,
"top": 56,
"width": 64,
"height": 30,
"label": "Animal"
},
{ ... },
{ ... }
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// populated task for convenience
...
}
}
Conditional attributes
Nested labels can be used when specifying conditional attributes as well.
As with the response object for a nested label, the condition should specify the “leaf” labels that it is conditional on.
For example, if in the above example you wanted an “occlusion” attribute for some mix of the “leaf” labels, you could specify
the label condition for the attribute as e.g. { "label": ["Car", "Truck", "Motorcycle", "Adult"] }
. If you wanted the attribute to be
conditional on all the subchoices of a given label (e.g., all “Vehicle"s), you would simply specify all of those subchoices in the array,
e.g. { "label": ["Car", "Truck", "Train", "Motorcycle"] }
.
Annotation Attributes
In many cases, it is useful to have more human-judged metadata on top of each annotation for a 2D image annotation task, for example, measuring the occlusion-level of all vehicles in an image.
To achieve this, we support annotation_attributes
, an object representing additional attributes that you’d like to capture per image annotation.
You may use annotation_attributes
to define categorical attributes, numerical attributes, angle attributes, or text attributes for each annotation. You define the type of the attribute using the type
property of the attribute, which is either category
, number
, angle
, or text
. It defaults to category
.
The format for annotation_attributes
is an object whose key-value pairs all specify attributes of each annotation that you want to capture. The schema differs slightly based on the type of the attribute:
Categorical Attributes
Example definition of a categorical attribute.
{
...
"annotation_attributes": {
"example_category": {
"type": "category",
"description": "What is the answer to this example question?",
"choices": [
"Answer",
"Other"
]
}
}
}
Each key-value defining a categorical attribute should have the following structure:
- key: a short string that serves as the identifier for the attribute
- value: an object that should have the following keys:
description
: a human-readable string describing the attribute to a labeler.choices
: a list of strings corresponding to the categorical choices.type
(defaultcategory
): a string describing the type of attribute. Must becategory
for categorical attributes.conditions
(optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information.
Numerical Attributes
Example definition of a numerical attribute.
{
...
"annotation_attributes": {
"example_number": {
"type": "number",
"description": "What is the answer to this example question?",
"min": 0,
"max": 100
}
}
}
Each key-value defining a numerical attribute should have the following structure:
- key: a short string that serves as the identifier for the attribute
- value: an object that should have the following keys:
description
: a human-readable string describing the attribute to a labeler.type
: a string describing the type of attribute. Must benumber
for numerical attributes.min
(optional): The minimum number that may be chosen as the attribute.max
(optional): The maximum number that may be chosen as the attribute.step
(optional, default1
): The legal number intervals for the attribute.conditions
(optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information
Angle Attributes
Example definition of a angle attribute.
{
...
"annotation_attributes": {
"example_angle": {
"type": "angle",
"description": "What is the angle of this object?",
}
}
}
An angle attribute allows you to receive some angular information of the attribute, such as the heading yaw angle or heading pitch angle. This will return a number from 0
to 360
, which is the value of the angle in degrees.
Each key-value defining an angle attribute should have the following structure:
- key: a short string that serves as the identifier for the attribute
- value: an object that should have the following keys:
description
: a human-readable string describing the attribute to a labeler.type
: a string describing the type of attribute. Must beangle
for angle attributes.conditions
(optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information
Text Attributes
Example definition of a text attribute.
{
...
"annotation_attributes": {
"example_text": {
"type": "text",
"description": "What does this sign say?",
}
}
}
A text attribute allows you to receive freeform text, such as a transcription of a road sign or a car’s license plate. This will return a string.
Each key-value defining a text attribute should have the following structure:
- key: a short string that serves as the identifier for the attribute
- value: an object that should have the following keys:
description
: a human-readable string describing the attribute to a labeler.type
: a string describing the type of attribute. Must betext
for text attributes.conditions
(optional): a JSON that describes the conditions under which this attribute should be collected. See below for more information
Conditionality
The conditions
JSON describes the set of conditions
under which the attribute should be collected. Each of the conditions described in the JSON are AND
’d together, meaning that adding more conditions is more restrictive.
The conditions
JSON can have up to two key-values: label_condition
and attribute_conditions
.
Definition: Condition
Example
Condition
objects
{ "label": "vehicle" } // condition for the label being equal to vehicle
{ "label": ["car", "truck"] } // condition for the label to be equal to either car or truck
{ "is_parked": "Yes" } // condition for the is_parked attribute to be equal to be Yes
A Condition
is an object defined by a single key and a value. The key describes the semantic string that you want to check, whether it is 'label'
for the label_condition, or the attribute identifier for attribute_conditions
.
The value is either a string or an array of strings. In the case of a string, the label or attribute will be checked for strict equality with the string. In the case of an array of strings, the label or attribute will be checked for membership within the array.
Condition Parameters
The label_condition
key-value will describe the label or labels that you want to collect this attribute for. The value is a Condition
object, which must have key equal to 'label'
.
The attribute_conditions
key-value will describe under what conditions with the other attributes you’d like to collect this attribute for. The value should a Condition
object or a list of Condition
objects.
attribute_conditions
are automatically verified for circular dependencies, and tasks will be rejected if any circular dependencies are found.
Request Format
Example task payload for an
annotation
task
{
"callback_url": "http://www.example.com/callback",
"instruction": "Draw boxes around the vehicles in the image.",
"attachment_type": "image",
"attachment": "http://i.imgur.com/v4cBreD.jpg",
"objects_to_annotate": [
"car",
"bus"
],
"with_labels": true,
"annotation_attributes": {
"parked": {
"description": "Is the car currently parked?",
"choices": [
"Yes",
"No"
]
},
"heading": {
"description": "Which direction is the car heading",
"choices": [
"left",
"right",
"back",
"front"
],
"conditions": {
"label_condition": {
"label": "car"
},
"attribute_conditions": [
{
"parked": "No"
}
]
}
}
}
}
To create a task with attributes, simply add the annotation_attributes
parameter to your task creation request using the format described above.
Response Format
An example response with attributes is below:
{
"response": {
"annotations": [
{
"left": 123,
"top": 10,
"width": 121,
"height": 39,
"label": "car",
"attributes": {
"parked": "Yes",
"heading": "left"
}
},
{
"left": 82,
"top": 56,
"width": 64,
"height": 30,
"label": "car",
"attributes": {
"parked": "No",
"heading": "front"
}
},
{ ... },
{ ... }
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// populated task for convenience
...
}
}
In addition to the standard attributes for the annotation response, if you specified annotation_attributes
in the request, each annotation object will contain an attributes
JSON. It will contain keys for each of the attributes you defined in the annotation_attributes
schema, and the values will be the chosen categorical value or numerical value by the Scaler.
Create Categorization Task
Multiple
curl "https://api.scale.ai/v1/task/categorization" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Categorize this movie trailer" \
-d attachment_type=website \
-d attachment="https://www.youtube.com/embed/6ZfuNTqbHE8" \
-d taxonomies[watch][type]=category \
-d taxonomies[watch][description]="Are you going to watch the full movie?" \
-d taxonomies[watch][choices][]="yes" \
-d taxonomies[watch][choices][]="no" \
-d taxonomies[genre][type]=category \
-d taxonomies[genre][description]="What genre does it belong to?" \
-d taxonomies[genre][choices][]="Action" \
-d taxonomies[genre][choices][]="Adventure" \
-d taxonomies[genre][choices][]="Comedy" \
-d taxonomies[genre][choices][]="Crime" \
-d taxonomies[genre][choices][]="Drama" \
-d taxonomies[genre][allow_multiple]=true
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_categorization_task(
callback_url='http://www.example.com/callback',
instruction='Categorize this movie trailer',
attachment_type='website',
attachment='https://www.youtube.com/embed/6ZfuNTqbHE8',
taxonomies={
'watch': {
'type': 'category',
'description': 'Are you going to watch the full movie?',
'choices': ['yes', 'no']
},
'genre': {
'type': 'category',
'description': 'What genre does it belong to?',
'choices': ['Action', 'Adventure', 'Comedy', 'Crime', 'Drama'],
'allow_multiple': True
}
}
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createCategorizationTask(
{
callback_url: 'http://www.example.com/callback',
instruction: 'Categorize this movie trailer',
attachment_type: 'website',
attachment: 'https://www.youtube.com/embed/6ZfuNTqbHE8',
taxonomies: {
watch: {
type: 'category',
description: 'Are you going to watch the full movie?',
choices: ['yes', 'no']
},
genre: {
type: 'category',
description: 'What genre does it belong to?',
choices: ['Action', 'Adventure', 'Comedy', 'Crime', 'Drama'],
allow_multiple: true
}
}
},
(err, task) => {
// do something with task
}
);
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_categorization_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Categorize this movie trailer',
attachment_type: 'website',
attachment: 'https://www.youtube.com/embed/6ZfuNTqbHE8',
taxonomies: {
watch: {
type: 'category',
description: 'Are you going to watch the full movie?',
choices: ['yes', 'no']
},
genre: {
type: 'category',
description: 'What genre does it belong to?',
choices: ['Action', 'Adventure', 'Comedy', 'Crime', 'Drama'],
allow_multiple: true
}
}
})
The above command returns an object structured like this:
{
"task_id": "5c3f9c25744e7d005052e319",
"created_at": "2019-01-16T21:03:33.166Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "pending",
"instruction": "Categorize this movie trailer",
"params": {
"attachment_type": "website",
"attachment": "https://www.youtube.com/embed/6ZfuNTqbHE8",
"taxonomies": {
"watch": {
"type": "category",
"description": "Are you going to watch the full movie?",
"choices": [
"yes",
"no"
]
},
"genre": {
"type": "category",
"description": "What genre does it belong to?",
"choices": [
"Action",
"Adventure",
"Comedy",
"Crime",
"Drama"
],
"allow_multiple": true
}
}
},
"is_test": false,
"urgency": "standard",
"metadata": {},
"processed_attachments": []
}
This endpoint creates a categorization
task. In this task, one of our workers will view the attachment and select the correct answer for every taxonomy/question. Example use cases are spam detection, copyright detection, product categorization, etc.
This task involves a markdown-enabled instruction
about how to make the categorization, an attachment
of what you’d like to categorize, an attachment_type
, and finally taxonomies
. The format for taxonomies
is an object whose key-value pairs specify taxonomies/questions for the attachment.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
The parameters attachment_type
, attachment
and taxonomies
will be stored in the params
object of the constructed task
object.
HTTP Request
POST https://api.scale.ai/v1/task/categorization
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
instruction |
string | A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type |
string | One of text , image , video , audio , website , or pdf . Describes what type of file the attachment is. |
attachment |
string | The attachment to be categorized. If attachment_type is text , then it should be plaintext. Otherwise, it should be a URL pointing to the attachment. |
taxonomies |
object | An object whose key-value pairs specify taxonomies/questions for the attachment. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
layers (optional) |
object | A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image ). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail. |
Categorical Taxonomies
Example definition of a categorical taxonomy.
{
...
"taxonomies": {
"genre": {
"type": "category",
"description": "What genre does it belong to?",
"choices": ["Action", "Adventure", "Comedy", "Crime", "Drama"],
"allow_multiple": true
}
}
}
Each key-value defining a categorical taxonomy should have the following structure:
- key: a short string that serves as the identifier for the taxonomy.
- value: an object that should have the following keys:
description
: a human-readable string describing the taxonomy to a categorizer.choices
: a list of strings or objects of nested choices.type
: a string describing the type of taxonomy. Must becategory
for categorical taxonomies.allow_multiple
(optional): allow multiple categories to be chosen.
Categorical Taxonomies: nested choices
There are often categorization tasks that have too many choices for a worker to efficiently sort through them all at once. In those cases, a solution is to use nested choices, where choices may have subchoices within them. A simple example illustrated in the example API call on the right is when selecting a movie genre. While there may be a large amount of total choices, using subchoices a worker can first categorize a movie as adventure, and based on that choice select the specific type of adventure.
For example, for a choices array of ["Action", "Adventure"]
, you could instead add a nested choice by passing an array like ["Action", { "choice": "Adventure", "subchoices": ["Anime", "Superhero"] }]
. Then, if a worker selected Adventure
, they would be further prompted to choose one of the corresponding subchoices.
These hierarchies can themselves be nested, so that the subchoices array may itself specify further nested choices.
Numerical Taxonomies
Example definition of a numerical taxonomy.
{
...
"taxonomies": {
"genre": {
"type": "number",
"description": "Rate the movie",
"min": 0,
"max": 10,
"step": 1
}
}
}
Each key-value defining a numerical taxonomy should have the following structure:
- key: a short string that serves as the identifier for the taxonomy.
- value: an object that should have the following keys:
description
: a human-readable string describing the taxonomy to a categorizer.type
: a string describing the type of taxonomy. Must benumber
for numerical taxonomies.min
(optional): The minimum number that may be chosen as the attribute.max
(optional): The maximum number that may be chosen as the attribute.step
(optional, default1
): allow multiple categories to be chosen.
Callback Format
The callback body will look like:
{
"response": {
"taxonomies": {
"genre": ["Action", "Adventure", "Comedy", "Crime", "Drama"],
"watch": ["yes"]
}
},
"task_id": "576ba74eec471ff9b01557cc",
"task": {
// populated task for convenience
...
}
}
The response
object, which is part of the callback POST request and permanently stored as part of the task object, will have a taxonomies
field.
For categorical taxonomies, the value will be an array of choices, each one being one of the original choices.
For numerical taxonomies, the value will be a number.
Nested choices are only intended for grouping a large set of choices together. Thus, the response will be the same as with plain choices, where only the final “leaf” choice that the worker selected is reported. In the example above, if a worker selected Adventure
and then Superhero
, the choice would be Superhero
.
Single (deprecated)
curl "https://api.scale.ai/v1/task/categorization" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Is this company public or private?" \
-d attachment_type=website \
-d attachment="http://www.google.com/" \
-d categories=public \
-d categories=private
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_categorization_task(
callback_url='http://www.example.com/callback',
instruction='Is this company public or private?',
attachment_type='website',
attachment='http://www.google.com/',
categories=['public', 'private']
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createCategorizationTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Is this company public or private?',
'attachment_type': 'website',
'attachment': 'http://www.google.com/',
'categories': ['public', 'private']
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_categorization_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Is this company public or private?',
attachment_type: 'website',
attachment: 'https://www.google.com',
categories: ['public', 'private']
})
=> #<Scale::Api::Tasks::Categorization:0x007fcc11819bf8 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"allow_multiple"=>false, "categories"=>["public", "private"], "attachment"=>"https://www.google.com", "attachment_type"=>"website"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "576ba74eec471ff9b01557cc",
"created_at": "2016-06-23T09:09:34.752Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "pending",
"instruction": "Is this company public or private?",
"urgency": "standard",
"params": {
"attachment_type": "website",
"attachment": "http://www.google.com/",
"categories": [
"public",
"private"
]
},
"metadata": {}
}
This endpoint creates a categorization
task. In this task, one of our workers will view the attachment and choose a category for it according to the instruction. You may allow multiple categories to be chosen by setting allow_multiple
to true
. Example use cases are spam detection, copyright detection, product categorization, etc.
This task involves a markdown-enabled instruction
about how to make the categorization, an attachment
of what you’d like to categorize, an attachment_type
, and finally a list of categories
.
There is an optional category_ids
parameter, which you can use to impose an id system over the categories. The format of this parameter should be a dictionary, where the keys are the ids (as strings), and then the values are the category values provided in categories
. An example is:
"category_ids": {
"123": "Blue Cross",
"124": "Red Cross"
}
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
The parameters attachment_type
, attachment
, categories
, and category_ids
(optional) will be stored in the params
object of the constructed task
object.
HTTP Request
POST https://api.scale.ai/v1/task/categorization
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
instruction |
string | A markdown-enabled string explaining how to categorize the item. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type |
string | One of text , image , video , audio , website , or pdf . Describes what type of file the attachment is. |
attachment |
string | The attachment to be categorized. If attachment_type is text , then it should be plaintext. Otherwise, it should be a URL pointing to the attachment. |
categories |
[string] | An array of strings for the categories which you’d like the object to be sorted between. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
category_ids (optional) |
dictionary | An optional dictionary where the keys are the optional ids, and the values are the category values provided in categories . |
allow_multiple (optional) |
boolean | Default is false . Determines whether you allow multiple categories to be chosen for the attachment |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
layers (optional) |
object | A set of existing read-only boxes, lines, polygons and/or cuboids to be pre-drawn on the image attachment (this only works if attachment_type is image ). This can be useful e.g. to draw a bounding box indicating which object to categorize within the image. See the Layers section in image annotation for more detail. |
Callback Format
If
allow_multiple
isfalse
, the callback body will look like:
{
"response": {
"category": "one_of_the_categories"
},
"task_id": "576ba74eec471ff9b01557cc",
"task": {
// populated task for convenience
...
}
}
If
allow_multiple
istrue
, the callback body will look like:
{
"response": {
"category": ["some_of", "the_categories"]
},
"task_id": "576ba74eec471ff9b01557cc",
"task": {
// populated task for convenience
...
}
}
Example if
category_ids
is provided andallow_multiple
isfalse
{
"response": {
"category": "Blue Cross",
"category_id": "123"
},
"task_id": "576ba74eec471ff9b01557cc",
"task": {
// populated task for convenience
...
}
}
Example if
category_ids
is provided andallow_multiple
istrue
{
"response": {
"category": ["Blue Cross", "Red Cross"],
"category_id": ["123", "124"]
},
"task_id": "576ba74eec471ff9b01557cc",
"task": {
// populated task for convenience
...
}
}
The response
object, which is part of the callback POST request and permanently stored as part of the task object, will have a category
field and potentially a category_id
field.
If allow_multiple
is false
, then the value will be a string equal to one of the original categories.
If allow_multiple
is true
, the value will be an array of categories, each one being one of the original categories.
If category_ids
was provided, there will be another field category_id
corresponding to the given id of the chosen category/categories.
Create Comparison Task
curl "https://api.scale.ai/v1/task/comparison" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Do the objects in these images have the same pattern?" \
-d attachment_type=image \
-d attachments="http://i.ebayimg.com/00/\$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG" \
-d attachments="http://images.wisegeek.com/checkered-tablecloth.jpg" \
-d choices="yes" \
-d choices="no"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_comparison_task(
callback_url='http://www.example.com/callback',
instruction='Do the objects in these images have the same pattern?',
attachment_type='image',
attachments=[
'http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG',
'http://images.wisegeek.com/checkered-tablecloth.jpg'
],
choices=['yes', 'no']
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createComparisonTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Do the objects in these images have the same pattern?',
'attachment_type': 'image',
'attachments': [
'http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG',
'http://images.wisegeek.com/checkered-tablecloth.jpg'
],
'choices': ['yes', 'no']
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_comparison_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Do the objects in these images have the same pattern?',
attachments: [
'http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG',
'http://images.wisegeek.com/checkered-tablecloth.jpg'
],
attachment_type: 'image',
choices: ['yes', 'no']
})
=> #<Scale::Api::Tasks::Comparison:0x007fcc109636e0 @task_id="58a6378aaa9d139b20a42532", @type="comparison", @instruction="Do the objects in these images have the same pattern?", @params={"choices"=>["yes", "no"], "attachment_type"=>"image", "attachments"=>["http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG", "http://images.wisegeek.com/checkered-tablecloth.jpg"]}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:42 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "5774cc78b02487c424f089dd",
"created_at": "2016-06-30T07:38:32.368Z",
"callback_url": "http://www.example.com/callback",
"type": "comparison",
"status": "pending",
"instruction": "Do the objects in these images have the same pattern?",
"urgency": "standard",
"params": {
"choices": [
"yes",
"no"
],
"attachment_type": "image",
"attachments": [
"http://i.ebayimg.com/00/$T2eC16dHJGwFFZKjy5ZjBRfNyMC4Ig~~_32.JPG",
"http://images.wisegeek.com/checkered-tablecloth.jpg"
]
},
"metadata": {}
}
This endpoint creates a comparison
task. In this task, one of our workers view the given attachments and do any comparison requested.
This task involves a markdown-enabled instruction
, an array of attachments
, and an attachment_type
.
At least of the fields
or choices
parameters must specified for the data to be returned. choices
is an array of strings from which the worker to choose, and fields
is useful for free-text response.
fields
is a dictionary where the keys are the keys you’d like the results to be returned under, and values are the descriptions you’d like to show the human worker.
If successful, Scale will immediately return the generated task object, of which you should store the task_id
.
The parameters attachment_type
, attachments
, choices
, and fields
will be stored in the params
object of the constructed task
object.
HTTP Request
POST https://api.scale.ai/v1/task/comparison
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
instruction |
string | A markdown-enabled string explaining how to compare the attachments. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type |
string | One of text , image , video , audio , website , or pdf . Describes what type of file the attachments are. |
attachments |
array | An array of attachments to compare. If attachment_type is text , then each attachment should be plaintext. Otherwise, they should be URLs pointing to the attachments. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
fields (optional) |
dictionary | A dictionary corresponding to the fields to be recorded. Keys are the keys you’d like the fields to be returned under, and values are descriptions to be shown to human workers. |
choices (optional) |
[string] | An array of strings for the choices to be given to the worker. One of choices or fields must be specified. |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
Callback Format
Example callback body:
{
"response": {
"choice": "yes",
"fields": {
"difference": "The patterns are identical."
}
},
"task_id": "576ba74eec471ff9b01557cc",
"task": {
// populated task for convenience
...
}
}
The response
object, which is part of the callback POST request and permanently stored as part of the task object, will have a fields
field and/or choice
field depending on the original request.
If your original call provided choices
, choice
will be one of the original choices.
If your original call provided fields
, fields
will have keys corresponding to the keys you provided in the parameters, with values the transcribed value.
Create OCR Transcription Task
curl "https://api.scale.ai/v1/task/transcription" \
-u "{{ApiKey}}:" \
-d callback_url="http://www.example.com/callback" \
-d instruction="Transcribe the given fields." \
-d attachment_type=website \
-d attachment="http://news.ycombinator.com/" \
-d fields[title]="Title of Webpage" \
-d fields[top_result]="Title of the top result"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
client.create_transcription_task(
callback_url='http://www.example.com/callback',
instruction='Transcribe the given fields.',
attachment_type='website',
attachment='http://news.ycombinator.com/',
fields={
'title': 'Title of Webpage',
'top_result': 'Title of the top result'
}
)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
client.createTranscriptionTask({
'callback_url': 'http://www.example.com/callback',
'instruction': 'Transcribe the given fields.',
'attachment_type': 'website',
'attachment': 'http://news.ycombinator.com/',
'fields': {
'title': 'Title of Webpage',
'top_result': 'Title of the top result'
}
}, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.create_transcription_task({
callback_url: 'http://www.example.com/callback',
instruction: 'Transcribe the given fields.',
attachment_type: 'website',
attachment: 'http://news.ycombinator.com/',
fields: {
title: 'Title of Webpage',
top_result: 'Title of the top result'
}
})
=> #<Scale::Api::Tasks::Transcription:0x007fcc1098d828 @task_id="58a6361eaa9d139b20a4252d", @type="transcription", @instruction="Transcribe the given fields.", @params={"fields"=>{"top_result"=>"Title of the top result", "title"=>"Title of Webpage"}, "attachment"=>"http://news.ycombinator.com/", "attachment_type"=>"website"}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:30:38 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}>
The above command returns an object structured like this:
{
"task_id": "576de9dc1ea5f917d56fc2a0",
"created_at": "2016-06-25T02:18:04.248Z",
"callback_url": "http://www.example.com/callback",
"type": "transcription",
"status": "pending",
"instruction": "Transcribe the given fields.",
"urgency": "standard",
"params": {
"fields": {
"title": "Title of Webpage",
"top_result": "Title of the top result"
},
"attachment": "http://news.ycombinator.com/",
"attachment_type": "website"
},
"metadata": {}
}
This endpoint creates a transcription
task. In this task, one of our workers will read an attachment and arbitrarily transcribe any information you’d like. Example use cases could be transcribing information from PDFs, manually scraping a web page for information, etc.
This task involves a markdown-enabled instruction
about how to transcribe the attachment, an attachment
of what you’d like to transcribe, an attachment_type
, fields
, and repeatable_fields
.
fields
is a dictionary which describes items you’d like transcribed for the attachment. Examples are phone numbers, names, etc. repeatable_fields
is a dictionary which describes items which you’d like transcribed for the attachment which appear in the attachment multiple times. Examples are the row-by-row items of an invoice or purchase order.
In addition, we now support choices for repeatable_fields
where you can specify an array of choices for the repeatable field.
At least one of fields
or repeatable_fields
is required. Both fields
and repeatable_fields
are dictionaries where the keys are the identifiers you’d like the results to be returned using, and values are plaintext descriptions you’d like to show the Scaler as they complete the task.
If successful, Scale will immediately return the generated task object, of which you should at least store the task_id
.
The parameters attachment_type
, attachment
, fields
, and repeatable_fields
will be stored in the params
object of the constructed task
object.
HTTP Request
POST https://api.scale.ai/v1/task/transcription
Parameters
Parameter | Type | Description |
---|---|---|
project (optional) |
string | The name of the project to associate this task with. See Projects section for more details. |
batch (optional) |
string | The name of the batch to associate this task with. Note that if a batch is specified, you need not specify the project, as the task will automatically be associated with the batch’s project. See Batches section for more details. |
callback_url |
string | The full url (including the scheme http:// or https:// ) of the callback when the task is completed. See the Callback section for more details about callbacks. |
instruction |
string | A markdown-enabled string explaining how to transcribe the attachment. You can use markdown to show example images, give structure to your instructions, and more. |
attachment_type |
string | One of image , pdf , or website . Describes what type of file the attachment is. |
attachment |
string | The attachment to be transcribed. If attachment_type is text , then it should be plaintext. Otherwise, it should be a URL pointing to the attachment. |
fields (optional if using repeatable_fields ) |
object | A dictionary corresponding to the fields to be transcribed. Keys are the identifiers you’d like the fields to be returned using, and values are descriptions to be shown to the Scalers as they complete the task. |
repeatable_fields (optional if using fields ) |
object | If your task requires a transcription of items which might be repeated within the attachment, such as rows of an invoice, then this dictionary describes those fields. With the addition of choices, there are now two acceptable formats. The first acceptable format is the same format as fields to generate repeatable text fields. The second acceptable format is a dictionary with keys of description (identifiers) and choices (the list of choices for that input). In addition, for repeatable_fields with the same format as fields , our internal task model will represent each repeatable field as a dictionary object with key description to the original value. |
urgency (optional, default standard ) |
string | A string describing the urgency of the response. One of express or standard . See the Urgency section for more details. |
metadata (optional, default {} ) |
object | A set of key/value pairs that you can attach to a task object. It can be useful for storing additional information about the task in a structured format. |
Callback Format
Example callback response sent on completion with
fields
{
"response": {
"fields": {
"title": "Some Title",
"top_result": "The Top Result or Something"
}
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// populated task for convenience
...
}
}
Example callback response sent on completion with
repeatable_fields
{
"response": {
"repeatable_fields": [
{
"description": "Espresso",
"amount": "10.00"
},
{
"description": "Ice Cream",
"amount": "9.99"
}
]
},
"task_id": "5774cc78b01249ab09f089dd",
"task": {
// populated task for convenience
...
}
}
The response
object, which is part of the callback POST request and permanently stored as part of the task object, will have a fields
field.
fields
will have keys corresponding to the keys you provided in the parameters, with values equal to the transcribed value.
If you requested repeatable_fields
, repeatable_fields
in the response
will be an array of such dictionaries, with keys corresponding to the keys you provided in the repeatable_fields
parameter, and values corresponding to the transcribed value. Each element of the array will correspond to one transcribed value in the attachment.
Projects
Projects are a way of organizing similar tasks, so that one can share parameters among tasks, or control what workers are allowed to work on certain classes of tasks.
The parameters associated with a project will be inherited by tasks created under that project. Currently the only project parameter supported is instruction
; instructions defined in a project will be concatenated to the end of the instructions of tasks created under the project.
To use a project for a given task, add a project
parameter when creating the task that references the project name; e.g., to associate a task with project my_project
, add project: my_project
to the task creation request. Note that projects created using your test API key can only be used for creating test tasks; likewise with the live API key and live tasks.
Project Creation
API Endpoint
https://api.scale.ai/v1/projects
Example API call
curl "https://api.scale.ai/v1/projects" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json" \
-X POST \
-d '
{
"type": "annotation",
"name": "kitten_labeling",
"params": {
"instruction": "please label the kittens in this image"
}
}'
import requests
import json
payload = {
'type': 'annotation',
'name': 'kitten_labeling',
'params': {
'instruction': 'please label the kittens in this image'
},
}
headers = {"Content-Type": "application/json"}
create_project_request = requests.post("https://api.scale.ai/v1/projects",
json=payload,
headers=headers,
auth=('{{ApiKey}}', ''))
print create_project_request.json()
const request = require('request');
const payload = {
type: 'annotation',
name: 'kitten_labeling',
params: {
instruction: 'please label the kittens in this image'
}
};
request.post({
url: 'https://api.scale.ai/v1/projects',
json: true,
auth: {user: '{{ApiKey}}', pass: ''},
body: payload,
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/projects")
req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
req.body = {
type: 'annotation',
name: 'kitten_labeling',
params: {
instruction: 'please label the kittens in this image'
}
}.to_json
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"type": "annotation",
"name": "kitten_labeling",
"param_history": [
{
"instruction": "please label the kittens in the image",
"version": 0,
"created_at": "2018-04-20T07:38:32.368Z"
}
],
"created_at": "2018-04-20T07:38:32.368Z"
}
HTTP Request
POST https://api.scale.ai/v1/projects
Parameters
Parameter | Type | Description |
---|---|---|
type |
string | The task type of all the tasks belonging to this project |
name |
string | Name identifying this project. Must be unique among all your projects. When creating tasks for this project, you should add a project: <name> parameter to the task creation request to associate the task with this project. |
params (optional) |
object | Default parameters for a task created under this project. Currently only supports params.instruction , which get appended to the instructions of any task created under this project. |
Retrieval
API Endpoint
https://api.scale.ai/v1/projects/:projectName
Example API call
curl "https://api.scale.ai/v1/projects/kitten_labeling" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json" \
import requests
import json
headers = {"Content-Type": "application/json"}
get_project_request = requests.get("https://api.scale.ai/v1/projects/kitten_labeling",
headers=headers,
auth=('{{ApiKey}}', ''))
print get_project_request.json()
const request = require('request');
const payload = {
instruction: 'please label the kittens and the cats in this image'
};
request.get({
url: 'https://api.scale.ai/v1/projects/kitten_labeling',
json: true,
auth: {user: '{{ApiKey}}', pass: ''},
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/projects/kitten_labeling")
req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"type": "annotation",
"name": "kitten_labeling",
"param_history": [
{
"instruction": "please label the kittens in the image",
"version": 0,
"created_at": "2018-04-20T07:38:32.368Z"
}
],
"created_at": "2018-04-20T07:38:32.368Z"
}
HTTP Request
GET https://api.scale.ai/v1/projects/:projectName
returns the information for a project with name :projectName
.
GET https://api.scale.ai/v1/projects
returns information for all projects owned by a user.
List Tasks
To retrieve a list of tasks that belong to a particular project, use the project
option in the List Tasks Endpoint.
Parameter Updating
API Endpoint
https://api.scale.ai/v1/projects/:projectName/setParams
Example API call
curl "https://api.scale.ai/v1/projects/kitten_labeling/setParams" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json" \
-X POST \
-d '
{
"instruction": "please label the kittens and the cats in this image"
}'
import requests
import json
payload = {
'instruction': 'please label the kittens and the cats in this image'
}
headers = {"Content-Type": "application/json"}
set_params_request = requests.post("https://api.scale.ai/v1/projects/kitten_labeling/setParams",
json=payload,
headers=headers,
auth=('{{ApiKey}}', ''))
print set_params_request.json()
const request = require('request');
const payload = {
instruction: 'please label the kittens and the cats in this image'
};
request.post({
url: 'https://api.scale.ai/v1/projects/kitten_labeling/setParams',
json: true,
auth: {user: '{{ApiKey}}', pass: ''},
body: payload
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/projects/kitten_labeling/setParams")
req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
req.body = {instruction: 'please label the kittens and the cats in this image'}.to_json
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"type": "annotation",
"name": "kitten_labeling",
"param_history": [
{
"instruction": "please label the kittens in the image",
"version": 0,
"created_at": "2018-04-20T07:38:32.368Z"
},
{
"instruction": "please label the kittens and the cats in the image",
"version": 1,
"created_at": "2018-04-20T10:38:32.368Z"
}
],
"created_at": "2018-04-20T07:38:32.368Z"
}
HTTP Request
POST https://api.scale.ai/v1/projects/:projectName/setParams
Projects keep a history of the parameters that they were set with. Tasks created under a project inherit the latest params of the project (the last entry in param_history
), one can also specify project_param_version
when creating a task to inherit from an older set of params (or use -1 to skip parameter inheritance altogether.)
Currently we only support params.instruction
, which get appended to the instructions of any task created under this project.
Batches
Batches are a way to logically group together tasks under the same project.
Note that the API key you use to create batches/projects/tasks with must all match: test projects can only be used with test batches, and test tasks; likewise with live projects, live batches, and live tasks.
To use a batch for a given task, add a batch
parameter when creating the task that references the batch name; e.g., to associate a task with batch my_batch
, add batch: my_batch
to the task creation request. Note that batchs created using your test API key can only be used for creating test tasks; likewise with the live API key and live tasks.
Workflow
- Batch is created (status
staging
) - Tasks are added to batch (must specify what batch to add to in task creation request), at this point in time tasks cannot be worked on
- Batch is finalized (status
inprogress
) - Tasks are worked on and completed (new tasks cannot be added to this batch)
- Final task is completed, batch is completed (status
completed
) and callback is fired
Batch Creation
API Endpoint
https://api.scale.ai/v1/batches
Example API call
curl "https://api.scale.ai/v1/batches" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json" \
-X POST \
-d '
{
"project": "kitten_labeling",
"name": "kitten_labeling_2018-07",
"callback": "http://mycallback.com/"
}'
import requests
import json
payload = {
'project': 'kitten_labeling',
'name': 'kitten_labeling_2018-07',
'callback': 'http://mycallback.com',
}
headers = {"Content-Type": "application/json"}
create_batch_request = requests.post("https://api.scale.ai/v1/batches",
json=payload,
headers=headers,
auth=('{{ApiKey}}', ''))
print create_batch_request.json()
const request = require('request');
const payload = {
project: 'kitten_labeling',
name: 'kitten_labeling_2018-07',
callback: 'http://mycallback.com',
};
request.post({
url: 'https://api.scale.ai/v1/batches',
json: true,
auth: {user: '{{ApiKey}}', pass: ''},
body: payload,
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/batches")
req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
req.body = {
project: 'kitten_labeling',
name: 'kitten_labeling_2018-07',
callback: 'http://mycallback.com',
}.to_json
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"project": "kitten_labeling",
"name": "kitten_labeling_2018-07",
"status": "staging",
"callback": "http://mycallback.com",
"created_at": "2018-08-01T09:09:10.108Z"
}
Create a batch under a project, so that you can associate new tasks with this batch.
HTTP Request
POST https://api.scale.ai/v1/batches
Parameters
Parameter | Type | Description |
---|---|---|
project |
string | The name of the project this batch (and its tasks) belong to |
name |
string | Name identifying this batch. Must be unique among all batches belonging to a customer. |
callback |
string | Email to notify, or URL to POST to when a batch is complete. |
Callback will be
POST
ed with the format below:
{
"batch": "kitten_labeling_2018-07", // the name of the batch that was just completed
"status": "completed", // the status of the batch
"stats": { // counts of the tasks of this batch, grouped by their status
"completed": 20,
"canceled": 3,
"error": 5,
}
}
Retrieval
Example API call
curl "https://api.scale.ai/v1/batches/kitten_labeling" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json"
import requests
import json
headers = {"Content-Type": "application/json"}
batch_info_request = requests.get("https://api.scale.ai/v1/batches/kitten_labeling",
headers=headers,
auth=('{{ApiKey}}', ''))
print batch_info_request.json()
const request = require('request');
request.post({
url: 'https://api.scale.ai/v1/batches/kitten_labeling',
json: true,
auth: {user: '{{ApiKey}}', pass: ''},
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/batches/kitten_labeling")
req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"project": "kitten_labeling",
"name": "kitten_labeling_2018-07",
"status": "staging",
"callback": "http://mycallback.com",
"created_at": "2018-08-01T09:09:10.108Z"
}
Get information about a batch with name :batchName
.
HTTP Endpoint
GET https://api.scale.ai/v1/batches/:batchName
Batch Status
Example API call
curl "https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json"
import requests
import json
headers = {"Content-Type": "application/json"}
status_request = requests.get("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status",
headers=headers,
auth=('{{ApiKey}}', ''))
print status_request.json()
const request = require('request');
request.post({
url: 'https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status',
json: true,
auth: {user: '{{ApiKey}}', pass: ''},
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/status")
req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"status": "staging",
"pending": 20
}
This endpoint returns the status of a batch with name :batchName
, as well as the counts of its tasks grouped by task status.
HTTP Request
GET https://api.scale.ai/v1/batches/:batchName/status
List All Batches
This is a paged endpoint for all of your batches. Batches will be returned in descending order based on created_at
. Pagination is based off limit
and offset
parameters, which determine the page size and how many results to skip.
Example API Call
curl "https://api.scale.ai/v1/batches?project=kitten_labeling" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json" \
import requests
import json
headers = {"Content-Type": "application/json"}
batch_list_request = requests.get("https://api.scale.ai/v1/batches?project=kitten_labeling",
headers=headers,
auth=('{{ApiKey}}', ''))
print batch_list_request.json()
const request = require('request');
request.get({
url: 'https://api.scale.ai/v1/batches?project=kitten_labeling',
json: true,
auth: {user: '{{ApiKey}}', pass: ''},
body: payload,
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/batches?project=kitten_labeling")
req = Net::HTTP::Get.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"docs": [
{
"project": "kitten_labeling",
"name": "kitten_labeling_2018-07",
"status": "staging",
"callback": "http://mycallback.com",
"created_at": "2018-08-01T09:09:10.108Z"
}
],
"total": 1,
"limit": 100,
"offset": 0,
"has_more": false
}
HTTP Request
GET https://api.scale.ai/v1/batches
Parameters (all optional)
Parameter | Type | Description |
---|---|---|
project |
string | Project name to filter batches by |
status |
string | Status to filter batches by (staging or in_progress or completed ) |
status |
string | Status to filter batches by (staging or in_progress or completed ) |
start_time |
ISO 8601 date | The minimum value of created_at for batches to be returned |
end_time |
ISO 8601 date | The maximum value of created_at for batches to be returned |
limit |
integer | Number between 1–100, number of results per page (default 100) |
offset |
integer | Number of results to skip for showing following pages (default 0) |
List Tasks
To retrieve a list of tasks that belong to a particular batch, see the batch
option in the List Tasks Endpoint.
Finalize Batch
Example API Call
curl "https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize" \
-u "{{ApiKey}}:" \
-H "Content-Type: application/json" \
-X POST
import requests
import json
headers = {"Content-Type": "application/json"}
finalize_request = requests.post("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize"
headers=headers,
auth=('{{ApiKey}}', ''))
print finalize_request.json()
const request = require('request');
request.get({
url: 'https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize',
json: true
auth: {user: '{{ApiKey}}', pass: ''},
body: payload,
}, (err, response, body) => {
console.log(body);
});
require "net/http"
require "uri"
uri = URI.parse("https://api.scale.ai/v1/batches/kitten_labeling_2018-07/finalize")
req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json')
req.basic_auth('{{ApiKey}}', '')
response = http.request('request')
print response.body
The above command returns an object structured like this:
{
"project": "kitten_labeling",
"name": "kitten_labeling_2018-07",
"status": "in_progress",
"callback": "http://mycallback.com",
"created_at": "2018-08-01T09:09:10.108Z"
}
Finalizes a batch with name :batchName
so its tasks can be worked on.
HTTP Request
POST https://api.scale.ai/v1/batches/:batchName/finalize
Callbacks
The
callback_url
will be POSTed withapplication/json
data of the following object form:
{
"task": {
"task_id": "576c41bf13e36b0600b02b34",
"completed_at": "2016-06-23T21:54:44.904Z",
"response": {
"category": "red"
},
"created_at": "2016-06-23T20:08:31.573Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "completed",
"instruction": "Is this object red or blue?",
"urgency": "standard",
"params": {
"attachment_type": "text",
"attachment": "tomato",
"categories": [
"red",
"blue"
]
},
"metadata": {}
},
"response": {
"category": "red"
},
"task_id": "576c41bf13e36b0600b02b34"
}
On your tasks, you will be required to supply a callback_url
, a fully qualified URL that we will POST with the results of the task when completed. The data will be served as a JSON body (application/json
). Alternately, you can set a default callback URL in your profile, which will be used for tasks that do not specify one.
Additionally, in order to simplify testing and add support for email automation pipelines, you may provide an email address as the callback_url
. In this case, each completed task will result in an email sent with the body as the task’s JSON payload.
You should respond to the POST request with a 2xx status code. If we do not receive a 2xx status code, we will continue to retry up to 20 times over the course of the next 24 hours.
If we receive a 2xx status code, the task will be populated with a true
value for the callback_succeeded
parameter. Otherwise, if we do not receive a 2xx status code on any retry, the task will be populated with a false
value for the callback_succeeded
parameter.
Getting Started
We have sample callback server implementations, which you can deploy to Heroku in seconds, in the following languages:
- Node.js
- Python (coming soon!)
If you’re just testing and want to try a few requests, the easiest way to get started is to use a RequestBin and send requests using your https://someHash.x.pipedream.net
URL as the callback_url
. You can also use ngrok to expose a local server to the internet for fast prototyping.
Feel free to chat us on Slack or email us if you have any trouble.
Authentication
If you’d like to authenticate our callbacks, we set a scale-callback-auth
HTTP header on each of our callbacks. The value will be equal to your Live Callback Auth Key
shown on your dashboard. If this header is not set, or it is set incorrectly, the callback is not from Scale.
POST Data
Attribute | Type | Description |
---|---|---|
task_id |
string | The task_id is the unique identifier for the task. |
status |
string | The status of the task when it was completed. Normally completed , but can also be error in the case that a task failed processing. |
response |
object | The response object of the completed request. For categorization , it will contain a category attribute of the assigned category. |
task |
object | The full task object for reference and convenience. |
Task Endpoints
Retrieve a Task
curl "https://api.scale.ai/v1/task/{task_id}" \
-u "{{ApiKey}}:"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
task_id = 'YOUR_TASK_ID'
task = client.fetch_task(task_id)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
var task_id = 'YOUR_TASK_ID';
client.fetchTask(task_id, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
scale.tasks.find("58a63795aa9d139b20a42535")
=> #<Scale::Api::Tasks::Categorization:0x007fcc10978ab8 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"attachment_type"=>"website", "attachment"=>"https://www.google.com", "categories"=>["public", "private"], "allow_multiple"=>false}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}, @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>>
The above command returns an object structured like this:
{
"task_id": "576ba74eec471ff9b01557cc",
"completed_at": "2016-06-23T09:10:02.798Z",
"created_at": "2016-06-23T09:09:34.752Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "completed",
"instruction": "Would you say this item is big or small?",
"urgency": "standard",
"params": {
"attachment_type": "text",
"attachment": "car",
"categories": [
"big",
"small"
]
},
"response": {
"category": "big"
},
"metadata": {}
}
This endpoint retrieves a specific task.
HTTP Request
GET https://api.scale.ai/v1/task/{TASK_ID}
URL Parameters
Parameter | Description |
---|---|
task_id |
The task_id of the task to retrieve |
Returns
Returns a task if a valid identifier was provided, and returns a 404 error otherwise.
Cancel a task
curl -X POST "https://api.scale.ai/v1/task/{task_id}/cancel" \
-u "{{ApiKey}}:"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
task_id = 'YOUR_TASK_ID'
task = client.cancel_task(task_id)
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
var task_id = 'YOUR_TASK_ID';
client.cancelTask(task_id, (err, task) => {
// do something with task
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
canceled_task = scale.tasks.cancel("YOUR_TASK_ID") # Returns the appropriate Task object with status set to canceled
=> #<Scale::Api::Tasks::Categorization:0x007fcc1108b288 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"attachment_type"=>"website", "attachment"=>"https://www.google.com", "categories"=>["public", "private"], "allow_multiple"=>false}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="canceled", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}, @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>>
canceled_task.cancelled?
=> true
The above command returns an object structured like this:
{
"task_id": "576ba74eec471ff9b01557cc",
"created_at": "2016-06-23T09:09:34.752Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "canceled",
"instruction": "Would you say this item is big or small?",
"urgency": "standard",
"params": {
"attachment_type": "text",
"attachment": "car",
"categories": [
"big",
"small"
]
},
"metadata": {}
}
This endpoint cancels a task so that it will not be completed.
You may only cancel pending tasks, and the endpoint will return a 500 error code if you attempt to cancel a completed task.
HTTP Request
POST https://api.scale.ai/v1/task/{TASK_ID}/cancel
URL Parameters
Parameter | Description |
---|---|
task_id |
The task_id of the task to cancel |
Returns
Returns the canceled task if a valid identifier for a pending task was provided, and returns a 404 error or 500 error otherwise.
List All Tasks
curl "https://api.scale.ai/v1/tasks" \
-u "{{ApiKey}}:"
import scaleapi
client = scaleapi.ScaleClient('{{ApiKey}}')
task_id = 'YOUR_TASK_ID'
# specify URL params as kwargs to tasks()
tasklist = client.tasks()
var scaleapi = require('scaleapi');
var client = scaleapi.ScaleClient('{{ApiKey}}');
// specify URL parameters as properties in the params object
var params = {};
client.tasks(params, (err, tasklist) => {
// do something with tasklist
});
require 'scale'
scale = Scale.new(api_key: '{{ApiKey}}')
# You can pass params like start_time or end_time as keys to filter results
scale.tasks.where
=> #<Scale::Api::TaskList:0x007fcc11822a28 @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>, @docs=[#<Scale::Api::Tasks::Categorization:0x007fcc118227f8 @task_id="58a63795aa9d139b20a42535", @type="categorization", @instruction="Is this company public or private?", @params={"attachment_type"=>"website", "attachment"=>"https://www.google.com", "categories"=>["public", "private"], "allow_multiple"=>false}, @urgency="standard", @response=nil, @callback_url="http://www.example.com/callback", @created_at=2017-02-16 23:36:53 UTC, @status="pending", @completed_at=nil, @callback_succeeded_at=nil, @metadata={}, @client=#<struct Scale::Api api_key="live_358440e50ba277654e847a079eda9614", callback_auth_key=nil, default_request_params={:callback_url=>nil}, logging=false>>], @limit=1, @offset=0, @has_more=true, @params={:start_time=>nil, :end_time=>nil, :limit=>1, :offset=>0, :status=>nil, :type=>nil}>
# Scale::Api::TaskList implements Enumerable, so it is Array-like.
The above command returns an object structured like this:
{
"docs": [
{
"task_id": "576b998b4628d1bfaed7d3a4",
"created_at": "2016-06-23T08:10:51.032Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "completed",
"instruction": "Is this object big or small?",
"urgency": "express",
"params": {
"attachment_type": "text",
"attachment": "ant",
"categories": [
"big",
"small"
]
},
"completed_at": "2016-06-23T19:36:23.084Z",
"response": {
"category": "small"
},
"metadata": {}
},
{
"task_id": "576ba301eed30241b0e9bbf7",
"created_at": "2016-06-23T08:51:13.903Z",
"callback_url": "http://www.example.com/callback",
"type": "categorization",
"status": "completed",
"instruction": "Is this object big or small?",
"urgency": "standard",
"params": {
"attachment_type": "text",
"attachment": "T-Rex",
"categories": [
"big",
"small"
]
},
"completed_at": "2016-06-23T09:09:10.108Z",
"response": {
"category": "big"
},
"metadata": {}
}
],
"total": 2,
"limit": 100,
"offset": 0,
"has_more": false
}
This is a paged endpoint retrieves a list of your tasks. The tasks will be returned in descending order based on created_at
time. The pagination is based on the limit
and offset
parameters, which determine the page size and how many results to skip.
HTTP Request
GET https://api.scale.ai/v1/tasks
URL Parameters
Parameter | Type | Description | Required |
---|---|---|---|
start_time |
ISO 8601 Date | The minimum value of created_at for tasks to be returned |
optional |
end_time |
ISO 8601 Date | The maximum value of created_at for tasks to be returned |
optional |
status |
string | The status of the task - can be: completed , pending , or canceled |
optional |
type |
string | The type of the task - can be: transcription , categorization , comparison , annotation , datacollection , audiotranscription , or any other task type. |
optional |
project |
string | The name of the project that the returned tasks must belong to. | optional |
batch |
string | The name of the batch that the returned tasks must belong to. | optional |
limit |
integer | A number between 1 and 100, the maximum number of results to display per page | optional, default 100 |
offset |
integer | The number of results to skip, for showing the next page | optional, default 0 |
Returns
Returns a list of your tasks.
Errors
// example task.response with failed attachment
{
"error": "One or more attachments could not be downloaded.", // reason for error
"attachments": [
{
"statusCode": 403, // HTTP code received when fetching attachment
"url": "http://example.com/kitten.png", // attachment URL
}
]
}
The Scale uses the following HTTP codes:
Error Code | Meaning |
---|---|
200 | OK – Everything worked as expected. |
400 | Bad Request – The request was unacceptable, often due to missing a required parameter. |
401 | Unauthorized – No valid API key provided. |
402 | Not enabled – Please contact [email protected] before creating this type of task. |
404 | Not Found – The requested resource doesn’t exist. |
429 | Too Many Requests – Too many requests hit the API too quickly. |
500 | Internal Server Error – We had a problem with our server. Try again later. |
In the event of one or more task attachments having invalid data, the task callback will be invoked with an error response detailing the problems, and the task’s status
will be set to error
. Additional detail about the error and failed attachments will be stored in the task response.