Type text

JavaScript

import GboxSDK from "gbox-sdk";

const gboxSDK = new GboxSDK({
  apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});

async function main() {
  const box = await gboxSDK.create({ type: "android" });

  await box.action.type({
    text: "Hello, World!"
  });

  // Type and press Enter
  await box.action.type({
    text: "search gbox",
    pressEnter: true
  });
}

main();

{
  "message": "Action executed successfully",
  "actionId": "c9bdc193-b54b-4ddb-a035-5ac0c598d32d",
  "screenshot": {
    "trace": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "before": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "after": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    }
  }
}

POST

boxes

{boxId}

actions

type

JavaScript

import GboxSDK from "gbox-sdk";

const gboxSDK = new GboxSDK({
  apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});

async function main() {
  const box = await gboxSDK.create({ type: "android" });

  await box.action.type({
    text: "Hello, World!"
  });

  // Type and press Enter
  await box.action.type({
    text: "search gbox",
    pressEnter: true
  });
}

main();

{
  "message": "Action executed successfully",
  "actionId": "c9bdc193-b54b-4ddb-a035-5ac0c598d32d",
  "screenshot": {
    "trace": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "before": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "after": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    }
  }
}

Authorizations

Authorization

string

header

required

Enter your API Key in the format: Bearer . Get it from https://gbox.ai

Path Parameters

boxId

string

required

Box ID

Example:

"c9bdc193-b54b-4ddb-a035-5ac0c598d32d"

Body

application/json

Text typing action configuration

text

string

required

Text to type

Example:

"Hello, World!"

mode

enum<string>

default:append

Text input mode: 'append' to add text to existing content, 'replace' to replace all existing text

Available options:

append,

replace

Example:

"append"

pressEnter

boolean

default:false

Whether to press Enter after typing the text

Example:

false

options

Action Common Options · object

Action options. When options.screenshot is provided, ALL deprecated screenshot fields (outputFormat, presignedExpiresIn, screenshotDelay, screenshotRange, includeScreenshot) will be completely ignored.

Show child attributes

options.screenshot

Screenshot options. Can be a boolean to enable/disable screenshots, or an object to configure screenshot options.

Show child attributes

options.screenshot.outputFormat

enum<string>

default:base64

Type of the URI. default is base64.

Available options:

base64,

storageKey

Example:

"base64"

options.screenshot.presignedExpiresIn

string

default:30m

Presigned url expires in. Only takes effect when outputFormat is storageKey.

Supported time units: ms (milliseconds), s (seconds), m (minutes), h (hours) Example formats: "500ms", "30s", "5m", "1h" Default: 30m

Example:

"30m"

options.screenshot.delay

string

default:500ms

Delay after performing the action, before taking the final screenshot.

Execution flow:

Take screenshot before action
Perform the action
Wait for screenshotDelay (this parameter)
Take screenshot after action

Example: '500ms' means wait 500ms after the action before capturing the final screenshot.

Supported time units: ms (milliseconds), s (seconds), m (minutes), h (hours) Example formats: "500ms", "30s", "5m", "1h" Default: 500ms Maximum allowed: 30s

Example:

"500ms"

options.screenshot.phases

enum<string>[]

Specify which screenshot phases to capture.

Available options:

before: Screenshot before the action
after: Screenshot after the action
trace: Screenshot with operation trace

Default captures all three phases. Can specify one or multiple in an array. If empty array is provided, no screenshots will be taken.

Available options:

before,

after,

trace

Example:

["before", "after"]

Example:

{
  "outputFormat": "base64",
  "presignedExpiresIn": "30m",
  "delay": "500ms",
  "phases": ["before", "after"]
}

options.model

enum<string>

default:gelato

Model to use for natural-language target resolution. Defaults to 'uitars'.

Available options:

gpt-5,

gpt-4o,

gelato,

ui-tars,

openai-computer-use

Example:

{
  "screenshot": {
    "outputFormat": "base64",
    "presignedExpiresIn": "30m",
    "delay": "500ms",
    "phases": ["before", "after"]
  }
}

Response

200 - application/json

Result of an UI action execution with optional screenshots

message

string

required

message

Example:

"Action executed successfully"

actionId

string

required

Unique identifier for each action. Use this ID to locate the action and report issues.

Example:

"c9bdc193-b54b-4ddb-a035-5ac0c598d32d"

screenshot

Action Result Screenshot · object

Optional screenshot data. Only present when screenshots are requested via options.screenshot.phases or deprecated fields

Show child attributes

screenshot.trace

Action Result Screenshot Operation Trace · object

URI of the screenshot before the action with operation trace

Show child attributes

screenshot.trace.uri

string

required

URI of the screenshot with operation trace

Example:

"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."

Example:

{
  "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
}

screenshot.before

Action Result Screenshot Before · object

URI of the screenshot before the action

Show child attributes

screenshot.before.uri

string

required

URI of the screenshot before the action

Example:

"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."

screenshot.before.presignedUrl

string

Presigned url of the screenshot before the action

Example:

"https://example.com/xxxxx/xxxxx/xxxxx"

Example:

{
  "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
}

screenshot.after

Action Result Screenshot After · object

URI of the screenshot after the action

Show child attributes

screenshot.after.uri

string

required

URI of the screenshot after the action

Example:

"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."

screenshot.after.presignedUrl

string

Presigned url of the screenshot before the action

Example:

"https://example.com/xxxxx/xxxxx/xxxxx"

Example:

{
  "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
}

Example:

{
  "trace": {
    "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
  },
  "before": {
    "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
  },
  "after": {
    "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
  }
}

Long press Move to position

⌘I

Model

Box

UI Action

Run Code

Command

File System

Browser

Android

Media

Authorizations

Path Parameters

Body

Response