Swipe

JavaScript

import GboxSDK from "gbox-sdk";

const gboxSDK = new GboxSDK({
  apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});

async function main() {
  const box = await gboxSDK.create({ type: "android" });

  // Swipe up
  await box.action.swipe({
    direction: "up",
    duration: "200ms"
  });

  // Swipe with enum distance
  await box.action.swipe({
    direction: "up",
    distance: "medium",
    duration: "500ms"
  });

  // Swipe with custom pixel distance
  await box.action.swipe({
    direction: "up",
    distance: 300,
    duration: "500ms"
  });

  // Swipe with natural language location
  await box.action.swipe({
    direction: "up",
    distance: 300,
    duration: "500ms",
    location: "screen bottom"
  });

  // Swipe with custom path
  await box.action.swipe({
    start: { x: 100, y: 150 },
    end: { x: 400, y: 300 },
    duration: "200ms"
  });

  // Swipe with natural language location
  await box.action.swipe({
    start: "screen bottom",
    end: "screen top",
    duration: "200ms"
  });
}

main();

{
  "message": "Action executed successfully",
  "actionId": "c9bdc193-b54b-4ddb-a035-5ac0c598d32d",
  "actual": {
    "start": {
      "x": 540,
      "y": 1000
    },
    "end": {
      "x": 540,
      "y": 500
    },
    "duration": "500ms"
  },
  "screenshot": {
    "trace": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "before": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "after": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    }
  }
}

POST

boxes

{boxId}

actions

swipe

JavaScript

import GboxSDK from "gbox-sdk";

const gboxSDK = new GboxSDK({
  apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});

async function main() {
  const box = await gboxSDK.create({ type: "android" });

  // Swipe up
  await box.action.swipe({
    direction: "up",
    duration: "200ms"
  });

  // Swipe with enum distance
  await box.action.swipe({
    direction: "up",
    distance: "medium",
    duration: "500ms"
  });

  // Swipe with custom pixel distance
  await box.action.swipe({
    direction: "up",
    distance: 300,
    duration: "500ms"
  });

  // Swipe with natural language location
  await box.action.swipe({
    direction: "up",
    distance: 300,
    duration: "500ms",
    location: "screen bottom"
  });

  // Swipe with custom path
  await box.action.swipe({
    start: { x: 100, y: 150 },
    end: { x: 400, y: 300 },
    duration: "200ms"
  });

  // Swipe with natural language location
  await box.action.swipe({
    start: "screen bottom",
    end: "screen top",
    duration: "200ms"
  });
}

main();

{
  "message": "Action executed successfully",
  "actionId": "c9bdc193-b54b-4ddb-a035-5ac0c598d32d",
  "actual": {
    "start": {
      "x": 540,
      "y": 1000
    },
    "end": {
      "x": 540,
      "y": 500
    },
    "duration": "500ms"
  },
  "screenshot": {
    "trace": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "before": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    },
    "after": {
      "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
    }
  }
}

Authorizations

Authorization

string

header

required

Enter your API Key in the format: Bearer . Get it from https://gbox.ai

Path Parameters

boxId

string

required

Box ID

Example:

"c9bdc193-b54b-4ddb-a035-5ac0c598d32d"

Body

application/json

Swipe Simple
Swipe Advanced

Simple swipe action configuration. The gesture will be performed from the center of the screen towards the specified direction.

direction

enum<string>

required

Direction to swipe. The gesture will be performed from the center of the screen towards this direction.

Available options:

up,

down,

left,

right,

upLeft,

upRight,

downLeft,

downRight

Example:

"up"

duration

string

default:500ms

Duration of the swipe

Supported time units: ms (milliseconds), s (seconds), m (minutes), h (hours) Example formats: "500ms", "30s", "5m", "1h" Default: 500ms

Example:

"500ms"

distance

Distance of the swipe. Can be either a number (in pixels) or a predefined enum value (tiny, short, medium, long). If not provided, the swipe will be performed from the center of the screen to the screen edge

Example:

300

location

string

Natural language description of the location where the swipe should originate. If not provided, the swipe will be performed from the center of the screen.

Example:

"Chrome App"

options

Action Common Options · object

Action options. When options.screenshot is provided, ALL deprecated screenshot fields (outputFormat, presignedExpiresIn, screenshotDelay, screenshotRange, includeScreenshot) will be completely ignored.

Show child attributes

options.screenshot

Screenshot options. Can be a boolean to enable/disable screenshots, or an object to configure screenshot options.

Show child attributes

options.screenshot.outputFormat

enum<string>

default:base64

Type of the URI. default is base64.

Available options:

base64,

storageKey

Example:

"base64"

options.screenshot.presignedExpiresIn

string

default:30m

Presigned url expires in. Only takes effect when outputFormat is storageKey.

Supported time units: ms (milliseconds), s (seconds), m (minutes), h (hours) Example formats: "500ms", "30s", "5m", "1h" Default: 30m

Example:

"30m"

options.screenshot.delay

string

default:500ms

Delay after performing the action, before taking the final screenshot.

Execution flow:

Take screenshot before action
Perform the action
Wait for screenshotDelay (this parameter)
Take screenshot after action

Example: '500ms' means wait 500ms after the action before capturing the final screenshot.

Supported time units: ms (milliseconds), s (seconds), m (minutes), h (hours) Example formats: "500ms", "30s", "5m", "1h" Default: 500ms Maximum allowed: 30s

Example:

"500ms"

options.screenshot.phases

enum<string>[]

Specify which screenshot phases to capture.

Available options:

before: Screenshot before the action
after: Screenshot after the action
trace: Screenshot with operation trace

Default captures all three phases. Can specify one or multiple in an array. If empty array is provided, no screenshots will be taken.

Available options:

before,

after,

trace

Example:

["before", "after"]

Example:

{
  "outputFormat": "base64",
  "presignedExpiresIn": "30m",
  "delay": "500ms",
  "phases": ["before", "after"]
}

options.model

enum<string>

default:gelato

Model to use for natural-language target resolution. Defaults to 'uitars'.

Available options:

gpt-5,

gpt-4o,

gelato,

ui-tars,

openai-computer-use

Example:

{
  "screenshot": {
    "outputFormat": "base64",
    "presignedExpiresIn": "30m",
    "delay": "500ms",
    "phases": ["before", "after"]
  }
}

Response

200 - application/json

Swipe action result with actual parameters used

Result of swipe action execution with actual parameters used

message

string

required

message

Example:

"Action executed successfully"

actionId

string

required

Unique identifier for each action. Use this ID to locate the action and report issues.

Example:

"c9bdc193-b54b-4ddb-a035-5ac0c598d32d"

actual

Swipe Action Actual Parameters · object

required

Actual parameters used when executing the swipe action

Show child attributes

actual.start

Swipe Path · object

required

Start point of the swipe

Show child attributes

actual.start.x

number

required

Start/end x coordinate of the swipe path

Example:

100

actual.start.y

number

required

Start/end y coordinate of the swipe path

Example:

150

Example:

{ "x": 540, "y": 1000 }

actual.end

Swipe Path · object

required

End point of the swipe

Show child attributes

actual.end.x

number

required

Start/end x coordinate of the swipe path

Example:

100

actual.end.y

number

required

Start/end y coordinate of the swipe path

Example:

150

Example:

{ "x": 540, "y": 500 }

actual.duration

string

default:500ms

required

Duration of the swipe

Supported time units: ms (milliseconds), s (seconds), m (minutes), h (hours) Example formats: "500ms", "30s", "5m", "1h" Default: 500ms

Example:

"500ms"

Example:

{
  "start": { "x": 540, "y": 1000 },
  "end": { "x": 540, "y": 500 },
  "duration": "500ms"
}

screenshot

Action Result Screenshot · object

Optional screenshot data. Only present when screenshots are requested via options.screenshot.phases or deprecated fields

Show child attributes

screenshot.trace

Action Result Screenshot Operation Trace · object

URI of the screenshot before the action with operation trace

Show child attributes

screenshot.trace.uri

string

required

URI of the screenshot with operation trace

Example:

"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."

Example:

{
  "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
}

screenshot.before

Action Result Screenshot Before · object

URI of the screenshot before the action

Show child attributes

screenshot.before.uri

string

required

URI of the screenshot before the action

Example:

"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."

screenshot.before.presignedUrl

string

Presigned url of the screenshot before the action

Example:

"https://example.com/xxxxx/xxxxx/xxxxx"

Example:

{
  "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
}

screenshot.after

Action Result Screenshot After · object

URI of the screenshot after the action

Show child attributes

screenshot.after.uri

string

required

URI of the screenshot after the action

Example:

"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."

screenshot.after.presignedUrl

string

Presigned url of the screenshot before the action

Example:

"https://example.com/xxxxx/xxxxx/xxxxx"

Example:

{
  "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
}

Example:

{
  "trace": {
    "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
  },
  "before": {
    "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
  },
  "after": {
    "uri": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
  }
}

Scroll Press key

⌘I

Model

Box

UI Action

Run Code

Command

File System

Browser

Android

Media

Authorizations

Path Parameters

Body

Response