# Basic
Source: https://docs.gbox.ai/android/basic
[GBOX](https://gbox.ai) provides cloud-based Android automation testing without the need to purchase, configure, or maintain any Android devices. GBOX supports both `virtual machines` and `physical devices`, letting you focus on testing:
* **No Device Management** - Eliminate hardware procurement, system configuration, and device maintenance hassles
* **Ready to Use** - Get clean Android testing environments in seconds
* **Flexible Options** - Choose between virtual machines or physical devices based on your testing needs
* **Auto Scaling** - Run multiple tests in parallel without hardware limitations
1. Go to [GBOX AI](http://gbox.ai)
2. Copy your API Key
3. Paste your GBOX API Key into your `env` file
```bash .env theme={null}
GBOX_API_KEY=gbox-******
```
Install gbox-sdk and other dependencies by running the following command in your terminal.
```bash Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npm install gbox-sdk dotenv typescript tsx @types/node
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
# Set up a virtual environment
python3 -m venv venv
source venv/bin/activate
pip install gbox-sdk python-dotenv
```
Create a basic Android box by running the following code.
```typescript Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
import GboxSDK from "gbox-sdk";
import * as dotenv from "dotenv";
dotenv.config();
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
console.log(`Android box created: ${box.data.id}`);
// take a screenshot of the box
await box.action.screenshot({
path: "screenshot.png",
});
// terminate the box
await box.terminate();
console.log("Box terminated");
}
main();
```
```python Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
import os
from dotenv import load_dotenv
from gbox_sdk import GboxSDK
# Load environment variables from .env file
load_dotenv()
def main():
api_key = os.getenv("GBOX_API_KEY")
gbox = GboxSDK(api_key=api_key)
box = gbox.create(type="android")
print(f"Android box created: {box.data.id}")
# take a screenshot of the box
box.action.screenshot(path="screenshot.png")
# terminate the box
box.terminate()
print("Box terminated")
if __name__ == "__main__":
main()
```
Run the box by running the following command in your terminal.
```bash Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npx tsx index.ts
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
python main.py
```
Explore the complete Android API reference to discover all available
operations and features.
# Install app
Source: https://docs.gbox.ai/android/install-app
[GBOX](https://gbox.ai) lets you easily install Android applications on cloud-based devices without needing physical hardware. You can install apps from APK files, URLs, or local files, making app testing and development seamless:
* **Flexible Installation** - Install apps from URLs, local APK files, or app stores
* **Instant Access** - Install and launch apps in seconds on clean Android environments
* **No Setup Required** - Skip device configuration and app sideloading complexities
* **Live Testing** - View and interact with installed apps through live browser sessions
1. Go to [GBOX AI](http://gbox.ai)
2. Copy your API Key
3. Paste your GBOX API Key into your `env` file
```bash .env theme={null}
GBOX_API_KEY=gbox-******
```
Install gbox-sdk and other dependencies by running the following command in your terminal.
```bash Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npm install gbox-sdk dotenv typescript tsx @types/node
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
# Set up a virtual environment
python3 -m venv venv
source venv/bin/activate
pip install gbox-sdk python-dotenv
```
Create a basic Android box by running the following code.
```typescript Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" highlight={16-19} theme={null}
import GboxSDK from "gbox-sdk";
import * as dotenv from "dotenv";
dotenv.config();
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
console.log(`Android box created: ${box.data.id}`);
// install markor (a open source markdown editor)
const app = await box.app.install({
apk: "https://github.com/gsantner/markor/releases/download/v2.14.1/net.gsantner.markor-v158-2.14.1-flavorDefault-release.apk"
})
// open the app
await app.open()
// you can also install an apk from your local machine
// const app = await box.app.install({
// apk: "/path/to/your/app.apk"
// })
const liveView = await box.liveView()
console.log(`Open the following URL in your browser to see the live view: ${liveView.url}`);
}
main();
```
```python Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" highlight={19} theme={null}
import os
from dotenv import load_dotenv
from gbox_sdk import GboxSDK
# Load environment variables from .env file
load_dotenv()
def main():
api_key = os.getenv("GBOX_API_KEY")
gbox = GboxSDK(api_key=api_key)
box = gbox.create(type="android")
print(f"Android box created: {box.data.id}")
# install markor (a open source markdown editor)
app = box.app.install(
apk="https://github.com/gsantner/markor/releases/download/v2.14.1/net.gsantner.markor-v158-2.14.1-flavorDefault-release.apk"
)
# open the app
app.open()
# you can also install an apk from your local machine
# app = box.app.install({
# "apk": "/path/to/your/app.apk"
# })
live_view = box.live_view()
print(f"Open the following URL in your browser to see the live view: {live_view.url}")
if __name__ == "__main__":
main()
```
Run the box by running the following command in your terminal.
```bash Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npx tsx index.ts
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
python main.py
```
Explore the complete Android API reference to discover all available
operations and features.
# Live View
Source: https://docs.gbox.ai/android/live-view
Live View provides real-time remote access to Android devices, allowing you to directly control and interact with cloud-based Android devices through your browser.
## Get Started
1. Go to [GBOX AI](http://gbox.ai)
2. Copy your API Key
3. Paste your GBOX API Key into your `env` file
```bash .env theme={null}
GBOX_API_KEY=gbox-******
```
Install gbox-sdk and other required dependencies by running the following command in your terminal.
```bash Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npm install gbox-sdk dotenv typescript tsx @types/node open
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
# Set up a virtual environment
python3 -m venv venv
source venv/bin/activate
pip install gbox-sdk python-dotenv
```
Create a Live View box by creating the following code file.
```typescript Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" highlight={42} theme={null}
import GboxSDK, { AndroidBoxOperator } from "gbox-sdk";
import * as dotenv from "dotenv";
import open from "open";
dotenv.config();
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
let currentBox: AndroidBoxOperator | null = null;
async function gracefulShutdown() {
if (currentBox) {
console.log("Shutting down program, deleting Android box...");
try {
await currentBox.terminate();
console.log("Box successfully terminated");
} catch (error) {
console.log(`Error terminating box: ${error}`);
}
}
process.exit(0);
}
// Listen for SIGINT signal (Ctrl+C)
process.on("SIGINT", gracefulShutdown);
process.on("SIGTERM", gracefulShutdown);
async function main() {
try {
console.log("\nAndroid Box Manager Started\n");
console.log("Creating Android box...");
const box = await gboxSDK.create({ type: "android" });
currentBox = box;
console.log("Android box created successfully!");
console.log(`Box ID: ${box.data.id}`);
console.log("Getting live view...");
const liveView = await box.liveView();
console.log("Live view is ready!");
console.log(`View URL: ${liveView.url}`);
console.log("Opening browser...");
console.log(
`You can try operate the android box in the browser, and then press Ctrl+C to stop and terminate box\n`
);
await open(liveView.url, {
wait: true,
});
// Keep the program running, waiting for SIGINT signal
await new Promise(() => {});
} catch (error) {
console.log(`An error occurred: ${error}`);
if (currentBox) {
await gracefulShutdown();
}
process.exit(1);
}
}
main();
```
```python Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" highlight={44} theme={null}
import os
import signal
import sys
import webbrowser
from dotenv import load_dotenv
from gbox_sdk import GboxSDK
# Load environment variables from .env file
load_dotenv()
current_box = None
def graceful_shutdown(signum, frame):
global current_box
if current_box:
print("Shutting down program, deleting Android box...")
try:
current_box.terminate()
print("Box successfully terminated")
except Exception as error:
print(f"Error terminating box: {error}")
sys.exit(0)
# Listen for SIGINT signal (Ctrl+C)
signal.signal(signal.SIGINT, graceful_shutdown)
signal.signal(signal.SIGTERM, graceful_shutdown)
def main():
global current_box
try:
print("\nAndroid Box Manager Started\n")
api_key = os.getenv("GBOX_API_KEY")
gbox = GboxSDK(api_key=api_key)
print("Creating Android box...")
box = gbox.create(type="android")
current_box = box
print("Android box created successfully!")
print(f"Box ID: {box.data.id}")
print("Getting live view...")
live_view = box.live_view()
print("Live view is ready!")
print(f"View URL: {live_view.url}")
print("Opening browser...")
print(
"You can try operate the android box in the browser, and then press Ctrl+C to stop and terminate box\n"
)
webbrowser.open(live_view.url)
# Keep the program running, waiting for SIGINT signal
while True:
pass
except Exception as error:
print(f"An error occurred: {error}")
if current_box:
graceful_shutdown(None, None)
sys.exit(1)
if __name__ == "__main__":
main()
```
Run the Live View box by running the following command in your terminal.
```bash Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npx tsx index.ts
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
python main.py
```
## Embed Live View in your website
You can embed Live View in your website by using the following code.
```html theme={null}
```
# Physical Device
Source: https://docs.gbox.ai/android/real-device
This is a **beta feature** currently in testing phase. Please note that physical
device availability may be limited, and some functionalities might experience
occasional instability. We appreciate your feedback to help us improve this
feature.
GBOX provides cloud physical devices through [https://gbox.ai](https://gbox.ai). You can instantly start cloud physical Android devices and operate through GBOX SDK/MCP/CLI. You can also register your local Android devices to gbox.ai and operate in the same way as cloud ones. [Register Local Android](https://docs.gbox.ai/cli/register-local-device)
*Note: Virtual devices created through Android Studio locally are treated as local physical devices.*
## Key Features
* **Real Hardware**: Access to actual physical Android devices, not emulators
* **Remote Control**: Full touch, swipe, and gesture support
* **Live View**: Real-time screen streaming with minimal latency
## Usage
### How to create a box using cloud physical device
```typescript Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" highlight={12-14} theme={null}
import GboxSDK from "gbox-sdk";
import * as dotenv from "dotenv";
dotenv.config();
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({
type: "android",
config: {
deviceType: "physical",
},
});
console.log(`Android box created: ${box.data.id}`);
const liveView = await box.liveView();
console.log(`Live view URL: ${liveView.url}`);
}
main();
```
```python Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" highlight={13-16} theme={null}
import os
from dotenv import load_dotenv
from gbox_sdk import GboxSDK
# Load environment variables from .env file
load_dotenv()
def main():
api_key = os.getenv("GBOX_API_KEY")
gbox = GboxSDK(api_key=api_key)
box = gbox.create(
type="android",
config={
"deviceType": "physical",
}
)
print(f"Android box created: {box.data.id}")
live_view = box.live_view()
print(f"Live view URL: {live_view.url}")
if __name__ == "__main__":
main()
```
### Create a box using local Android device
If you want to connect to a **local real device**,here is the tutorial:
1. Refer to the [Register Local Device](/cli/register-local-device) to register your real device to the cloud and get the corresponding device ID.
2. Enter the device ID in the SDK parameter below and execute:
```typescript Typescript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" highlight={12-18} theme={null}
import GboxSDK from "gbox-sdk";
import * as dotenv from "dotenv";
dotenv.config();
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({
type: "android",
config: {
deviceType: "physical",
labels: {
"gbox.ai/device-id": "YOUR_DEVICE_ID", // Replace with your device ID
},
},
});
console.log(`Android box created: ${box.data.id}`);
const liveView = await box.liveView();
console.log(`Live view URL: ${liveView.url}`);
}
main();
```
```python Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" highlight={13-19} theme={null}
import os
from dotenv import load_dotenv
from gbox_sdk import GboxSDK
# Load environment variables from .env file
load_dotenv()
def main():
api_key = os.getenv("GBOX_API_KEY")
gbox = GboxSDK(api_key=api_key)
box = gbox.create(
type="android",
config={
"deviceType": "physical",
"labels":{
"gbox.ai/device-id": "YOUR_DEVICE_ID" # Replace with your device ID
}
}
)
print(f"Android box created: {box.data.id}")
live_view = box.live_view()
print(f"Live view URL: {live_view.url}")
if __name__ == "__main__":
main()
```
# API Key
Source: https://docs.gbox.ai/api-key
## How to get your API key
1. Go to [GBOX AI](https://gbox.ai)
2. Login to your account
3. Click on "API Keys"
4. Create / Copy the API key
5. Use the API key in your requests
# Close all apps
Source: https://docs.gbox.ai/api-reference/android/close-all-apps
post /boxes/{boxId}/android/packages/close-all
Terminates all running Android applications inside the box
# Close app
Source: https://docs.gbox.ai/api-reference/android/close-app
post /boxes/{boxId}/android/packages/{packageName}/close
Forces the specified Android application to close inside the box
# Appium Connection
Source: https://docs.gbox.ai/api-reference/android/generate-appium-connection-url
post /boxes/{boxId}/android/connect-url/appium
Generate a pre-signed proxy URL for Appium server of a running Android box.
# Get app
Source: https://docs.gbox.ai/api-reference/android/get-app
get /boxes/{boxId}/android/apps/{packageName}
Get installed app info by package name
# Get pkg
Source: https://docs.gbox.ai/api-reference/android/get-pkg
get /boxes/{boxId}/android/packages/{packageName}
# Get pkg activities
Source: https://docs.gbox.ai/api-reference/android/get-pkg-activities
get /boxes/{boxId}/android/packages/{packageName}/activities
Retrieves the list of activities defined in a specific Android package
# Install app
Source: https://docs.gbox.ai/api-reference/android/install-app
post /boxes/{boxId}/android/packages
Install an Android app on the box
# List apps
Source: https://docs.gbox.ai/api-reference/android/list-apps
get /boxes/{boxId}/android/apps
List all installed apps on the launcher
# List pkg
Source: https://docs.gbox.ai/api-reference/android/list-pkg
get /boxes/{boxId}/android/packages
Retrieves detailed information for all installed pkgs. This endpoint provides comprehensive pkg details.
# List pkg simple
Source: https://docs.gbox.ai/api-reference/android/list-pkg-simple
get /boxes/{boxId}/android/packages/simple
A faster endpoint to quickly retrieve basic pkg information. This API provides better performance for scenarios where you need to get essential pkg details quickly.
# Open app
Source: https://docs.gbox.ai/api-reference/android/open-app
post /boxes/{boxId}/android/packages/{packageName}/open
Launches a specific Android application within the box
# Restart app
Source: https://docs.gbox.ai/api-reference/android/restart-app
post /boxes/{boxId}/android/packages/{packageName}/restart
Closes and immediately reopens the specified Android application inside the box
# Uninstall app
Source: https://docs.gbox.ai/api-reference/android/uninstall-app
delete /boxes/{boxId}/android/packages/{packageName}
Uninstalls an Android app from the box
# Clear proxy
Source: https://docs.gbox.ai/api-reference/box/clear-proxy
delete /boxes/{boxId}/proxy
Clears the HTTP proxy for the box
# Create android box
Source: https://docs.gbox.ai/api-reference/box/create-android-box
post /boxes/android
Provisions a new Android box that you can operate through the GBOX SDK. Use this endpoint when you want to create a fresh Android environment for testing, automation, or agent execution.
# Create linux box
Source: https://docs.gbox.ai/api-reference/box/create-linux-box
post /boxes/linux
Provisions a new Linux box that you can operate through the GBOX SDK. Use this endpoint when you want to create a fresh Linux environment for testing, automation, or agent execution.
# Create presigned url
Source: https://docs.gbox.ai/api-reference/box/create-presigned-url
post /boxes/{boxId}/storage/presigned-url
Create a presigned url for a storage key. This endpoint provides a presigned url for a storage key, which can be used to download the file from the storage.
# Get box
Source: https://docs.gbox.ai/api-reference/box/get-box
get /boxes/{boxId}
This endpoint retrieves information about a box
# Get box display
Source: https://docs.gbox.ai/api-reference/box/get-box-display
get /boxes/{boxId}/display
Retrieve the current display properties for a running box. This endpoint provides details about the box's screen resolution, orientation, and other visual properties.
# Get proxy
Source: https://docs.gbox.ai/api-reference/box/get-proxy
get /boxes/{boxId}/proxy
Retrieves the HTTP proxy settings for a specific box. Use this endpoint to route traffic through the box's network.
# List box
Source: https://docs.gbox.ai/api-reference/box/list-box
get /boxes
Returns a paginated list of box instances. Use this endpoint to monitor environments, filter by status or type, or retrieve boxes by labels or device type.
# Live view url
Source: https://docs.gbox.ai/api-reference/box/live-view-url
post /boxes/{boxId}/live-view-url
This endpoint allows you to generate a pre-signed URL for accessing the live view of a running box. The URL is valid for a limited time and can be used to view the box's live stream.
# Set proxy
Source: https://docs.gbox.ai/api-reference/box/set-proxy
post /boxes/{boxId}/proxy
Configures the HTTP proxy settings for a specific box. Use this endpoint when you need the box's outbound network traffic to pass through a proxy server.
# Set screen resolution
Source: https://docs.gbox.ai/api-reference/box/set-screen-resolution
post /boxes/{boxId}/resolution
Set the screen resolution
# Terminate box
Source: https://docs.gbox.ai/api-reference/box/terminate-box
post /boxes/{boxId}/terminate
Terminate a running box. This action will stop the box and release its resources.
# Web terminal url
Source: https://docs.gbox.ai/api-reference/box/web-terminal-url
post /boxes/{boxId}/web-terminal-url
This endpoint allows you to generate a pre-signed URL for accessing the web terminal of a running box. The URL is valid for a limited time and can be used to access the box's terminal interface.
# Close a browser tab
Source: https://docs.gbox.ai/api-reference/browser/close-a-browser-tab
delete /boxes/{boxId}/browser/tabs/{tabId}
Close a specific browser tab identified by its id. This endpoint will permanently close the tab and free up the associated resources. After closing a tab, the ids of subsequent tabs may change.
# Close browser
Source: https://docs.gbox.ai/api-reference/browser/close-browser
delete /boxes/{boxId}/browser/close
Close the browser in the specified box
# Generate CDP url
Source: https://docs.gbox.ai/api-reference/browser/generate-cdp-url
post /boxes/{boxId}/browser/connect-url/cdp
This endpoint allows you to generate a pre-signed URL for accessing the Chrome DevTools Protocol (CDP) of a running box. The URL is valid for a limited time and can be used to interact with the box's browser environment
# List all browser tabs
Source: https://docs.gbox.ai/api-reference/browser/list-all-browser-tabs
get /boxes/{boxId}/browser/tabs
Retrieve a comprehensive list of all currently open browser tabs in the specified box. This endpoint returns detailed information about each tab including its id, title, current URL, and favicon. The returned id can be used for subsequent operations like navigation, closing, or updating tabs. This is essential for managing multiple browser sessions and understanding the current state of the browser environment.
# Open a new browser tab
Source: https://docs.gbox.ai/api-reference/browser/open-a-new-browser-tab
post /boxes/{boxId}/browser/tabs
Create and open a new browser tab with the specified URL. This endpoint will navigate to the provided URL and return the new tab's information including its assigned id, loaded title, final URL (after any redirects), and favicon. The returned tab id can be used for future operations on this specific tab. The browser will attempt to load the page and will wait for the DOM content to be loaded before returning the response. If the URL is invalid or unreachable, an error will be returned.
# Open browser
Source: https://docs.gbox.ai/api-reference/browser/open-browser
post /boxes/{boxId}/browser/open
Open the browser in the specified box. If the browser is already open, repeated calls will not open a new browser.
# Switch to browser tab
Source: https://docs.gbox.ai/api-reference/browser/switch-to-browser-tab
post /boxes/{boxId}/browser/tabs/{tabId}/switch
Switch to a specific browser tab by bringing it to the foreground (making it the active/frontmost tab). This operation sets the specified tab as the currently active tab without changing its URL or content. The tab will receive focus and become visible to the user. This is useful for managing multiple browser sessions and controlling which tab is currently in focus.
# Update browser tab URL
Source: https://docs.gbox.ai/api-reference/browser/update-browser-tab-url
put /boxes/{boxId}/browser/tabs/{tabId}
Navigate an existing browser tab to a new URL. This endpoint updates the specified tab by navigating it to the provided URL and returns the updated tab information. The browser will wait for the DOM content to be loaded before returning the response. If the navigation fails due to an invalid URL or network issues, an error will be returned. The updated tab information will include the new title, final URL (after any redirects), and favicon from the new page.
# Exec command
Source: https://docs.gbox.ai/api-reference/command/exec-command
post /boxes/{boxId}/commands
Execute a command on a running box. This endpoint allows you to send commands to the box and receive the output
# Check if file/dir exists
Source: https://docs.gbox.ai/api-reference/file-system/check-if-filedir-exists
post /boxes/{boxId}/fs/exists
# Delete box file/dir
Source: https://docs.gbox.ai/api-reference/file-system/delete-box-filedir
delete /boxes/{boxId}/fs
Deletes a file or a directory. If target path doesn't exist, the delete will fail.
# Get file/dir
Source: https://docs.gbox.ai/api-reference/file-system/get-filedir
get /boxes/{boxId}/fs/info
Retrieves metadata for a specific file or directory inside a box
# List box files
Source: https://docs.gbox.ai/api-reference/file-system/list-box-files
get /boxes/{boxId}/fs/list
Lists files and directories in a box. You can specify the directory path and depth, and optionally a working directory. The response includes metadata such as type, size, permissions, and last modified time.
# Read box file
Source: https://docs.gbox.ai/api-reference/file-system/read-box-file
get /boxes/{boxId}/fs/read
Reads the contents of a file inside the box and returns it as a string. Supports absolute or relative paths, with `workingDir` as the base for relative paths.
# Rename box/dir
Source: https://docs.gbox.ai/api-reference/file-system/rename-boxdir
post /boxes/{boxId}/fs/rename
Renames a file or a directory. If the target newPath already exists, the rename will fail.
# Write box file
Source: https://docs.gbox.ai/api-reference/file-system/write-box-file
post /boxes/{boxId}/fs/write
Creates or overwrites a file. Creates necessary directories in the path if they don't exist. If the target path already exists, the write will fail.
# Create album
Source: https://docs.gbox.ai/api-reference/media/create-album
post /boxes/{boxId}/media/albums
Create a new album with media files
# Delete album
Source: https://docs.gbox.ai/api-reference/media/delete-album
delete /boxes/{boxId}/media/albums/{albumName}
Delete an album and all its media files
# Delete media from album
Source: https://docs.gbox.ai/api-reference/media/delete-media-from-album
delete /boxes/{boxId}/media/albums/{albumName}/media/{mediaName}
Delete a specific media file from an album
# Download media
Source: https://docs.gbox.ai/api-reference/media/download-media
get /boxes/{boxId}/media/albums/{albumName}/media/{mediaName}/download
Download a specific media file from an album
# Get album detail
Source: https://docs.gbox.ai/api-reference/media/get-album-detail
get /boxes/{boxId}/media/albums/{albumName}
Get detailed information about a specific album including its media files
# Get media detail
Source: https://docs.gbox.ai/api-reference/media/get-media-detail
get /boxes/{boxId}/media/albums/{albumName}/media/{mediaName}
Get detailed information about a specific media file
# Get media support extensions
Source: https://docs.gbox.ai/api-reference/media/get-media-support-extensions
get /boxes/{boxId}/media/support
Get supported media file extensions for photos and videos
# List albums
Source: https://docs.gbox.ai/api-reference/media/list-albums
get /boxes/{boxId}/media/albums
Get a list of albums in the box
# List media in album
Source: https://docs.gbox.ai/api-reference/media/list-media-in-album
get /boxes/{boxId}/media/albums/{albumName}/media
Get a list of media files in a specific album
# Update album
Source: https://docs.gbox.ai/api-reference/media/update-album
patch /boxes/{boxId}/media/albums/{albumName}
Add media files to an existing album
# Generate Coordinates
Source: https://docs.gbox.ai/api-reference/model/generate-coordinates-for-a-model
post /model
Generate precise UI element coordinates using the **gbox-handy-1** model.
This specialized model analyzes screenshots and instructions to identify exact coordinates for UI operations.
## Supported Actions
The model supports three core actions that cover nearly all coordinate-based UI interactions:
* **Click**: Identify precise tap/click coordinates for buttons, links, and interactive elements
* **Drag**: Calculate start and end coordinates for drag operations (e.g., swipe, scroll bars)
* **Scroll**: Determine optimal scroll coordinates and directions
# Run code on the box
Source: https://docs.gbox.ai/api-reference/run-code/run-code-on-the-box
post /boxes/{boxId}/run-code
Executes code inside the specified box. Supports multiple languages (bash, Python, TypeScript) and allows you to configure environment variables, arguments, working directory, and timeouts.
# Click
Source: https://docs.gbox.ai/api-reference/ui-action/click
post /boxes/{boxId}/actions/click
Simulates a click action on the box.
# Detect UI elements
Source: https://docs.gbox.ai/api-reference/ui-action/detect-ui-elements
post /boxes/{boxId}/actions/elements/detect
Detect and identify interactive UI elements in the current screen. Note: This feature currently only supports element detection within a running browser. If the browser is not running, the Elements array will be empty.
# Disable rewind recording
Source: https://docs.gbox.ai/api-reference/ui-action/disable-rewind-recording
delete /boxes/{boxId}/actions/recording/rewind
Disable the device's background screen rewind recording.
# Drag
Source: https://docs.gbox.ai/api-reference/ui-action/drag
post /boxes/{boxId}/actions/drag
Simulates a drag gesture, moving from a start point to an end point over a set duration. Supports simple start/end coordinates, multi-point drag paths, and natural-language targets.
# Enable rewind recording
Source: https://docs.gbox.ai/api-reference/ui-action/enable-rewind-recording
post /boxes/{boxId}/actions/recording/rewind
Enable the device's background screen rewind recording.
# Extract rewind recording
Source: https://docs.gbox.ai/api-reference/ui-action/extract-rewind-recording
post /boxes/{boxId}/actions/recording/rewind/extract
Rewind and capture the device's background screen recording from a specified time period.
# Get clipboard
Source: https://docs.gbox.ai/api-reference/ui-action/get-clipboard
get /boxes/{boxId}/actions/clipboard
Get the clipboard content
# Get screen layout
Source: https://docs.gbox.ai/api-reference/ui-action/get-screen-layout
get /boxes/{boxId}/actions/screen-layout
Get the current structured screen layout information. This endpoint returns detailed structural information about the UI elements currently displayed on the screen, which can be used for UI automation, element analysis, and accessibility purposes. The format varies by box type: Android boxes return XML format with detailed UI hierarchy information including element bounds, text content, resource IDs, and properties, while other box types may return different structured formats.
# Get settings
Source: https://docs.gbox.ai/api-reference/ui-action/get-settings
get /boxes/{boxId}/actions/settings
Get the action settings for the box
# Long press
Source: https://docs.gbox.ai/api-reference/ui-action/long-press
post /boxes/{boxId}/actions/long-press
Perform a long press action at specified coordinates for a specified duration. Useful for triggering context menus, drag operations, or other long-press interactions.
# Move to position
Source: https://docs.gbox.ai/api-reference/ui-action/move-to-position
post /boxes/{boxId}/actions/move
Moves the focus to a specific coordinate on the box without performing a click or tap. Use this endpoint to position the cursor, hover over elements, or prepare for chained actions such as drag or swipe.
# Press button
Source: https://docs.gbox.ai/api-reference/ui-action/press-button
post /boxes/{boxId}/actions/press-button
Press device buttons like power, volume, home, back, etc.
# Press key
Source: https://docs.gbox.ai/api-reference/ui-action/press-key
post /boxes/{boxId}/actions/press-key
Simulates pressing a specific key by triggering the complete keyboard key event chain (keydown, keypress, keyup). Use this to activate keyboard key event listeners such as shortcuts or form submissions.
# Reset settings
Source: https://docs.gbox.ai/api-reference/ui-action/reset-settings
delete /boxes/{boxId}/actions/settings
Resets the box settings to default
# Rotate screen
Source: https://docs.gbox.ai/api-reference/ui-action/rotate-screen
post /boxes/{boxId}/actions/screen-rotation
Rotates the screen orientation. Note that even after rotating the screen, applications or system layouts may not automatically adapt to the gravity sensor changes, so visual changes may not always occur.
# Scroll
Source: https://docs.gbox.ai/api-reference/ui-action/scroll
post /boxes/{boxId}/actions/scroll
Performs a scroll action. Supports both advanced scroll with coordinates and simple scroll with direction.
# Set clipboard
Source: https://docs.gbox.ai/api-reference/ui-action/set-clipboard
post /boxes/{boxId}/actions/clipboard
Set the clipboard content
# Start recording
Source: https://docs.gbox.ai/api-reference/ui-action/start-recording
post /boxes/{boxId}/actions/recording/start
Start recording the box screen. Only one recording can be active at a time. If a recording is already in progress, starting a new recording will stop the previous one and keep only the latest recording.
# Stop recording
Source: https://docs.gbox.ai/api-reference/ui-action/stop-recording
post /boxes/{boxId}/actions/recording/stop
Stop recording the box screen
# Swipe
Source: https://docs.gbox.ai/api-reference/ui-action/swipe
post /boxes/{boxId}/actions/swipe
Performs a swipe in the specified direction
# Take screenshot
Source: https://docs.gbox.ai/api-reference/ui-action/take-screenshot
post /boxes/{boxId}/actions/screenshot
Captures a screenshot of the current box screen
# Tap
Source: https://docs.gbox.ai/api-reference/ui-action/tap
post /boxes/{boxId}/actions/tap
Tap action for Android devices using ADB input tap command
# Touch
Source: https://docs.gbox.ai/api-reference/ui-action/touch
post /boxes/{boxId}/actions/touch
Performs more advanced touch gestures. Use this endpoint to simulate realistic behaviors.
# Type text
Source: https://docs.gbox.ai/api-reference/ui-action/type-text
post /boxes/{boxId}/actions/type
Directly inputs text content without triggering physical key events (keydown, etc.), ideal for quickly filling large amounts of text when intermediate input events aren't needed.
# Update settings
Source: https://docs.gbox.ai/api-reference/ui-action/update-settings
put /boxes/{boxId}/actions/settings
Update the action settings for the box
# Playwright with GBOX
Source: https://docs.gbox.ai/browser/playwright
[Playwright](https://playwright.dev/) is a browser automation library that allows you to automate browser actions.
**GBOX provides scalable, cloud-based remote environments for Playwright automation.** Instead of running browsers locally, GBOX spins up isolated browser instances in the cloud that you can control remotely through Playwright's API. This enables you to:
* **Scale your automation** - Run multiple Playwright tests in parallel without hardware limitations
* **Consistent environments** - Execute tests in standardized browser environments, eliminating "works on my machine" issues
* **Resource efficiency** - Offload browser processes to the cloud, freeing up local resources
* **Cross-platform testing** - Access different operating systems and browser versions without local setup
* **Remote debugging** - Debug and inspect automated browser sessions through live view capabilities
With GBOX's remote browser environments, your Playwright scripts gain enterprise-grade scalability and reliability while maintaining the same familiar API you're used to.
1. Go to [GBOX AI](http://gbox.ai)
2. Copy your API Key
3. Paste your GBOX API Key into your `env` file
```bash .env theme={null}
GBOX_API_KEY=gbox-******
```
Install gbox-sdk and other dependencies by running the following command in your terminal.
```bash TypeScript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npm install gbox-sdk dotenv typescript tsx playwright @types/node
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
# Set up a virtual environment
python3 -m venv venv
source venv/bin/activate
pip install gbox-sdk python-dotenv playwright
```
Create a new Playwright test by running the following code.
```typescript TypeScript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" highlight={17} theme={null}
import GboxSDK from "gbox-sdk";
import { chromium } from "playwright";
import * as dotenv from "dotenv";
dotenv.config();
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "linux" });
console.log(`Box created: ${box.data.id}`);
// Get browser CDP URL for Chrome DevTools Protocol connection
const cdpUrl = await box.browser.cdpUrl();
console.log(`Browser CDP URL: ${cdpUrl}`);
const browser = await chromium.connectOverCDP(cdpUrl);
// Use the browser as usual
const context = await browser.newContext();
const page = await context.newPage();
await page.goto("https://gbox.ai");
// Take a screenshot of the page
await page.screenshot({ path: "screenshot.png" });
// Perform actions on the page
console.log(await page.title());
// Close the browser
await browser.close();
console.log("Browser closed");
// Terminate the box
await box.terminate();
console.log("Box terminated");
}
main();
```
```python Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" highlight={19} theme={null}
import os
from dotenv import load_dotenv
from gbox_sdk import GboxSDK
from playwright.sync_api import sync_playwright
# Load environment variables from .env file
load_dotenv()
def main():
api_key = os.getenv("GBOX_API_KEY")
gbox = GboxSDK(api_key=api_key)
box = gbox.create(type="linux")
print(f"Box created: {box.data.id}")
# Get browser CDP URL for Chrome DevTools Protocol connection
cdp_url = box.browser.cdp_url()
print(f"Browser CDP URL: {cdp_url}")
with sync_playwright() as p:
browser = p.chromium.connect_over_cdp(cdp_url)
# Use the browser as usual
context = browser.new_context()
page = context.new_page()
page.goto("https://gbox.ai")
# Take a screenshot of the page
page.screenshot(path="screenshot.png")
# Perform actions on the page
print(page.title())
# Close the browser
browser.close()
print("Browser closed")
# Terminate the box
box.terminate()
print("Box terminated")
if __name__ == "__main__":
main()
```
Run the test by running the following command in your terminal.
```bash TypeScript icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
npx tsx index.ts
```
```bash Python icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
python main.py
```
# Changelog
Source: https://docs.gbox.ai/changelog
These features require upgrading to [TypeScript SDK version
43](https://www.npmjs.com/package/gbox-sdk/v/0.43.0) or later.
## š±ļø Click with Modifier Keys Support (Linux Only)
The `click` action now supports modifier keys, enabling keyboard combinations during click operations. This is useful for actions like Shift+Click for multi-select, Control+Click for opening links in new tabs, or other keyboard-mouse combinations.
This feature is currently only available for **Linux boxes**. Android support is not yet available.
**Supported Modifier Keys:**
* `control` - Control key
* `shift` - Shift key
* `alt` - Alt key
### Example
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "linux" });
// Shift+Click for multi-select
await box.click({
x: 100,
y: 100,
modifierKeys: ["shift"]
});
// Control+Click to open link in new tab
await box.click({
x: 200,
y: 200,
modifierKeys: ["control"]
});
// Multiple modifiers: Control+Shift+Click
await box.click({
x: 300,
y: 300,
modifierKeys: ["control", "shift"]
});
```
These features require upgrading to [TypeScript SDK version
42](https://www.npmjs.com/package/gbox-sdk/v/0.42.0) or later.
## š Action Response with Actual Field
All UI actions (Click, Tap, Scroll, Swipe, Drag, Long Press, Touch) now return an `actual` field with detailed execution information, showing the exact coordinates and parameters used during action execution.
### Example
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "linux" });
const result = await box.action.drag({
start: "Chrome App",
end: "Trash",
});
console.log(result);
```
Returns:
```json wrap theme={null}
{
"message": "Action executed successfully",
"actionId": "61649fee-1887-4f74-87b8-e50ee6f0c967",
"actual": {
"start": { "x": 987, "y": 1039 },
"end": { "x": 921, "y": 1038 },
"duration": "500ms"
}
}
```
The `actual` field helps you verify action execution, debug automation, and understand how natural language targets are translated into screen coordinates.
These features require upgrading to [TypeScript SDK version
41](https://www.npmjs.com/package/gbox-sdk/v/0.41.0) or later.
### š¤ Model API for Coordinate Generation
Introducing the **gbox-handy-1** model - a specialized AI model that generates precise UI element coordinates from screenshots using natural language. The Model API analyzes screenshots and returns exact coordinates for click, drag, and other UI actions based on descriptive targets.
[Learn more ā](api-reference/model/generate-coordinates-for-a-model)
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const result = await gboxSDK.model.call({
model: "gbox-handy-1",
screenshot: "https://gru-activate2-public-assets.s3.us-west-2.amazonaws.com/jessica/screenshot-1759332945616-pu0ovj.png",
action: {
type: "click",
target: "the VSCode app icon on the bottom dock"
}
});
// Returns coordinates for the target element
console.log(result);
}
main();
```
### ā³ Keep Alive on Activity
Automatically extend box expiration time when there's activity on the box. When `keepAlive` is set (e.g., "5m"), any operation (UI Action, File System, Browser, Command, Media, or Run Code) will ensure at least the specified duration remains.
**Example:** If `keepAlive` is "5m" and the box has 2 minutes remaining, any operation extends it back to 5 minutes.
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({
type: "android",
config: {
keepAlive: "30m" // Set to "0ms" to disable (default)
}
});
// Each operation extends the expiration time
await box.action.tap({ target: "chrome app" });
await box.action.screenshot();
// Box automatically stays alive as long as you're using it
}
main();
```
These features require upgrading to [TypeScript SDK version
40](https://www.npmjs.com/package/gbox-sdk/v/0.40.0) or later.
## š¤ Customizable AI Model Selection for UI Actions
The UI Action system now supports specifying custom AI models through the `options.model` parameter. This enhancement gives you the flexibility to choose different computer vision models based on your specific needs, performance requirements, or accuracy preferences.
The `model` parameter is only effective when using natural language-driven UI actions (e.g., `click`, `type`, `scroll` with descriptive targets). It does not apply to coordinate-based or other non-AI-driven interactions.
You can now choose from different AI models (e.g., `openai-computer-use`) for UI element detection by passing the `model` parameter in action options.
### Example Usage
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
// Use a specific AI model for UI action
await box.action.click({
target: "chrome app",
options: {
model: "openai-computer-use"
}
})
}
main()
```
## š Appium Integration Support
Gbox now provides native Appium connection support, enabling seamless integration with the Appium ecosystem for advanced automation workflows. The new `appiumURL()` method generates a ready-to-use Appium connection URL with optimized default configurations.
### Key Features
* **One-Click Connection**: Get an Appium connection URL with a single method call
* **Pre-configured Options**: Default capabilities and settings optimized for Gbox Android boxes
* **Full Appium Compatibility**: Use any Appium client library (WebdriverIO, Appium Python Client, etc.)
* **Advanced Automation**: Access native Appium features like XML layout inspection, element finding, and complex gestures
### Use Cases
* Extract and analyze UI element hierarchies
* Implement complex automation workflows
* Integrate with existing Appium-based testing frameworks
* Debug UI layouts and element properties
### Example Usage
```typescript wrap theme={null}
import { remote } from "webdriverio";
import GboxSDK from "gbox-sdk";
import * as fs from "fs";
import * as path from "path";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
// Generate Appium connection URL and default options
const { url, defaultOption } = await box.appiumURL();
console.log("Appium connection URL:", url);
// Connect to Appium with defaultOption from backend
console.log("Connecting to Appium server...");
const ac = await remote(defaultOption);
console.log("ā Successfully connected to Appium server");
console.log("Session ID:", ac.sessionId);
try {
// Get current page XML layout
console.log("Fetching XML layout...");
const xmlLayout = await ac.getPageSource();
console.log("ā XML layout fetched successfully!");
console.log("XML length:", xmlLayout.length, "characters");
// Display a preview of the XML (first 500 characters)
console.log("\n--- XML Layout Preview (first 500 chars) ---");
console.log(xmlLayout.substring(0, 500) + "...\n");
// Save XML to file for easier viewing
const outputDir = path.join(__dirname, "output");
if (!fs.existsSync(outputDir)) {
fs.mkdirSync(outputDir, { recursive: true });
}
const timestamp = new Date().toISOString().replace(/[:.]/g, "-");
const filename = `layout_${timestamp}.xml`;
const filepath = path.join(outputDir, filename);
fs.writeFileSync(filepath, xmlLayout, "utf-8");
console.log(`š XML layout saved to: ${filepath}`);
// Optional: Parse and display some basic info
const elementMatches = xmlLayout.match(/<[\w.-]+/g);
if (elementMatches) {
const elementSet = new Set(elementMatches.map(e => e.substring(1)));
const uniqueElements = Array.from(elementSet);
console.log("\n--- UI Elements Found ---");
console.log("Total elements:", elementMatches.length);
console.log("Unique element types:", uniqueElements.length);
console.log("Element types:", uniqueElements.slice(0, 10).join(", "), "...");
}
} catch (error) {
console.error(
"ā Error fetching XML layout:",
error instanceof Error ? error.message : String(error)
);
} finally {
console.log("\nClosing session...");
await ac.deleteSession();
console.log("Session closed.");
}
}
main()
```
These features require upgrading to [TypeScript SDK version
38](https://www.npmjs.com/package/gbox-sdk/v/0.38.0) or later.
## š Enhanced Scroll Action with Natural Language Location Support
The `scroll` action now supports natural language descriptions for the `location` parameter, allowing you to specify where on the screen to perform the scroll gesture. This enhancement makes it easier to interact with specific areas of the UI, such as scrolling within a particular region or component.
### Key Features
* **Natural Language Location**: Use descriptive text to specify scroll locations (e.g., "screen bottom", "toolbar area", "middle of the screen")
* **Flexible Targeting**: Perfect for scrolling within specific UI regions or components
* **Improved Precision**: Better control over scroll behavior in complex layouts
### Example Usage
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({
type: "android",
});
// Scroll up from the bottom of the screen
const result = await box.action.scroll({
direction: 'up',
location: "screen bottom",
});
console.info(result);
}
```
### Additional Examples
```typescript wrap theme={null}
// Scroll down in the toolbar area
await box.action.scroll({
direction: 'down',
location: "toolbar area",
});
// Scroll left in the center of the screen
await box.action.scroll({
direction: 'left',
location: "middle of the screen",
});
```
These features require upgrading to [TypeScript SDK version
37](https://www.npmjs.com/package/gbox-sdk/v/0.37.0) or later.
## š Enhanced Browser API
New browser API with full-screen support and window control options, designed to prevent AI agents from accidentally closing or minimizing browsers.
### Key Features & Use Cases
* **š Full-Screen Control**: Seamlessly maximize browser windows for immersive experiences
* **šŖ Window Control Management**: Hide browser minimize, maximize, and close buttons to prevent AI or automation from accidentally closing browsers
* **š¤ AI Agent Automation**: Perfect for AI agents and LLM applications requiring focused browser sessions
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
import { chromium } from "playwright";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "linux" });
const result = await box.browser.open({
maximize: true,
showControls: false,
})
const browser = await chromium.connectOverCDP(result.cdpUrl);
// Use the browser as usual
const context = await browser.contexts()[0];
const page = await context.newPage();
await page.goto("https://example.com");
// Perform actions on the page
console.log(await page.title());
await box.browser.close()
}
main();
```
These features require upgrading to [TypeScript SDK version
36](https://www.npmjs.com/package/gbox-sdk/v/0.36.0) or later.
## š UI Elements Detection
Automatically detect and extract interactive elements from web pages for precise automation and AI-driven interactions.
Currently only supported for **Linux boxes with browser**. Android support coming soon.
### Key Features
* **Element Detection**: Identifies buttons, links, inputs, and other interactive elements
* **Rich Metadata**: Position, size, text content, and HTML attributes
* **Annotated Screenshots**: Returns screenshots with visual annotations highlighting detected elements
* **AI Ready**: Perfect for LLM integration and intelligent automation
### Usage Example
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "linux" });
await box.browser.openTab({
url: "https://gbox.ai",
});
const { screenshot, elements } = await box.action.elements.detect({
screenshot: {
outputFormat: 'storageKey'
}
});
console.info(`Screenshot: ${JSON.stringify(screenshot, null, 2)}`);
console.info(`Detected elements length: ${elements.list().length}`);
// You can send the screenshot to an LLM or Agent to decide which element to click
// here we just click the first element
const firstElement = elements.get("1");
await box.action.click({
target: firstElement,
});
console.info(
`Clicked element: ${JSON.stringify(firstElement, null, 2)}`
);
}
main();
```
These features require upgrading to [TypeScript SDK version
35](https://www.npmjs.com/package/gbox-sdk/v/0.35.0) or later.
## š Clipboard Support
New clipboard management capabilities for Android boxes, enabling programmatic control over device clipboard content.
### Key Features
* **Set/Get Content**: Read and write text to device clipboard
* **Automation Integration**: Seamlessly integrate with UI automation workflows
* **Cross-App Data**: Share data between applications through clipboard
### Usage Example
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
// Set clipboard content
await box.action.clipboard.set("Hello, world!");
// Get current clipboard content
const clipboardContent = await box.action.clipboard.get();
console.log("Clipboard content:", clipboardContent);
}
main();
```
These features require upgrading to [TypeScript SDK version
34](https://www.npmjs.com/package/gbox-sdk/v/0.34.0) or later.
## šø Screenshot ScrollCapture
Enhanced screenshot functionality now supports automatic scrolling to capture tall content like long web pages, documents, or chat conversations in a single image.
### Key Features
* **Automatic Scrolling**: Intelligently scrolls through content to capture everything in one screenshot
* **Height Control**: Configurable maximum height to manage memory usage and file size
* **Position Restoration**: Optional scroll-back functionality to return to original position
* **Memory Optimization**: Built-in limits to prevent excessive memory consumption
### Configuration Options
* **`maxHeight`**: Maximum height in pixels (default: 4000px) - limits the total height of captured content
* **`scrollBack`**: Whether to scroll back to original position after capture (default: false)
### Usage Example
```typescript theme={null}
const result = await box.action.screenshot({
scrollCapture: {
maxHeight: 5000,
scrollBack: true
}
})
```
### Use Cases
* **Web Page Documentation**: Capture entire web pages for documentation or analysis
* **Chat History**: Save complete conversation threads
* **Long Documents**: Capture full document content in one screenshot
* **Social Media Feeds**: Capture extended social media timelines
* **App Content**: Document complete app screens or settings pages
These features require upgrading to [TypeScript SDK version
31](https://www.npmjs.com/package/gbox-sdk/v/0.31.0) or later.
## š Rewind Recording
New Rewind functionality that automatically preserves the last 5 minutes of screen recording, allowing you to extract video clips from any time period at any moment.
### Key Features
* **Automatic Recording**: Continuously preserves up to 5 minutes of recent box screen recording
* **Flexible Extraction**: Extract video clips from any time period (up to 5 minutes maximum)
* **Instant Access**: No need to pre-start recording - access historical clips anytime
### Use Cases
* **Debug Analysis**: Review recent operation recordings when automation scripts encounter issues
* **Error Reproduction**: Quickly capture screen recordings from before problems occurred for easier troubleshooting
* **Operation Logging**: Automatically save operation history without manual recording management
* **Performance Monitoring**: Observe application behavior during specific operations
### Performance Considerations
Enabling Rewind functionality will have some performance impact. It's recommended to enable only when needed and disable when not required to optimize performance.
### Basic Usage
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({
type: "android",
config: {
deviceType: "physical",
},
});
// Enable Rewind functionality
await box.action.recording.rewind.enable();
// Perform some operations...
await box.action.tap({ target: "chrome app" });
await box.action.swipe({ direction: "up" });
// Extract the last 10 seconds of recording
const result = await box.action.recording.rewind.extract({
duration: "10s"
});
console.log("Recording result:", result);
console.log("Download URL:", result.presignedUrl);
// Disable Rewind functionality when not needed to save performance
// await box.action.recording.rewind.disable();
}
main();
```
### Advanced Usage
```typescript wrap theme={null}
// Extract different time periods
const shortClip = await box.action.recording.rewind.extract({
duration: "5s" // Last 5 seconds
});
const longClip = await box.action.recording.rewind.extract({
duration: "2m" // Last 2 minutes
});
const maxClip = await box.action.recording.rewind.extract({
duration: "5m" // Last 5 minutes (maximum)
});
```
## šø Save Screenshots to Album
Enhanced screenshot functionality now supports saving screenshots directly to the device's media album, making it easier to organize and access captured images.
### Key Features
* **Album Integration**: Screenshots are automatically saved to the device's media gallery
* **Easy Access**: Screenshots can be accessed through the device's photo app
* **Organized Storage**: All screenshots are properly categorized in the media library
### Usage
```typescript wrap theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"] // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
// Take a screenshot and save it to the device album
await box.action.screenshot({
saveToAlbum: true,
});
// Screenshot is now available in the device's photo gallery
}
main();
```
These features require upgrading to [TypeScript SDK version
30](https://www.npmjs.com/package/gbox-sdk/v/0.30.0) or later.
### š¼ļø Enhanced Screenshot Options
New unified screenshot configuration with `options.screenshot` parameter for better control over screenshot behavior.
#### Basic Usage
```typescript wrap theme={null}
// Simple boolean
await box.action.click({
x: 100,
y: 100,
options: {
screenshot: true,
},
});
// Detailed configuration
await box.action.click({
x: 100,
y: 100,
options: {
screenshot: {
outputFormat: "storageKey",
presignedExpiresIn: "1h",
delay: "1s",
phases: ["before", "after"],
},
},
});
```
#### Key Features
**šø Screenshot Phases**
* `before`: Screenshot before the action
* `after`: Screenshot after the action
* `trace`: Screenshot with operation trace
* Default captures all three phases
**ā±ļø Configurable Delay**
* `delay`: Wait time after action before taking final screenshot
* Default: `500ms`, Maximum: `30s`
**š Output Format**
* `base64`: Direct image data (default)
* `storageKey`: Storage key with presigned URL access
* `presignedExpiresIn`: Custom expiration for storageKey URLs (default: `30m`)
#### Usage Examples
**Capture specific phases:**
```typescript wrap theme={null}
await box.action.click({
target: "login button",
options: {
screenshot: {
phases: ["before", "after"],
},
},
});
```
**Custom delay for UI state capture:**
```typescript wrap theme={null}
await box.action.click({
target: "submit button",
options: {
screenshot: {
delay: "2s",
phases: ["after"],
},
},
});
```
**Disable screenshots:**
```typescript wrap theme={null}
await box.action.click({
target: "button",
options: {
screenshot: false,
},
});
```
Please use the new `options.screenshot` parameter instead of the old
screenshot fields, as they will be deprecated in future versions.
These features require upgrading to [TypeScript SDK version
29](https://www.npmjs.com/package/gbox-sdk/v/0.29.0) or later.
### šÆ Natural Language UI Actions
UI Actions now support natural language descriptions for targets and locations, making automation more intuitive and human-readable.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
// Tap on app icons or UI elements using natural language
await box.action.tap({
target: "chrome app",
});
await box.action.click({
target: "login button",
});
// Swipe with natural language location descriptions
await box.action.swipe({
direction: "up",
distance: 300,
duration: "500ms",
location: "screen bottom",
});
// Drag and drop using natural language
await box.action.drag({
start: "Chrome App",
end: "Trash",
});
// Long press with natural language targets
await box.action.longPress({
target: "Chrome icon",
duration: "1s",
});
```
### š Action Scale Settings
Added support for scaling UI actions to adjust the size of screenshots and coordinate calculations without changing the actual screen resolution.
#### Key Features
* **Scale Range**: 0.1 to 1.0 (10% to 100% of original size)
* **Screenshot Scaling**: Output screenshots are scaled according to the setting
* **Coordinate Scaling**: All action coordinates and distances are automatically scaled
* **No Resolution Change**: The box's actual screen resolution remains unchanged
#### Usage Examples
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
// Get current action settings
const currentSettings = await box.action.getSettings();
console.log("Current settings:", currentSettings);
// Set scale to 50% for smaller screenshots and scaled coordinates
await box.action.updateSettings({
scale: 0.5,
});
// With scale = 0.5, this click at (100, 100) becomes equivalent to (50, 50) at full scale
await box.action.click({
x: 100,
y: 100,
});
// Reset all settings to default values
await box.action.resetSettings();
// Verify settings have been reset
const resetSettings = await box.action.getSettings();
console.log("Settings after reset:", resetSettings);
```
#### Scale Examples
| Scale Value | Screenshot Size | Coordinate Example |
| --------------- | --------------- | ---------------------------------- |
| `1.0` (default) | Full size | `Click({x: 100, y: 100})` |
| `0.5` | 50% size | `Click({x: 50, y: 50})` equivalent |
| `0.25` | 25% size | `Click({x: 25, y: 25})` equivalent |
**Note**: Scale affects both the output screenshot dimensions and the coordinate system for all UI actions, making it useful for optimizing performance and storage when full resolution isn't needed.
### š Proxy Configuration
Added proxy configuration support for Android boxes.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
// Set proxy
await box.proxy.set({
host: "127.0.0.1",
port: 9090,
});
// Get current proxy settings
console.info(await box.proxy.get());
// Clear proxy
await box.proxy.clear();
```
These features require upgrading to [TypeScript SDK version
27](https://www.npmjs.com/package/gbox-sdk/v/0.27.0) or later.
### š±ļø UI Actions
* **Added**: `tap` and `longPress` actions for precise coordinate taps and long-press interactions.
* **Enhanced**: `swipe` / `scroll` now support semantic `distance` values (`"tiny" | "short" | "medium" | "long"`), so you no longer need to provide pixel values.
#### New action: tap
```typescript theme={null}
const box = await gboxSDK.create({ type: "android" });
await box.action.tap({
x: 100,
y: 100,
});
```
#### New action: longPress
```typescript theme={null}
const box = await gboxSDK.create({ type: "android" });
await box.action.longPress({
x: 100,
y: 100,
});
```
#### Semantic distance (swipe/scroll)
```typescript theme={null}
const box = await gboxSDK.create({ type: "android" });
await box.action.swipe({
direction: "up",
distance: "long",
});
await box.action.scroll({
direction: "up",
distance: "long",
});
```
* Docs: [`tap`](/api-reference/ui-action/tap) Ā· [`longPress`](/api-reference/ui-action/long-press)
These features require upgrading to [TypeScript SDK version
26](https://www.npmjs.com/package/gbox-sdk/v/0.26.0) or later.
### š„ Android Screen Recording
Added screen recording capabilities to Android boxes for creating tutorials and debugging UI interactions.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
console.info("start recording");
await box.action.screenRecordingStart();
console.info("swipe up");
await box.action.swipe({
direction: "up",
});
console.info("sleep 5 seconds...");
// you can do anything you want here
await new Promise((resolve) => setTimeout(resolve, 5000));
const result = await box.action.screenRecordingStop();
// you can download the video from the result
console.info(`recording result: ${JSON.stringify(result, null, 2)}`);
```
### š± Android Media Management
Added media file management capabilities for Android devices, including album listing and media operations.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
const albums = await box.media.listAlbums();
console.info(albums);
```
For more media API endpoints, see [Media API Docs](/api-reference/media/list-albums).
### šŖ AI Action Progress Callbacks
Since AI Action execution takes time, we provide a series of callbacks to monitor the progress of the entire operation process.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
await box.action.ai("open the youtube app", {
onActionStart: () => {
console.info("action start");
},
onActionEnd: () => {
console.info("action end");
},
});
```
These features require upgrading to [TypeScript SDK version
25](https://www.npmjs.com/package/gbox-sdk/v/0.25.0) or later.
### UI Action Support for StorageKey Output Format
UI Action now supports `outputFormat: "storageKey"`, allowing GBOX.AI to directly store screenshot information. Compared to returning image data directly, StorageKey provides more flexible storage and access options:
* **presigned URL**: System-generated temporary access link with a default validity period of 30 minutes
* **storageKey**: A storage key that remains valid throughout the box lifecycle (until the box is deleted)
* **Custom presigned URL**: Use `createPresignedUrl` to create presigned URLs with specified expiration times, convenient for returning to LLM models for image analysis
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
const res = await box.action.screenshot({
outputFormat: "storageKey",
});
console.info(`presigned url: ${res.presignedUrl}`);
// create custom presigned url
const customPresignedUrl = await box.storage.createPresignedUrl({
storageKey: res.uri,
expiresIn: "1h",
});
console.info(`custom presigned url: ${customPresignedUrl}`);
```
### Android Multiple APK Installation Support
Added support for installing multiple APK files using ZIP archives, enabling installation of split APKs and bundle applications.
#### Installation Modes
**Single APK Installation** (default):
* Upload and install a single APK file
* Traditional installation method for standalone applications
**Multiple APK Installation** (ZIP-based):
* Upload a ZIP archive containing multiple APK files
* Automatically extracts and installs all APK files in the correct order
* Essential for modern Android apps that use App Bundle distribution
#### ZIP Archive Structure
When using multiple APK installation, organize your files as follows:
```
app-bundle.zip
āāā app-folder/
āāā base.apk (base application)
āāā config.arm64_v8a.apk (architecture-specific)
āāā config.en.apk (language resources)
āāā config.xxxhdpi.apk (density-specific resources)
```
#### Usage Examples
**Single APK Installation:**
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
const app = await box.app.install({
apk: "/path/to/single-app.apk",
});
await app.open();
```
**Multiple APK Installation (ZIP):**
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
const app = await box.app.install({
apk: "/path/to/app-bundle.zip",
});
await app.open();
```
These features require upgrading to [TypeScript SDK version
22](https://www.npmjs.com/package/gbox-sdk/v/0.22.0) or later.
### Android Box Command Support
Added direct command execution support for Android boxes, enabling full system-level control through shell commands.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "android" });
await box.command({
commands: ["logcat"],
onStdout: (data) => {
console.info("---stdout---");
console.info(data);
},
onStderr: (data) => {
console.info("---stderr---");
console.error(data);
},
});
```
### Linux Box Support for UI Action / AI Action
Added UI automation and AI-powered action capabilities to Linux boxes, enabling desktop application interaction and programmatic UI operations.
```typescript wrap theme={null}
await gboxSDK.create({ type: "linux" });
await box.action.click({
x: 100,
y: 100,
});
await box.action.ai("Open the Chrome browser");
```
### Linux Box Command / Run Code Streaming Support
Added real-time streaming support for command execution and code running on Linux boxes, providing immediate feedback and live output monitoring.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "linux" });
const result = await box.command({
commands: ["for i in {10..1}; do echo $i; echo $i >&2; sleep 1; done"],
onStdout: (data) => {
console.info("---stdout---");
console.info(data);
},
onStderr: (data) => {
console.info("---stderr---");
console.error(data);
},
});
console.info(result);
const result2 = await box.runCode({
code: `
console.log("something xxxx");
console.error("something error")
`,
language: "typescript",
onStdout: (data) => {
console.info("stdout");
console.info(data);
},
onStderr: (data) => {
console.info("stderr");
console.error(data);
},
});
console.info(result);
```
### Browser Action Support
Added browser automation capabilities to Linux boxes, enabling programmatic control and interaction with web browsers and tabs.
```typescript wrap theme={null}
const box = await gboxSDK.create({ type: "linux" });
const tabs = await box.browser.listTabInfo();
console.info(tabs);
```
# GBOX CLI
Source: https://docs.gbox.ai/cli/gbox-cli
## Installation
### System Requirements
* macOS 10.15 or later
* [Docker Desktop for Mac](https://docs.docker.com/desktop/setup/install/mac-install/)
* [Homebrew](https://brew.sh)
> Note: Please check [https://github.com/babelcloud/gbox](https://github.com/babelcloud/gbox) for support of other platforms.
### Installation Steps
```bash theme={null}
# Install GBOX CLI
brew install gbox
# Login to GBOX.cloud (GitHub authorization required)
gbox login
```
### Update Steps
```bash theme={null}
# Update GBOX to the latest version
brew update && brew upgrade gbox
```
### Command Line Usage
The project provides a command-line tool `gbox` for managing sandbox containers:
```bash theme={null}
# Available Commands:
gbox login Login using GitHub OAuth
gbox box Manage box resources
gbox device-connect Manage remote connections for local Android development devices
gbox port-forward Forward one or more ports from a remote box to your local machine (multi-port, kubectl style)
gbox mcp Manage MCP configuration operations
gbox profile Manage configuration profiles
gbox version Print the client version information
gbox completion Generate the autocompletion script for the specified shell # preview merge result
```
# Register Local Device
Source: https://docs.gbox.ai/cli/register-local-device
This guide explains how to register your local Android device with GBOX for local testing and development.
## Prerequisites
Before proceeding, ensure you have:
* An Android device with developer options enabled
* A Mac computer
* USB cable for device connection
* Active internet connection
## Step-by-Step Registration Process
> **Note**: If your local device is created by Android Studio, you should skip this step 1-3.
### 1. Enable Developer Options on Android Device
> **Reference**: For detailed instructions, refer to the [Pixel Developer Options Guide](https://developer.android.com/studio/debug/dev-options) and [Samsung Developer Options Guide](https://developer.samsung.com/health/android/data/guide/dev-mode.html).
### 2. Configure Device Settings
To ensure optimal performance during testing:
* Disable **Auto screen timeout**
* Disable **Auto lock screen**
* Keep the device screen on during testing sessions
### 3. Connect Device to Mac
1. Connect your Android device to your Mac using a USB cable
2. On your Android device, allow **USB debugging** when prompted
3. Ensure the device is recognized by your Mac
### 4. Install and Configure GBOX CLI
Open Terminal on your Mac and execute the following commands:
```bash theme={null}
# Install GBOX CLI
brew install gbox
# Update GBOX if already installed
brew update && brew upgrade gbox
# Login to GBOX.cloud (GitHub authorization required)
gbox login
# Register local device to GBOX.AI
gbox device-connect
```
### 5. Install GBOXKeyboard (Required)
Your device may prompt you to install **GBOXKeyboard** - this is essential for Agent input operations. Please install it when prompted.
### 6. Verify Device Connection
After successful registration, you should see your connected device in the device list:
You can also see the registered devices in the [GBOX Dashboard](https://gbox.ai/dashboard):
> **Note**: You may see the newly registered device status is "Offliine", this maybe caused by some initialize work still in progress. Waitfor a few minutes and it should be changed to "Online". Or try to register the device again.
## Troubleshooting
### Connection Issues
* Ensure USB debugging is enabled
* Try different USB cables
* Restart both device and Mac if needed
### Performance Note
> **Note**: Due to our servers being located in the United States, users in other locations may experience some latency during operations.
## Next Steps
Once your device is successfully registered, you can:
* Run automated tests
* Execute UI actions
* Debug applications
* Perform device automation tasks
For more information on using your registered device, refer to the [Android Basic Guide](/android/basic) and [Real Device Testing](/android/real-device) documentation.
# Concepts
Source: https://docs.gbox.ai/concepts
This conceptual guide introduces the two core abstractions in GBOX: **Devices** and **Boxes**
## Device
A **Device** is the underlying hardware resource (cloud virtual/physical Android devices, registered local Android device, Linux desktop/sandbox).
* Devices are the foundation for creating boxes.
* A device can only host **one active box at a time**.
## Box
A **Box** is an operable environment created from a device.
* Think of it as a running session on that device.
* Each device can create multiple boxes **over time**, but you must terminate the current one before starting another.
* Boxes expose the APIs and SDK actions your AI agents use to interact with the device environment.
# Android MCP Server
Source: https://docs.gbox.ai/docs-mcp/android-mcp-server
> MCP server exposing GBOX Android control tools via Model Context Protocol
[](https://www.npmjs.com/package/@gbox.ai/mcp-server) [](https://github.com/babelcloud/gbox/blob/HEAD/LICENSE)
## Description
This package provides an MCP (Model Context Protocol) server for controlling Android devices via GBOX tools. It exposes a set of tools and APIs for automation, device management, and integration with the GBOX ecosystem.
## Usage
Use GBOX CLI to configure GBOX Android MCP Server for Cursor and Claude Code.
```bash theme={null}
#Using GBOX CLI to configure GBOX Android MCP Server for Cursor
gbox mcp export --merge-to cursor
#Using GBOX CLI to configure GBOX Android MCP Server for Claude Code
gbox mcp export --merge-to claude-code --scope user
```
Copy the following configuration into your Cursor or Claude code MCP config file:
> `GBOX_API_KEY` needs to be obtained through [GBOX Dashboard](https://gbox.ai/dashboard).
```json theme={null}
"gbox-android": {
"command": "npx",
"args": [
"-y",
"@gbox.ai/mcp-server@latest"
],
// NOTE: You can omit the 'env' section if you have successfully run 'gbox login' in cli.
"env": {
"GBOX_API_KEY": "gbox_xxxx",
}
}
```
For detailed instructions on connecting IDE to GBOX MCP, please see the following: [Cursor Guide](/integrations/ide/cursor), [Claude Code Guide](/integrations/ide/claude-code) and [VSCode Guide](/integrations/ide/vscode).
If you need the agents to control your local
Android devices, please check [https://docs.gbox.ai/cli/register-local-device](https://docs.gbox.ai/cli/register-local-device).
# Docs MCP Server
Source: https://docs.gbox.ai/docs-mcp/docs-mcp-server
This MCP server provides local access to `GBOX documentation`, enabling seamless AI-powered interactions with your product. Once installed, AI assistants can directly query and reference GBOX documentation to help you build, debug, and optimize your applications.
## Installation
Install the MCP server by running the following command in your terminal.
```bash theme={null}
npx mint-mcp add docs.gbox.ai
```
# Introduction
Source: https://docs.gbox.ai/index
GBOX provides environments for AI Agents to operate computer and mobile devices.
# Environments
GBOX supports four types of environments: Mobile, Browser, Desktop and Sandbox.
**Environment OS**
**Highlights**
**Mobile**
Android
ā¢ Supports physical devices running on cloud
ā¢ Supports registration of your own Android device(s)
ā¢ Instant creation under 150ms
ā¢ Run any code or command
The GBOX SDK supports operating UI environments via natural language, such as:
```typescript theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
// Natural language click
await box.action.click({
target: "login button on the upper right corner",
});
}
main();
```
# Why GBOX
Traditional AI development is fragmented and complex. You need separate tools for mobile testing, desktop automation, web scraping, and sandboxed execution. Each environment requires different SDKs, authentication methods, and deployment processes.
GBOX unifies everything under one interface. Whether you're building an AI agent that needs to interact with Android apps, Linux desktops, web browsers, or sandboxed execution environmentsāitās all accessible through the same SDK and API.
**Key Benefits**
* **Rapid Prototyping**: Go from idea to working agent in minutes, not days. Spin up a sandbox environment instantly to test your agent's logic, then seamlessly deploy to real devices.
* **True Cross-Platform**: Your agent code works identically across mobile, desktop, browser, and sandbox environments. Write once, run anywhere.
* **Cloud-Ready Deployment**: Start locally, then run the same agent code on hosted devices without modification. GBOX manages provisioning and environment setup for you.
* **Natural Language Control**: No need to learn platform-specific automation libraries. Describe what your agent should do in plain English - GBOX translates that into precise device interactions.
# Scenarios
## QA Agent
You can use the GBOX SDK to build your own QA Agents.
## Develop/Test Android App with Claude Code
Enable your Claude Code session to operate Android through GBOX Android MCP.
## Mobile Automation with Claude Code
Assign almost any task to Claude Code, good luck!
# Claude Code
Source: https://docs.gbox.ai/integrations/ide/claude-code
Integrate GBOX with Claude Code for seamless Android app testing and development workflow.
If you have not installed Claude Code, please check the [installation](https://docs.anthropic.com/en/docs/claude-code/overview) instructions before proceeding.
## Getting Started
Install the GBOX CLI tool to manage your GBOX configuration.
```bash theme={null}
# Install GBOX CLI
brew install gbox
# Update GBOX if already installed
brew update && brew upgrade gbox
```
Authenticate with your GBOX account and set up the configuration.
```bash theme={null}
gbox login
```
This command will prompt you to enter your GBOX API key and configure your account.
Enter the command in the command line
```bash theme={null}
#Using GBOX CLI
#scope param: MCP server scope for claude-code (local|project|user) (default "user")
gbox mcp export --merge-to claude-code --scope user
#Check merging result
claude mcp list
#Expected Output: gbox-android: npx -y @gbox.ai/mcp-server
```
If the installation is correct, you will see the gbox-android mcp server when you type **/mcp** in Claude.
Then you can **give Claude Code instructions** using **natural language**, for example:
By default, GBOX MCP uses a cloud android VM. To connect GBOX to your own Android devices, see [Register Local Devices](https://docs.gbox.ai/cli/register-local-device).
Demo
# Cursor
Source: https://docs.gbox.ai/integrations/ide/cursor
Integrate GBOX with Cursor IDE for seamless Android app testing and development workflow.
If you have not installed Cursor, please check the [installation](https://docs.cursor.com/en/get-started/installation) instructions before proceeding.
## Getting Started
Install the GBOX CLI tool to manage your GBOX configuration.
```bash theme={null}
# Install GBOX CLI
brew install gbox
# Update GBOX if already installed
brew update && brew upgrade gbox
```
Enter the command in the command line.
```bash theme={null}
gbox login
```
This command will prompt you to enter your GBOX API key and configure your account.
Set up the Model Context Protocol (MCP) server to enable communication between Cursor and GBOX.
```bash theme={null}
#Using GBOX CLI
#NOTE: This command will not affect your original configuration.
gbox mcp export --merge-to cursor
#Check merging result
cursor /Users/@your-username/.cursor/mcp.json
```
Or copy and paste the following content into your Cursor configuration file, you'll find the MCP configuration in `~/.cursor/mcp.json`:
```json theme={null}
{
"mcpServers": {
"gbox-android": {
"command": "npx",
"args": [
"-y",
"@gbox.ai/mcp-server@latest"
]
}
}
}
```
Download and configure the pre-built rule file for Android app testing: [Rule File Link](https://github.com/babelcloud/gbox-mcp-server/blob/main/src/prompts/gbox-manual.md)
This rule file contains predefined instructions for the AI agent to effectively test Android applications using GBOX.
Now you can use Cursor's Agent Mode to test your Android applications.
1. **Enable Agent Mode** in Cursor
2. **Check** whether the rule is enabled
3. **Assign testing tasks** to the AI agent:
By default, GBOX MCP uses a cloud android VM. To connect GBOX to your own Android devices, see [Register Local Devices](https://docs.gbox.ai/cli/register-local-device).
## Demo
# VSCode
Source: https://docs.gbox.ai/integrations/ide/vscode
Integrate GBOX with VSCode for seamless Android app testing and development workflow.
## Getting Started
Install the GBOX CLI tool to manage your GBOX configuration.
```bash theme={null}
# Install GBOX CLI
brew install gbox
# Update GBOX if already installed
brew update && brew upgrade gbox
```
Authenticate with your GBOX account and set up the configuration.
```bash theme={null}
gbox login
```
This command will prompt you to enter your GBOX API key and configure your account.
Set up the Model Context Protocol (MCP) server to enable communication between VSCode and GBOX.
You can configure the MCP server either globally or at the project level:
**Global Configuration:**
Create or modify your MCP configuration file at `~/.vscode/mcp.json`:
**Project-Level Configuration:**
Alternatively, you can create the configuration file at `.vscode/mcp.json` in your project directory for project-specific settings.
```json theme={null}
{
"servers": {
"gbox-android": {
"command": "npx",
"args": [
"-y",
"@gbox.ai/mcp-server@latest"
]
}
}
}
```
Create a project-level instruction file to guide Copilot Agents during testing. You can start by copying our template and then customizing it:
```bash theme={null}
cd
mkdir -p .github
curl -fsSL https://raw.githubusercontent.com/babelcloud/open-webui/refs/heads/main/.github/instructions/android-testing.instructions.md -o .github/android-testing.instructions.md
```
You can include:
* Test account credentials
* Project-specific setup guides
* Feature specs and limitations
In VSCode with GitHub Copilot **Agent Mode** enabled, you can now use AI agents to test your Android applications.
By default, GBOX MCP uses a cloud android VM. To connect GBOX to your own Android devices, see [Register Local Devices](https://docs.gbox.ai/cli/register-local-device).
# OSWorld
Source: https://docs.gbox.ai/integrations/leader-board/os-world
Learn how to use GBOX as a provider in OSWorld to build and run agents.
This tutorial teaches you how to use **GBOX as a provider** in OSWorld to build and run agents that can interact with operating systems.
## What is OSWorld?
[OSWorld](https://github.com/xlang-ai/OSWorld) is a benchmark framework for evaluating multimodal agents on open-ended tasks in real computer environments. It supports multiple providers for running virtual environments, including VMware, VirtualBox, Docker, and AWS. By using **GBOX as a provider**, you can leverage cloud-native infrastructure without managing local virtual machines, making it easier to scale your agent evaluations and reduce setup complexity.
## Architecture
The following diagram illustrates the architecture of OSWorld using GBOX as a provider:
```mermaid theme={null}
graph TB
A[OSWorld Agent] -->|Actions & Observations| B[OSWorld Framework]
B -->|Provider Interface| C[GBOX Provider]
C -->|API Calls| D[GBOX Cloud API]
D -->|Manage & Control| E[GBOX Box Environment]
E -->|Screenshots & UI State| D
D -->|Response| C
C -->|Environment State| B
B -->|Task Results| A
style A fill:#7139ee,color:#fff
style B fill:#7139ee,color:#fff
style C fill:#7139ee,color:#fff
style D fill:#7139ee,color:#fff
style E fill:#7139ee,color:#fff
```
## Benefits of Using GBOX Provider
Using GBOX as a provider in OSWorld offers several advantages:
### š **Cloud-Native Infrastructure**
* No need to set up and manage local virtual machines
* Works seamlessly across different development environments
* **Setup time reduced from \~2 hours to \~5 minutes**: Start evaluating agents immediately without downloading large VM images (often dozens of GB) or waiting for installations
### ā” **Easy Scaling & Parallelization**
* Run multiple environments in parallel without local resource constraints
* Significantly reduce evaluation time through parallel execution
### š§ **Simplified Setup**
* No need to check KVM support or install Docker Desktop
* Works on any platform without virtualization requirements
* No downloading VM images, installing virtualization software, or troubleshooting compatibility issues
### š **Accessibility**
* Access your environments from anywhere
* Consistent performance regardless of your local hardware
## Prerequisites
Before getting started, make sure you have:
* A GBOX account with an API key ([Get your API key](/api-key))
* An OpenAI API key (or another compatible LLM provider)
* Python 3.10 or higher installed
* Git installed
## Getting Started
### Step 1: Clone the Repository
Clone the OSWorld provider repository:
```bash theme={null}
# Clone the OSWorld provider repository
git clone https://github.com/babelcloud/OSWorld-provider
# Change directory into the cloned repository
cd OSWorld-provider
# Optional: Create a Conda environment for OSWorld
# conda create -n osworld python=3.10
# conda activate osworld
# Install required dependencies
pip install -r requirements.txt
```
### Step 2: Configure API Keys
Create a `.env` file in the repository root and add your GBOX API Key and OpenAI API Key:
```bash .env theme={null}
GBOX_API_KEY=your_gbox_api_key
OPENAI_API_KEY=your_openai_api_key
```
> **Note:** You can obtain your GBOX API key from the [API Key page](/api-key). Make sure to keep your API keys secure and never commit them to version control.
### Step 3: Run the Provider
Execute the following command to start the provider with GBOX:
```bash theme={null}
python run_multienv.py \
--provider_name gbox \
--model gpt-4o \
--region us-east-1 \
--max_steps 15 \
--observation_type screenshot \
--action_space pyautogui \
--result_dir ./results_gbox \
--num_envs 1 \
--test_all_meta_path evaluation_examples/test_small.json
```
**Command Parameters Explained:**
* `--provider_name gbox`: Use GBOX as the provider
* `--model gpt-4o`: Specify the LLM model for the agent
* `--region us-east-1`: GBOX region (adjust based on your preference)
* `--max_steps 15`: Maximum number of steps the agent can take
* `--observation_type screenshot`: Use screenshots for environment observation
* `--action_space pyautogui`: Use PyAutoGUI for action execution
* `--result_dir ./results_gbox`: Directory to save evaluation results
* `--num_envs 1`: Number of parallel environments to run. **Increasing this value can significantly improve evaluation efficiency** by running multiple tasks concurrently
* `--test_all_meta_path`: Path to the test configuration file
### Step 4: Monitor Agent Execution
Once the agent starts running, you can monitor its progress in real-time through the VNC viewer. The agent will interact with the OS environment, performing tasks based on the evaluation configuration.
> **Tip:** The default VNC password is `osworld-public-evaluation`. You can access the VNC viewer URL from the GBOX dashboard or API response.
### Step 5: View Results
After the evaluation completes, you can find the results in the `results_gbox` directory. The results include:
* Task execution logs
* Screenshots of key actions
* Performance metrics
* Success/failure status for each task
You can now start building your own agents by modifying the test configuration files or creating custom evaluation scenarios.
## Next Steps
* Explore the [OSWorld documentation](https://github.com/xlang-ai/OSWorld) to learn more about creating custom evaluation tasks
* Check out the [GBOX API reference](/api-reference) for advanced configuration options
* Experiment with different models and parameters to optimize agent performance
* Scale up your evaluations by increasing the `--num_envs` parameter to run multiple environments in parallel
# AgentKit
Source: https://docs.gbox.ai/integrations/platform/agentkit
Develop AI Agents that can browse the web autonomously using GBOX and AgentKit (by inngest).
## Overview
AgentKit is a framework for building AI agents with tools, memory, and autonomous behavior. It provides a structured way to define what an agent can do and how it makes decisions, enabling developers to build flexible, task-oriented agents.
By integrating with GBOX, AgentKit agents gain the ability to **control managed headless browsers**, enabling them to autonomously search the web, extract data, and interact with websites. This makes them well-suited for **real-time information retrieval and robust web automation**.
## Getting Started
Install the necessary dependencies for your project.
```bash npm theme={null}
npm install gbox-sdk @inngest/agent-kit playwright-core dotenv typescript tsx @types/node
```
```bash pnpm theme={null}
pnpm install gbox-sdk @inngest/agent-kit playwright-core dotenv typescript tsx @types/node
```
```bash yarn theme={null}
yarn add gbox-sdk @inngest/agent-kit playwright-core dotenv typescript tsx @types/node
```
Create a `.env` file in your project root and add your GBOX API key:
```dotenv theme={null}
GBOX_API_KEY=your_gbox_api_key
ANTHROPIC_API_KEY=your_anthropic_api_key
```
Create a new file `agent.ts` and set up your AgentKit network with an agent that can use GBOX:
```typescript theme={null}
import {
anthropic,
createAgent,
createNetwork,
createTool,
} from "@inngest/agent-kit";
// Create the search agent
const searchAgent = createAgent({
name: "reddit_searcher",
description:
"An intelligent agent that searches Reddit for relevant information and provides summarized insights",
system: `You are a helpful Reddit search assistant. When searching for information:
1. Use the search_reddit tool to find relevant posts
2. Summarize the key findings from multiple posts
3. Highlight different perspectives or opinions found
4. Provide context about the discussions
5. If no results are found, suggest alternative search terms`,
tools: [searchReddit],
});
// Create the network with proper API key configuration
const redditSearchNetwork = createNetwork({
name: "reddit_search_network",
description:
"A network that intelligently searches Reddit and provides insights",
agents: [searchAgent],
maxIter: 5, // Allow more iterations for better results
defaultModel: anthropic({
model: "claude-3-5-sonnet-latest",
apiKey: requiredEnvVars.ANTHROPIC_API_KEY,
defaultParameters: {
max_tokens: 4096,
},
}),
});
```
Define the `searchReddit` tool that uses GBOX to search Reddit:
```typescript theme={null}
import GboxSDK from "gbox-sdk";
import { chromium } from "playwright-core";
import dotenv from "dotenv";
import { z } from "zod";
// Load environment variables
dotenv.config();
// Validate required environment variables
const requiredEnvVars = {
GBOX_API_KEY: process.env.GBOX_API_KEY,
ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY,
};
// Create a tool to search Reddit using GBOX SDK
if (!requiredEnvVars.GBOX_API_KEY || !requiredEnvVars.ANTHROPIC_API_KEY) {
throw new Error(
"Missing required environment variables: GBOX_API_KEY and ANTHROPIC_API_KEY"
);
}
const searchReddit = createTool({
name: "search_reddit",
description: "Search Reddit posts and comments using PullPush API",
parameters: z.object({
query: z.string().describe("The search query for Reddit"),
}),
handler: async ({ query }) => {
console.log(`š Searching Reddit for: "${query}"`);
const gboxSDK = new GboxSDK({
apiKey: requiredEnvVars.GBOX_API_KEY,
});
const box = await gboxSDK.create({ type: "linux" });
const cdpUrl = await box.browser.cdpUrl();
try {
const browser = await chromium.connectOverCDP(cdpUrl);
const browserContext = await browser.newContext();
const page = await browserContext.newPage();
// Construct the search URL
const searchUrl = `https://search-new.pullpush.io/?type=submission&q=${query}`;
await page.goto(searchUrl);
// Wait for results to load
await page.waitForSelector("div.results", { timeout: 10000 });
// Extract search results
const results = await page.evaluate(() => {
const posts = document.querySelectorAll(
"div.results div:has(h1)"
);
return Array.from(posts).map((post) => ({
title: post.querySelector("h1")?.textContent?.trim(),
content: post.querySelector("div")?.textContent?.trim(),
}));
});
// Close the browser
await browser.close();
console.log("Browser closed");
// Terminate the box
await box.terminate();
console.log("Box terminated");
return results.slice(0, 5); // Return top 5 results
} catch (error) {
console.error("Error connecting to browser:", error);
throw new Error("Failed to connect to browser for Reddit search");
}
},
});
```
Now, combine everything in your `agent.ts` file to create a complete AgentKit network that can search Reddit:
```typescript theme={null}
// Helper function to extract the final agent response from AgentKit's response format
function extractFinalResponse(response) {
try {
// Check if response has the new state structure
if (response && response.state && response.state._results) {
// Get the last result from _results array
const results = response.state._results;
if (results.length > 0) {
const lastResult = results[results.length - 1];
// Extract the content from the output array
if (lastResult.output && Array.isArray(lastResult.output)) {
for (const outputItem of lastResult.output) {
if (outputItem.type === "text" && outputItem.content) {
return outputItem.content;
}
}
}
}
}
// If we can't find a clear text response, return a message
return "Unable to extract response content from agent output.";
} catch (error) {
console.error("Error extracting response:", error);
return "Error occurred while processing agent response.";
}
}
// Main execution function with combined formatting and search
async function searchRedditWithAgent(
query = "Best programming languages for beginners"
) {
try {
console.log(`\n${"=".repeat(75)}`);
console.log(`\t Query: ${query}`);
console.log(`${"=".repeat(75)}`);
console.log(`š Starting Reddit search for: "${query}"`);
console.log("ā³ This may take a moment...\n");
// Run the agent network
const response = await redditSearchNetwork.run(query);
console.log("ā Search completed!");
// Extract the final text response using our helper function
const finalResponse = extractFinalResponse(response);
console.log("š Results:", finalResponse.slice(0, 50) + "...");
// Log the clean result for user consumption
console.log("\n" + "-".repeat(60));
console.log("\t\t šÆ Final Clean Response");
console.log("-".repeat(60));
console.log(finalResponse);
return finalResponse;
} catch (error) {
console.error(`ā Failed to process query "${query}":`, error.message);
throw error;
}
}
// Run the example
searchRedditWithAgent().catch(console.error);
```
You can run your agent using the following command:
```bash theme={null}
npx tsx agent.ts
```
This will execute the agent, which will search Reddit for the specified query and return summarized results.
# Introduction
Source: https://docs.gbox.ai/integrations/platform/intro
Explore how to connect GBOX with your favorite platforms and services.
Integrations create powerful connections between our headless browser infrastructure and your preferred platforms, APIs, and services. Whether youāre building sophisticated AI agents, automating complex transactions, or optimizing workflows, our integration suite provides everything you need to expand your application's capabilities.
# Custom Base URL
Source: https://docs.gbox.ai/sdk/base-url
Configure custom API endpoints for different environments
## Overview
By default, the GBOX SDK connects to the production API at `https://gbox.ai/api/v1`. You can customize this base URL to connect to different environments or self-hosted instances.
## Configuration
### Global Configuration
Set the base URL when initializing the SDK:
```typescript Production (Default) theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
baseURL: "https://gbox.ai/api/v1", // This is the default
});
```
```typescript Development Environment theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
baseURL: "https://dev.gbox.ai/api/v1",
});
```
```typescript Local Development theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
baseURL: "http://localhost:3000/api/v1",
});
```
### Environment-Based Configuration
For better flexibility, use environment variables to manage different base URLs:
```typescript theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
baseURL: process.env["GBOX_BASE_URL"] || "https://gbox.ai/api/v1",
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
console.log("Device created:", box.id);
}
main();
```
## Environment Variables
Set up your environment variables in a `.env` file:
```bash theme={null}
GBOX_API_KEY=your_api_key_here
GBOX_BASE_URL=https://your-custom-endpoint.com/api/v1
```
# Quick Start
Source: https://docs.gbox.ai/sdk/index
Get started with the GBOX SDK to create and manage virtual devices
## Installation
Install the GBOX SDK using your preferred package manager:
```bash npm theme={null}
npm install gbox-sdk
```
```bash pnpm theme={null}
pnpm install gbox-sdk
```
```bash yarn theme={null}
yarn add gbox-sdk
```
## Basic Usage
### 1. Initialize the SDK
First, import and initialize the SDK with your API key for authentication:
```typescript Basic Setup theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // Recommended: use environment variables
});
```
```typescript Full Configuration theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
baseURL: "https://gbox.ai/api/v1", // Optional: custom API base URL
timeout: 60 * 1000, // Optional: request timeout in milliseconds
maxRetries: 0, // Optional: maximum number of retries
});
```
### 2. Create Virtual Devices
Use the SDK to create different types of virtual devices:
```typescript Android Device theme={null}
async function createAndroidBox() {
try {
const box = await gboxSDK.create({
type: "android",
// Optional: add other configuration parameters
});
console.log("Android device created successfully:", box.id);
return box;
} catch (error) {
console.error("Failed to create Android device:", error);
}
}
createAndroidBox();
```
```typescript Linux Device theme={null}
async function createLinuxBox() {
try {
const box = await gboxSDK.create({
type: "linux",
});
console.log("Linux device created successfully:", box.id);
return box;
} catch (error) {
console.error("Failed to create Linux device:", error);
}
}
createLinuxBox();
```
### 3. Complete Example
Here's a complete example showing how to create a device and perform basic operations:
```typescript theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
});
async function main() {
try {
// Create an Android device
const box = await gboxSDK.create({ type: "android" });
console.log(`Device created successfully, ID: ${box.id}`);
// Start the device
await gboxSDK.startBox({ id: box.id });
console.log("Device started successfully");
// Wait for device to be ready...
// Stop the device
await gboxSDK.stopBox({ id: box.id });
console.log("Device stopped");
} catch (error) {
console.error("Operation failed:", error);
}
}
main();
```
## Next Steps
* Learn how to [configure custom base URLs](/sdk/base-url)
* Understand [timeout settings](/sdk/timeout) and [retry mechanisms](/sdk/retries)
* Explore the complete [TypeScript SDK documentation ](https://babelcloud.github.io/gbox-sdk-ts/classes/GboxSDK.html)
# Python SDK
Source: https://docs.gbox.ai/sdk/python
# Retry
Source: https://docs.gbox.ai/sdk/retries
Configure automatic retry behavior for failed requests
## Overview
The GBOX SDK automatically retries certain types of failed requests to improve reliability. By default, retry is disabled (0 retries), but you can configure this behavior based on your needs.
## Automatic Retry Conditions
The SDK will automatically retry requests that fail due to:
* **Connection errors** (network connectivity issues)
* **408 Request Timeout**
* **409 Conflict**
* **429 Rate Limit**
* **5xx Server Errors** (500, 502, 503, etc.)
## Configuration
### Global Retry Settings
Configure retry behavior when initializing the SDK:
```typescript Basic Retry Configuration theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
maxRetries: 3, // Retry up to 3 times (default is 0)
});
```
```typescript Advanced Configuration theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
maxRetries: 5,
timeout: 30 * 1000, // 30 seconds timeout per request
});
```
### Per-Request Retry Settings
Override retry settings for specific requests:
```typescript Override for Device Creation theme={null}
// Create device with custom retry settings
const box = await gboxSDK.create(
{ type: "android" },
{
maxRetries: 5, // Retry up to 5 times for this request
}
);
```
```typescript Override for Critical Operations theme={null}
// Use higher retry count for important operations
await gboxSDK.startBox(
{ id: boxId },
{
maxRetries: 10,
timeout: 60 * 1000, // 60 seconds timeout
}
);
```
## Retry Behavior
### Exponential Backoff
The SDK uses exponential backoff between retries:
* **1st retry**: \~1 second delay
* **2nd retry**: \~2 seconds delay
* **3rd retry**: \~4 seconds delay
* And so on...
### Example Usage
```typescript theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
maxRetries: 3,
});
async function createDeviceWithRetry() {
try {
const box = await gboxSDK.create({ type: "android" });
console.log("Device created successfully:", box.id);
return box;
} catch (error) {
console.error("Failed to create device after retries:", error);
throw error;
}
}
createDeviceWithRetry();
```
## Best Practices
* **Start with low retry counts** (2-3) and increase if needed
* **Use higher retry counts** for critical operations
* **Set appropriate timeouts** to avoid long waits
* **Handle final failures** gracefully in your application
* **Monitor retry patterns** to identify infrastructure issues
## Disable Retries
To disable automatic retries completely:
```typescript theme={null}
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
maxRetries: 0, // Disable retries
});
```
# Timeout
Source: https://docs.gbox.ai/sdk/timeout
Configure request timeout settings for optimal performance
## Overview
By default, all requests timeout after 60 seconds. You can customize timeout settings globally or per-request.
## Global Configuration
Set default timeout when initializing the SDK:
```typescript Basic Setup theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
timeout: 30 * 1000, // 30 seconds (default is 60 seconds)
});
```
```typescript Longer Timeout theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"],
timeout: 2 * 60 * 1000, // 2 minutes for longer operations
});
```
## Per-Request Configuration
Override timeout for specific operations:
```typescript Quick Operations theme={null}
// Short timeout for fast operations like listing
const boxes = await gboxSDK.listBoxes(
{},
{ timeout: 10 * 1000 } // 10 seconds
);
```
# Typescript SDK
Source: https://docs.gbox.ai/sdk/typescript
# Basic
Source: https://docs.gbox.ai/ui-action/basic
## Overview
**UI actions** are one of GBOX's core features, enabling AI agents to interact with devices just like humans do.
Through simple API calls, you can perform various operations including:
* **Clicking** on specific coordinates or elements
* **Scrolling** through content
* **Typing text** into input fields
* **Dragging** and other gesture-based interactions
> š” **[Explore More UI Actions ā](/api-reference/ui-action/click)**
>
> Discover all available UI operations and advanced features.
## Quick Start Example
Here's a basic UI action example showing how to create an Android box and perform a click operation:
```typescript TypeScript highlight={10-13} icon="https://cdn.worldvectorlogo.com/logos/typescript.svg" theme={null}
import GboxSDK from "gbox-sdk";
const gboxSDK = new GboxSDK({
apiKey: process.env["GBOX_API_KEY"], // This is the default and can be omitted
});
async function main() {
const box = await gboxSDK.create({ type: "android" });
await box.action.click({
x: 100,
y: 100,
});
await box.action.screenshot({
path: "screenshot.png",
});
}
main();
```
```python Python highlight={11} icon="https://cdn.worldvectorlogo.com/logos/python-5.svg" theme={null}
import os
from gbox_sdk import GboxSDK
def main():
api_key = os.getenv("GBOX_API_KEY")
gbox = GboxSDK(api_key=api_key)
box = gbox.create(type="android")
box.action.click(x=100, y=100)
box.action.screenshot(path="screenshot.png")
if __name__ == "__main__":
main()
```
Explore the complete UI Action API reference to discover all available
operations and features.
You can also see the registered devices in the [GBOX Dashboard](https://gbox.ai/dashboard):
> **Note**: You may see the newly registered device status is "Offliine", this maybe caused by some initialize work still in progress. Waitfor a few minutes and it should be changed to "Online". Or try to register the device again.
## Troubleshooting
### Connection Issues
* Ensure USB debugging is enabled
* Try different USB cables
* Restart both device and Mac if needed
### Performance Note
> **Note**: Due to our servers being located in the United States, users in other locations may experience some latency during operations.
## Next Steps
Once your device is successfully registered, you can:
* Run automated tests
* Execute UI actions
* Debug applications
* Perform device automation tasks
For more information on using your registered device, refer to the [Android Basic Guide](/android/basic) and [Real Device Testing](/android/real-device) documentation.
# Concepts
Source: https://docs.gbox.ai/concepts
This conceptual guide introduces the two core abstractions in GBOX: **Devices** and **Boxes**
## Device
A **Device** is the underlying hardware resource (cloud virtual/physical Android devices, registered local Android device, Linux desktop/sandbox).
* Devices are the foundation for creating boxes.
* A device can only host **one active box at a time**.
## Box
A **Box** is an operable environment created from a device.
* Think of it as a running session on that device.
* Each device can create multiple boxes **over time**, but you must terminate the current one before starting another.
* Boxes expose the APIs and SDK actions your AI agents use to interact with the device environment.
# Android MCP Server
Source: https://docs.gbox.ai/docs-mcp/android-mcp-server
> MCP server exposing GBOX Android control tools via Model Context Protocol
[](https://www.npmjs.com/package/@gbox.ai/mcp-server) [](https://github.com/babelcloud/gbox/blob/HEAD/LICENSE)
## Description
This package provides an MCP (Model Context Protocol) server for controlling Android devices via GBOX tools. It exposes a set of tools and APIs for automation, device management, and integration with the GBOX ecosystem.
## Usage
Use GBOX CLI to configure GBOX Android MCP Server for Cursor and Claude Code.
```bash theme={null}
#Using GBOX CLI to configure GBOX Android MCP Server for Cursor
gbox mcp export --merge-to cursor
#Using GBOX CLI to configure GBOX Android MCP Server for Claude Code
gbox mcp export --merge-to claude-code --scope user
```
Copy the following configuration into your Cursor or Claude code MCP config file:
> `GBOX_API_KEY` needs to be obtained through [GBOX Dashboard](https://gbox.ai/dashboard).
```json theme={null}
"gbox-android": {
"command": "npx",
"args": [
"-y",
"@gbox.ai/mcp-server@latest"
],
// NOTE: You can omit the 'env' section if you have successfully run 'gbox login' in cli.
"env": {
"GBOX_API_KEY": "gbox_xxxx",
}
}
```
For detailed instructions on connecting IDE to GBOX MCP, please see the following: [Cursor Guide](/integrations/ide/cursor), [Claude Code Guide](/integrations/ide/claude-code) and [VSCode Guide](/integrations/ide/vscode).
If you need the agents to control your local
Android devices, please check [https://docs.gbox.ai/cli/register-local-device](https://docs.gbox.ai/cli/register-local-device).
# Docs MCP Server
Source: https://docs.gbox.ai/docs-mcp/docs-mcp-server
This MCP server provides local access to `GBOX documentation`, enabling seamless AI-powered interactions with your product. Once installed, AI assistants can directly query and reference GBOX documentation to help you build, debug, and optimize your applications.
## Installation
Install the MCP server by running the following command in your terminal.
```bash theme={null}
npx mint-mcp add docs.gbox.ai
```
# Introduction
Source: https://docs.gbox.ai/index
GBOX provides environments for AI Agents to operate computer and mobile devices.
# Environments
GBOX supports four types of environments: Mobile, Browser, Desktop and Sandbox.
Then you can **give Claude Code instructions** using **natural language**, for example:
By default, GBOX MCP uses a cloud android VM. To connect GBOX to your own Android devices, see [Register Local Devices](https://docs.gbox.ai/cli/register-local-device).
Demo
This rule file contains predefined instructions for the AI agent to effectively test Android applications using GBOX.
3. **Assign testing tasks** to the AI agent:
> **Tip:** The default VNC password is `osworld-public-evaluation`. You can access the VNC viewer URL from the GBOX dashboard or API response.
### Step 5: View Results
After the evaluation completes, you can find the results in the `results_gbox` directory. The results include:
* Task execution logs
* Screenshots of key actions
* Performance metrics
* Success/failure status for each task
You can now start building your own agents by modifying the test configuration files or creating custom evaluation scenarios.
## Next Steps
* Explore the [OSWorld documentation](https://github.com/xlang-ai/OSWorld) to learn more about creating custom evaluation tasks
* Check out the [GBOX API reference](/api-reference) for advanced configuration options
* Experiment with different models and parameters to optimize agent performance
* Scale up your evaluations by increasing the `--num_envs` parameter to run multiple environments in parallel
# AgentKit
Source: https://docs.gbox.ai/integrations/platform/agentkit
Develop AI Agents that can browse the web autonomously using GBOX and AgentKit (by inngest).
## Overview
AgentKit is a framework for building AI agents with tools, memory, and autonomous behavior. It provides a structured way to define what an agent can do and how it makes decisions, enabling developers to build flexible, task-oriented agents.
By integrating with GBOX, AgentKit agents gain the ability to **control managed headless browsers**, enabling them to autonomously search the web, extract data, and interact with websites. This makes them well-suited for **real-time information retrieval and robust web automation**.
## Getting Started
)
)
)