Computer Use

Use Vibengine sandboxes to give AI agents the ability to interact with desktop environments, browsers, and graphical applications in a secure, isolated setting.

Overview

Computer-use agents need a visual environment to operate in. Vibengine sandboxes provide a full Linux environment where you can run headless browsers, take screenshots, and automate UI interactions — all without affecting your host machine.

Running Headless Browsers

Puppeteer (JavaScript)

Create a sandbox with a browser pre-installed and control it programmatically.

import { Sandbox } from "vibengine"

const sandbox = await Sandbox.create({ template: "browser-tools" })

// Install Puppeteer inside the sandbox
await sandbox.runCode(`
  const puppeteer = require("puppeteer");
  const browser = await puppeteer.launch({ headless: true, args: ["--no-sandbox"] });
  const page = await browser.newPage();
  await page.goto("https://example.com");
  const title = await page.title();
  console.log("Page title:", title);
  await browser.close();
`)

Playwright (Python)

from vibengine import Sandbox

sandbox = Sandbox.create(template="browser-tools")

sandbox.run_code("""
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.goto("https://example.com")
    print("Page title:", page.title())
    browser.close()
""")

Taking Screenshots

Screenshots are essential for computer-use agents — they allow the LLM to "see" the current state of the screen.

import { Sandbox } from "vibengine"

const sandbox = await Sandbox.create({ template: "browser-tools" })

const result = await sandbox.runCode(`
  const puppeteer = require("puppeteer");
  const fs = require("fs");
  const browser = await puppeteer.launch({ headless: true, args: ["--no-sandbox"] });
  const page = await browser.newPage();
  await page.setViewport({ width: 1280, height: 720 });
  await page.goto("https://vibengine.ai");
  await page.screenshot({ path: "/tmp/screenshot.png" });
  console.log("Screenshot saved");
  await browser.close();
`)

// Download the screenshot from the sandbox
const screenshotBytes = await sandbox.downloadFile("/tmp/screenshot.png")

from vibengine import Sandbox

sandbox = Sandbox.create(template="browser-tools")

sandbox.run_code("""
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    page = browser.new_page()
    page.set_viewport_size({"width": 1280, "height": 720})
    page.goto("https://vibengine.ai")
    page.screenshot(path="/tmp/screenshot.png")
    print("Screenshot saved")
    browser.close()
""")

screenshot_bytes = sandbox.download_file("/tmp/screenshot.png")

Creating a Browser Tools Template

Pre-install all browser dependencies in a custom template so sandboxes start instantly with everything ready.

# vibengine.Dockerfile
FROM vibengine/base:latest

RUN apt-get update && apt-get install -y \
    chromium-browser \
    fonts-liberation \
    libappindicator3-1 \
    libasound2 \
    libatk-bridge2.0-0 \
    libdrm2 \
    libgbm1 \
    libnspr4 \
    libnss3 \
    libxdamage1 \
    xdg-utils

RUN npm install -g puppeteer
RUN pip install playwright && playwright install chromium

Build and deploy the template:

ve template build --name browser-tools --dockerfile vibengine.Dockerfile

Agent Loop Example

A complete computer-use agent that browses the web and completes a task.

import { Sandbox } from "vibengine"

async function computerUseAgent(task, llm) {
  const sandbox = await Sandbox.create({
    template: "browser-tools",
    timeoutMs: 10 * 60 * 1000,
  })

  let screenshot = await takeScreenshot(sandbox, "https://example.com")

  for (let step = 0; step < 10; step++) {
    const action = await llm.decide(task, screenshot)

    if (action.type === "done") break

    await sandbox.runCode(action.code)
    screenshot = await takeScreenshot(sandbox)
  }

  await sandbox.kill()
}

from vibengine import Sandbox

def computer_use_agent(task: str, llm):
    sandbox = Sandbox.create(template="browser-tools", timeout=600)

    screenshot = take_screenshot(sandbox, "https://example.com")

    for step in range(10):
        action = llm.decide(task, screenshot)

        if action["type"] == "done":
            break

        sandbox.run_code(action["code"])
        screenshot = take_screenshot(sandbox)

    sandbox.kill()

Security — Always use sandboxes for computer-use tasks. Never let an AI agent control a browser on your host machine directly, as it could access sensitive data or perform unintended actions.

Computer Use

On this page