xg.glass SDK

Contributor Guide

How to extend the xg.glass SDK (new devices, new APIs, build tooling).

Scope

This guide is for people who want to extend the SDK itself:

  • add a new glasses model
  • add a new capability/API to the unified Kotlin surface
  • improve build tooling (RayNeo host generation, Flutter embedding, templates)

If you just want to build an app using the SDK, use the Developer Guide.

The SDK repository also keeps a detailed contributor checklist in CONTRIBUTING.md.

Repository layout

The SDK repo is organized to keep public API separate from vendor adapters and tooling:

  • kotlin/: shared SDK modules (core, core-android, app-contract, universal) plus internal scaffolding (universal-full, app).
    • :core: the public Kotlin API surface (GlassesClient, models, events, audio primitives). No vendor SDK dependencies.
    • :app-contract: the “universal app” contract used by host apps to load app logic (important for RayNeo’s two-host model).
    • :universal: the published Android aggregate for Maven consumers.
  • devices/: vendor-specific adapters implementing core:
    • devices/device-rokid/ → Gradle module :device-rokid
    • devices/device-frame-flutter/ → :device-frame-flutter (Kotlin ↔ Flutter contract, no Flutter dependency)
    • devices/device-frame-embedded/ → :device-frame-embedded (SDK-owned FlutterEngine + bridge; only included when the Flutter module exists)
    • devices/device-meta/ → :device-meta (optional Android artifact; requires Meta GitHub Packages access)
    • devices/device-omi/ and devices/device-omi-ios/ → Omi adapters
    • devices/device-rayneo-installer/ → :device-rayneo-installer (phone-side installer via ADB-over-TCP)
    • devices/device-rayneo-runtime/ → :device-rayneo-runtime (on-glasses runtime implementation)
    • devices/device-even/ → Even Realities G1 Android/iOS adapters
    • devices/device-inmo-runtime/ → INMO Air3 on-glasses runtime adapter
  • third_party/:
    • third_party/frame/frame_module/: no-UI Flutter module template for Frame (Dart + Lua assets)
    • third_party/rayneo/aar/: RayNeo vendor AAR drop-in folder (Mercury + IPC SDK AARs)
  • build-logic/: Gradle plugins (included via includeBuild("build-logic") in settings.gradle.kts)
    • com.xgglass.rayneo.settings: generates + includes the on-glasses RayNeo host module
    • com.xgglass.rayneo.app: builds the host APK and copies it into phone app assets
  • templates/: project templates used by xg-glass init (currently templates/kotlin-app)
  • tools/ + xg-glass: the repo-local CLI used to generate and run developer projects

Note: Gradle module names are kept stable (:core, :app-contract, :device-rokid, etc.), while shared implementations live under kotlin/ and device implementations live under devices/. See settings.gradle.kts.

Local development workflow

For the end-to-end “run an app” workflow, we intentionally reuse the app-developer docs:

Contributor-specific notes:

  • The SDK repo root ships ./gradlew for repo-level verification.
  • Use pip install -e tools/ or the repo-root ./xg-glass launcher for local CLI development. The published CLI package is xg-glass on PyPI for app developers.
  • Flutter is required today because Frame is embedded via a Flutter module.

Adding a new glasses model

1) Extend the model enum and capabilities

Update :core:

  • add a new entry in kotlin/core/src/commonMain/kotlin/.../Models.kt (GlassesModel)
  • add/update capability flags in DeviceCapabilities if needed

2) Add a new device adapter module

Create a new Android library module under devices/, e.g.:

  • devices/device-<vendor>/

Then wire it in:

  • settings.gradle.kts: include(":device-<vendor>")
  • settings.gradle.kts: map its projectDir:
    • project(":device-<vendor>").projectDir = file("devices/device-<vendor>")

Implement GlassesClient:

  • connect()/disconnect() should be idempotent
  • return Result<T> with actionable errors
  • set capabilities correctly (and enforce them)

3) Export it from the universal entry point

Update kotlin/universal/ so third-party apps can depend on a single artifact and still get the new device implementation.

If the new device should be selectable in the default host UI, update:

  • templates/kotlin-app/app/.../MainActivity.kt

Adding a new API / capability

Follow this order:

1) Add it to core

Update:

  • kotlin/core/.../GlassesClient.kt: add a new method (or extend an existing options model)
  • kotlin/core/.../Models.kt / kotlin/core/.../Audio.kt: add models/options/chunk formats if needed
  • DeviceCapabilities: add a capability flag if this is not universally supported

2) Implement per-device

  • Rokid: implement in devices/device-rokid
  • RayNeo: decide where it belongs
    • installer (phone-side) should focus on deployment/connectivity
    • runtime (on-glasses) should implement operations requiring glasses-side permissions (camera, sensors, etc.)
  • Frame: update both the Kotlin contract and the Flutter module (see below)

3) Keep backward compatibility

  • avoid breaking existing signatures
  • use additive options with safe defaults

Frame-specific: Flutter module + bridge

Frame integration is split into:

  • Kotlin contract (no Flutter dependency): devices/device-frame-flutter/.../FrameFlutterChannelContract.kt
  • Embedded runtime (depends on Flutter): devices/device-frame-embedded/.../EmbeddedFrameFlutterBridge.kt
  • Flutter module template:
    • Dart: third_party/frame/frame_module/lib/universal_frame_bridge.dart
    • Lua assets: third_party/frame/frame_module/assets/lua/*

The MethodChannel name is:

  • xgglass/frame/methods

To add a new operation for Frame:

  1. Add/extend method names + args in FrameFlutterChannelContract
  2. Implement in Dart (universal_frame_bridge.dart)
  3. If needed, update the Lua app logic (assets/lua/ug_frame_app.lua and helpers)
  4. Update the embedded bridge to route events/data back to Kotlin

RayNeo-specific: two-process design + build tooling

RayNeo contributions usually fall into one of three places:

  • Installer (phone-side): devices/device-rayneo-installer/
    • deploys an APK onto glasses via ADB-over-TCP
    • focus: reliable install flows, reachability, clear logs on failure
  • Runtime (on-glasses): devices/device-rayneo-runtime/
    • implements capturePhoto() / display() inside the glasses app process
    • focus: runtime permissions, performance, correct threading
  • Build tooling: build-logic/.../rayneo/
    • Settings plugin generates the on-glasses host module (default :xgglass_rayneo_glass_host)
    • Project plugin builds the host APK and copies it into phone app assets (default name rayneo_glass_app.apk)

If you change the RayNeo host template, bump:

  • RayneoHostTemplate.TEMPLATE_VERSION in build-logic/.../rayneo/TemplateFiles.kt

Vendor AARs (Mercury + IPC SDK) are expected under:

  • third_party/rayneo/aar/ (or configured via xgRayneo.mercuryAarDir)

Code style (high level)

  • use Kotlin coroutines (suspend, Flow, StateFlow) for async and events
  • keep vendor callbacks behind suspend wrappers (suspendCancellableCoroutine)
  • keep error messages actionable (what failed, where, and what to try next)