xg.glass SDK

Contributor Guide

How to extend the xg.glass SDK (new devices, new APIs, build tooling).

Scope

This guide is for people who want to extend the SDK itself:

  • add a new glasses model
  • add a new capability/API to the unified Kotlin surface
  • improve build tooling (RayNeo host generation, Flutter embedding, templates)

If you just want to build an app using the SDK, use the Developer Guide.

Repository layout

The SDK repo is organized to keep public API separate from vendor adapters and tooling:

  • core/: the public Kotlin API surface (GlassesClient, models, events, audio primitives). No vendor SDK dependencies.
  • app-contract/: the “universal app” contract used by host apps to load app logic (important for RayNeo’s two-host model).
  • devices/: vendor-specific adapters implementing core:
    • devices/device-rokid/ → Gradle module :device-rokid
    • devices/device-frame-flutter/ → :device-frame-flutter (Kotlin ↔ Flutter contract, no Flutter dependency)
    • devices/device-frame-embedded/ → :device-frame-embedded (SDK-owned FlutterEngine + bridge; only included when the Flutter module exists)
    • devices/device-rayneo-installer/ → :device-rayneo-installer (phone-side installer via ADB-over-TCP)
    • devices/device-rayneo-runtime/ → :device-rayneo-runtime (on-glasses runtime implementation)
  • universal/: single entry-point module that re-exports core + devices (third-party apps ideally depend on this).
  • third_party/:
    • third_party/frame/frame_module/: no-UI Flutter module template for Frame (Dart + Lua assets)
    • third_party/rayneo/aar/: RayNeo vendor AAR drop-in folder (Mercury + IPC SDK AARs)
  • build-logic/: Gradle plugins (included via includeBuild("build-logic") in settings.gradle.kts)
    • com.universalglasses.rayneo.settings: generates + includes the on-glasses RayNeo host module
    • com.universalglasses.rayneo.app: builds the host APK and copies it into phone app assets
  • templates/: project templates used by xg-glass init (currently templates/kotlin-app)
  • tools/ + xg-glass: the repo-local CLI used to generate and run developer projects

Note: module names are kept stable (:device-rokid, etc.), while implementations live under devices/. See settings.gradle.kts.

Local development workflow

For the end-to-end “run an app” workflow, we intentionally reuse the app-developer docs:

Contributor-specific notes:

  • The SDK repo root does not ship a Gradle wrapper; use xg-glass init to generate a host project (it includes ./gradlew) when you want to validate changes end-to-end.
  • Flutter is required today because Frame is embedded via a Flutter module.

Adding a new glasses model

1) Extend the model enum and capabilities

Update core:

  • add a new entry in core/src/main/java/.../Models.kt (GlassesModel)
  • add/update capability flags in DeviceCapabilities if needed

2) Add a new device adapter module

Create a new Android library module under devices/, e.g.:

  • devices/device-<vendor>/

Then wire it in:

  • settings.gradle.kts: include(":device-<vendor>")
  • settings.gradle.kts: map its projectDir:
    • project(":device-<vendor>").projectDir = file("devices/device-<vendor>")

Implement GlassesClient:

  • connect()/disconnect() should be idempotent
  • return Result<T> with actionable errors
  • set capabilities correctly (and enforce them)

3) Export it from the universal entry point

Update universal/ so third-party apps can depend on a single artifact and still get the new device implementation.

If the new device should be selectable in the default host UI, update:

  • templates/kotlin-app/app/.../MainActivity.kt

Adding a new API / capability

Follow this order:

1) Add it to core

Update:

  • core/.../GlassesClient.kt: add a new method (or extend an existing options model)
  • core/.../Models.kt / core/.../Audio.kt: add models/options/chunk formats if needed
  • DeviceCapabilities: add a capability flag if this is not universally supported

2) Implement per-device

  • Rokid: implement in devices/device-rokid
  • RayNeo: decide where it belongs
    • installer (phone-side) should focus on deployment/connectivity
    • runtime (on-glasses) should implement operations requiring glasses-side permissions (camera, sensors, etc.)
  • Frame: update both the Kotlin contract and the Flutter module (see below)

3) Keep backward compatibility

  • avoid breaking existing signatures
  • use additive options with safe defaults

Frame-specific: Flutter module + bridge

Frame integration is split into:

  • Kotlin contract (no Flutter dependency): devices/device-frame-flutter/.../FrameFlutterChannelContract.kt
  • Embedded runtime (depends on Flutter): devices/device-frame-embedded/.../EmbeddedFrameFlutterBridge.kt
  • Flutter module template:
    • Dart: third_party/frame/frame_module/lib/universal_frame_bridge.dart
    • Lua assets: third_party/frame/frame_module/assets/lua/*

The MethodChannel name is:

  • universal_glasses/frame/methods

To add a new operation for Frame:

  1. Add/extend method names + args in FrameFlutterChannelContract
  2. Implement in Dart (universal_frame_bridge.dart)
  3. If needed, update the Lua app logic (assets/lua/ug_frame_app.lua and helpers)
  4. Update the embedded bridge to route events/data back to Kotlin

RayNeo-specific: two-process design + build tooling

RayNeo contributions usually fall into one of three places:

  • Installer (phone-side): devices/device-rayneo-installer/
    • deploys an APK onto glasses via ADB-over-TCP
    • focus: reliable install flows, reachability, clear logs on failure
  • Runtime (on-glasses): devices/device-rayneo-runtime/
    • implements capturePhoto() / display() inside the glasses app process
    • focus: runtime permissions, performance, correct threading
  • Build tooling: build-logic/.../rayneo/
    • Settings plugin generates the on-glasses host module (default :ug_rayneo_glass_host)
    • Project plugin builds the host APK and copies it into phone app assets (default name rayneo_glass_app.apk)

If you change the RayNeo host template, bump:

  • RayneoHostTemplate.TEMPLATE_VERSION in build-logic/.../rayneo/TemplateFiles.kt

Vendor AARs (Mercury + IPC SDK) are expected under:

  • third_party/rayneo/aar/ (or configured via ugRayneo.mercuryAarDir)

Code style (high level)

  • use Kotlin coroutines (suspend, Flow, StateFlow) for async and events
  • keep vendor callbacks behind suspend wrappers (suspendCancellableCoroutine)
  • keep error messages actionable (what failed, where, and what to try next)