Blog

xg.glass SDK

<a
  href="https://github.com/hkust-spark/xg-glass-sdk"
  target="_blank"
  rel="noreferrer"
  className="inline-flex items-center gap-2"
>
  <GitHubIcon className="w-4 h-4" title="GitHub" />
  <span className="underline">hkust-spark/xg-glass-sdk</span>
</a>

# Why Did We Build xg.glass?

The growth of smart glasses depends on a healthy ecosystem and truly **breakthrough applications**.  
However, while new smart-glasses manufacturers keep emerging, developers are facing two major challenges—both of which we have personally experienced:

**1️⃣ Overly complex low-level interfaces**  
If I simply want to design an algorithm or build an application, I often get stuck dealing with Bluetooth, WiFi connections, and other low-level details that are hard to understand and even harder to debug.

**2️⃣ Extremely high cross-platform migration cost**  
An app developed for Rokid usually needs to be almost completely redesigned and reimplemented to run on RayNeo.

xg.glass was created to solve these exact problems.  
We want to turn our hard-earned experience into a shared foundation for the broader developer community.

We strongly believe that:  
> If the smart-glasses industry enters its next growth wave, it will be driven by truly iconic applications.  
> And those applications **cannot be built without the power of the community**.

xg.glass aims to lower the barrier for smart-glasses development and help the community discover the next breakout app.

---

# What Does xg.glass Do?

At their core, smart glasses only require four fundamental interfaces:

- 🎥 Video input (camera)  
- 🎙️ Audio input (microphone)  
- 🖥️ Video output (display)  
- 🔊 Audio output (speaker)

xg.glass exposes **only these four interfaces** to developers.  
All other complexities are hidden inside our SDK.

This means:

- You can build a glasses-based application **as fast as possible**
- You don’t need to worry about Bluetooth, connectivity, or device-specific quirks
- You don’t have to guess which hardware platform will have enough users

Under the hood, xg.glass already wraps SDKs from multiple smart-glasses vendors and device types.

---

# A Simple Example

With just **10 lines of code**, you can build a vision-based translation application.

📦 Source code:  
https://github.com/hkust-spark/xg-glass-app

```kotlin
override suspend fun run(ctx: UniversalAppContext): Result<Unit> {
    val img = ctx.client.capturePhoto().getOrThrow()
    val b64 = Base64.getEncoder().encodeToString(img.jpegBytes)
    val req = chatCompletionRequest {
        model = ModelId("gpt-4o-mini")
        messages { user { content { text("Translate the text in this image to Chinese. Output only the result."); image("data:image/jpeg;base64,$b64") } } }
    }
    val text = openAI.chatCompletion(req).choices.firstOrNull()?.message?.content.orEmpty().ifBlank { "No text" }
    return ctx.client.display(text, DisplayOptions())
}
```

After installing xg.glass, simply run:
```bash
xg-glass run TranslationEntry.kt
```

This will generate an APK automatically.  
Download the APK to your phone, and as long as the glasses are supported by xg.glass, the app will connect and run—regardless of the specific hardware model.

---

# Which Glasses Are Currently Supported?

We currently support selected models from:

- Rokid  
- RayNeo  
- Frame  

We also support a simulator for development and testing that can run on your PC with a webcam.

More devices are coming soon. Please check the GitHub repository for the latest development progress.  
We also warmly welcome community contributions to help us support more brands and devices.

---

# What Does **xg.glass** Mean?

- **XG** stands for **neXt Generation** (next-generation glasses)  

---

If you’re interested in smart glasses, AI applications, or cross-platform development, we’d love for you to join us and help shape what comes next 🚀


Jiayang Xu, Yufan Zhuang, Zijie Li, Jun Zhang, Zili Meng

xg.glass: Develop an AI Glasses Applications in 10 Lines of Code and Easily Deploy onto Different Glasses 

<a
  href="https://github.com/hkust-spark/xg-glass-release"
  target="_blank"
  rel="noreferrer"
  className="inline-flex items-center gap-2"
>
  <GitHubIcon className="w-4 h-4" title="GitHub" />
  <span className="underline">hkust-spark/xg-glass-release</span>
</a>

![Image alt text](/images/glass+gpt.png "")

During a stressful final exam week, an inconspicuous "AI glasses" — smart AI glasses developed based on Rokid Glasses and equipped with OpenAI’s latest GPT-5.2 model released last week — achieved an astonishing score of 92.5 in the undergraduate final exam for Computer Communication Networks at the Hong Kong University of Science and Technology. 
This score successfully placed it among the top five in a class of over 100 students, significantly exceeding the student average of 72. 
Students could simply look down at the exam paper and quickly receive the answers.

This test, led by Professor Jun Zhang and Professor Zili Meng’s team at HKUST, not only verified the deep reasoning capabilities of AI glasses in complex knowledge tasks but also sounded an alarm to the global education community: the era of smart glasses for education has arrived. 
How to better use and regulate this technology will be crucial in the age of AI glasses.

<video
  controls
  playsInline
  className="w-full rounded-lg shadow"
  src="/videos/recording.mp4"
>
  Sorry, your browser doesn't support embedded videos.
</video>

<video
  controls
  playsInline
  className="w-full rounded-lg shadow"
  src="/videos/rokid-demo-short.mp4"
>
  Sorry, your browser doesn't support embedded videos.
</video>

# Research Team's Test Results

The research team’s test results are shocking:
| Test Subject | Score (out of 100) |
| ------------- | ------------------ |
| Rokid Glasses (GPT 5.2) | 92.5 |
| Student Highest Score | 97.5 |
| Student Average Score | 72 |  

**AI Glasses Answering Workflow:**
* Image Capture: The student looks down at the exam paper, and the AI glasses' camera (such as the Rokid Glasses' 12-megapixel camera) quickly takes a photo.
* Remote Inference: The exam photo is transmitted to the remote LLM via the glasses → phone → cloud pipeline.
* Result Return: The answer is inferred by remote LLM, delivered via the reverse route (cloud → phone → glasses), and finally displayed on the glasses for the student to transcribe.
![Image alt text](/images/workflow.png "Workflow")

# Platform Selection

To achieve automatic answering with AI glasses, the glasses must satisfy both camera input and display output. 
A display is preferred over a speaker as it allows the answer to remain visible for a longer time, facilitating transcription.

We evaluated 12 mainstream commercial glasses:
| Company | Brand Model | Camera | Display |
| ------- | ------------ | ------ | ------- |
| Meta | Ray-Ban Gen2 | ✅ | ❌ |
| Meta | Ray-Ban Display | ✅ | ✅ |
| Xingji Meizu | StarV Air2 | ❌ | ✅ |
| Xiaomi | Xiaomi AI Glasses | ✅ | ❌ |
| Based Hardware | Omi | ✅ | ❌ |
| Mentra | Even Realities G1 | ❌ | ✅ |
| Even Realities | Even Realities Vuzix 100 | ❌ | ✅ |
| Gyges Labs | Halliday | ❌ | ✅ |
| Solos | Solos AirGo 3 | ❌ | ❌ |
| OPPO | Air Glass 3 | ❌ | ✅ |
| Brilliant Labs | Frame | ✅ | ✅ |
| Rokid | Rokid Glasses | ✅ | ✅ |

Only the Meta Ray-Ban, Frame, and Rokid smart glasses offer both an onboard camera and an integrated display. 
In addition, the glasses must provide an SDK for follow-on development. 
Although Meta provides a Device Access Toolkit, it does not expose an SDK for controlling the display. 
Therefore, only Frame and Rokid satisfy all three requirements. 
Because Frame’s camera image quality was insufficient, we ultimately selected Rokid Glasses as the test platform.

# Large Model Selection

We evaluated mainstream large models. Although accuracy was similar, GPT-5.2 was chosen because it was significantly more responsive. GPT-5.2 is a powerful professional knowledge model newly released by OpenAI on December 11th of this year.

Here is an example video showing GPT-5.2’s faster response time than Gemini 3 Pro on Poe. GPT-5.2 produces the answer quickly, whereas Gemini 3 Pro takes noticeably longer to respond.
<video
  controls
  playsInline
  className="w-full rounded-lg shadow"
  src="/videos/gpt_vs_gemini.mp4"
>
  Sorry, your browser doesn't support embedded videos.
</video>

# Detailed Performance Evaluation

| Question Type | Score | Full Score |
|---------------|-------|------------|
| Multiple Choice | 29 | 29 |
| Single-Page SAQ | 18 | 18 |
| Multi-Page SAQ | 45.5 | 53 |
| Total Score | 92.5 | 100 |

**Rokid scored full marks on both multiple-choice questions and single-page short-answer questions. Even on cross-page short-answer questions, it still earned most of the points.**
Cross-page SAQs are especially tricky because a single main problem can be split across several pages -- meaning the AI glasses can’t see the entire question at once. 
Sub-questions on later pages may depend on key information from earlier pages, which makes the whole workflow much harder. 
Despite that, Rokid handled these cases with ease.

Overall, the questions Rokid missed were usually the toughest sub-parts within a larger problem. 
They required non-trivial inferential calculation and put serious pressure on the model’s reasoning ability. 
And even when the final answer was wrong, the model still produced a substantial portion of the underlying reasoning steps.

Examples of correct answers:

![Image alt text](/images/correct1.png "")
![Image alt text](/images/correct2.png "")

Examples of wrong answers:

![Image alt text](/images/wrong1.png "")

In the incorrect example above (caused by a knowledge discrepancy), the AI’s answer (111.123.15.254) was technically correct but did not match the simplified model used in the course exam (111.123.15.255). 
his mismatch occurred because the AI applied knowledge beyond what was covered in class.

# Existing Issues and Challenges

Although the Rokid platform proved the software's feasibility, current commercial hardware still has the following shortcomings:
- Image Quality Bottleneck: Camera clarity is the decisive factor for answering performance, as it directly determines whether AI can clearly read the question.
- Power Consumption and Connection Stability: These factors are also critical.
  - Rokid: Enabling Wi-Fi for high-resolution photo transmission significantly increases power consumption. The battery level dropped from 100% to 58% after 30 minutes of answering.
  - Frame: Bluetooth connection stability is poor, especially at low battery levels. It disconnected 4 times within 15 minutes during testing, requiring a manual reconnection each time.

![Image alt text](/images/outage.png "")

# Impact on Education and Regulation

The rise of AI glasses in education is a double-edged sword.
- Positive Impacts: Potential for educational technology innovation, such as personalized learning, providing tailored assistance and resources, reducing learning resource inequality, and helping students needing extra support improve academic performance.
- Negative Impacts: Severe damage to academic integrity when used during exams, calling into question the fairness of the entire educational assessment system. 

Educators and policymakers must act immediately to establish reasonable regulatory mechanisms to ensure technology is used for good:
- Formulate Clear Policies and Regulations: Establish a legal framework that clearly defines the boundaries for smart device use in educational and examination settings. At the same time, enforce mandatory requirements for glasses manufacturers to indicate the presence of a camera through lights or other prompts.
- Technical Monitoring and Detection: Employ signal jamming or specific monitoring software during exams to detect unauthorized use of smart devices.
- Education and Publicity: Advocate for the legal and effective use of smart devices as aids during the learning process, but strictly prohibit their use during exams.

Only by finding a balance between ethics and technology, through comprehensive regulation at the legal, technical, educational, and cultural levels, can academic integrity be protected while truly realizing innovation and breakthroughs in educational technology.


Blog

xg.glass: Develop an AI Glasses Applications in 10 Lines of Code and Easily Deploy onto Different Glasses

Rokid Glasses + GPT-5.2 in a Real-World Test: How AI Glasses Will Impact the Future of Education