# Input

> Buttons, touch gestures, keyboards, and controllers such as the Meta Neural Band.

Source: https://docs.glassly.ai/app-devs/core-concepts/input/

`session.input` delivers the physical controls on the glasses and paired
controllers: button presses, touchpad gestures, keys, and the Meta Neural Band's
air mouse. The starter scaffold wires a button press to the display, so a
short press shows a text wall.

```typescript src/background/index.ts
import { registerMiniapp } from "@glassly/miniapp/background";

registerMiniapp((session) => {
  session.input.onButtonPress((press) => {
    session.display.render([{type: "text", id: "msg", box: {x: 0, y: 0, w: 576, h: 288}, text: `${press.buttonId}: ${press.pressType}`}]);
  });
});
```

Button events only reach glasses that have a button. Declare a `BUTTON` hardware
requirement in your [manifest](https://docs.glassly.ai/app-devs/core-concepts/miniapp-manifest.md) to gate
the miniapp to those glasses.

## Button presses

`onButtonPress` fires on every press of a physical button:

```typescript
session.input.onButtonPress((press) => {
  if (press.pressType === "long") session.display.render([]);
});
```

Each `ButtonPressData` has:

| Field | Type | Notes |
|---|---|---|
| `buttonId` | `string` | Which button fired. |
| `pressType` | `"short" \| "long"` | Press duration. |

## Keyboard

Even G2 glasses running Glassly CFW revision 39 or later can host a Bluetooth
keyboard themselves (the user pairs it from the Glassly App under
**Controllers**). `onKey` delivers every key transition to every subscribed
miniapp, the same way button presses are broadcast:

```typescript
session.input.onKey((e) => {
  if (e.phase !== "down") return;
  if (e.text) type(e.text);            // printable characters, layout applied
  else if (e.key === "Enter") submit();
  else if (e.key === "c" && e.modifiers.ctrl) cancel();
});
```

Each `KeyboardEventData` has:

| Field | Type | Notes |
|---|---|---|
| `phase` | `"down" \| "up"` | Press or release. |
| `key` | `string` | Logical key under the host layout (US today): `"a"`, `"A"`, `"Enter"`, `"ArrowUp"`, `"F5"`, `"AudioVolumeUp"`. |
| `code` | `string` | Physical key: `"KeyA"`, `"Digit1"`, `"Space"`, `"ControlLeft"`. |
| `modifiers` | `{shift, ctrl, alt, meta}` | Modifier state at the time of the event. |
| `text` | `string?` | The character a plain press types. Down and repeat only; absent for Ctrl/Alt/Meta chords and named keys. |
| `repeat` | `boolean` | Synthesized auto-repeat while the key stays held. |
| `timestamp` | `number` | Phone receipt time in ms. |

A dropped link releases every held key with matching `up` events, so a
modifier can never stay stuck. On glasses without a keyboard bridge the
subscription never fires.

The Meta Neural Band's handwriting arrives through `onKey` too: each change to
the written text comes as `down`/`up` pairs that type it, with `Backspace` for
corrections and `Enter` for new lines. The band only listens for writing while a
miniapp has a text field open, so tell the host when yours is on the glasses:

```typescript
session.input.setTextFieldOpen(true).catch(() => {});  // search box on screen
session.input.setTextFieldOpen(false).catch(() => {}); // and gone
```

The host closes the field when your miniapp stops. Wearers can turn handwriting
off in the Glassly App's Neural Band settings; older Glassly App versions reject
the request, hence the `catch`.

## Touch gestures

`onTouch` subscribes to touchpad gestures. Pass a handler for every gesture, a
single gesture name to filter, or an array of names for one subscription across
several gestures.

```typescript
// Every touch gesture.
session.input.onTouch((touch) => {
  session.display.render([{type: "text", id: "msg", box: {x: 0, y: 0, w: 576, h: 288}, text: touch.kind}]);
});

// One gesture.
session.input.onTouch("single_tap", () => {
  session.display.render([{type: "text", id: "msg", box: {x: 0, y: 0, w: 576, h: 288}, text: "Tap"}]);
});

// Several gestures, one subscription.
session.input.onTouch(["swipe_up", "swipe_down"], (touch) => {
  // touch.kind tells you which one.
});
```

Each `TouchData` has:

| Field | Type | Notes |
|---|---|---|
| `kind` | `string` | The gesture. See the values below. |
| `index` | `number?` | On the Meta Neural Band's pinch gestures (`single_tap`, `double_tap`, `long_press`) and their `touch_down` / `tap_start` / `tap_end` edges: the finger pinching the thumb, `0` index, `1` middle. On the IINE Gamebrick's D-pad (`touch_down`, its swipe, `tap_end`): `12`-`15` for up, down, left, right. Absent elsewhere. |
| `controllerId` | `string?` | The controller that made the gesture (an R1 ring or a Neural Band), stable while it is paired. Several controllers can be connected at once; use it to tell them apart. Absent for glasses input, and for ring input the G2 relays when the phone can't tell which ring it came from. |
| `input2d` | `true?` | The gesture came from a controller that swipes in two dimensions (the Meta Neural Band). Absent for the temples, the R1 ring and the phone's mirror controls. |
| `distance` | `number?` | On `scroll_up` / `scroll_down`: travel since the previous step, out of the R1 ring's 200-unit touch span. |
| `speed` | `number?` | On `scroll_up` / `scroll_down`: the ring's smoothed slide speed, 0-255. |

The gesture values the runtime reports:

| `kind` | Gesture |
|---|---|
| `"single_tap"` | A single tap on the touchpad. |
| `"double_tap"` | Two quick taps. |
| `"triple_tap"` | Three quick taps. |
| `"long_press"` | A press and hold. |
| `"swipe_up"` | A swipe up. |
| `"swipe_down"` | A swipe down. |
| `"swipe_left"` / `"swipe_right"` | The Meta Neural Band's thumb swipes, a watch trackpad, and the IINE Gamebrick's D-pad. |
| `"tap_start"` | Press start on hardware that reports it. On the R1/G2 and Neural Band, a held press: use with `tap_end` for hold-to-talk. The band emits it once after a 400 ms pinch, and the IINE Gamebrick's A button once after a 400 ms hold; quick taps have `touch_down` / `tap_end` edges without `tap_start`. The Apple Watch sends it on every finger-down. |
| `"tap_end"` | The raw finger-up edge. Brackets every gesture on hardware that reports it. |
| `"tap_then_hold"` | The G2 2.2.9+ tap-then-hold (Menu) gesture. |
| `"touch_down"` | A controller's finger-down: the R1 ring's (forwarded by the Glassly CFW, GLASSLYCFW/37+), the Meta Neural Band's pinch, with `index` for the finger, and the IINE Gamebrick's A button and D-pad. The temples report `tap_start`. |
| `"dial_up"` / `"dial_down"` | One step of the Meta Neural Band's pinch-and-turn dial. |
| `"scroll_up"` / `"scroll_down"` | One step of travel while an R1 ring slide is in progress, for smooth scrolling. See [Smooth scrolling on the ring](#smooth-scrolling-on-the-ring). |
| `"head_up"` / `"head_down"` | The wearer lifted their head past the head-up angle, or brought it back down. See [Head position](#head-position). |

Glasses can send other `kind` strings, so `kind` is typed as `string`. Match the
values you handle and ignore the rest.

### Smooth scrolling on the ring

The R1 ring sends one `swipe_up` / `swipe_down` per press, however far the
finger slides, like the temples. Reversing before the finger lifts sends no
second swipe. To scroll with the finger, subscribe to `scroll_up` /
`scroll_down` instead: one arrives for every step of travel during the slide,
the first included (it comes with the swipe), carrying `distance` and `speed`.
The direction follows the kind. Claiming a swipe does not claim its scroll
steps, so claim `scroll_up` / `scroll_down` too when only one app should see them.

```typescript
session.input.onTouch(["scroll_up", "scroll_down"], ({kind, distance = 0}) => {
  list.scrollBy(kind === "scroll_up" ? -distance : distance);
});
```

### One- and two-dimensional swipes

The G2 temples and the R1 ring swipe along one axis (`swipe_up` / `swipe_down`).
While any controller that swipes in two dimensions is connected (the Meta Neural
Band), `session.capabilities.button.buttons` includes an entry with
`type: "swipe2d"`, and `swipe_left` / `swipe_right` arrive too.

The temples and the ring keep working while the band is connected, under the same
`swipe_up` / `swipe_down` names, so decide per gesture: the band's touches carry
`input2d: true`. A game can steer in four absolute directions on the band and
fall back to relative turns on the temples; a menu can walk every item on a 1D
swipe and move by row or column on a 2D one:

```typescript
session.input.onTouch(["swipe_up", "swipe_down"], (touch) => {
  if (touch.input2d) {
    // The band: absolute up and down.
  } else {
    // The temples or the ring: step forward and back.
  }
});

// Whether a 2D controller is connected at all, for hints and layout.
const has2d = () => session.capabilities?.button?.buttons?.some((b) => b.type === "swipe2d") === true;
session.onCapabilitiesChange(() => {
  // The band connected or disconnected.
});
```

### Head position

Head position rides the touch stream: `head_up` and `head_down` are gestures
like any other, so they go through the [claims](#claiming-gestures). The
dashboard claims both while it runs; claim them yourself to receive them while
it is up. An unclaimed `head_up` on glasses that leave the dashboard to the
phone opens the glasses' native dashboard.
[`session.imu.onHeadPosition`](https://docs.glassly.ai/app-devs/core-concepts/imu.md#head-position) is a
convenience over `onTouch(["head_up", "head_down"], ...)`.

```typescript
await session.input.setClaimedGestures(["head_up", "head_down"], {scope: "page"});
session.input.onTouch(["head_up", "head_down"], (touch) => {
  if (touch.kind === "head_up") showDetails();
  else hideDetails();
});
```

## Air mouse

`onPointer` delivers the Meta Neural Band's air mouse. Nothing arrives unless
the wearer turns on **Air mouse** in the Glassly App's Neural Band settings.

```typescript
session.input.onPointer((e) => {
  if (e.kind === "move") moveCursor(e.dx!, e.dy!);
  else if (e.kind === "buttons") setPressed((e.buttons! & 1) !== 0);
});
```

Each `PointerEventData` has:

| Field | Type | Notes |
|---|---|---|
| `kind` | `"move" \| "buttons"` | Aim change or pinch state change. |
| `dx`, `dy` | `number?` | On `move`: the forearm's aim change in degrees since the last event, right and down positive. |
| `buttons` | `number?` | On `buttons`: the pinch state as a bit mask, `1` index, `2` middle. |
| `timestamp` | `number` | Phone receipt time in ms. |
| `controllerId` | `string` | The band the event came from, when several are connected. |

## Claiming gestures

`setClaimedGestures(gestures, {scope?})` registers the gestures your miniapp
handles. Claims stack: the most recent registration receives the gesture
**alone**, through `onTouch`, and the host skips its default behavior. This
also applies to background miniapps. Subscribing with `onTouch` alone does not
claim a gesture or override another miniapp's claim.

```typescript
const offTouch = session.input.onTouch("double_tap", () => {
  // Handle an in-miniapp back gesture.
});
const offClaim = session.input.onClaimChange(({gesture, held}) => {
  console.log(gesture, held ? "received here" : "held by another miniapp");
});

await session.input.setClaimedGestures(["double_tap"], {scope: "page"});

// When the page closes, restore the previous claimant.
await session.input.setClaimedGestures([], {scope: "page"});
offTouch();
offClaim();
```

Each call replaces the selected scope's set; omitting `scope` uses the default
scope. Re-registering a nonempty set takes priority again. Use separate scopes
for persistent shortcuts and temporary pages so closing a page releases only
its claims. Stopping the miniapp releases every scope. `onClaimChange` reports
`{gesture: string, held: boolean}` to the miniapps gaining and losing a gesture;
a handoff between two scopes of the same miniapp does not fire it.

One host rule runs first: double-tap leaves a miniapp visibly drawing on the
main view if that miniapp has not registered `double_tap` in any scope. Claim it
if you use it while drawing there. Main content covered by a dashboard view
is not visible, so the rule does not exit the miniapp underneath. A main
miniapp that registered double-tap is not exited even if another claimant
currently has priority. The rule covers both of the Neural Band's fingers. The
middle finger's double tap (`index: 1`) skips the host's other double-tap
defaults, and claims match it by name like the index finger's.

## Cleaning up

`onButtonPress`, `onKey`, `onPointer`, and every `onTouch` form return an
unsubscribe function:

```typescript
const off = session.input.onTouch("single_tap", handler);
off(); // stop listening
```

Calling `onTouch` with an array returns one unsubscribe function that drops all of
the gestures it registered.
