macotron.ax

Use the macOS accessibility API to read the selected text in any app, find buttons and fields in the front app, press them, and change their values.

macotron.command("Shout selection", "Upper-case the selected text", async () => {
  const text = await macotron.ax.selectedText();
  if (!text) return macotron.notify.toast("Shout", "Nothing selected");
  macotron.clipboard.set(text.toUpperCase());
});

Every call needs Accessibility permission. Declare permissions: ["accessibility"] in macotron.plugin(). Without it, calls return null, [], or false.

Elements

Calls that find elements return plain objects:

Field	Meaning
`id`	A handle to pass to `children`, `parent`, `press`, and `setValue`.
`role`	The accessibility role, such as `"AXButton"` or `"AXTextField"`.
`title`	The element's title, or `""`.
`value`	The element's value as a string, or `""`. Numbers, such as a slider's position, are turned into strings.
`frame`	`{ x, y, width, height }` in screen points, with the origin at the top-left of the main display.

Each call hands out new ids, even for an element you have seen before. Ids stop working when plugins reload, so do not store them.

Reading

selectedText(): Promise<string | null>

The text selected in the focused element of any app. Resolves to null when nothing is selected or the app does not expose its selection. It runs off the main thread, so other plugins keep running while it waits.

Chrome, Edge, and Electron apps turn on accessibility for web pages only when asked. The first time it meets such an app, selectedText() asks, then waits up to 300 ms for the page to answer.

focused(): Element | null

The element with keyboard focus in any app, such as the text field you are typing in, or null.

children(id: number): Element[]

The element's children, in order. Returns [] for an unknown id.

parent(id: number): Element | null

The element that contains it, or null at the top or for an unknown id.

find(opts: { role?: string, title?: string }): Element | null

Searches the frontmost app, depth first, and returns the first element that matches every field you give. role ignores case and the AX prefix, so "button" matches "AXButton". title must match the whole title, ignoring case. Returns null when nothing matches or when you give neither field.

The search reads the whole tree until it finds a match, which can take a while in a large app.

Acting

press(id: number): boolean

Presses the element, as a click would. Returns false for an unknown id or when the element cannot be pressed.

setValue(id: number, value: string): boolean

Sets the element's value, such as the text in a field. Returns false for an unknown id or when the app refuses.

const ok = macotron.ax.find({ role: "button", title: "OK" });
if (ok) macotron.ax.press(ok.id);