macotron.ocr

Read the text in an image with Apple's Vision framework. Recognition runs on the Mac; nothing leaves it.

macotron.plugin({
  title: "OCR",
  description: "Drag a box on screen and copy the text inside it.",
  permissions: ["screenRecording"],
});

macotron.command("OCR Selection", "Copy the text in a dragged box", async () => {
  const image = await macotron.screen.capture({ selection: true });
  if (!image) return;
  const text = await macotron.ocr.recognize({ image });
  if (!text) return macotron.notify.toast("No text found");
  macotron.clipboard.set(text);
});

Recognition

recognize(opts: { path?: string; image?: string }): Promise<string>

Resolves with the recognized text, one line per line of text found, joined with "\n". Resolves with "" when the image has no text. It uses Vision's accurate mode with language correction.

Pass one of:

• path: an image file. A leading ~ expands to the home folder.
• image: image data as base64, or as a data:image/…;base64,… URL. The PNG from macotron.screen.capture() works as is.

When both are given, path wins. The call throws a TypeError right away when neither is usable, and the promise rejects when Vision cannot read the image.

See also

• macotron.screen to capture the screen to read.
• macotron.qr to read QR codes instead of text.
• The ocr.js plugin.