Diary | Local Browser AI Experiment

Local AI Workbench: Building a Private AI Side Panel for Chrome

A personal builder diary on Nano Workbench, a local-first Chrome extension experiment for page summaries, documents, OCR, screenshots, voice notes, and private browser memory.

Experimental note. This is a local learning and product exploration page. The repository is public to read and try locally. If the code is opened for reuse, MIT is the preferred default license; diary text and screenshots remain separate.

Chrome tab sourceExtension side panelOn-device AILocal memory and Doctor

I built Nano Workbench to test a specific idea: can Chrome's on-device AI become useful inside the browser without creating a developer backend for the user's page text, prompts, screenshots, notes, or transcripts?

The browser is already where much of the work happens: reading a page, comparing information, saving notes, checking a document, capturing a screen, or extracting text from an image. The side panel keeps the AI surface beside that work.

The hard question is not whether AI can summarize a page. The hard question is whether the tool can show the exact source, explain the browser boundary, and keep local records under user control.

This page records the build as a learning diary: what the extension does, how it can be tried locally, what is hard, and why the product needs diagnostics as much as features.

Build Model

The extension has to answer one question before every response.

What source is being used right now? The answer could be the active page, selected text, an attached file, an image, a screenshot, a voice transcript, local memory, or no page context at all.

Implementation Track

Page

Read the active tab

Use the current tab as the starting point, but make it clear when Chrome blocks the page or when context is empty.

Panel

Work beside the source

Keep the assistant in Chrome's side panel so the answer stays next to the page, document, image, or screenshot being used.

Inputs

Route many source types

Support page text, selected text, PDFs, DOCX, images, screenshots, voice memos, and generic chat without hiding the source.

Doctor

Explain every boundary

Show model readiness, blocked pages, storage state, screenshot status, OCR support, voice state, and conversation evidence.

User-Facing Surfaces

Page and selected text

Summarize the tab, ask questions, or send a highlighted paragraph through the right-click menu.

Files and images

Attach documents, run image OCR when supported, and keep the source type visible.

Screenshots and voice

Capture visible or long pages, record voice notes, and save only transcript text locally.

Doctor and memory

Use local memos, tab sessions, and diagnostics to make the assistant more explainable.

Working Model

The product is a context router before it is an AI assistant.

Nano Workbench is useful only if it can name the source, check the browser boundary, route the selected input, and tell the user what was actually available to the local model.

Open page

A normal http or https tab is the safest starting point.

Select context

Choose page, selection, file, image, screenshot, voice, or chat.

Check readiness

Chrome local model, APIs, page access, and permissions must be ready.

Answer in panel

The result appears beside the source rather than in a detached tool.

Record evidence

Doctor and local memory keep the state reviewable.

Context is not background

The source must be named: page, selected text, file, image, screenshot, voice note, memory, or generic chat.

Local-first must be observable

The product has to show where records live and whether any feature has a browser or service caveat.

Chrome decides some boundaries

Internal pages, model availability, image support, and Web Speech behavior are browser realities the UI must explain.

Doctor is a trust surface

A local tool earns trust when it can say what worked, what failed, and what source was actually used.

Source Modes

The side panel is fed by different browser sources.

This is the part that should be visible in the design. The app is not a single chat box. It is a small browser workbench that routes page text, chosen text, files, images, screenshots, and voice notes through different paths.

Active tab

The basic mode: summarize or ask questions about the current page when Chrome allows the page to be read.

Selected text

Right-click a highlighted paragraph and send only that chosen text into the side panel workflow.

Local files

Attach text, markdown, CSV, JSON, PDF, DOCX, or image files for local reading and summarization.

Image OCR

Try extracting text from an image when Chrome's local image capability is available.

Screenshots

Capture visible pages or full pages, save them locally, and use them as visual work records.

Voice notes

Record a memo or meeting note. Audio is not stored; saved transcript text remains local.

Why This Exists

A private AI layer for the page, document, image, or note already open.

A lot of AI workflows still require copying material into another window. That breaks the flow of reading and working, and it can make the context boundary unclear.

Nano Workbench tries a different path: let the user choose the context inside Chrome, keep the assistant beside the page, and make it clear what is local and what is not available.

The user chooses the context. The browser does the work. The developer should not see the user's content.

What It Does Today

One click opens the side panel, but the source path matters.

01Open a normal tab
02Choose a source
03Check model readiness
04Ask beside the page
05Save local notes
06Use Doctor evidence

The extension supports page summaries, tab questions, local files, image OCR, screenshots, voice notes, memos, history, tab sessions, exports, right-click actions, and Doctor diagnostics.

Animated Pipeline

From browser source to local answer to Doctor evidence.

A click on the extension icon should not hide the path. The page needs to show that Nano Workbench first identifies a source, then checks the local model, then uses helper tools such as OCR, screenshots, Web Speech, storage, and Doctor.

Tab

The extension starts from the active tab, selected text, or the current browser task.

Source

The user chooses page text, file, image, screenshot, voice note, or generic chat.

Model

Chrome on-device AI runs only when the local model and browser APIs are ready.

Tools

OCR, screenshots, downloads, Web Speech, and local parsers handle the extra source types.

Doctor

Doctor records readiness and failure evidence without exposing private page content.

Build History

From active-tab summary to a multi-source browser workbench.

The project kept expanding as each source type introduced a new boundary: page access, file parsing, image support, screenshot capture, voice transcription, local records, and diagnostic proof.

MVP

Make the active tab useful

The first milestone was to open the side panel, read the current page when allowed, and keep the answer beside the source.

Context

Add source routing

The project then separated active tab, selected text, file upload, image OCR, screenshot capture, voice memo, and generic chat.

Trust

Keep collection out

No developer backend, no analytics, no telemetry, no ads, and no account became design constraints, not footer language.

Diagnostics

Make Doctor evidence visible

Doctor became the screen for model readiness, page access, storage, screenshot, OCR, voice, and conversation state.

Publishing

Separate public from reusable

For reusable code, MIT is the practical default. Personal writing, screenshots, and diary material remain separate unless clearly permitted.

Local First Architecture

No developer backend, no analytics, no telemetry.

The architecture is intentionally modest. The extension tries to keep page text, prompts, notes, screenshots, conversations, memos, and transcripts local unless the user deliberately exports or shares something.

Active tab

The page the user is already working with

Side panel

A Chrome workspace beside the page

On-device AI

Chrome local model when available

Documents

PDF, DOCX, text, CSV, JSON, markdown, and images

OCR

Image text extraction when Chrome supports it

Screenshots

Visible and full-page capture experiments

Local memory

Memos, sessions, notes, and work history

Doctor

Evidence for model, page, storage, voice, and capture health

Engineering Insights

The difficult parts are the trust boundaries.

A browser assistant becomes useful only when it handles context, readiness, screenshots, OCR, voice, and local memory without making hidden assumptions.

Context freshness

Focus: The assistant must not silently answer from the wrong tab or stale page text.
Design choice: Show the active tab/source state and explain blocked, internal, empty, ignored, and generic-chat modes.
Learning: The central question is: what source is the model using right now?

On-device readiness

Focus: Chrome built-in AI depends on browser version, local model availability, and feature readiness.
Design choice: Show setup and status screens instead of pretending every browser is ready.
Learning: A local AI product must have a graceful waiting room for model download, availability, and unsupported states.

Screenshot and OCR workflow

Focus: Screenshots and image OCR look simple, but long pages, scroll position, stitching, and OCR quality add complexity.
Design choice: Save screenshots under Downloads/Local AI Workbench/Screenshots and make OCR capability visible.
Learning: Visual context is powerful, but it needs careful product language and failure handling.

Local memory

Focus: Notes, history, memos, and tab sessions become useful only if the user trusts where they live.
Design choice: Use browser-local storage and keep exports intentional.
Learning: Memory is not just storage. It is a trust and control surface.

Actual Working Contract

The app should never leave the user guessing about context.

The working contract is concrete: if the page cannot be read, say so. If the model is not ready, show setup. If the source is a screenshot or image, make the visual path clear. If voice uses Web Speech, explain the caveat. If anything is saved, keep the local record visible.

Show which source is active: page, selection, file, image, screenshot, voice, memory, or chat.
Show whether Chrome local AI is available before asking for trust.
Keep screenshots, exports, memos, and sessions as deliberate user actions.
Use Doctor to make blocked pages, stale context, and weak OCR visible.

Engineering Diary

The browser surface turned edge cases into product decisions.

Chrome internal pages, host permissions, model setup, OCR support, long-page capture, Web Speech behavior, and local storage are not footnotes. They shape the actual experience.

The side panel changed the feeling

The side panel keeps the answer beside the current tab, so the source and the response remain in the same working frame.

Doctor became part of the product

Readiness checks for page access, model state, OCR, screenshots, voice, storage, and conversation state make the tool more trustworthy.

Permissions need explanation

Browser permissions are powerful. The interface and documentation must say why access is needed and when it is used.

Iconography creates orientation

Page, selection, file, image, screenshot, voice, memory, and Doctor need distinct signals so the user knows the current mode quickly.

Broad host access

The extension needs page access to be useful, but that same permission must be explained plainly and used only for user-chosen context.

Long-page screenshots

Full-page capture involves scrolling, splitting, stitching, saving, and checking whether the output is actually complete.

Voice caveat

Voice transcription uses Chrome Web Speech API, which may use a Google speech service depending on the setup. That boundary must be visible.

License default

MIT is the preferred default for reusable code. Diary writing, screenshots, and personal notes should stay separate unless permission is clearly given.

Publishing path

The public store path should be linked only after review is complete. The diary should explain the route without listing internal release details.

Diagnostics without leakage

Evidence should help debugging without exposing private page text, prompts, notes, screenshots, or transcripts.

Data Layer

The local memory layer is part of the product.

The app stores local memos, history, tab sessions, notes, and diagnostic evidence in browser-local storage. That is useful only if the user can understand what is saved and when export is a deliberate choice.

Local browser records

Memos, history, tab sessions, and work notes begin in browser-local storage so the prototype can be tested without a server.

Why structure matters

As documents, OCR, screenshots, voice notes, and sessions grow, the data model becomes part of the product experience.

Export is intentional

The user may export or share selected material, but the default direction is local control, not automatic upload.

Download And Try Locally

Load the GitHub folder as an unpacked Chrome extension.

There is no separate frontend build step for normal local use. The folder that contains manifest.json is the folder Chrome loads.

Get the repository

Clone the public repository or download the ZIP from GitHub.

Open Chrome extensions

Go to chrome://extensions and turn on Developer mode.

Load unpacked

Select the folder that contains manifest.json. There is no separate frontend build step for normal local use.

Open a normal page

Use a regular http or https page, then click the extension icon to open the side panel.

Git Option

git clone https://github.com/Najariya/nano-workbench.git
cd nano-workbench

ZIP Option

Open GitHub, choose Code, download the ZIP, unzip it, and then load the extracted folder through Chrome Developer mode.

Open GitHub repository

Google Chrome 138 or newer
Desktop Chrome with built-in on-device AI support
Chrome on-device AI model enabled or downloaded
Developer mode for local unpacked install
A normal web page, not a Chrome internal page

Permissions

Powerful browser access should be explained plainly.

The extension uses permissions such as side panel, storage, scripting, tabs, active tab, downloads, context menus, tab groups, optional history, and broad host access. These permissions are not decorative. They decide whether the workbench can see the user selected page, save local notes, add right-click actions, capture screenshots, and open beside the current page.

The important design rule is that access should follow the user selected context and should be explained in ordinary language.

sidePanel
storage
scripting
tabs
activeTab
downloads
contextMenus
tabGroups
optional history
<all_urls>

What To Try First

A practical path from install to confidence.

01Summarize an article already open in Chrome
02Ask a follow-up question about that tab
03Right-click selected text and ask Local AI
04Try OCR on an image
05Capture a visible screenshot
06Try a full-page screenshot on a long page

Attach a PDF or DOCX and ask for a simple summary
Record a voice memo and check where the transcript is stored
Open Doctor before trusting a complex result

Future Roadmap

From page helper toward a private browser assistant.

The direction is not to make a dramatic claim. The direction is to make the current browser task more understandable, more private, and more useful, one layer at a time.

Phase 1

Private page workbench

Make page summary, tab questions, selected text, and generic chat reliable and easy to understand.

Phase 2

Richer context

Improve documents, images, screenshots, OCR, voice memos, and meeting notes with clearer status and safer defaults.

Phase 3

Local work memory

Turn browser-local notes, sessions, and history into a useful memory layer that remains under user control.

Phase 4

Personal browser assistant

Move toward a refined, private assistant for the browser that can explain context, actions, and boundaries.

Add safe approved screenshots
Add Chrome Web Store link after public review
Add MIT license to the source repo when reuse is intended
Improve Doctor evidence
Refine full-page screenshot checks
Improve OCR feedback
Clarify Web Speech boundary
Consider structured local database
Keep privacy claims testable

Current Boundary

Public to learn from, careful about what it promises.

The current working candidate is being validated locally and through Doctor evidence. The public store path should be added only after review is complete, without turning this diary into a release tracker.

For the source code, MIT is the preferred default license when reuse is intended. The writing, screenshots, and personal diary material remain separate unless permission is clearly stated.

Public repo: https://github.com/Najariya/nano-workbench
Current candidate: Current working candidate, validated locally and documented through Doctor evidence
Publishing path: Public store link will be added after review; internal release details are not shown on this diary page
License default: MIT is the preferred default for reusable code; diary text and screenshots remain separate
Privacy posture: No developer backend, analytics, telemetry, ads, or account layer in the prototype

This page is a record of a browser experiment: a side panel that tries to keep AI close to the user's chosen source, keep records local, and make failure states visible instead of hiding them behind a polished answer.

Follow the Nano Workbench build on GitHub