[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-94717":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":18,"rankGlobal":10,"rankLanguage":10,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":41,"readmeContent":42,"aiSummary":43,"trendingCount":15,"starSnapshotCount":15,"syncStatus":44,"lastSyncTime":45,"discoverSource":46},94717,"dsh-vision-toolkit","Anionex\u002Fdsh-vision-toolkit","Anionex","让纯文本模型更好地做视觉任务的DeepSeek Harness插件：带意图的图片问答、长截图 OCR、UI 还原等｜DeepSeek Harness-native integration for agent-vision-toolkit: image Q&A, long-screenshot OCR, UI restoration, grounding, pixel diff, Artifacts, and Web UI.","",null,"TypeScript",733,32,1,0,122,617,78.56,"MIT License",false,"main",true,[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40],"agent-skills","agent-vision-toolkit","computer-vision","deepseek","deepseek-harness","dsh","dsh-plugin","gui-automation","ocr","plugin","python","screenshot-testing","text-only-llm","typescript","ui-restoration","vision-language-model","vision-tools","2026-08-25 04:01:22","\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fhero-v2.png\" alt=\"DSH Vision Toolkit helps text-only DeepSeek Harness agents understand images and complete visual tasks\" \u002F>\n\u003C\u002Fp>\n\n\u003Cdiv align=\"center\">\n\n# DSH Vision Toolkit\n\n[![Recommended by dshfind](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Frecommended%20by-dshfind-FFD700?style=flat-square)](https:\u002F\u002Fdshfind.com\u002Fen\u002Fplugins\u002FAnionex\u002Fdsh-vision-toolkit)\n[![npm](https:\u002F\u002Fimg.shields.io\u002Fnpm\u002Fv\u002F@anionex\u002Fdsh-vision-toolkit?style=flat-square&color=5B4CF0)](https:\u002F\u002Fwww.npmjs.com\u002Fpackage\u002F@anionex\u002Fdsh-vision-toolkit)\n[![MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-0B7285?style=flat-square)](LICENSE)\n[![DSH](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDSH-Web%20%2B%20Headless-5B4CF0?style=flat-square)](cordis.patch.yml)\n\n**Give text-only DSH agents eyes: paste an image, ask a question, locate exact elements, extract assets, and verify UI restoration with measurable results.**\n\n🌐 **English** | [中文](README.zh.md)\n\n\u003C\u002Fdiv>\n\nWhen you run DeepSeek or another text-only model in DeepSeek Harness (DSH), familiar problems appear quickly: the model cannot see a screenshot, generic image descriptions miss the point, buttons have no usable coordinates, and a rebuilt page may look “close enough” without any way to measure the remaining difference.\n\nDSH Vision Toolkit packages [`agent-vision-toolkit`](https:\u002F\u002Fgithub.com\u002FAnionex\u002Fagent-vision-toolkit) as a native DSH plugin. It helps an agent do more than describe an image: the agent can read, locate, crop, trace, rebuild, and verify visual work around the task at hand.\n\n> **Install and use it immediately.** The default setup includes a free Gemma 4 vision service and requires no API key. Cropping, pixel diffing, color analysis, foreground extraction, SVG tracing, and HTML screenshots run locally without spending vision API requests.\n\n```sh\ndsh plugin --profile web add @anionex\u002Fdsh-vision-toolkit\n```\n\n**Upstream toolkit:** [Anionex\u002Fagent-vision-toolkit](https:\u002F\u002Fgithub.com\u002FAnionex\u002Fagent-vision-toolkit) · **Project website:** [agent-vision.anionex.me](https:\u002F\u002Fagent-vision.anionex.me)\n\n\u003Cdetails>\n\u003Csummary>\u003Cstrong>Table of contents\u003C\u002Fstrong>\u003C\u002Fsummary>\n\n- [Recent updates](#recent-updates)\n- [Problems it solves](#problems-it-solves)\n- [See it in action](#see-it-in-action)\n- [Highlights](#highlights)\n- [Quick start: three steps](#quick-start-three-steps)\n- [Common workflows](#common-workflows)\n- [Toolbox](#toolbox)\n- [Configuration and limits](#configuration-and-limits)\n- [Troubleshooting](#troubleshooting)\n- [Development and community](#development-and-community)\n\n\u003C\u002Fdetails>\n\n## Recent updates\n\n- **2026-08-16 · Windows Python:** Added Microsoft Store Python support, fixing first-time isolated-runtime setup failures for affected Windows users.\n- **2026-08-16 · Better free vision:** Switched the default model to Gemma 4, improving the no-key image-understanding path.\n- **2026-08-16 · Image paste:** Text-only routes now switch to a `(Vision Toolkit)` variant and keep a workspace path, fixing blocked pastes and images that could not be reused later.\n- **2026-08-16 · Higher free quotas:** Raised per-client, global, and burst limits to `100\u002Fday`, `400\u002Fday`, and `20\u002Fminute`, reducing avoidable rate-limit failures while the shared capacity is lightly used.\n- **2026-08-16 · Real model test:** Added a full image-request test in Settings, fixing the false confidence caused by a successful `\u002Fmodels` request to a model that still cannot process images.\n\n## Problems it solves\n\n| The problem | What Vision Toolkit delivers |\n|---|---|\n| **A text-only model cannot see a screenshot** | Paste an image in DSH Web; the plugin obtains visual evidence and returns the task-relevant parts to the text model |\n| **The description is long but misses the point** | Ask “Where is the error?” or “What color is the submit button?” and receive an answer focused on that question |\n| **The model knows an element exists but cannot act on it** | Get original-image pixel coordinates and an optional labeled or numbered preview |\n| **Long-screenshot OCR skips or duplicates lines** | Split and audit the image while preserving Markdown, chunks, manifests, and resumable run state |\n| **UI restoration is judged by feel** | Compare the reference and implementation screenshots to get a difference percentage, ranked regions, a heatmap, and JSON |\n| **Screenshot assets cannot be reused** | Produce a crop, transparent PNG, color palette, or editable SVG instead of stopping at prose |\n\n## See it in action\n\n### Paste an image directly into DSH\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fdsh-view-example.png\" width=\"82%\" alt=\"A text-only DeepSeek model answering a question about a pasted image through Vision Toolkit in DSH Web\" \u002F>\n\u003C\u002Fp>\n\n*Paste an image into the conversation. A text-only model can switch to its `Vision Toolkit` variant and inspect the image in the context of the user's question.*\n\n### Screenshot to editable page\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fupstream\u002Finfographic-reference.webp\" width=\"49%\" alt=\"Reference infographic screenshot used for restoration\" \u002F>\n  \u003Cimg src=\"assets\u002Fupstream\u002Finfographic-result.webp\" width=\"49%\" alt=\"Editable HTML and CSS reconstruction created from the reference screenshot\" \u002F>\n\u003C\u002Fp>\n\n*Left: the reference screenshot. Right: an editable HTML\u002FCSS result. The result can continue into screenshot rendering and pixel comparison instead of ending as an image description.*\n\n### Sketch to working interface\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fupstream\u002Fui-sketch.webp\" width=\"49%\" alt=\"Hand-drawn JupyterLab interface used as the restoration reference\" \u002F>\n  \u003Cimg src=\"assets\u002Fupstream\u002Fui-result.webp\" width=\"49%\" alt=\"Working JupyterLab-style interface reconstructed from the sketch\" \u002F>\n\u003C\u002Fp>\n\n*Left: a hand-drawn reference. Right: the working interface reconstructed from it.*\n\n### Turn “looks close” into a verifiable result\n\nThe repository includes a reproducible UI-restoration example. The first implementation differs from the reference by **6.04%**. After the highlighted regions are corrected, the final `1200 × 720` render reaches **0% pixel difference**.\n\n\u003Cp>\n  \u003Cimg src=\"examples\u002Fui-restoration\u002Fassets\u002Finitial.png\" width=\"49%\" alt=\"Initial UI implementation with measurable layout and styling differences\" \u002F>\n  \u003Cimg src=\"examples\u002Fui-restoration\u002Fassets\u002Fimplementation.png\" width=\"49%\" alt=\"Final UI implementation after visual diagnosis, reaching zero pixel difference\" \u002F>\n\u003C\u002Fp>\n\n## Highlights\n\n- **Free by default.** New installations use the built-in Gemma 4 service without requiring another account or API key.\n- **Focused on the current task.** The agent sends the reason it needs to inspect the image, so the result emphasizes useful evidence instead of producing a generic caption.\n- **Outputs you can keep working with.** Coordinates, OCR, transparent PNGs, SVGs, screenshots, heatmaps, and JSON can feed directly into the next step.\n- **Built for UI and screenshot work.** Reference analysis, element location, asset extraction, HTML rendering, and pixel comparison form one continuous workflow.\n- **Local where possible.** Crop, trace, pixel diff, color, foreground, and HTML screenshot operations do not need a remote vision model.\n- **The same capabilities in Web and Headless.** Web users can preview and download artifacts; Headless runs still receive replayable structured results and workspace paths.\n\n## Quick start: three steps\n\n### 1. Install\n\n```sh\ndsh plugin --profile web add @anionex\u002Fdsh-vision-toolkit\n```\n\nYou can install it into a Headless Profile too:\n\n```sh\ndsh plugin --profile headless add @anionex\u002Fdsh-vision-toolkit\n```\n\n### 2. Restart and check it\n\nRestart a running Web Profile, then open **Settings → Vision Toolkit**. The free provider is already configured; run **Test vision model** to confirm it is reachable.\n\nThe first start prepares an isolated runtime, so it needs access to the Python package cache or the network. A normal installation does not require an `agent-vision-toolkit` source checkout or a local path setting.\n\n### 3. Paste an image and describe the outcome you want\n\nPaste a screenshot into the conversation or place an image in the session workspace, then invoke `\u002Fvision-tools`. For example:\n\n```text\nInspect this screenshot. Explain the error and tell me what to fix first.\nFind the login button in the top-right corner, return original pixel coordinates, and make a boxed preview.\nCrop this icon and convert it to SVG.\nRebuild the page from reference.png. After each pass, render it and run a pixel diff until the major differences are gone.\n```\n\n## Common workflows\n\n| Task | Recommended workflow |\n|---|---|\n| Image Q&A or screenshot debugging | Inspect → answer around the current question → locate details when needed |\n| Find a button, icon, or text region | Ground the target → return pixel box → create a labeled preview |\n| Extract an icon from a screenshot | Ground → crop → trace to SVG |\n| Read a long webpage screenshot | Split → OCR → merge Markdown → audit boundaries |\n| Recreate a page or component | Reference → implementation → HTML screenshot → pixel diff → iterate |\n| Extract brand visuals | Crop region → analyze dominant colors → extract foreground → export transparent PNG |\n\n## Toolbox\n\nThe plugin provides 10 tools that can be called independently or composed into a workflow:\n\n| Tool | Best question to ask | Main result |\n|---|---|---|\n| `vision_glance` | “What is happening in this image?” | Focused answer, description, OCR, or multi-image comparison |\n| `vision_ground` | “Where is the thing I need?” | Original pixel coordinates and optional boxed preview |\n| `vision_detect` | “Which buttons, icons, or elements are present?” | Numbered element inventory, coordinates, and optional preview |\n| `vision_crop` | “Extract this region as its own image” | PNG or JPEG crop |\n| `vision_trace` | “Turn this shape into an editable vector” | SVG |\n| `vision_pixel_diff` | “Where does the implementation differ from the reference?” | Difference percentage, ranked regions, heatmap, and JSON |\n| `vision_long_screenshot_ocr` | “Read this entire long screenshot” | Markdown, chunks, manifest, and audit output |\n| `vision_extract_foreground` | “Remove the background from this subject” | Transparent PNG |\n| `vision_dominant_colors` | “Which colors dominate this area?” | Palette or ranked candidate colors |\n| `vision_html_screenshot` | “Render this local page at an exact viewport or capture the full page” | PNG and optional CSS `pageHeight` |\n\nCoordinates always use original-image pixels in `x1,y1,x2,y2` form, so grounding output can feed directly into cropping, tracing, or later automation.\n\nFor a long HTML document, pass `fullPage=true`. The requested width and height remain the layout viewport, while the resulting PNG covers the complete document and reports `pageHeight` in CSS pixels.\n\n## How it works\n\nThe plugin keeps image understanding and deterministic local image processing in one Agent workflow. Expand the flow below for the implementation boundary.\n\n\u003Cdetails>\n\u003Csummary>\u003Cstrong>Architecture and image-input behavior\u003C\u002Fstrong>\u003C\u002Fsummary>\n\n```mermaid\nflowchart LR\n    Image[\"Screenshot or local HTML\"] --> Skill[\"vision-tools Skill\"]\n    Skill --> Agent[\"Text agent selects a task\"]\n    Agent --> Vision[\"Use a vision model when image understanding is needed\"]\n    Agent --> Local[\"Run crop, SVG, and pixel work locally\"]\n    Vision --> Result[\"Answer, OCR, coordinates\"]\n    Local --> Artifact[\"PNG, SVG, heatmap, JSON\"]\n    Result --> Session[\"Continue reasoning and acting\"]\n    Artifact --> Session\n```\n\nThe visual capabilities come from a packaged, pinned `agent-vision-toolkit` snapshot. The DSH plugin handles installation, session-scoped tool exposure, Credentials, path checks, cancellation, timeouts, result files, and Web presentation. The runtime never fetches upstream `main` in the background.\n\nFor routes that DSH positively identifies as text-only, the plugin registers a sibling `\u003Cmodel> (Vision Toolkit)` variant. By default, pasting an image in DSH Web switches to that variant and gives the model both a reusable workspace path and a visual description focused on the current task.\n\n\u003C\u002Fdetails>\n\n## Configuration and limits\n\n### Built-in free service\n\nThe default setup uses:\n\n```text\nBase URL: https:\u002F\u002Fvision.anionex.me\u002Fv1\nModel:    gemma-4-26b-a4b-it\nAPI Key:  no user configuration required\n```\n\nThis is a shared zero-configuration entry point, not an unlimited private endpoint. Current limits are:\n\n| Limit | Current value |\n|---|---:|\n| Per client | 100 requests per UTC day |\n| Whole service | 400 requests per UTC day |\n| Burst | 20 requests per 60 seconds |\n| Image size | 4 MiB per image |\n| Decoded pixels | 20,000,000 per image |\n| Output | 512 tokens per request |\n\nThe limits protect shared capacity and prevent unusually large images from monopolizing memory or request time. When a limit is reached, the service returns a readable reason and error code. Rate-limit responses also include `Retry-After` instead of collapsing into an unexplained model failure.\n\n### Bring your own vision model\n\nFor higher quotas, private endpoints, or another model, change the provider in **Settings → Vision Toolkit** and store the API key as a DSH Credential. Settings stores the Credential reference and never reads the saved secret back into the browser.\n\nYou can also configure a Profile patch:\n\n```yaml\n- id: vision-toolkit\n  config:\n    provider:\n      baseUrl: https:\u002F\u002Fapi.example.com\u002Fv1\n      credential: MY_VISION_KEY\n      model: your-vision-model\n      protocol: openai\n```\n\nOpenAI Chat Completions-compatible endpoints and Anthropic Messages are supported. The Web Settings panel exposes the full provider, runtime, timeout, image-limit, and image-input-variant configuration.\n\n### Requirements\n\n- A DeepSeek Harness Web or Headless Profile.\n- Node.js `^22.19.0` or `>=24.0.0`.\n- Python 3.11+; the plugin prepares an isolated environment by default.\n- Only `vision_html_screenshot` requires Chrome, Chromium, or Edge.\n- Inputs must be PNG, JPEG, GIF, or WebP files in the session workspace or an explicitly allowed directory.\n\n\u003Cdetails>\n\u003Csummary>\u003Cstrong>Install, upgrade, disable, and uninstall\u003C\u002Fstrong>\u003C\u002Fsummary>\n\n```sh\ndsh plugin --profile web update @anionex\u002Fdsh-vision-toolkit\ndsh plugin --profile web remove @anionex\u002Fdsh-vision-toolkit\n```\n\nIf you are migrating from the retired `@dsh-external\u002Fdsh-vision-toolkit`, remove the old package first and install `@anionex\u002Fdsh-vision-toolkit`.\n\nTo disable the bundle temporarily, set this in the Profile patch:\n\n```yaml\n- id: vision-toolkit\n  disabled: true\n```\n\nRestart the Web Profile and refresh the page after enabling or upgrading the Web plugin.\n\n\u003C\u002Fdetails>\n\n### Plugin updates\n\nIn **Settings → Vision Toolkit**, **Check for updates** queries the Profile's npm registry. For a direct registry installation, **Update and restart** installs only the exact version you confirmed, verifies it, and restarts an explicitly opted-in POSIX Web process on a fixed `--port`. Local\u002Fworkspace\u002Ffile\u002Fgit\u002FURL installs, Windows, dynamic ports, read-only Profiles, and manager-owned processes remain check-only.\n\nThe updater revalidates the Profile before mutation, snapshots the original manifest and lockfile, and holds a token-owned cross-process lock. The current Web process exits only after the restart helper confirms that the backup is readable and the lock handoff succeeded. When the Profile was already operational, the replacement must report both the target plugin version and a ready runtime; failed replacements restore the original manifest\u002Flockfile and rebuild dependencies with a frozen lockfile before retrying the previous exact version. If automatic recovery itself fails, the backup and lock are preserved and their paths are written to `$DSH_HOME\u002Flogs\u002Fvision-toolkit-restart.log`. Detached restart requires `DSH_VISION_TOOLKIT_ALLOW_DETACHED_RESTART=1`; unsaved Settings or API-key input blocks installation.\n\n## Troubleshooting\n\n| Problem | What to do |\n|---|---|\n| Pasting an image still says the model does not support image input | Restart the Web Profile, refresh the page, and confirm the selected route has the `(Vision Toolkit)` suffix. You can also place the image in the session workspace and invoke `\u002Fvision-tools` |\n| The free service returns 429 | Wait for the `Retry-After` interval, or switch to your own endpoint when you need stable higher volume |\n| The image exceeds a size or pixel limit | Crop or resize it first; the error identifies whether bytes or decoded pixels caused the rejection |\n| A custom Credential is missing | Enter the API key in **Settings → Vision Toolkit** and confirm the Credential name matches the provider configuration |\n| First-time runtime setup fails | Check Python 3.11+, network or package-cache access, and disk permissions, then retry the model test in Settings |\n| Chrome is not found | Install Chrome, Chromium, or Edge. Only HTML screenshot rendering is unavailable; the other tools still work |\n| An artifact cannot be previewed | Use **Open file** or the workspace path in the result. Preview URLs exist only while the Web route is available |\n\n## Project status and limitations\n\nThe current release focuses on screenshot understanding, visual grounding, OCR, asset extraction, UI restoration, and pixel-level verification. It is not a video, audio, or camera-input system and does not automatically click GUI controls. Interactive box editing, remote service clusters, model voting, and cross-session visual caches are also outside the current scope.\n\n## Development and community\n\n```sh\npnpm install --frozen-lockfile --trust-lockfile\npnpm run verify:portable\npnpm run build\npnpm test\nTSX_TSCONFIG_PATH=tsconfig.json pnpm dlx tsx scripts\u002Fui-restoration-example.ts --check\n```\n\n- Read [CONTRIBUTING.md](CONTRIBUTING.md) before contributing.\n- Use [GitHub Issues](https:\u002F\u002Fgithub.com\u002FAnionex\u002Fdsh-vision-toolkit\u002Fissues) for bugs, focused feature requests, and usage questions; see [SUPPORT.md](SUPPORT.md) for channel guidance.\n- Report vulnerabilities privately through [SECURITY.md](SECURITY.md).\n- See [CHANGELOG.md](CHANGELOG.md) for releases and [FUNDING.md](FUNDING.md) for sponsorship details.\n- Visit upstream [agent-vision-toolkit](https:\u002F\u002Fgithub.com\u002FAnionex\u002Fagent-vision-toolkit) for the general toolkit, cross-agent integrations, and visual-task playbooks.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fcommunity-group-qr.png\" alt=\"QR code for the agent-vision-toolkit community group\" width=\"240\" \u002F>\n\u003C\u002Fp>\n\n[`agent-vision-toolkit`](https:\u002F\u002Fgithub.com\u002FAnionex\u002Fagent-vision-toolkit) was created by [Anionex](https:\u002F\u002Fanionex.me\u002F). This repository maintains its native DeepSeek Harness integration.\n\n## License\n\nThe plugin is available under the [MIT License](LICENSE). The packaged upstream snapshot retains its original MIT license in [`vendor\u002Fagent-vision-toolkit\u002FLICENSE`](vendor\u002Fagent-vision-toolkit\u002FLICENSE).\n","DSH Vision Toolkit 是一个为 DeepSeek Harness（DSH）设计的 TypeScript 插件，旨在赋能纯文本大模型执行视觉任务。它支持带意图的图片问答、长截图 OCR、UI 界面还原、像素级差异比对、SVG 矢量化追踪、资产提取与网页 UI 验证等核心能力；依托本地运行的 Gemma-4 视觉模型及零 API 密钥的免费方案，所有图像处理（如裁剪、前景提取、HTML 截图）均可离线完成。适用于自动化测试、GUI 自动化、无障碍辅助、前端回归验证及低资源视觉代理开发等场景。",2,"2026-08-14 02:30:08","CREATED_QUERY"]