BB code

Code: 
[HEADING=1]heading[/HEADING]
[HEADING=2]heading[/HEADING]
[HEADING=3]heading[/HEADING]

[SIZE=1]size[/SIZE]
[SIZE=2]size[/SIZE]
[SIZE=3]size[/SIZE]
[SIZE=4]size[/SIZE]
[SIZE=5]size[/SIZE]
[SIZE=6]size[/SIZE]
[SIZE=7]size[/SIZE]
 
Search to see if any forum contributors or tech journalists or bloggers are especially interested un gui scraping

Yes—there are people interested in precisely your “robot seated at a computer” approach. The best matches I found are practitioners in visual automation and OCR communities, plus a technical blogger who closely follows screenshot-driven computer use. They are more directly relevant than most articles marketed as “visual web scraping,” which can mean a graphical interface for an ordinary HTML scraper.[1][2][3]

People worth following​


1. ulrich on the Ui.Vision forum — the closest match for hands-on screen scraping​


This contributor explicitly says, “I use OCR screen scraping often,” and provides concrete advice about reading text from the screen. For example:

• In “Using !OCRScale=true”, they explain how scaling affects recognition of small text and numbers, including cases where scaling makes recognition worse. August 2020.[2]

• In “How to Scrape Pop-Up”, they test another user’s screenshot and recommend a different OCR engine to correctly recognize the text. May 2020.[4]

• Search results also show contributions on dynamic-table scraping and a difficult date-picker control, the latter in July 2024. Those demonstrate broader UI-automation experience, although not every thread uses purely visual extraction.[5][6]

Why this is useful: this is someone discussing the actual mechanics of extracting information from displayed pixels—not just writing generic scraping explainers. The strongest directly relevant posts I found are older, so I would not assume current availability or Instagram-specific experience.

2. Ken Soh, GitHub handle kensoh — detailed visual-automation implementations​


Ken Soh’s contributions to TagUI include an unusually explicit explanation of the difference between webpage extraction and screen-based extraction.

In “TagUI for Desktop Applications → use visual automation”, he explains that TagUI uses image recognition to operate desktop applications and provides examples involving Outlook, a PDF viewer, and Word. He also distinguishes:

• Reading a webpage’s text.

• Using visual recognition and OCR to read the whole screen into a variable.

• Saving that recognized text to a file.[1]

The thread contains practical discussion of image-based clicking, typing, visual checks, and Sikuli integration. It dates from 2018, so it establishes substantial involvement in the technique—not necessarily his current focus.[1]

Why this is useful: his examples closely match your architecture: find something on the screen, interact with it, read the resulting screen, and continue.

3. Simon Willison — the strongest independent blogger match​


Willison maintains a dedicated computer-use topic archive, covering models that operate computers through GUIs, including screenshot interpretation and coordinate-based clicking. That recurring coverage is stronger evidence of sustained interest than a single article.[7]

His October 2024 hands-on exploration of Anthropic’s computer use describes the precise loop you have in mind: send screenshots, receive proposed mouse/keyboard actions, execute them, and repeat.[3]

He also published an October 2025 correction after initially attributing CAPTCHA handling to a computer-use model when the browser infrastructure had actually performed it. That is particularly relevant to your investigation because it illustrates why “the agent did this visually” claims need careful checking.[8]

Why this is useful: follow him for independent technical evaluation of the “software robot” approach. He is not, on the evidence I found, an Instagram GUI-scraping specialist.

Communities and blogs with a relevant focus​


PlaceWhy it fitsQualification
Ui.Vision community forumDedicated discussion of OCR, visual desktop/browser automation, and screen scraping. This is the strongest community lead for practical implementation questions. [9]Ui.Vision combines multiple automation methods; check whether a particular answer uses pixels/OCR or DOM selectors.
Olostep: “Screen Scraping: Web Scraping, APIs, and AI Agents” — April 2026Explicitly distinguishes reading rendered pixels through OCR/computer vision from extracting HTML, DOM, or network data. Very close to your terminology. [10]Vendor-authored explanatory material, not independent evidence of production reliability.
Padiso: “Computer Use for Legacy ERP Screen Scraping…” — May 2026Describes a screenshot → interpretation → mouse/keyboard action → new screenshot loop for extracting data from legacy software. [11]Business/ERP context, not Instagram; a company blog rather than an independent practitioner account.
Mosco.ai: “Why we don’t use Computer Use” — April 2026A skeptical discussion of screen-driven agents, unexpected dialogs, session timeouts, layout changes, and per-action latency. [12]The company advocates its API-first approach. Its numerical reliability claims should not be treated as independent benchmarks.

Where I would start for your specific idea​


For actual screenshot/OCR extraction, start with ulrich and the Ui.Vision forum. For concrete image-driven workflow examples, read Ken Soh’s TagUI discussion. For ongoing independent coverage of newer AI “robot at the computer” systems, follow Simon Willison’s computer-use archive.[1][2][7]

I did not find a comparably strong, identifiable journalist or forum contributor specifically documenting long-running Instagram scraping through exclusively screen-based mouse/keyboard automation. The matches above are credible leads for the underlying technique, rather than evidence that someone has already validated your exact Instagram workflow.

Citations:

[1] TagUI for Desktop Applications --> use visual automation · Issue #113 github.com/aisingapore/TagUI/issues/113

[2] Using !OCRScale=true - #3 by ulrich - General Discussion forum.ui.vision/t/using-ocrscale-true/5914/3

[3] Initial explorations of Anthropic's new Computer Use ... simonwillison.net/2024/Oct/22/computer-use/

[4] How to Scrape Pop-Up - #7 by ulrich - General Discussion forum.ui.vision/t/how-to-scrape-pop-up/5319/7

[5] Dynamic Table Scraping - #15 by ulrich forum.ui.vision/t/dynamic-table-scraping/6037/15

[6] Need help for a tricky day control - #12 by ulrich - HowTo forum.ui.vision/t/need-help-for-a-tricky-day-control/15317/12

[7] Simon Willison on computer-use simonwillison.net/tags/computer-use/

[8] Retraction simonwillison.net/2025/Oct/7/gemini-25-computer-use-captchas/

[9] Categories - Ui.Vision, AI & OCR Community Forums forum.ui.vision/categories

[10] Screen Scraping: Web Scraping, APIs, and AI Agents - Olostep www.olostep.com/blog/screen-scraping

[11] Computer Use for Legacy ERP Screen Scraping in PE Carve-Outs www.padiso.co/blog/computer-use-legacy-erp-screen-scraping-pe-carve-outs/

[12] Why we don't use Computer Use - Mosco.ai mosco.ai/blog/why-we-dont-use-computer-use

[13] Our Open-Source RPA Blog - UI.Vision ui.vision/blog/blog/page3/

[14] 2026 Open-Source RPA Software/Web Automation/Computer Use ... ui.vision/scraping

[15] Screen Scraping - New Media and Technology Law Blog newmedialaw.proskauer.com/category/screen-scraping/page/2/

[16] A browser/GUI tool that you can select what to scrape, and ... www.reddit.com/r/webscraping/comments/1en9gkp/a_browsergui_tool_that_you_can_select_what_to/

[17] Best AI web scraping tools I've tried recently (and what I ... - Reddit www.reddit.com/r/automation/comments/1uf6g6v/best_ai_web_scraping_tools_ive_tried_recently_and/

[18] I built Supacrawler, an lightweight Go service for web ... www.reddit.com/r/opensource/comments/1ng57ol/i_built_supacrawler_an_lightweight_go_service_for/

[19] Blog | link.sc link.sc/blog/

[20] AI Agent Workflow Automation Patterns: Why Your Workflows Still Suck coasty.ai/blog/ai-agent-workflow-automation-patterns-why-youre-failing

[21] I built a screenshot as a service API to capture ... www.reddit.com/r/SideProject/comments/1q14hzp/i_built_a_screenshot_as_a_service_api_to_capture/

[22] WinTuner GUI Tool : r/Intune www.reddit.com/r/Intune/comments/1ryxfpx/wintuner_gui_tool/

[23] Ui Vision RPA Open Source Automation Tool for Browser and Desktop openresource.eu.com/uivision/

[24] Archive for Friday, 25th October 2024 simonwillison.net/2024/Oct/25/

[25] From Prompt Injection to C2 with Claude Computer Use simonwillison.net/2024/Oct/25/zombais/

[26] Scraping URL from Salesforce Button - #6 by ulrich forum.ui.vision/t/scraping-url-from-salesforce-button/8352/6

[27] How to Scrape Pop-Up - #9 by ulrich - General Discussion forum.ui.vision/t/how-to-scrape-pop-up/5319/9

[28] How to Scrape Pop-Up - #2 by ulrich - General Discussion forum.ui.vision/t/how-to-scrape-pop-up/5319/2

[29] How to search and add pictures - #2 by ulrich - HowTo forum.ui.vision/t/how-to-search-and-add-pictures/7037/2

[30] OCRExtract Variable - #5 by ulrich - General Discussion forum.ui.vision/t/ocrextract-variable/1871/5
 
Last edited:
You must log in or register to reply here.
Top