SecondSight: How Does a Browser Finding Become a Source-Code Fix?
Follow an icon button from an accessibility finding to a proposed patch. Learn why mapping, editing, browser verification, and pull-request submission are separate milestones.
The browser knows the element. The developer needs the file.
Imagine a shopping-cart button that shows only an icon. It looks clear to a sighted reader, but its accessible name is missing. An automated audit can identify the rendered button. The developer still has to find its component, choose the right wording, change the source, and check the result.
SecondSight explores that whole journey. The intended loop is audit → map → patch → verify → prepare a pull request, coordinated with LangGraph. It combines browser-audit tooling, Gemini-assisted mapping and editing, and Git operations.
The important distinction is between the intended loop and the demo paths in the published revision. A Playwright/axe runner exists, but the graph's audit node currently supplies fixed demonstration violations. Known issues also have deterministic mappings and patches. We'll use one of those examples to understand the boundaries, without treating the demo as a measured audit of arbitrary websites.
Follow the icon button
From unnamed button to reviewable change
Choose a step to see what happens and why it matters.
The graph currently supplies a fixed button-name violation. A separate Playwright and axe runner exists for live browser audits, but is not wired into that node.
The graph makes the sequence explicit. Amber supplies the observation, indigo locates the source, teal proposes a change, and slate carries review and submission state. Verification can route back to coding. A reliable workflow needs that decision to reflect actual evidence.
Why a selector isn't a source location
A browser selector describes a rendered element. A React source file describes how elements are produced. One component may render many buttons, and a button may be assembled from several components.
SecondSight's mapper uses three approaches. Known demo violations have fixed component mappings. A data-component attribute can suggest a component name. For other cases, Gemini receives the violation details and proposes a likely file and line range.
That ordering is useful: use a deterministic match when the application provides one, then consider a model for less direct cases. However, a plausible file path still needs to be checked against the repository. The attribute-based demo helper constructs a path without checking that it exists.
A stable component marker makes a known target easier to map. Inference is more flexible for uninstrumented applications, but uncertainty moves into file selection. A confidence value is a claim to validate, rather than a substitute for opening the file.
The smallest useful change needs context
For the demo button, the prepared patch adds aria-label="Add to cart". The intended improvement is concrete: expose a meaningful name for an icon-only control.
Compare two illustrative names:
| Name | What it tells the reader |
|---|---|
| “Button” | The kind of control, without its purpose |
| “Add to cart” | The action the control performs |
Choosing the second name requires knowing what the button does. The presence of an aria-label alone doesn't establish that its wording is correct.
The coding node has fixed patches for the known demo issues. Its model-assisted path reads the mapped file and asks Gemini to change exactly one violation, preserve exports, avoid unrelated refactoring, and return the full patched file. Those constraints aim to keep review manageable.
Returning a complete file also increases the importance of comparing it with the original. A one-attribute fix shouldn't silently remove another handler, change a component's inputs, or replace unrelated content.
What would count as verification?
There are at least three questions to ask after generating a patch:
- Does the source still build and preserve the intended behavior?
- Does the rendered page resolve the targeted automated finding?
- Does the interaction make sense to the people using it?
A static model review can help with the first question, but it does not run the changed page. An automated browser re-scan provides different evidence, and it still doesn't replace interaction checks such as keyboard use or understanding the control's purpose.
Pause and predict: if the model says the patch is correct, should the run be marked verified?
The model has assessed the candidate source, not demonstrated the changed behavior in a browser. A useful status would distinguish “static review passed” from “browser check passed.” If the browser check never ran, the report should say so rather than inheriting the model's certainty.
In the inspected verifier, model-based static review exists, but the dynamic browser check is a placeholder set to succeed. Missing model configuration or a model error can also return a skipped-verification result that allows progress. Those paths explain why a success message needs to be tied to the check that actually ran.
A production loop would also need a stopping rule enforced across every retry path. Comments about a maximum attempt count aren't enough if the routing conditions don't enforce it for each kind of failure.
A compare link isn't an opened pull request
The submitter writes the candidate file, attempts to create a Git branch and commit, and attempts a push. It then constructs a GitHub compare URL for the branch.
That URL is useful: it gives a person a place to inspect the change and open a pull request. It is a different milestone from an API-confirmed, opened pull request. A failed push can also leave a local commit without a usable remote comparison.
For an agent interface, the status vocabulary matters. Generated, applied, committed, pushed, and submitted should each describe a confirmed event. Otherwise, the final “done” screen can hide the very step the user still needs to finish.
What I would validate next
Wire the existing browser runner into the graph's audit node. Apply a candidate in an isolated checkout, run the changed target, and re-check the same element. Test what happens when the mapping points to a missing file, model review is unavailable, the browser check fails, or a push is rejected.
These are proposed next checks. No detection-rate or end-to-end autonomy benchmark is established by this walkthrough.
SecondSight's strongest idea is the bridge from a passive report to a reviewable change. Each stage earns a different kind of confidence: the observation identifies a problem, the mapping identifies the source, and verification tests the proposed repair. Keeping those stages visible makes the automation easier to understand and improve.
Explore the implementation
Source reviewed at revision fcf2249: