bingoss/owned-growth-engine
Archived
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

LLM Visibility Specialist records measured adapter results only #14

New issue
Closed
opened 2026-05-28 18:14:46 +01:00 by bingoss · 1 comment
bingoss commented 2026-05-28 18:14:46 +01:00
Owner
Copy link

What to build

Type: AFK.

Create the LLM Visibility Monitoring slice of Weekly Intelligence. Configured prompts should run only against configured adapters, and named visibility claims must be backed by a measured visibility result for the exact adapter, provider, model, prompt, and timestamp.

Acceptance criteria

  • A Visibility run records configured prompt results as measured, skipped, failed, blocked, or insufficient evidence rows.
  • The product never claims ChatGPT, Claude, Gemini, Perplexity, or other named model visibility unless that exact adapter ran.
  • Dashboard exposes prompt results, mentions, competitors, citations, and gaps without raw secrets or unsafe payloads.
  • Tests are written first for measured results, skipped adapter, failed required adapter, and no-false-claim behavior.

Blocked by

None - can start immediately

## What to build Type: AFK. Create the LLM Visibility Monitoring slice of Weekly Intelligence. Configured prompts should run only against configured adapters, and named visibility claims must be backed by a measured visibility result for the exact adapter, provider, model, prompt, and timestamp. ## Acceptance criteria - [ ] A Visibility run records configured prompt results as measured, skipped, failed, blocked, or insufficient evidence rows. - [ ] The product never claims ChatGPT, Claude, Gemini, Perplexity, or other named model visibility unless that exact adapter ran. - [ ] Dashboard exposes prompt results, mentions, competitors, citations, and gaps without raw secrets or unsafe payloads. - [ ] Tests are written first for measured results, skipped adapter, failed required adapter, and no-false-claim behavior. ## Blocked by None - can start immediately
bingoss commented 2026-05-28 23:51:01 +01:00
Author
Owner
Copy link

Issue #14 completion audit

Implemented and verified:

  • Added a Postgres-writing LLM Visibility Monitoring worker entrypoint, run_visibility_monitoring_worker_once.
  • Visibility runs now read configured prompts/adapters from the Run Request payload and write normalized visibility_results rows.
  • The worker records measured, skipped_not_configured, failed_adapter, blocked_required_adapter, and insufficient_evidence rows.
  • Raw adapter payloads are stored only in raw_provider_payloads; dashboard output remains normalized and does not expose raw payload content.
  • Adapter execution snapshots are written to run_stage_adapter_results with required/optional status, safe error codes, and diagnostics.
  • Required adapter failures block Weekly Intelligence promotion through the existing required adapter gate.
  • Named provider/model claims are pinned to the configured adapter/provider/model identity, so adapter output cannot create a false ChatGPT/Claude/Gemini/Perplexity claim for an adapter that did not run.
  • Dashboard visibility output exposes prompt results, mentions, competitors, citations, and gaps without raw provider payloads.

Tests run on codex-server:

  • python3 -m unittest tests.test_visibility_worker tests.test_postgres_dashboard tests.test_postgres_schema tests.test_technical_audit_worker
  • Result: 131 tests OK
  • git diff --check: clean
Issue #14 completion audit Implemented and verified: - Added a Postgres-writing LLM Visibility Monitoring worker entrypoint, run_visibility_monitoring_worker_once. - Visibility runs now read configured prompts/adapters from the Run Request payload and write normalized visibility_results rows. - The worker records measured, skipped_not_configured, failed_adapter, blocked_required_adapter, and insufficient_evidence rows. - Raw adapter payloads are stored only in raw_provider_payloads; dashboard output remains normalized and does not expose raw payload content. - Adapter execution snapshots are written to run_stage_adapter_results with required/optional status, safe error codes, and diagnostics. - Required adapter failures block Weekly Intelligence promotion through the existing required adapter gate. - Named provider/model claims are pinned to the configured adapter/provider/model identity, so adapter output cannot create a false ChatGPT/Claude/Gemini/Perplexity claim for an adapter that did not run. - Dashboard visibility output exposes prompt results, mentions, competitors, citations, and gaps without raw provider payloads. Tests run on codex-server: - python3 -m unittest tests.test_visibility_worker tests.test_postgres_dashboard tests.test_postgres_schema tests.test_technical_audit_worker - Result: 131 tests OK - git diff --check: clean
Commenting is not possible because the repository is archived.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
codex-token-write-test-1780004082
Temporary token write test; safe to delete.

No labels
codex-token-write-test-1780004082
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
bingoss/owned-growth-engine#14
No description provided.