LLM Prompt and Response Visibility Step 1 Solution Assessment

Problem Statement

Operators and approved users need to inspect the exact prompt sent to the LLM and the exact response received when an LLM call occurs, but the current implementation offers only partial diagnostics and no complete-exchange view.

Option A: Extend the existing operator CLI diagnostics

Extend the existing operator diagnostics to include the prompt and response.

Pros:

• Smallest scope and reuses existing operator diagnostics.

• Useful for operators during troubleshooting.

• Keeps sensitive LLM context out of the public application UI.

Cons:

• Does not satisfy approved users who need to inspect calls in the browser.

• Does not guarantee that reconstructed data is the exact payload sent.

• Does not provide a durable, searchable exchange history across all LLM call types.

Option B: Add an approved-user/operator exchange view backed by exact-call records

Add a restricted read-only exchange view that records the exact prompt and response for each LLM call, with the existing operator diagnostics as a secondary access path.

Pros:

• Directly satisfies both requested audiences and preserves the exact exchange.

• Supports both approved-user access and operator troubleshooting.

• Preserves a durable, auditable record instead of relying on reconstruction.

Cons:

• Requires a new access-controlled view and durable exchange records.

• Prompts may contain user-authored content and other sensitive context, so access control, redaction of credentials, retention, and payload-size handling must be explicit.

• Requires care to ensure failed, successful, and malformed responses are all recorded without exposing API keys.

Option C: Log exchanges only to server files

Write each prompt/response exchange to operator-readable server logs.

Pros:

• Avoids a database migration and is quick to add.

• Simple operational capture.

Cons:

• Poor discoverability and no approved-user access.

• Log rotation, permissions, retention, and concurrent writes become operational concerns.

• Easy to lose the audit record or expose secrets through ordinary log access.

Recommendation

Recommend Option B, with the existing CLI extended as a secondary operator view.

The requirement covers both operators and approved users and requires exact call-time data. Option B best satisfies that scope and provides a durable audit trail; Step 2 should define authorization, redaction/retention, supported call types, and view organization.