Security and access control

Security and access control

An AI agent must not be a shortcut to a database or administrative permission. It operates within boundaries set by the application and the specific process. Before information reaches an AI task, trusted application components check the user, their access and the purpose of the task.

Four distinct access questions

| Question | Application responsibility | |---|---| | Who is the user? | authentication | | What may they read or do? | authorisation and roles | | Which data may they see? | organisation, customer, tenant or document scope | | Which action may they take? | action validation and approval permission |

Permission to read a record does not automatically allow its change, a payment refund or another user’s access to be changed. Human approval may be required, but it does not replace technical controls.

Data for the task, and no more

For a hypothetical support ticket, AI may receive the message, product and current status to draft a summary. It does not need a password, session token, unrelated invoices or another customer’s records. Data minimisation reduces the input, but does not replace enforcement of access before that input is selected.

User → [application: identity, permission, data scope]
                            │
                            ▼
                    selected task data
                            │
                            ▼
                       AI provider
                            │
                            ▼
             untrusted output → application validation → permitted action

For a knowledge assistant, the application first limits search to documents available to the current person. For an invoice, AI may suggest extracted values, but the application checks format and rules and an authorised employee verifies them before the existing approval. These are illustrative examples, not GiSoft client deployments.

Treat external content and AI output with care

Prompt injection is an attempt to put instructions into a customer message or document so that AI ignores the intended rules. Such content is untrusted data, not an application instruction. Useful controls include limited context, separation of instructions from content, output validation and limited tool access. No single control eliminates all prompt-injection risk.

Categories, summaries, invoice values and proposed actions require checking before they affect a business record. Structured JSON or a high confidence score does not guarantee correctness. Integration secrets must not appear in prompts, retrieved context or responses; a separate integration layer does not guarantee confidentiality on its own.

Audit and exceptions

Useful operational records may include case identifier, user, sources used, process version, validation outcome and approved action. They need not include complete prompts or documents. Relevant permissions should be checked again before a delayed result is shown or an action runs.

Provider timeout, invalid output, revoked access, unavailable source or cancellation should produce a visible state and behaviour designed for that workflow. A manual fallback is useful only where it has been designed and verified. AI can assist work; authoritative data, decisions and actions remain with the application and authorised people.