API Design for Curie Attachments

Design ticket: PLFM-9827

1. Overview

Curie is the Synapse conversational agent. Today a chat turn carries only text (AgentChatRequest.chatText). This design adds attachments: a list of existing Synapse files a user can attach to a chat message so Curie can read and process them (e.g. inspect a CSV, summarize a PDF). Attachments are references to existing file handles — not a new upload mechanism — and are loaded into the agent's code-interpreter session for the turn.

Scope note: Curie is being migrated to the new Spring AI + Bedrock AgentCore supervisor/specialist framework. This design targets that framework, where files are staged into a shared code-interpreter session. It does not modify the legacy Bedrock Agent Runtime return-control path.

2. Supported attachment types

Attachments reference existing Synapse files via the standard FileHandleAssociation (fileHandleId + associateObjectId + associateObjectType). Only S3-backed file handles are supported (external / proxy handles are not staged). Approved content types:

Format	Content type

Format	Content type
Text	`text/plain`
PDF	`application/pdf`
CSV	`text/csv`
JSON	`application/json`

Any other content type is rejected per-attachment with UNSUPPORTED_TYPE.

3. How attachments are added to a chat message

Attachments are referenced, not uploaded inline. The client uploads or obtains the file through the normal Synapse file-handle APIs first, then passes the resulting FileHandleAssociations on the chat request. This keeps chat request bodies small and reuses Synapse's existing authorization model.

Request schema — add an optional attachments array to AgentChatRequest:

"attachments": {
    "description": "Optional. Files to attach as context for this message. Each attachment is a FileHandleAssociation the user must be authorized to download. Attached files are loaded into the agent's code interpreter session. Limits: at most 20 attachments per request; each file must be <= 100 MB and one of the approved content types (text/plain, application/pdf, text/csv, application/json).",
    "type": "array",
    "items": { "$ref": "org.sagebionetworks.repo.model.file.FileHandleAssociation" }
}

Why FileHandleAssociation and not a bespoke object: it carries exactly what authorization needs (fileHandleId + the associated object + its type), and it is the established pattern for "a request that references a list of files" (BatchFileRequest.requestedFiles, BulkFileDownloadRequest, etc.). File name / content type / size are read from the resolved FileHandle server-side — never trusted from the client.

4. Attachment metadata

The request carries only the reference. Name, content type, and size are resolved server-side from the FileHandle during authorization (FileHandleManager.getFileHandleAndUrlBatch returns the full handle). This resolved metadata is used to (a) enforce limits, (b) inform the agent (§9), and (c) populate the per-attachment status (§6).

5. Authentication & permission checks

The chat session itself is gated by the existing owner check — only the user who started the AgentSession may send messages to it.
Each attachment is authorized individually using Synapse's standard DOWNLOAD check (AuthorizationManager.canDownLoadFile via FileHandleManager.getFileHandleAndUrlBatch). A user must be authorized to download a file to attach it. Files the user cannot download are reported as UNAUTHORIZED and not staged.
The session's AgentAccessLevel is unchanged by attachments; attaching a file the user can already download does not require write access.

6. Processing & upload status

Attachment handling is fail-soft: a bad attachment does not abort the chat turn. Each attachment gets an outcome, returned to the caller and made available to the agent.

New schema AgentChatAttachmentStatus (modeled on FileDownloadSummary):

{
    "description": "The processing outcome for a single chat attachment.",
    "properties": {
        "fileHandleId":        { "type": "string", "description": "The file handle id of the attachment." },
        "associateObjectId":   { "type": "string", "description": "The ID of the object associated with the file." },
        "associateObjectType": { "$ref": "org.sagebionetworks.repo.model.file.FileHandleAssociateType" },
        "status": {
            "type": "string", "name": "AgentChatAttachmentStatusType",
            "enum": [
                { "name": "STAGED", "description": "The attachment was authorized and loaded into the agent's session." },
                { "name": "FAILED", "description": "The attachment could not be loaded." }
            ]
        },
        "failureMessage": { "type": "string", "description": "Human-readable failure reason. Null when status is STAGED." },
        "failureCode": {
            "type": "string", "name": "AgentChatAttachmentFailureCode",
            "enum": [
                { "name": "NOT_FOUND",           "description": "The file handle or associated object could not be found." },
                { "name": "UNAUTHORIZED",        "description": "The user is not authorized to download the file." },
                { "name": "EXCEEDS_SIZE_LIMIT",  "description": "The file exceeds the 100 MB per-file limit." },
                { "name": "UNSUPPORTED_TYPE",    "description": "The file's content type is not one of the approved types (txt, PDF, CSV, JSON)." },
                { "name": "TOO_MANY_ATTACHMENTS","description": "The request exceeded the maximum of 20 attachments." },
                { "name": "UNKNOWN_ERROR",       "description": "An unspecified error occurred while processing the attachment." }
            ]
        }
    }
}

Response schema — add an optional attachmentStatuses array to AgentChatResponse, in request order:

"attachmentStatuses": {
    "description": "Per-attachment processing results, in the same order as the request's attachments. Present only when the request included attachments.",
    "type": "array",
    "items": { "$ref": "org.sagebionetworks.repo.model.agent.AgentChatAttachmentStatus" }
}

7. Limits

Limit	Value	Enforcement

Limit	Value	Enforcement
Max attachments / request	20	over-count rejects the whole request → `TOO_MANY_ATTACHMENTS`
Max size / file	100 MB	per-file → `EXCEEDS_SIZE_LIMIT` (reuse `FileConstants.MAX_FILE_SIZE_ELIGIBLE_FOR_PACKAGING`)
Approved content types	txt, PDF, CSV, JSON	per-file → `UNSUPPORTED_TYPE`

The count limit is a new constant (proposed MAX_AGENT_CHAT_ATTACHMENTS = 20).

8. Error handling

Per-attachment (fail-soft): NOT_FOUND, UNAUTHORIZED, EXCEEDS_SIZE_LIMIT, UNSUPPORTED_TYPE, UNKNOWN_ERROR — reported in attachmentStatuses; the turn proceeds with whatever staged successfully, and the agent is told which files are available vs. which failed.
Whole-request (fail-fast): exceeding the 20-attachment count is a request validation error (TOO_MANY_ATTACHMENTS) — rejected before any staging.

9. How attachments reach the agent

Authorize and resolve all attachments in one batch (FileHandleManager.getFileHandleAndUrlBatch), applying the size / type limits.
Stage each authorized S3-backed file into the chat's code-interpreter session via CodeInterpreterFileManager.pushFileHandlesToSession(user, requests, sessionId) — the same primitive EntityMetadataSpecialistTools.addFilesToSession already uses.
Make the supervisor aware of staged files via a prompt session attribute (analogous to the existing access_level attribute) listing each staged file's session path, original filename, content type, and size — so Curie knows what is available and can reference files by path when running Python.

10. Display & referencing in responses

Curie references staged files by their session filename / path in responseText. The response's attachmentStatuses gives the client a machine-readable per-file outcome to render (e.g. show which attachments were accepted).

11. Retention, deletion & audit

Attachments are references to existing Synapse file handles; no new Synapse file is created for an inbound attachment.
Staging copies live only in the ephemeral code-interpreter staging bucket ({stack}.code-interpreter.staging.sagebase.org) and are scoped to the session lifecycle; they are not user-visible Synapse objects.
Existing file-handle download authorization and audit apply unchanged.

12. Open items for the implementation ticket

Session lifecycle for chat. The new framework currently starts/stops a code-interpreter session per curation task. Chat needs a session bound to the chat turn (or AgentSession) lifecycle — to be defined during the Curie migration.
Curie migration. Routing AgentChatRequest to the supervisor/specialist framework has not started; this design assumes that migration and the attachment work rides on it.
Schema files to add / modify: AgentChatRequest.json (add attachments), AgentChatResponse.json (add attachmentStatuses), and a new AgentChatAttachmentStatus.json, all under lib/lib-auto-generated/.../repo/model/agent/.

Acceptance-criteria coverage

Ticket criterion	Section

Ticket criterion	Section
Request & response schemas defined	§3, §6
Permission & security requirements	§5
File limits & supported formats	§2, §7
Processing statuses & error responses	§6, §8
Reviewed by Curie frontend + backend engineers	pending review

Technical Design Document: API Design for Curie Attachments