Contract Intelligence on AWS: Extract With One Model, Verify With Another, Query With SQL

AWS's reference platform pulls eight fields from every contract with Claude Sonnet 4.6, checks them with Claude Haiku 4.5 and settles signature disputes with Textract. AI inference costs $0.014 per contract.

Contract Intelligence on AWS: Extract With One Model, Verify With Another, Query With SQL

Ask an AI chat tool a question about one contract and it will often produce a useful answer. Ask the same tool to calculate the total value of 250 contracts, identify the number of agreements expiring within a particular period or determine the organisation’s overall contractual exposure, and the result becomes considerably less reliable. The answer may arrive quickly and sound confident, but it can be wrong because the model has not necessarily examined every relevant document or extracted every required value consistently [1].

This is a fundamental limitation of applying conversational retrieval directly to large document collections. A language model can interpret a contract, explain a clause and locate relevant information, but that does not automatically make it a dependable portfolio-wide calculation engine.

AWS’s reference architecture for contract intelligence approaches the problem differently. Instead of asking a model to repeatedly search documents and calculate answers from retrieved passages, it extracts structured information from every contract, verifies the extracted values with a second model, stores the results in a relational database and uses analytics tools to answer portfolio-level questions [1:1].

The architecture combines Amazon S3, Amazon Bedrock AgentCore, Claude Sonnet 4.6, Claude Haiku 4.5, Amazon Textract, Aurora PostgreSQL and Amazon Quick. Each component has a defined responsibility, separating document ingestion, information extraction, verification, visual inspection, structured storage and analytics.

The most interesting part is not simply that AWS uses multiple AI models. It is how the architecture divides the work between probabilistic language models, a visual detection service and a relational database. Rather than expecting one model to perform every task, it assigns each operation to a component suited to that responsibility.

Why it matters

Retrieval-augmented generation (RAG) is a common approach to making large document collections accessible to language models. Documents are divided into smaller chunks, indexed and retrieved when a user asks a question. The system selects the most relevant passages and supplies them to the model as context for generating an answer.

This works well when the question concerns a particular document or a small number of relevant passages.

For example, a user might ask, “What are the payment terms in this contract?” The retrieval system can locate the relevant section, provide it to the model and ask it to explain the payment conditions. The model does not need to process every contract in the collection to answer that question.

The difficulty emerges when the question changes from document-level interpretation to portfolio-wide analysis.

Consider the difference between these two requests:

  • “What are the payment terms in this contract?”
  • “What is the total financial value of our 250 contracts?”

The first request is primarily an information-retrieval and interpretation problem. The second requires comprehensive coverage of the contract portfolio, consistent extraction of relevant values and an accurate aggregation of those values.

A conventional RAG pipeline retrieves only the top matching chunks for a particular question. It does not automatically guarantee that every relevant contract or every required value has been included in the model’s context [1:2].

Consequently, asking a RAG-based chatbot to calculate the total value of an entire portfolio creates a coverage problem. The model may receive several relevant passages and produce a plausible answer, but that does not establish that it has considered all 250 contracts.

The same issue applies to questions about contract counts, financial exposure and other portfolio-level measurements. A response generated from a limited set of retrieved passages cannot be treated as a complete aggregation unless the underlying retrieval and calculation process establishes that completeness.

AWS’s architectural argument is that this is not something that can be reliably solved through prompt engineering alone. The problem requires a different processing architecture [1:3].

Instead of retrieving information on demand for every portfolio question, the system processes the documents first. Each contract is converted into a structured representation containing the fields needed for subsequent analysis. Once those values have been extracted and verified, they can be stored as database records.

This changes the role of the language model.

The model is no longer expected to read hundreds of contracts every time someone asks for a total. It performs the document-understanding work during ingestion. The database subsequently handles operations such as counting, filtering and summing the extracted values.

For example, a question about the total value of a portfolio can be translated into a database aggregation over the stored contract records. The answer is then based on the structured dataset rather than an LLM attempting to calculate a total from a collection of retrieved text passages.

The distinction is particularly relevant to organisations managing procurement agreements, insurance claims, HR documents and leases. These document collections contain information that is useful both for individual-document questions and for broader operational analysis [1:4].

The architecture therefore supports two complementary ways of working with documents:

  • Structured extraction makes portfolio-wide questions and analytics possible.
  • The original documents remain available for questions that require interpreting specific clauses or contractual language.

This is a useful design principle for enterprise AI systems. Not every question should be answered through document retrieval, and not every document-related task should be delegated to SQL. The architecture needs to distinguish between interpreting unstructured information and calculating over structured records.

How it works

AWS’s reference architecture organises contract processing into five stages: upload, extraction, verification, tiebreaking and storage with querying [1:5].

1. Upload

The workflow begins when contract PDFs are uploaded to Amazon S3.

S3 acts as the document storage layer and provides the starting point for the processing pipeline. Uploading a contract triggers the subsequent workflow, allowing the system to process documents as they enter the collection rather than waiting for someone to initiate an individual extraction request [1:6].

This event-driven approach separates document submission from document processing. The storage layer receives the original PDF, while the downstream components handle the work of understanding and converting its contents into structured information.

For a contract intelligence application, retaining the original document is important because the extracted database record is not a replacement for the underlying agreement. It is a structured representation that makes selected information easier to retrieve and analyse.

The original PDF remains relevant for verification, visual inspection and questions about contractual language that cannot be answered adequately from the extracted fields alone.

2. Extract

Once a document enters the pipeline, an agent running Claude Sonnet 4.6 reads the PDF directly and extracts eight key fields.

The extraction process does not require a separate OCR stage in the evaluated architecture. The model reads the PDF directly and returns the extracted information as JSON, with a confidence score for each field [1:7].

The use of structured JSON is central to the design. Instead of returning a conversational summary of the agreement, the extraction agent produces information in a predictable format that can be passed to the verification and storage stages.

Consider the difference between receiving a response such as:

“This agreement is valued at a particular amount and contains the relevant contractual details.”

And receiving structured output in which individual fields are represented separately, each with its own extracted value and confidence score.

The second representation is much more useful for downstream processing. Individual values can be checked, stored and queried without having to interpret a new paragraph of generated text every time the system needs a particular piece of information.

The confidence score adds another layer of information. It gives the verification workflow a field-level indication of how confident the extraction model is about the values it has produced.

However, the architecture does not rely on the extraction model’s confidence alone to establish correctness. A separate model performs an independent verification step.

This distinction is important because an extraction system can produce plausible-looking structured output even when it has misinterpreted part of a document. Converting information into JSON makes it easier to process, but structured formatting does not automatically guarantee that the values are accurate.

The extraction stage therefore focuses on reading the contract and producing the initial structured representation. The next stage independently examines that result.

3. Verify

A separate agent running Claude Haiku 4.5 independently reads the same PDF and checks the extraction produced by Claude Sonnet 4.6 [1:8].

This creates a dual-model verification pattern.

The first model performs the primary extraction. The second model independently examines the document and checks whether the extracted information is supported by its contents.

Using a different model is a deliberate architectural decision. Simply asking the original model to repeat its extraction or review its own output may reproduce the same interpretation error. An independent model provides another opportunity to identify mistakes that the first model has overlooked.

The two models therefore have different responsibilities:

  • Claude Sonnet 4.6 produces the initial structured extraction.
  • Claude Haiku 4.5 independently checks the extracted information against the original document.

The second model is not merely asked to generate another summary. Its role is to examine the extraction and identify whether the values are consistent with the contract.

This creates an additional verification boundary before information reaches the database.

For data engineering teams, the important architectural idea is that extraction and verification are separate processing stages. The output of the first stage is not automatically accepted as authoritative simply because it has been returned in a structured format.

There is also an economic consideration. The architecture uses Claude Haiku 4.5 as the verification model rather than relying on the same extraction model for both tasks. This allows the workflow to combine a primary extraction model with a different model for checking the result.

The approach does not eliminate the possibility of errors, but it introduces independent scrutiny without requiring every processing stage to use the same model.

4. Tiebreak

The verification process exposed an interesting problem involving contract signatures.

When the two models disagree about whether a contract has been signed, the architecture uses Amazon Textract to inspect the page visually [1:9].

This introduces a third component, but it does not simply add another language model to the verification chain. Textract is used for a specific task: resolving a disagreement about the visual presence of a signature.

The distinction between reading contractual text and detecting a signature visually is important. A language model may understand the meaning of a document’s contents while still making an incorrect judgement about whether a signature is physically present.

The signature problem revealed by the evaluation demonstrates why this additional verification mechanism was necessary.

5. Store and query

Once the extracted information has been verified, the results are stored in Aurora PostgreSQL [1:10].

This moves the workflow from document processing into structured data management.

Each processed contract contributes information to a relational database, where the extracted fields can be queried independently of the original document-reading process.

The database becomes the foundation for portfolio-level analysis. Instead of asking a language model to retrieve information from hundreds of PDFs every time a user requests a total, the application can query the structured records.

Amazon Quick provides the analytics interface. AWS’s architecture uses it for live dashboards and a chat agent connected to two information sources:

  • The Aurora PostgreSQL database, which supports questions involving totals and counts.
  • A knowledge base containing the original documents, which supports questions requiring clause lookups [1:11].

This separation is particularly useful because the two sources serve different purposes.

A question such as “What is the total value of the contracts?” requires structured aggregation. A question such as “What does this agreement say about a particular clause?” requires access to the original contractual content.

Connecting the chat agent to both sources allows users to interact with the same application while the underlying architecture routes the question to the appropriate information layer.

The agent and security layer

The agents are built using the Strands Agents SDK and run on Amazon Bedrock AgentCore. The architecture also uses Cedar-based policies to control which contracts users and agents are authorised to access [1:12].

This is an important part of the overall design because contract intelligence is not simply a document-processing problem. It also involves controlling access to potentially sensitive business information.

Different users and agents may have different permissions. A system that extracts information from an entire contract collection must ensure that those records are not automatically exposed to every person interacting with the analytics interface.

Cedar-based policies provide the access-control mechanism described in the reference architecture. They establish an authorisation layer so that users and agents only see contracts they are permitted to access.

The result is a workflow in which ingestion, extraction, verification, storage, querying and access control are handled by separate architectural components.

The signature problem

One of the most revealing findings from the testing was not related to extracting financial values or interpreting complex contractual language. It involved something that might appear straightforward: determining whether a contract had actually been signed.

The verification model sometimes marked a contract as “signed” with 95 to 100% confidence even when no signature existed [1:13].

The reason was particularly interesting. The model interpreted an empty line labelled “Signature: ____” as evidence that the contract had been signed.

From a document-understanding perspective, the model had recognised the signature section and associated it with the concept of a completed agreement. However, recognising a signature field is not the same as detecting an actual signature.

An empty placeholder can appear in a contract template even when the agreement has not been executed. Treating the presence of the field as proof of signing creates a false positive.

The confidence scores made the issue more concerning. The model was not merely uncertain about the result; it sometimes expressed 95 to 100% confidence in an incorrect classification [1:14].

This illustrates a limitation of relying exclusively on model-generated confidence when validating important document attributes. A high-confidence answer is not necessarily a correct answer, particularly when the task requires recognising a visual condition rather than interpreting text.

AWS addressed the issue by introducing Amazon Textract’s computer-vision signature detection. The service is invoked only when the two models disagree about whether the contract is signed [1:15].

This is a targeted use of a specialised service rather than an additional model being applied indiscriminately to every document.

The approach also reveals an important design principle: different verification problems may require different techniques.

Language models are useful for reading and interpreting contractual content. However, a visual detection problem can be better addressed through a service designed to inspect document images.

Similarly, counting and arithmetic should not depend on a language model generating an answer from textual context when deterministic processing tools are available.

The lesson AWS draws is to use LLMs for reading and understanding, while using deterministic services for visual detection, counting and arithmetic [1:16].

For data and AI engineers, this is a useful distinction when designing agentic workflows. The objective should not be to maximise the number of tasks handled by an LLM. It should be to assign each task to the component that can perform it most reliably.

A model can extract and interpret information, another model can independently verify it, a computer-vision service can inspect a visual condition, and a database can perform aggregation.

The signature example demonstrates why this division of responsibilities matters. Adding another prompt to the same model might not address the underlying problem because the error originates in how the document is interpreted. Introducing a visual detection mechanism addresses a different aspect of the task.

The conditional invocation of Textract also keeps the additional visual check focused on cases where the two models disagree, rather than making it a mandatory step for every contract.

What it costs

AWS estimates that processing 1,000 contracts per month through the reference architecture costs approximately $349 in total [1:17].

The estimated monthly cost breakdown is:

| Component | Estimated monthly cost | |—|—:| | Amazon Quick licensing | $290 | | Aurora Serverless v2 | $44 | | Amazon Bedrock | $12 | | Amazon Textract | $2 | | Lambda, API Gateway, S3 and CloudFront | Under $1 | | Total | Approximately $349 |

The most important observation is how the costs are distributed.

Amazon Quick licensing accounts for approximately $290 of the estimated $349 monthly total. Aurora Serverless v2 contributes another $44, while Amazon Bedrock accounts for approximately $12. Textract is estimated at $2, with the remaining infrastructure services collectively costing less than $1 [1:18].

This means the AI inference component is not the dominant cost in the reference architecture.

AWS estimates the Bedrock inference cost at approximately $0.014 per contract [1:19]. At 1,000 contracts, that works out to approximately $14, although the published monthly estimate for Bedrock is $12. The figures are estimates from the reference architecture and should be understood in that context.

The distinction between per-document inference costs and the fixed or usage-related costs of the surrounding platform is important when evaluating the economics of an enterprise AI application.

A common assumption is that adding multiple language models to a workflow will make it prohibitively expensive. In this example, the architecture uses Claude Sonnet 4.6 for extraction and Claude Haiku 4.5 for verification, yet the estimated inference expense remains a relatively small part of the overall monthly cost.

The larger expense is the analytics licensing component.

This changes how organisations should approach cost optimisation. Reducing inference expenditure may have limited impact on the total bill if the analytics licence accounts for most of the monthly expense.

The cost structure also suggests that the number of users and the number of processed documents should be considered separately when planning deployment.

The document-processing component introduces costs associated with inference and document handling, while the analytics layer contributes a substantial licensing expense. As a result, the overall economics may change more significantly with the number of users than with the number of contracts processed.

For teams considering a similar implementation, the $0.014 inference figure is an interesting reference point, but it should not be interpreted as the total cost of operating a contract intelligence platform.

Storage, database infrastructure, analytics licensing and access to the original documents remain part of the complete architecture.

What data and AI engineers should do

AWS’s reference architecture offers several practical lessons for teams building enterprise document-processing systems. The most important is to design the workflow around the type of question the application needs to answer, rather than assuming that a single conversational retrieval pipeline can handle everything.

Extract first, then query.

For portfolio-level questions, convert documents into structured database records and let SQL perform the aggregation. Retain the original documents for questions that require interpreting individual clauses or locating specific contractual language [1:20].

This separates two different workloads.

Document understanding involves identifying relevant information in unstructured PDFs and converting it into consistent fields. Portfolio analytics involves filtering, counting and calculating over those fields.

Trying to perform both operations repeatedly through an LLM introduces unnecessary uncertainty. A structured extraction pipeline performs the document-reading work once, while the database makes the resulting information available for repeated queries.

The original PDFs should remain accessible because not every question can be answered through the extracted schema.

Verify with a different model.

A second model from a different family can provide an inexpensive independent check on the initial extraction. Disagreement between the two models should be treated as a signal that the result needs further examination [1:21].

The key is independence. Asking the same model to review its own output may reproduce the original interpretation error. A separate model provides another opportunity to identify inconsistencies.

However, agreement between two models should not automatically be interpreted as proof of correctness. The signature problem shows that models can share misleading interpretations of document structure. Independent verification improves the process, but specialised checks may still be required for particular fields.

Route model disagreements to deterministic tools or humans.

Not every disagreement should trigger another LLM call.

AWS’s use of Textract for signature disputes illustrates how a specialised visual detection service can handle a problem that language models struggled to resolve reliably.

For cases that remain unresolved, AWS suggests sending them to a human reviewer [1:22].

This creates a practical escalation strategy. Models handle the primary reading and independent verification, specialised services address specific detection problems, and human review handles cases that cannot be resolved confidently.

The objective is not to eliminate human involvement from every workflow. It is to direct human attention towards the cases where automated processing cannot establish a reliable result.

Run your own evaluation.

Before choosing a model combination, teams should hand-label a sample of their own documents and compare the performance of different configurations [1:23].

This is especially important because contract structures and field complexity can vary. A model combination that performs well on one collection may behave differently when the documents contain different formats or more complicated information.

A small evaluation dataset can provide an initial indication of extraction quality and reveal which fields require additional verification.

It also helps teams understand whether the second model is catching meaningful extraction errors or simply increasing processing overhead.

AWS’s own testing demonstrates why evaluation should examine individual fields and specific failure cases rather than relying only on an overall impression of answer quality.

Limitations and open questions

The reference architecture is a useful demonstration of how to combine multiple AI and data services, but the evaluation has limitations that engineering teams should consider before adopting it.

The first limitation is the size of the accuracy evaluation.

AWS tested 20 contracts containing 160 hand-labelled values. The company itself describes these results as directional, and it does not publish per-combination accuracy numbers [1:24].

That qualification matters. A small evaluation can reveal useful patterns and expose specific failure modes, but it does not establish that the same performance will hold across a much larger and more diverse contract collection.

Contract documents can differ in their formats and in the complexity of the information they contain. The extraction challenge may also vary between fields. A value that is straightforward to identify in one contract may be more difficult to interpret in another.

AWS found that extraction quality mattered more than the verifier, but the results will vary with contract formats and field complexity [1:25].

This observation is particularly relevant to teams deciding where to invest engineering effort. A strong verification stage cannot completely compensate for a weak initial extraction. If the primary model consistently misinterprets a field, the overall pipeline will remain limited even when a second model is introduced.

The extraction and verification stages should therefore be evaluated independently, with attention to the quality of the initial values and the errors detected by the second model.

Another limitation is the scope of the schema.

The reference architecture extracts eight key fields from each contract. This provides a manageable starting point for structured analysis, but it is a modest representation of what a complete contract may contain.

Contracts with nested obligations or amendments may require a more extensive schema [1:26].

This raises an important data-modelling consideration. A relational representation needs to capture the information required by the application’s analytical questions without assuming that every contractual relationship can be represented by a small set of flat fields.

The eight-field approach demonstrates the value of structured extraction, but organisations with more complex requirements may need to extend the schema and evaluate how additional fields affect extraction and verification quality.

The cost model also deserves attention.

Although AI inference costs only $0.014 per contract, the overall estimated monthly cost is dominated by Amazon Quick licensing. This means costs may scale more closely with the number of users than with the number of documents processed [1:27].

Organisations should therefore evaluate the economics of the complete application rather than focusing exclusively on token consumption or per-document inference.

The reference architecture provides a useful cost breakdown, but the $349 estimate should be understood in relation to its stated workload and licensing assumptions.

Finally, the signature example highlights a broader limitation of multi-model verification. Two models can disagree, but they can also make similar mistakes. Independent model checking reduces reliance on a single interpretation, yet it does not guarantee that the resulting classification is correct.

The Textract integration addresses one specific failure mode, but other document attributes may require different verification mechanisms. The appropriate approach will depend on whether a field requires semantic interpretation, exact value matching or visual detection.

These considerations do not undermine the architecture. They show why its most valuable principle is the separation of responsibilities rather than the assumption that a particular combination of models will work for every contract-processing task.

Key takeaways

  • RAG retrieves relevant document chunks but does not automatically provide complete portfolio-wide coverage. Structured extraction into a database makes reliable aggregation and analytics possible. [1:28]
  • AWS’s architecture uses Claude Sonnet 4.6 to extract eight fields from each contract, Claude Haiku 4.5 to independently verify them and Amazon Textract to resolve signature disputes. [1:29]
  • The signature evaluation exposed a significant failure: an LLM interpreted empty “Signature: ____” lines as evidence of signing, sometimes with 95 to 100% confidence. A visual detection service addressed the problem. [1:30]
  • Aurora PostgreSQL stores verified information, while Amazon Quick supports dashboards and conversational access to both structured database records and the original documents. [1:31]
  • AI inference costs approximately $0.014 per contract. The complete architecture is estimated at approximately $349 per month for 1,000 contracts, with analytics licensing representing the largest expense. [1:32]
  • For data and AI engineers, the broader lesson is to use LLMs for extraction and interpretation, independent models for verification, specialised services for visual detection, SQL for aggregation and human review for unresolved cases. [1:33]

Sources


  1. AWS Machine Learning Blog, “Building an AI-powered contract intelligence platform with Amazon Quick and Amazon Bedrock AgentCore”, 2026-09-29. https://aws.amazon.com/blogs/machine-learning/building-an-ai-powered-contract-intelligence-platform-with-amazon-quick-and-amazon-bedrock-agentcore/ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

Discussion

Join the discussion: create a free account or sign in to comment.

No comments yet.

Keep reading

Evaluating Multi-Agent Systems With Amazon Bedrock AgentCore: Helpfulness, Business Accuracy and Explainability
AI Engineering

Evaluating Multi-Agent Systems With Amazon Bedrock AgentCore: Helpfulness, Business Accuracy and Explainability

AWS shows a three-layer way to test agents on AgentCore. Built-in evaluators set a baseline, custom evaluators check business rules, and separate explainability evaluators check whether agents can say why.

Nov 16, 2026 · 5 min read
OpenAI Dots: A Persistent Cloud Agent for 4,000+ Apps
AI Engineering

OpenAI Dots: A Persistent Cloud Agent for 4,000+ Apps

Dots gives Pro and Business Premium users a dedicated cloud computer powered by GPT-6 Astra, but its high default cost makes the cheaper GPT-6.1 Sol a critical consideration for engineers.

Oct 26, 2026 · 4 min read
GPT-6.1 Sol on Amazon Bedrock: Near-Astra performance at a fraction of the cost
AI Engineering

GPT-6.1 Sol on Amazon Bedrock: Near-Astra performance at a fraction of the cost

OpenAI's GPT-6.1 Sol brings near-Astra performance to coding, computer use and professional work at one-fifth of Astra's standard token prices. Here's what Amazon Bedrock adds, from pricing and a 1M-token context to endpoints and code.

Oct 06, 2026 · 14 min read