Parameters

• Text only​
• Internal users​
• Non conversational​
• Leave the coolness aside – GraphRAG for another day​

Problems v1

• Where did this info come from??
• Is it even accurate??
• Is it plagiarizing 😮?

Prompt v1

Chunking strategy

How to combine stuff to make a prompt

Problems v2

• Corpus imbalance
  • 32 gov't docs, 3 charity docs
  • Under/oversampling
• Chunks source unclear
  • What does [Kyle] think about....
  • Filename is just more text in each chunk...
• Chunks have perspectives / inconsistent parts of speech
  • There are no problems in social care... the first of our problems is...
  • Need homogenous voice

Problems v2 cont

• RAG only knows similarity, not recency
  • Old chunks with higher semantic similarity
  • Recent things are often more important!
• Even worse... It mixes old and new chunks
  • 4 chunks from 2023, 1 chunk from 2007
  • Time changes things
• Chunks don’t have context within the doc
  • Headings like "DO NOT" aren't encoded
  • Position in doc matters

Prompt v2

This is a Retrieval Augmented Generation (RAG) model that will generate paragraphs of text suitable for inclusion in a web portal about Social Care for the general public. Answer the question based on the context chunks provided, and list the source, which you can find in the metadata for each chunk. Highlight inconsistencies between the sources. Also explicitly state your confidence at the end. DO NOT COPY TEXT VERBATIM, that's plaigerism! There should be no sentence that can be found in the source material. Instead, paraphrase, but stick to the sources.Quote the sources as (source) along the material, don't put them at the end. Write in British English, not American English, and consider that the report will be read by a British audience, take that into consideration when considering tone. When the question refers to a specific source, make sure to look at the 'source' property in the 'metadata' key provided in each chunk.

Tech Stack v3

• UI: Streamlit
• embeddings: HuggingFace Transformers
  • all-MiniLM-L6-v2
  • ChromaDB
    • Structured metadata queries
• generation: ChatGPT
  • GPT-4 (with more prompt engineering)

So what?

• ✅ Plaigerism (P)
• ✅ Corpus imbalance (Q)
• ✅ Chunks understand sources (M)
• ✅ NLG has "its own" voice (P)
• ✅ RAG understands recency (M/P)
• ✅ RAG doesn't mix old/new chunks (M/P)
• ✅ Better at selecting from multiple sources (Q/P)
• ❌Chunks still don't have document context

(P = prompt; M = metadata; Q = query)

Proof ∈ Pudding

Prompt v3

This is a Retrieval Augmented Generation (RAG) model that will generate paragraphs of text suitable for inclusion in a web portal about Social Care for the general public. Answer the question based on the context chunks provided, and list the source, which you can find in the metadata for each chunk. Highlight inconsistencies between the sources. Also explicitly state your confidence at the end. DO NOT COPY TEXT VERBATIM, that's plaigerism! There should be no sentence that can be found in the source material. Instead, paraphrase, but stick to the sources. Quote the sources as (source) along the material, don't put them at the end. Write in British English, not American English, and consider that the report will be read by a British audience, take that into consideration when considering tone. When the question refers to a specific source, make sure to look at the 'source' property in the 'metadata' key provided in each chunk. Make sure that you don't overuse chunks from the same source (don't use a source more than twice) even if that means using less chunks of data. Also, look at the 'published' field, and prefer recent chunks should there not be too much of a disparity in accuracy.

UI code

results = collection.query(

            query_texts = search_query,

            n_results = 10,

            where = {"publisherType": {"$in": file_source_type}}

        )

Next steps

• Chunk reranking
  • Keyword matching?
• LlamaIndex (smart chunking, multimodal)
• Knowledge Graph

Bonus problems

• Trust
  • Who has access to what knowledge (auth)?
  • Who is promulgating WRONG facts?