# Boost Accuracy with Generated Knowledge Prompting Canonical URL: https://www.adaline.ai/blog/boost-accuracy-with-generated-knowledge-prompting LLM text URL: https://www.adaline.ai/blog/boost-accuracy-with-generated-knowledge-prompting/llms.txt Published: 2025-06-13T00:00:00.000Z Modified: 2025-06-22T20:06:00.962Z Author: Nilesh Barla Category: Research Visibility: public Reading time: 13 min Topics: Research, Adaline, AI agent observability, agent evals, self-improving agents ## Summary LLM Optimization Through Strategic Knowledge Prompting Methods ## Article # What is Generated Knowledge Prompting? Generated Knowledge Prompting transforms how AI models approach complex tasks. It’s a prompt engineering technique that makes models smarter by building on their existing knowledge. Image: https://a-us.storyblok.com/f/1023026/2208x1580/ea7b52ac75/generated-knowledge-1.png _Overview of generated-knowledge prompting_ | **Source**: [Generated Knowledge Prompting for Commonsense Reasoning](https://arxiv.org/abs/2110.08387) The process works in two straightforward steps. First, the model generates relevant facts about a question or task. Second, it uses those facts as additional context to answer the original question. **Core Components:** - Knowledge generation phase - Knowledge integration phase - Zero-shot capability without external databases This approach differs from traditional prompting methods. Standard prompts ask models to answer directly from their training knowledge. Generated Knowledge Prompting makes models **pause** and **think** first. They actively construct relevant information before attempting solutions. The technique excels because it requires no task-specific training. Models don't need access to structured knowledge bases or domain-specific databases. They leverage their existing knowledge more effectively through strategic information retrieval. **Key Benefits:** - No external knowledge sources required. - Works across diverse domains. - Improves reasoning quality. - Reduces hallucination risks. Consider a complex question about historical events. Traditional prompting might yield incomplete answers. Generated Knowledge Prompting first asks the model to **list relevant historical facts**. Then it uses those facts to c**onstruct a comprehensive response**. This method proves especially powerful for knowledge-intensive tasks. The approach is to combine what the model knows and how it applies that knowledge. The result is more accurate, comprehensive, and reliable AI responses across various applications. # Why use Generated Knowledge Prompting over other Prompting techniques? Generated Knowledge Prompting stands out among prompt engineering methods for compelling reasons. It delivers measurable improvements while maintaining simplicity. ## Benefit 1: Enhanced Accuracy and Context-Aware Reasoning [Research](https://arxiv.org/pdf/2110.08387) demonstrates 7-10% accuracy improvements in zero-shot settings. This technique transforms implicit reasoning into explicit procedures. Models break down complex problems through clear deduction, induction, and analogy steps. **Accuracy benefits:** - More precise answers through additional context. - Reduced hallucination rates. - Better handling of multi-step reasoning. - Improved performance on knowledge-intensive tasks. The method helps LLMs provide context-aware responses. Instead of relying solely on training knowledge, models actively construct relevant information. This creates a richer foundation for answering complex questions. ## Benefit 2: High Flexibility without Custom Infrastructure Unlike **knowledge prompting** vs **RAG** systems, this approach needs no external databases. Teams can implement it immediately without custom infrastructure or retrieval systems. **Implementation advantages:** - Zero external dependencies. - Works across any domain. - Requires only few-shot demonstrations. - No task-specific training needed. This flexibility makes it superior to retrieval-augmented generation for many use cases. Teams avoid the complexity of maintaining knowledge bases while gaining performance benefits. ## Benefit 3: Amplification of Model Capabilities Generated knowledge prompting makes smaller models punch above their weight. Even LLM benefit from the knowledge they generate themselves. This creates an amplification effect that improves LLM with generated knowledge approaches. The technique essentially makes models smarter by helping them think more systematically. It’s like giving LLM a structured way to organize thoughts before responding. ## When to avoid it? Generated Knowledge Prompting offers significant advantages over traditional prompt engineering approaches. It excels when tasks require deep reasoning and comprehensive context building. **When to choose Generated Knowledge Prompting:** - Complex multi-step reasoning tasks. - Questions requiring broad domain knowledge. - Scenarios where external databases aren't available. - Tasks benefiting from systematic knowledge construction. The technique shines in knowledge-intensive scenarios. Unlike simple prompting that relies on immediate recall, this method builds comprehensive context first. It improves LLM performance by creating structured reasoning paths. However, Generated Knowledge Prompting isn’t always optimal. Several scenarios call for alternative approaches. **Avoid When:** - High-quality domain databases exist (QASC dataset scenarios). - Simple tasks requiring quick answers. - Computational efficiency is critical. - Real-time applications demand low latency. - Factual accuracy is paramount, and models hallucinate frequently. The method adds generation overhead. Each task requires two processing steps instead of one. **This doubles computational costs and increases response time**. For straightforward questions, the knowledge generation step becomes unnecessary complexity. "What's 2+2?" doesn't benefit from preliminary knowledge construction. Domain-specific databases often provide more accurate information than generated knowledge. **Medical or legal applications should prioritize verified sources over model-generated facts**. Real-time systems suffer from the added latency. **Customer service chatbots need immediate responses, not extended reasoning cycles**. **The technique works best for complex reasoning where the benefits outweigh the computational costs**. It transforms how LLMs approach challenging problems by making knowledge construction explicit and systematic. # How Generated Knowledge Prompting Works — Step by Step Generated Knowledge Prompting follows a systematic “generate knowledge prompting” process. **Phase 1: Knowledge Generation** The first phase creates relevant background information. The prompt contains three key elements: - Clear instructions for knowledge generation. - Few-shot demonstrations showing the desired output format. - Question placeholders for dynamic content insertion. Human-written demonstrations prove crucial here. They transform implicit commonsense reasoning into explicit procedures. The model learns to generate structured knowledge statements rather than jumping directly to conclusions. **Phase 2: Knowledge Integration** Generated knowledge statements become input for the inference model. The system evaluates multiple knowledge candidates using a max-scoring approach. For each knowledge statement: 1. Combine with original question 2. Generate final answer 3. Score response quality 4. Select highest-scoring result This evaluation process ensures that the best knowledge is utilized for final reasoning. **Implementation Parameters:** - Generate M=20 knowledge statements per question. - Use nucleus sampling (p=0.95) for diversity. - Apply temperature settings for controlled randomness. - Set termination conditions based on response quality. The multiple knowledge approach mitigates the risk of a single poor generation. Even if some statements contain errors, the scoring mechanism identifies the most helpful information. This methodology improves LLM performance by making reasoning explicit. Models build a comprehensive context before attempting solutions. The result is more accurate and reliable responses across knowledge-intensive tasks. The two-phase structure separates knowledge construction from answer generation, creating cleaner reasoning pathways. # Prompt Templates Effective prompt engineering requires structured templates that guide knowledge generation across different domains. Each template follows the same core structure while adapting to specific reasoning requirements. Image: https://a-us.storyblok.com/f/1023026/2810x1510/9f7e0633ce/generated-knowledge-3.png _Generated-knowledge prompt example._ | **Source**: [Generated Knowledge Prompting for Commonsense Reasoning](https://arxiv.org/abs/2110.08387) ```markdown Universal Template Structure. Instruction: Generate relevant knowledge about {{topic}} Demonstrations: {{2-3 examples showing format}} Question: {{Dynamic placeholder}} ``` ```markdown Numerical Reasoning Template (NumerSense). Generate numerical facts related to the following question. Example: How many wheels does a bicycle have? Knowledge: Bicycles are two-wheeled vehicles designed for human transportation. Question: {{INSERT_QUESTION}} Knowledge: ``` ```markdown Commonsense QA Template (CSQA). Provide background knowledge that helps answer this question. Example: What happens when you drop a glass? Knowledge: Glass is brittle and breaks when it hits hard surfaces due to impact force. Question: {{INSERT_QUESTION}} Knowledge: ``` ```markdown Scientific Reasoning Template (QASC). Generate scientific facts relevant to this question. Example: Why do plants need sunlight? Knowledge: Photosynthesis converts sunlight into chemical energy for plant growth. Question: {{INSERT_QUESTION}} Knowledge: ``` The demonstrations avoid directly answering questions. Instead, they show how to extract relevant background information. This prevents the model from jumping to conclusions during the knowledge generation phase. Each template maintains consistency while targeting domain-specific reasoning patterns. Numerical templates focus on quantitative relationships. Commonsense templates emphasize everyday reasoning. Scientific templates highlight cause-and-effect relationships. These structured approaches improve LLM performance by creating predictable knowledge generation patterns. Models learn to produce relevant, focused information that supports subsequent reasoning steps. # Choosing the right LLM for Generated Knowledge Prompting in 2025 Selecting the optimal[ ](https://artificialanalysis.ai/leaderboards/models)[LLM for Generated Knowledge Prompting](https://artificialanalysis.ai/leaderboards/models) requires evaluating specific capabilities that support this two-phase reasoning approach. Generated Knowledge Prompting demands models with **strong context retention**, **reasoning abilities**, and **knowledge synthesis capabilities**. ## Context Window Requirements Large context windows prove essential for Generated Knowledge Prompting. The technique requires models to maintain generated knowledge while processing subsequent reasoning steps. ```csv Requirement Level,Token Capacity,Use Case Minimum,32K tokens,Basic implementation, simple tasks Recommended,128K+ tokens,Complex reasoning, multi-step analysis Optimal,1M+ tokens,Document analysis, extended reasoning ``` Models like Gemini 2.5 Pro offer 1 million token windows, while Llama 4 Scout extends to 10 million tokens, enabling the processing of entire codebases or document collections. ## Model Comparison for Generated Knowledge Prompting ```csv Model,Context Window,Reasoning Strength,Cost Level,Deployment,Best For OpenAI o3/o4-mini,200K,Excellent,High,Cloud,Complex reasoning tasks Claude 4 Sonnet,200K,Excellent,Medium,Cloud,Fast processing needs Gemini 2.5 Pro,1M,Excellent,Medium,Cloud,Massive context analysis Llama 3.3-70B,128K,Good,Low,Local/Cloud,Cost-sensitive deployments DeepSeek R1,128K,Excellent,Low,Local/Cloud,Open-source reasoning Mistral-Large,128K,Good,Medium,Local/Cloud,Enterprise balance Qwen3,131K,Good,Low,Local/Cloud,Multilingual tasks ``` ## Model Categories OpenAI's o3 and DeepSeek R1 excel at Chain-of-Thought reasoning, breaking problems into multiple steps before generating responses. These models naturally align with Generated Knowledge Prompting's sequential approach. Open-source models like Llama 3.3-70B, Mistral-Large-Instruct-2407, and DeepSeek R1 provide strong performance at lower costs. These options suit organizations requiring customization and data privacy. ## Selection Decision Matrix ```csv Priority,Recommended Model,Rationale Maximum Performance,OpenAI o3 or Gemini 2.5 Pro,Superior reasoning with large context Cost Efficiency,DeepSeek R1 or Llama 3.3,Open-source with strong capabilities Data Privacy,Local Llama 3.3 or Mistral,Self-hosted deployment options Balanced Approach,Claude 4 Sonnet,Fast, reliable, moderate cost ``` Knowledge generation requires models that can effectively separate knowledge creation from answer synthesis. Models with Mixture-of-Experts architecture, like Qwen3 and DeepSeek V3, efficiently handle this dual-phase processing. The key is matching model capabilities to your Generated Knowledge Prompting use cases while balancing performance, cost, and deployment requirements. # Empirical Performance Generated Knowledge Prompting demonstrates significant performance gains across multiple knowledge-intensive benchmarks. Research results show consistent improvements over traditional prompting methods. Image: https://a-us.storyblok.com/f/1023026/2382x944/2ab6495e21/generated-knowledge-2.png _Generated-knowledge showing significant performance gains across multiple knowledge-intensive benchmarks_ | **Source**: [Generated Knowledge Prompting for Commonsense Reasoning](https://arxiv.org/abs/2110.08387) ## Performance Improvements ```csv Dataset,Baseline Accuracy,Generated Knowledge,Improvement NumerSense,64.2%,70.1%,+6% CSQA2,72.8%,74.9%,+2% QASC,61.3%,64.5%,+3% ``` These results represent state-of-the-art performance across diverse reasoning domains. NumerSense requires numerical understanding. CSQA2 tests commonsense reasoning. QASC evaluates scientific knowledge application. ## Baseline Comparisons Generated Knowledge Prompting outperformed multiple control methods: - **Random sentence insertion**: No improvement. - **Context sentence additio**n: Minimal gains. - **Template-based knowledge**: Moderate improvement. - **Retrieval-based methods**: Competitive but inconsistent. The technique proved particularly effective against retrieval methods. While RAG systems depend on external databases, Generated Knowledge works with the model's internal knowledge. ## Knowledge Quantity Analysis Performance improvements plateau around M=20 knowledge statements. Additional statements beyond this threshold provide diminishing returns. This finding helps optimize computational efficiency. ## Model Type Performance Both zero-shot and fine-tuned models showed improvements. Zero-shot models gained more from knowledge generation, suggesting the technique fills reasoning gaps effectively. Fine-tuned models achieved smaller but consistent gains. The empirical evidence confirms that Generated Knowledge Prompting improves LLM performance across multiple domains. It provides a reliable method to enhance reasoning without external dependencies or extensive fine-tuning requirements. # Pros, Cons & Common Pitfalls Generated Knowledge Prompting offers powerful advantages while presenting specific challenges. Understanding both sides helps optimize prompt engineering implementations. **Advantages:** - **High flexibility:** Adapts across diverse domains without retraining. - **No external dependencies:** Works without structured knowledge bases. - **Significant improvements:** Delivers 2-6% accuracy gains consistently. - **Model compatibility:** Enhances both zero-shot and fine-tuned models. - **Resource efficiency:** Amplifies smaller model capabilities. The technique excels in scenarios where external knowledge sources aren't available. It improves LLM performance by making reasoning explicit and systematic. **Disadvantages:** - **Computational overhead:** Doubles processing requirements. - **Quality dependency:** Results rely on generated knowledge accuracy. - **Limited advantage:** May underperform domain-specific databases. - **Hallucination risk:** Incorrect knowledge can cascade into wrong answers. - **Latency impact:** Additional generation step increases response time. **Critical Pitfalls:** ```csv Pitfall,Problem,Solution Poor demonstrations,Directly answer questions,Design reasoning-focused examples Wrong quantity,Too few/many statements,Use M=20 as optimal baseline No validation,Accept all generated knowledge,Implement quality scoring Overuse,Apply to simple tasks,Reserve for complex reasoning ``` # Conclusion Generated Knowledge Prompting represents a significant advancement in prompt engineering techniques for enhancing LLM performance. This two-phase approach delivers consistent 2-6% accuracy improvements across knowledge-intensive tasks without requiring external databases or specialized infrastructure. **Key Takeaways:** - Works across diverse domains without retraining. - Requires no external knowledge sources. - Amplifies smaller model capabilities effectively. - Balances performance gains with computational overhead. **Product Applications:** Product leaders can leverage this technique to improve LLM with generated knowledge in user-facing applications. Customer support chatbots become more accurate by generating relevant context before responding. Content creation tools produce higher-quality outputs through systematic knowledge construction. Educational platforms deliver more comprehensive explanations by building foundational knowledge first. The approach particularly benefits products requiring complex reasoning, multi-step analysis, or domain-specific expertise where traditional prompting falls short. # FAQ ### What is an example of generated knowledge prompt? A typical generated knowledge prompt follows a two-step structure. First, ask the LLM to generate relevant facts: "Generate three key facts about renewable energy sources." The model might respond with information about solar efficiency, wind power costs, and battery storage limitations. Second, use this knowledge in your main prompt: "Based on these facts about renewable energy, explain why solar panels are becoming more popular for residential use." This approach differs from single-step prompting, where you’d ask directly about solar panel popularity. The generated knowledge prompting steps create a better context for more accurate responses. ### What is generated knowledge? Generated knowledge refers to factual information that an LLM creates about a specific topic before answering the main question. Rather than relying solely on training data recall, the model actively constructs relevant background information. This self-generated context serves as additional input for final reasoning. The technique improves LLM with generated knowledge by making implicit reasoning explicit. Models perform better when they first establish foundational facts, then build conclusions upon that knowledge base. ### What does the generated knowledge approach ask the LLM to do? The generated knowledge approach asks the LLM to pause and think systematically before responding. Instead of jumping directly to conclusions, models must first generate relevant background information about the topic. This creates a structured reasoning process where knowledge construction precedes answer generation. This method represents advanced prompt engineering that transforms how models approach complex tasks. Unlike knowledge prompting vs RAG systems that retrieve external information, this technique uses the model's internal knowledge more effectively through strategic information organization.