AI / GenAI Engineer

AI refers to systems that can perform tasks typically requiring human intelligence, such as learning, reasoning, and perception. Unlike traditional software that follows explicit rules, AI systems learn patterns from data and can generalise to new inputs.

Narrow AI (ANI) is designed for a specific task like image classification or language translation. General AI (AGI) would have human-level reasoning across all domains. All current AI systems, including ChatGPT and AlphaGo, are narrow AI. AGI remains a theoretical concept.

AI is the broadest term covering any technique that enables machines to mimic human intelligence. ML is a subset of AI where models learn from data without explicit programming. Deep Learning is a subset of ML using neural networks with multiple layers to learn hierarchical representations.

The Turing Test, proposed by Alan Turing in 1950, evaluates whether a machine can exhibit behaviour indistinguishable from a human in conversation. While historically significant, modern AI researchers consider it insufficient because a system can pass via tricks without true understanding, and it does not measure reasoning or generalisation.

A neural network is a computational model composed of layers of interconnected nodes (neurons) that process inputs through weighted connections. It is inspired by biological neurons but the analogy is loose. Each artificial neuron applies a weighted sum followed by a non-linear activation function.

Indian companies use AI extensively in fintech (credit scoring for underbanked populations), agriculture (crop disease detection), healthcare (radiology screening at scale), e-commerce (recommendation engines and vernacular search), and customer support (multilingual chatbots supporting Indian languages).

AI is like teaching a computer to learn from examples rather than following fixed rules. For instance, instead of writing rules to detect fraudulent UPI transactions, we show the system millions of past transactions labelled as fraud or legitimate, and it learns the patterns itself. It gets better with more data and can handle cases we never explicitly programmed.

An LLM is a neural network trained on massive text corpora to predict the next token in a sequence. It generates text autoregressively by sampling from a probability distribution over its vocabulary, one token at a time, using the previously generated tokens as context. Models like GPT-4 and Claude use the Transformer architecture.

Pre-training involves training a model on massive unlabelled text data with a self-supervised objective like next-token prediction. This gives the model general language understanding. Fine-tuning then adapts this pre-trained model to specific downstream tasks using smaller labelled datasets, transferring the knowledge learned during pre-training.

Python offers a rich ecosystem of ML libraries (NumPy, pandas, scikit-learn, PyTorch, TensorFlow), clean syntax that enables rapid prototyping, strong community support, and excellent interoperability with C/C++ for performance-critical components. Its interpreted nature allows interactive development through Jupyter notebooks, which is ideal for experimentation.

Use NumPy: def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)). This computes the dot product of the vectors divided by the product of their magnitudes. The result ranges from -1 (opposite) to 1 (identical direction). Add epsilon to the denominator to prevent division by zero.

Type hints annotate function parameters and return types (e.g., def predict(input: np.ndarray) -> np.ndarray). They improve code readability, enable static analysis tools like mypy to catch bugs before runtime, and provide better IDE support. They are critical for ML production code where data type mismatches cause silent errors.

Prompt engineering is the practice of designing and refining inputs to LLMs to elicit desired outputs. It matters because LLMs are highly sensitive to how instructions are framed. Good prompts can dramatically improve output quality, accuracy, and relevance without any model changes, making it a cost-effective alternative to fine-tuning for many use cases.

Zero-shot gives the model a task description without examples. One-shot includes one example of the expected input-output pair. Few-shot provides multiple examples to establish the pattern. Few-shot typically improves performance on complex or ambiguous tasks by helping the model understand the desired format and reasoning style through demonstration.

The system prompt sets the AI's behaviour, persona, and constraints across the conversation. User prompts contain the human's messages and queries. Assistant prompts contain the AI's responses and can be pre-filled to guide output format. This separation enables consistent behaviour while allowing dynamic user interactions. Most API providers support this three-role structure.

Temperature controls output randomness. Use low temperature (0-0.3) for factual, deterministic tasks like classification and extraction. Use higher temperature (0.7-1.0) for creative tasks like brainstorming. Top-p and top-k further shape the probability distribution. These parameters interact with prompt design; well-constrained prompts can tolerate higher temperatures.

Delimiters (like triple backticks, XML tags, or separators) clearly mark boundaries between instructions, context, and user input. They prevent prompt injection by isolating user-provided content, improve parsing of structured prompts, and help the model distinguish between what to process and how to process it. Consistent delimiter usage improves output reliability.

RAG combines information retrieval with LLM generation. It retrieves relevant documents from a knowledge base and includes them in the prompt context, allowing the LLM to generate answers grounded in specific data. RAG reduces hallucination, enables access to private/current data without fine-tuning, and is more cost-effective than retraining models.

A RAG pipeline consists of: (1) document ingestion and chunking, (2) embedding generation for chunks, (3) vector storage in a database, (4) query embedding and similarity search at inference time, (5) context assembly with retrieved chunks, and (6) LLM generation using the augmented prompt. Each component can be optimised independently.

A vector database stores and indexes high-dimensional vector embeddings for efficient similarity search. Unlike traditional databases that use exact matching on structured fields, vector databases use approximate nearest neighbour (ANN) algorithms like HNSW or IVF for fast similarity search. Popular options include Pinecone, Weaviate, Qdrant, ChromaDB, and pgvector.

MCP is an open protocol that standardises how AI applications connect to external data sources and tools. It solves the M x N integration problem where each AI app would need custom integrations with each tool. With MCP, tools implement the protocol once and any MCP-compatible AI application can use them, similar to how USB standardised device connections.

Tool discovery is the process by which an MCP client learns what tools a server offers. The client calls the tools/list method, and the server responds with a list of available tools including their names, descriptions, and input schemas (JSON Schema). The host then presents these tools to the LLM so it can decide when and how to use them.

Official SDKs exist for TypeScript/JavaScript and Python, which are the most popular choices. Community SDKs are available for Kotlin, Go, Rust, C#, and other languages. The TypeScript SDK is the most mature with the best documentation. Choose based on your team's expertise and the ecosystem of the services you are integrating with.

An AI agent is an autonomous system that can perceive its environment, reason about goals, plan actions, and execute them using tools. Unlike a chatbot that responds to individual messages, an agent maintains state across interactions, can break complex tasks into sub-tasks, invoke external tools, and iterate on results until a goal is achieved.

Fine-tuning adapts a pre-trained model to a specific task by continuing training on task-specific data. Use it when prompt engineering alone is insufficient, you need consistent output formatting, domain-specific knowledge integration, or significant behaviour changes. It is more expensive than prompting but cheaper than training from scratch.

Evals are systematic methods for measuring AI system performance across quality, safety, and reliability dimensions. They are critical because they prevent deploying underperforming or harmful models, enable data-driven decision making between model versions, catch regressions early, and build stakeholder confidence. Without evals, AI development is essentially flying blind.

Offline evaluation tests models on static datasets before deployment using metrics like accuracy, F1, and perplexity. Online evaluation measures model performance in production with real users through A/B testing, user satisfaction, and business metrics. Offline catches obvious issues early; online captures real-world performance that test sets cannot fully represent. Both are essential.

AI bias occurs when a model produces systematically unfair outcomes due to biased training data, flawed feature selection, or skewed labelling. This matters because biased models can discriminate against underrepresented groups, especially critical in applications like loan approvals, hiring, and criminal justice.

Transfer learning involves taking a model pre-trained on a large dataset and fine-tuning it on a smaller, task-specific dataset. It reduces training time and data requirements dramatically. Models like BERT and ResNet are commonly used as pre-trained bases for NLP and computer vision tasks.

Embeddings are dense, low-dimensional vector representations of high-dimensional data like words, images, or users. They capture semantic relationships so that similar items have nearby vectors. They enable efficient similarity search, are used in recommendation systems, and form the foundation of modern NLP models.

An activation function determines whether a neuron should fire by applying a transformation to the weighted sum of inputs. Without non-linear activations, a multi-layer network would collapse to a single linear transformation, unable to learn complex patterns. Common non-linear activations include ReLU, sigmoid, and tanh.

Key considerations include ensuring fairness across diverse Indian demographics (caste, language, region), respecting data privacy under India's DPDP Act, maintaining transparency in automated decision-making, preventing displacement of workers without reskilling programs, and ensuring AI systems work equitably for rural and urban populations.

AutoML automates the process of selecting models, tuning hyperparameters, and engineering features. Tools like Google AutoML, H2O, and Auto-sklearn make ML accessible to non-experts and accelerate prototyping. However, domain expertise and understanding of results remain essential for production deployments.

Tokenisation splits text into units (tokens) the model can process. Byte Pair Encoding (BPE) starts with individual characters and iteratively merges the most frequent adjacent pairs to create a vocabulary of subword units. This handles rare words by breaking them into known subwords, balancing vocabulary size with representation quality.

Encoder-only models like BERT process entire input bidirectionally and are great for classification and embeddings. Decoder-only models like GPT generate text autoregressively using causal masking. Encoder-decoder models like T5 use a full encoder for input understanding and a decoder for generation, excelling at translation and summarisation.

Temperature controls randomness by scaling logits before softmax; lower values make output more deterministic. Top-k sampling restricts selection to the k most probable tokens. Top-p (nucleus sampling) selects from the smallest set of tokens whose cumulative probability exceeds p. These parameters trade off between creativity and coherence.

The context window is the maximum number of tokens an LLM can process in a single forward pass. Extending it is challenging because self-attention has O(n^2) computational complexity and memory usage. Techniques like FlashAttention, sparse attention, sliding window attention, and RoPE scaling help extend context windows to 100K+ tokens.

Hallucination occurs when LLMs generate plausible-sounding but factually incorrect information. It happens because LLMs are trained to predict probable next tokens, not verified facts. Mitigation strategies include RAG for grounding in verified sources, chain-of-thought prompting, constrained decoding, and implementing fact-checking pipelines.

In-context learning is the ability of LLMs to learn new tasks from examples provided in the prompt without updating model weights. Fine-tuning permanently modifies model weights through gradient descent on task-specific data. In-context learning is faster and more flexible but limited by context window size and generally less reliable for complex tasks.

Evaluate on task-specific benchmarks with your actual data, not just public leaderboards. Compare latency, throughput, cost per token, context window size, and output quality. Consider factors like API reliability, data privacy policies, and hosting options (cloud vs self-hosted). Run A/B tests and collect human evaluation scores on representative queries.

Greedy decoding always selects the most probable next token, which is fast but can produce repetitive text. Beam search maintains multiple candidate sequences and selects the highest-probability one, better for translation. Sampling-based methods (top-k, top-p) introduce controlled randomness for more creative and diverse outputs.

Key risks include generating harmful content, producing convincing misinformation, leaking private training data, being manipulated through prompt injection, encoding and amplifying biases, and enabling malicious uses like phishing. Mitigations include RLHF alignment, content filtering, red-teaming, and responsible deployment practices.

PyTorch uses eager execution by default, making it intuitive for debugging and research with a Pythonic API. TensorFlow uses graph-based execution for better production optimisation and has a more mature serving ecosystem (TF Serving, TFLite). PyTorch dominates research and is increasingly popular in production; TensorFlow is still strong in mobile and edge deployment.

Decorators are functions that modify the behaviour of other functions, applied using the @decorator syntax. In ML codebases, they are used for logging (tracking experiments), timing (measuring function performance), caching (memoising expensive computations), and in frameworks like PyTorch for torch.no_grad() and custom autograd functions.

asyncio provides an event loop for concurrent IO-bound operations using async/await syntax. In AI applications, use it for making concurrent API calls to LLM providers, handling multiple WebSocket connections for real-time inference, or orchestrating parallel data fetches in a RAG pipeline. It does not help with CPU-bound computation.

Use FastAPI or Flask to create endpoints. Load the model at startup (not per request). Define Pydantic models for request/response validation. Implement async handlers for IO-bound operations. Add health checks, logging, and error handling. Use gunicorn/uvicorn with multiple workers for production. Containerise with Docker and add rate limiting.

Dataclasses provide a decorator to auto-generate __init__, __repr__, and comparison methods for data-holding classes. Pydantic models add runtime validation, serialisation, and type coercion. Use dataclasses for internal data structures like model configs. Use Pydantic for API request/response schemas and configuration files where validation of external inputs is critical.

Chain-of-thought prompting instructs the model to show its reasoning step by step before giving the final answer. It is most effective for complex reasoning tasks like math, logic, multi-step analysis, and code debugging. Adding 'Let's think step by step' or providing worked examples significantly improves accuracy on tasks requiring multi-step reasoning.

Define the role (banking support agent), scope (account queries, transactions, card services), tone (professional, helpful, in English and Hindi). Include guardrails against providing financial advice or sharing account details in responses. Specify escalation paths for complaints. Include context about Indian banking terminology (UPI, NEFT, RTGS) and RBI regulations relevant to customer interactions.

Prompt injection occurs when user input manipulates the LLM into ignoring its system instructions or performing unintended actions. Defences include separating system and user prompts with clear delimiters, input sanitisation, output validation, using structured output formats, implementing allowlists for permitted actions, and layered defence with multiple validation steps.

Specify the exact schema in the prompt with examples. Use delimiters like code blocks to frame the output. Many APIs support function calling or structured output modes that constrain generation to valid JSON. Implement parsing with error handling and retry logic. For critical applications, validate against a JSON schema and use type-safe parsing libraries.

ReAct combines reasoning (thinking about what to do) and acting (taking actions like tool calls) in an interleaved fashion. The model reasons about the problem, decides on an action, observes the result, and repeats. This pattern powers most AI agent frameworks and is more reliable than pure reasoning because it grounds decisions in real observations.

Create a diverse test set of inputs with expected outputs. Measure quantitative metrics (accuracy, relevance scores, format compliance). Use LLM-as-judge for qualitative evaluation. Track prompt versions systematically. A/B test in production. Iterate by analysing failure cases and refining instructions. Tools like promptfoo, LangSmith, and Braintrust help automate this process.

Prompt templates are parameterised prompts with placeholders filled at runtime (e.g., 'Summarise this {document_type} in {language}'). In production, manage them as versioned configuration with template engines like Jinja2. Store in a prompt registry, track performance per version, and implement rollback capability. LangChain and similar frameworks provide prompt template abstractions.

Provide 3-5 diverse examples per category covering Indian telecom issues (network outages, billing disputes, SIM-related, data pack queries, number portability). Include edge cases and examples in both English and Hinglish. Structure examples with clear input-output pairs. Add instructions for ambiguous cases and a fallback 'Other' category. Test with regional variations.

Strategies include chunking the document and processing each chunk separately, using map-reduce patterns (summarise chunks then synthesise), implementing rolling summaries, using RAG to retrieve only relevant sections, and leveraging models with longer context windows. Choose based on whether the task needs global context (use map-reduce) or local information (use RAG).

Prompt engineering crafts inputs to guide a model's existing capabilities without changing weights. Instruction tuning fine-tunes the model on instruction-following data to make it inherently better at following prompts. Instruction-tuned models (ChatGPT, Claude) respond better to natural language prompts compared to base models that require more carefully engineered prompts.

Instruct the model to only use provided context and say 'I don't know' when uncertain. Use structured extraction from grounding documents rather than open generation. Ask for citations or evidence. Implement confidence scoring in the output format. Use chain-of-thought to make reasoning transparent. Combine with RAG for factual grounding.

Be specific about the role, capabilities, and limitations. Define the output format explicitly. Provide examples of good and bad responses. Set clear boundaries on what the model should and should not do. Use numbered steps for complex tasks. Place the most important instructions at the beginning and end (primacy/recency effect). Keep instructions concise and unambiguous.

Specify the programming language, framework, and version. Describe the function signature, inputs, outputs, and edge cases. Provide relevant context like existing code patterns and constraints. Ask for error handling and comments. Include test cases the code should pass. Use few-shot examples of similar functions. Request explanations of key design decisions.

Remove redundant instructions and examples. Use concise, precise language. Cache common prompt prefixes where the API supports it. Replace verbose few-shot examples with clear instructions when possible. Use shorter output formats (abbreviations, codes instead of full text). Batch similar requests. Evaluate whether a smaller, cheaper model suffices for the task.

Prompt chaining breaks a complex task into sequential simpler prompts where each output feeds the next. A single complex prompt tries to handle everything at once. Chaining is more reliable for multi-step tasks, easier to debug, and allows different models or parameters per step. Single prompts are faster and cheaper but may fail on complex reasoning.

Function calling is a structured way for LLMs to indicate they want to invoke a specific function with typed arguments, returning JSON matching a schema. Tool use is a broader concept where the model selects from available tools and generates appropriate calls. Function calling is a specific implementation of tool use. Both enable LLMs to interact with external systems reliably.

Fixed-size chunking splits by token/character count with overlap. Semantic chunking splits at natural boundaries (paragraphs, sections). Recursive chunking tries multiple separators hierarchically. Document-structure-based chunking uses headings and sections. Choose based on document type: structured documents benefit from semantic chunking, while unstructured text works with fixed-size plus overlap.

Evaluate retrieval quality using recall@k, precision@k, and MRR (Mean Reciprocal Rank). Evaluate generation quality using faithfulness (is the answer grounded in retrieved context?), relevancy (does it answer the question?), and correctness. Use frameworks like RAGAS for automated evaluation. Also measure end-to-end latency and user satisfaction.

Hybrid search combines dense vector search (semantic similarity) with sparse keyword search (BM25/TF-IDF). Dense search captures semantic meaning but can miss exact terms. Sparse search is excellent for specific keywords, names, and codes. Combining both with reciprocal rank fusion (RRF) or weighted scoring captures both semantic and lexical matches.

Reranking applies a more expensive cross-encoder model to rescore initially retrieved documents based on their relevance to the query. The initial retrieval uses fast bi-encoder embeddings to get candidate documents, then the reranker (like Cohere Rerank or a cross-encoder) scores each query-document pair jointly. This two-stage approach balances speed and accuracy.

Naive RAG follows a simple retrieve-then-generate pipeline. Advanced RAG adds pre-retrieval optimisation (query rewriting, expansion), retrieval optimisation (hybrid search, reranking, iterative retrieval), and post-retrieval processing (context compression, lost-in-the-middle mitigation). Advanced RAG also includes feedback loops and self-evaluation for continuous improvement.

Query decomposition breaks a complex question into simpler sub-questions, retrieves information for each, and synthesises the results. Use it when questions require information from multiple domains or documents. For example, 'Compare the AI policies of India and the US' decomposes into separate retrievals for each country's policy, then a comparison synthesis step.

Implement incremental indexing that detects changed documents via checksums or timestamps. Use metadata to track document versions and update timestamps. For critical applications, maintain version history and allow querying against specific versions. Implement a document deletion pipeline that removes stale vectors. Schedule periodic full re-indexing for consistency.

LLMs tend to pay more attention to information at the beginning and end of the context, potentially ignoring relevant information in the middle. Mitigate by placing the most relevant chunks first and last, limiting the number of retrieved chunks, using reranking to prioritise relevance, or restructuring retrieved content into a focused summary before generation.

Consider the MTEB leaderboard for benchmark performance. Evaluate on your specific domain data using retrieval metrics. Compare dimensionality (trade-off between quality and storage), latency, and cost. Test multilingual capability if needed. Consider whether a domain-specific fine-tuned embedding outperforms general-purpose ones. Popular choices include OpenAI ada-002, Cohere embed, and open-source models like BGE and E5.

Dense retrieval uses learned vector embeddings from neural models to represent queries and documents as continuous vectors, capturing semantic similarity. Sparse retrieval uses traditional methods like BM25 with term frequency and inverse document frequency. Dense is better for semantic matching; sparse excels at exact term matching. Combining both often yields the best results.

Include source metadata (date, authority, reliability) with retrieved chunks. Instruct the LLM to identify and flag contradictions. Prioritise more recent or authoritative sources. Present multiple viewpoints when appropriate. Implement a confidence scoring mechanism. For critical applications, surface contradictions to users rather than silently choosing one version.

Monitor retrieval latency, embedding computation time, and end-to-end response time. Track retrieval hit rate and relevance scores. Measure answer faithfulness and hallucination rate. Monitor index size growth and query volume. Track user feedback (thumbs up/down, corrections). Alert on unusual query patterns, error rates, or degradation in response quality.

Use table detection models or specialised PDF parsers (like Unstructured, Camelot, or Tabula) to extract tables. Convert tables to markdown or structured text preserving column relationships. Store tables as separate chunks with table-specific metadata. Consider embedding tables alongside their captions and surrounding context. For complex tables, create text descriptions.

There is no universal optimal size; it depends on document type, query patterns, and embedding model. Typical ranges are 256-1024 tokens. Smaller chunks give more precise retrieval but may lack context. Larger chunks provide more context but may include irrelevant information. Experiment with different sizes and evaluate retrieval quality. Use overlap (10-20%) to maintain context.

Parent-child chunking stores documents at two granularity levels. Small child chunks are embedded and used for retrieval (precise matching). When a child is retrieved, its larger parent chunk is returned to the LLM (richer context). This combines the precision of small-chunk retrieval with the context richness of larger passages, improving both retrieval accuracy and generation quality.

Include source metadata (document title, page, URL) with each chunk. Instruct the LLM to reference specific sources using identifiers. Parse citations from the output and validate them against retrieved chunks. Display inline citations with links to source documents. For accuracy, use structured output to separate the answer from citations and validate each reference.

Metadata filtering narrows the search space before or after vector search, improving relevance and performance. Design metadata fields based on common query patterns: date ranges for recency, categories for topic scoping, access levels for permissions, and source types for authority. Keep metadata normalised and indexed. Implement pre-filtering for mandatory constraints and post-filtering for preferences.

Use RAG when you need access to frequently updated data, require citations, or have a large knowledge base. Use fine-tuning when you need to change the model's behaviour, style, or reasoning patterns, or when knowledge is stable and fits in training data. RAG is cheaper and more flexible; fine-tuning gives better performance for specific formats. Many production systems combine both approaches.

MCP uses a client-server model where the AI application (host) contains an MCP client that connects to MCP servers. Each server exposes capabilities like tools, resources, and prompts. The host manages multiple client connections to different servers. Communication happens via JSON-RPC 2.0 over transports like stdio or SSE. The client initiates connections and the server responds to requests.

The three primitives are: Tools (model-controlled functions the LLM can invoke, like API calls or computations), Resources (application-controlled data the client can read, like files or database records), and Prompts (user-controlled templates for common interactions). Tools are called by the model, resources are read by the application, and prompts are selected by users.

Create an MCP server that exposes search and retrieval tools for the knowledge base. Implement tools like search_documents(query), get_document(id), and list_categories(). Use resources to expose document content. Add authentication via the transport layer. Handle pagination for large result sets. Include proper error handling and rate limiting. Test with multiple MCP clients.

Stdio transport communicates via standard input/output streams and is used for local server processes spawned by the client. SSE (Server-Sent Events) transport uses HTTP for remote servers, with SSE for server-to-client messages and HTTP POST for client-to-server messages. Stdio is simpler for local tools; SSE enables remote and shared server deployments.

MCP implements security through transport-level encryption (TLS for SSE), OAuth 2.0 for authentication with remote servers, and a human-in-the-loop approval model where the host application can require user confirmation before tool execution. Servers should validate inputs, implement rate limiting, and follow least-privilege principles. Sensitive operations should always require explicit user consent.

Write clear, specific descriptions that explain what the tool does, when to use it, and what inputs it expects. Include parameter descriptions with examples. Specify constraints and valid values. Add negative instructions (when NOT to use the tool). Use consistent naming conventions. Test with actual LLMs to verify they invoke tools correctly in various scenarios.

Resources are read-only data sources exposed by MCP servers, identified by URIs (like file:///path or db://table/id). Unlike tools which are model-controlled and perform actions, resources are application-controlled and provide data. The client application decides when to read resources, often to provide context. Resources support subscriptions for real-time updates.

Expose tools like query_database(sql) with read-only access, list_tables(), and describe_table(name). Use resources to expose schema information. Implement SQL injection prevention by parameterising queries. Add result size limits and timeout handling. Restrict to SELECT queries only for safety. Include proper error messages for invalid queries. Support connection pooling for performance.

JSON-RPC 2.0 is the messaging format used for all MCP communication. It provides a standardised structure for requests (method + params), responses (result or error), and notifications (one-way messages). It supports batch requests and has a simple error handling model. MCP uses it as the application-level protocol over different transport layers.

Return structured error responses with clear error codes and human-readable messages. Implement isRetryable flags for transient errors. The host application should implement exponential backoff for retryable errors. Design tools to be idempotent where possible. Log errors for debugging. The LLM should receive informative error messages to adjust its approach rather than raw stack traces.

MCP prompts are pre-defined interaction templates exposed by servers that users can select to initiate specific workflows. They can include dynamic arguments and generate structured messages with roles. Unlike system prompts which set global behaviour, MCP prompts are task-specific templates that combine instructions, context, and tool usage patterns for common operations.

During the initialise handshake, client and server exchange their supported capabilities. The client declares what features it supports (like sampling or roots), and the server declares its capabilities (tools, resources, prompts, logging). This negotiation ensures both sides understand what features are available and prevents errors from using unsupported features.

Write unit tests for individual tool handlers with mock inputs. Use the MCP Inspector tool for interactive testing. Test with actual MCP clients like Claude Desktop. Verify schema validation for all inputs. Test error handling with invalid parameters. Load test for performance under concurrent connections. Test transport layer reliability with connection drops and reconnects.

Implement per-client rate limiting using token buckets or sliding window counters. Return appropriate error codes (429) with retry-after headers. Track usage per authenticated user. Set tool-specific rate limits based on the cost of the underlying operation. Communicate limits through tool descriptions. Cache frequent requests. Consider implementing priority queues for different user tiers.

An MCP server is specifically designed for AI consumption with structured tool descriptions, input schemas, and standardised discovery. A REST API requires custom integration code, prompt engineering to teach the LLM the API format, and manual error mapping. MCP provides a universal protocol so any AI host can use any MCP server without custom integration code.

Use JSON Schema with clear type definitions and descriptions for every parameter. Mark required vs optional parameters. Include validation constraints (min/max, patterns, enums). Provide default values where sensible. Keep schemas simple; break complex operations into multiple simpler tools. Include examples in descriptions. Validate inputs server-side regardless of schema enforcement.

Use the protocol version field in the initialise handshake. Add new tools without removing old ones. Deprecate tools by updating their descriptions before removal. Make new parameters optional with defaults. Use semantic versioning for the server itself. Test new versions against existing clients. Document breaking changes clearly. Support multiple versions simultaneously during transitions.

MCP specifies that the host application should have the ability to require user approval before executing tool calls, especially for sensitive operations. This is crucial because LLMs may make incorrect tool calls or attempt unintended actions. The host acts as a gatekeeper, presenting tool calls to the user for confirmation, ensuring safety and maintaining user control.

Use MCP's built-in logging capability to send structured log messages to the client. Log all tool invocations with parameters, execution time, and outcomes. Implement correlation IDs to trace requests across the system. Export metrics (latency, error rate, usage) to monitoring systems. Set up alerts for unusual patterns. Store logs with appropriate retention for audit compliance.

Multiple servers follow the Unix philosophy of doing one thing well, enabling independent development, deployment, and scaling. A monolithic server is simpler to deploy but harder to maintain. Compose multiple servers when tools have different security requirements, update frequencies, or team ownership. Use a monolithic server for tightly coupled tools that share state.

MCP adds a layer of abstraction and latency that may be unnecessary for simple, single-tool integrations. It requires both client and server support. The ecosystem is still maturing with limited SDKs in some languages. For high-throughput, low-latency requirements, direct API integration may be more appropriate. If you only integrate with one or two tools, the protocol overhead may not be justified.

An AI agent typically consists of: (1) an LLM as the reasoning engine, (2) a planning module that breaks tasks into steps, (3) a memory system (short-term context and long-term storage), (4) tools/actions the agent can execute, and (5) an observation/feedback loop that processes tool results and decides next steps. The orchestrator ties these components together.

ReAct (Reasoning + Acting) interleaves thinking and doing in a loop: Thought (reason about the situation), Action (select and execute a tool), Observation (process the result). The agent continues this cycle until it reaches the goal or determines it cannot proceed. This pattern grounds reasoning in real observations and is the foundation of most production agent frameworks.

Define tools with clear names, descriptions, and parameter schemas. Present available tools to the LLM in the system prompt or through function calling APIs. Parse the LLM's tool selection and arguments. Execute the tool with validation and error handling. Feed the result back as an observation. Implement safeguards like confirmation prompts for destructive actions and rate limiting.

Single-agent systems use one LLM to handle all reasoning and actions. Multi-agent systems use multiple specialised agents that collaborate, each with different roles, tools, or expertise. Multi-agent systems handle complex tasks better through division of labour but add coordination complexity. Examples include manager-worker patterns, debate systems, and assembly line pipelines.

Implement short-term memory as the conversation context window. Use long-term memory with vector databases for past interactions and learned information. Implement working memory for the current task state. Summarise older context to stay within token limits. Use structured memory stores (key-value) for facts and episodic memory for past experiences. Implement memory retrieval based on relevance.

Common failures include infinite loops (agent keeps retrying failed actions), tool misuse (wrong tool or parameters), hallucinated tool calls, context window overflow, and goal drift. Handle with maximum iteration limits, tool call validation, structured error recovery, context summarisation, periodic goal re-evaluation, and human-in-the-loop checkpoints for critical decisions.

Planning is how agents decompose complex goals into executable steps. Approaches include: task decomposition (breaking into sub-tasks upfront), iterative planning (plan one step at a time based on observations), hierarchical planning (plans at multiple abstraction levels), and plan-and-execute (generate full plan, then execute with replanning on failure). Choice depends on task predictability.

Agent orchestration frameworks provide structure for building agent systems. LangGraph uses a graph-based approach where nodes are agent steps and edges define control flow, offering fine-grained control. CrewAI uses role-based agents in a crew metaphor with higher-level abstractions. LangGraph is more flexible but complex; CrewAI is easier to start with but less customisable.

Measure task completion rate, step efficiency (number of steps to complete a task), tool usage accuracy, cost per task, latency, and error recovery success. Create benchmark tasks with known solutions. Track trajectories to understand reasoning quality. Use human evaluation for subjective quality. Compare against non-agent baselines. Monitor for regressions when changing models or prompts.

Deterministic workflows follow predefined paths with fixed step sequences, like pipelines. Non-deterministic workflows let the LLM decide which tools to use and in what order based on the situation. Deterministic workflows are more predictable and debuggable; non-deterministic workflows are more flexible and can handle unexpected scenarios. Most production agents combine both approaches.

Function calling is the mechanism by which LLMs generate structured tool invocations. OpenAI uses a tools parameter with JSON schema definitions. Anthropic (Claude) uses a similar tool_use format. Google Gemini has function declarations. The LLM outputs a structured call with function name and arguments rather than free text, which the application executes and returns results.

In the supervisor pattern, a central agent (supervisor) receives tasks, delegates sub-tasks to specialised worker agents, collects their results, and synthesises the final output. The supervisor decides which worker to invoke based on the task requirements. This provides clear control flow, easier debugging, and the ability to use different models for different sub-tasks.

Reactive agents respond to stimuli (user messages, events) and take action. Proactive agents monitor conditions and initiate actions autonomously without explicit triggers, such as detecting anomalies and alerting users. Most current AI agents are reactive, but proactive capabilities are emerging through scheduled checks, event-driven triggers, and continuous monitoring patterns.

Agent loops consume many LLM tokens through repeated reasoning, tool calls, and observations. Costs multiply with retries, long context, and multi-agent coordination. Optimise by using cheaper models for simple tool selection, caching frequent operations, limiting iteration counts, compressing context, and batching similar tasks. Monitor cost per completed task and set budgets per workflow.

Reflection is when an agent critiques its own output or reasoning before finalising. The agent generates a response, then evaluates it against criteria like accuracy, completeness, and relevance. If the evaluation identifies issues, the agent revises its approach. This self-improvement loop can significantly enhance output quality at the cost of additional LLM calls.

Implement try-catch blocks around tool execution. Provide clear error messages back to the agent with context for recovery. Define fallback tools for common failures. Allow the agent to retry with modified parameters. Implement escalation paths when retries fail. Distinguish between retryable errors (network timeout) and permanent errors (invalid input). Track error patterns for tool improvement.

Computer use enables AI agents to interact with computer interfaces the same way humans do, through screenshots, mouse clicks, and keyboard input. The agent sees the screen, reasons about the interface, and performs actions like clicking buttons and typing text. This enables automation of tasks in any application without needing APIs, though it is slower and less reliable than tool-based approaches.

Agent chains execute steps in a fixed linear sequence, like a pipeline. Agent graphs define nodes (steps) and edges (transitions) that can include branching, parallel execution, loops, and conditional routing. Graphs are more expressive for complex workflows with dynamic control flow. LangGraph and similar frameworks implement graph-based agent orchestration.

Structured output (JSON, function calls) ensures agent decisions are parseable and actionable by the orchestration system. Without structure, the system must extract tool calls and parameters from free text, which is error-prone. Modern LLM APIs support constrained output modes that guarantee valid JSON. This reliability is essential for production agent systems.

Full fine-tuning updates all model parameters, requiring significant GPU memory and risking catastrophic forgetting. PEFT methods like LoRA, QLoRA, and adapters only modify a small subset of parameters, reducing compute and memory needs by 10-100x while achieving comparable performance. PEFT is preferred for most practical use cases due to its efficiency.

LoRA freezes the pre-trained weights and injects trainable low-rank decomposition matrices into each layer. Instead of updating a full weight matrix W (d x d), it learns two smaller matrices A (d x r) and B (r x d) where r is much smaller than d. This reduces trainable parameters by 100-1000x while maintaining performance. Multiple LoRA adapters can be swapped for different tasks.

Create instruction-response pairs in the model's expected format. Ensure diversity in instructions, response lengths, and complexity. Include edge cases and negative examples. Clean data for quality: remove duplicates, fix formatting, validate factual accuracy. Aim for 1K-100K high-quality examples. Use consistent formatting with clear instruction, input, and output fields.

Catastrophic forgetting occurs when fine-tuning causes the model to lose previously learned general capabilities. Prevent it by using a low learning rate, keeping training epochs low, mixing in general-purpose data, using PEFT methods like LoRA which preserve base weights, applying elastic weight consolidation, and validating on both the target task and general benchmarks.

Key hyperparameters include learning rate (typically 1e-5 to 5e-5), batch size, number of epochs (1-3 for LLMs), warmup ratio, weight decay, and for LoRA: rank (4-64), alpha (typically 2x rank), and target modules. Start with recommended defaults, use a validation set to monitor overfitting, and adjust learning rate first as it has the largest impact.

Compare against the base model on your task-specific evaluation set using relevant metrics. Check for regression on general capabilities with standard benchmarks. Compute perplexity on a held-out set. Use human evaluation for subjective quality. A/B test in production. Verify the model handles edge cases in your domain. Monitor both task performance and safety metrics.

Create training examples with consistent JSON output format. Include diverse examples covering all possible output structures. Add examples with edge cases (null values, empty arrays, special characters). Validate that training data contains only valid JSON. Fine-tune with a focus on format compliance. Evaluate using both format validity (parse success rate) and content accuracy metrics.

Load the base model with appropriate quantisation config (BitsAndBytesConfig for QLoRA). Apply LoRA config specifying target modules, rank, and alpha. Prepare the dataset in the instruction format using a tokeniser. Use SFTTrainer from trl library with training arguments (learning rate, epochs, batch size). Monitor training loss and validation metrics. Save and merge LoRA weights after training.

Use a stronger model (like GPT-4 or Claude) to generate instruction-response pairs. Provide diverse seed prompts covering your domain. Apply quality filters to remove low-quality generations. Validate with domain experts for a subset. Augment with paraphrasing and variation generation. Be careful about model distillation terms of service. Combine synthetic data with real examples for best results.

Overfitting is common with small fine-tuning datasets. Detect it by monitoring training loss vs validation loss divergence, checking for verbatim memorisation of training examples, and testing on held-out data. Prevent it by using fewer epochs (1-3), applying dropout, using weight decay, increasing dataset diversity, and employing early stopping based on validation metrics.

Consider: model size vs available compute, pre-existing knowledge in your domain, licensing restrictions (commercial vs research), language support (especially for Indian languages), community support and tooling, and baseline performance on your task. Start with the smallest model that meets quality requirements. Test 2-3 candidates on your evaluation set before committing to full fine-tuning.

Use the merge_and_unload() method from the PEFT library, which adds the LoRA weight deltas (A * B scaled by alpha/r) to the corresponding base model weights. The merged model has the same architecture as the base model with no inference overhead. Save the merged model for deployment. Verify that the merged model produces identical outputs to the LoRA model before deployment.

Create a diverse test set covering common cases, edge cases, and adversarial inputs. Define metrics for each quality dimension: correctness, relevance, completeness, formatting, latency, and safety. Use automated metrics where possible, LLM-as-judge for subjective qualities, and human evaluation for ground truth. Track metrics across model versions and prompt changes. Automate evaluation in CI/CD.

LLM-as-judge uses a powerful LLM to evaluate the quality of another model's outputs by scoring or comparing responses. It scales better than human evaluation and correlates well with human judgments for many criteria. Limitations include self-preference bias, inconsistency across runs, sensitivity to evaluation prompt phrasing, and inability to catch factual errors without ground truth.

For retrieval: precision@k, recall@k, MRR, and NDCG to measure document relevance. For generation: faithfulness (is the answer grounded in context?), answer relevance (does it address the question?), and completeness. End-to-end: accuracy against ground truth, hallucination rate, latency, and cost per query. Use frameworks like RAGAS or custom evaluation scripts.

Collect diverse, representative examples from real usage patterns. Include edge cases, adversarial inputs, and failure modes. Have domain experts create ground truth answers. Ensure coverage across different input types, lengths, and topics. Version the test set and keep it separate from training data. Update periodically to reflect evolving usage. Aim for 100-1000 examples depending on task complexity.

Compare against known ground truth databases or verified sources. Use claim decomposition to break responses into individual facts and verify each. Implement automated fact-checking pipelines using knowledge bases. Use NLI (natural language inference) models to check consistency between claims and source documents. Track hallucination rate as a key metric. Human spot-checking remains essential for novel claims.

A/B testing compares two model versions with real users by randomly assigning traffic. Design: define a clear hypothesis and primary metric, calculate required sample size for statistical power, randomise at the user level to avoid cross-contamination, run for sufficient duration to capture temporal patterns, and use proper statistical tests (not just point estimates) for decision making.

Red teaming involves deliberately trying to make AI systems fail or produce harmful outputs through creative adversarial inputs. This includes prompt injection attempts, jailbreaking, edge case exploitation, and bias probing. It uncovers vulnerabilities that standard testing misses. Effective red teaming requires diverse evaluators with different backgrounds and perspectives to cover a wide attack surface.

Measure P50, P95, and P99 latency at each pipeline stage (preprocessing, model inference, postprocessing). Profile with tools like NVIDIA Nsight for GPU utilisation. Optimise by reducing model size (quantisation, distillation), batching requests, using faster hardware, implementing caching, and optimising tokenisation. Set latency SLAs based on user experience requirements.

An evaluation rubric defines criteria and scoring scales for assessing output quality. For LLM outputs, define dimensions like correctness, completeness, clarity, formatting, and safety. Create clear scoring guidelines with examples for each level (e.g., 1=incorrect, 3=partially correct, 5=fully correct). Use the rubric consistently across human evaluators and as guidance for LLM-as-judge prompts.

Run evaluations multiple times and report mean and variance. Use temperature 0 for reproducible baseline comparisons. For sampling-based evaluation, use enough samples for statistical significance. Set random seeds where possible. Use robust metrics that tolerate minor variations. Compare distributions of scores rather than individual outputs. Report confidence intervals alongside point estimates.

Evaluate task completion rate (does it achieve the goal?), step efficiency (how many steps?), tool usage accuracy (correct tool selection and parameters), cost (total tokens consumed), error recovery (can it handle failures?), safety (no harmful actions), and reasoning quality (is the thinking sound?). Also measure user experience metrics like response time and transparency of reasoning.

A model card documents the model's intended use, training data, evaluation results across subgroups, limitations, and ethical considerations. Include performance metrics disaggregated by demographic groups. Document known failure modes and biases. Specify the evaluation methodology and datasets used. Include environmental impact of training. Follow templates from Google or Hugging Face for consistency.

Development evaluation uses curated test sets to make go/no-go decisions before deployment, focusing on model quality and safety. Production monitoring tracks live performance metrics, user feedback, latency, error rates, and data drift continuously. Production monitoring catches issues that test sets miss, like distribution shifts, real-world edge cases, and gradual performance degradation over time.

LLMOps adapts MLOps practices for LLM-based applications. Key differences include prompt management and versioning instead of feature engineering, evaluation through LLM-as-judge and human preference, fine-tuning with PEFT instead of full training, managing API costs and rate limits, monitoring for hallucination and safety issues, and handling much larger model artifacts and compute requirements.

Evaluate: query performance (latency at scale), scalability (data size and QPS), index types supported (HNSW, IVF), filtering capabilities (metadata filters), managed vs self-hosted, cost model, backup and replication, cloud region availability (Indian regions), API compatibility with your stack, and community/support quality. Popular choices include Pinecone, Weaviate, Qdrant, Milvus, and pgvector.

Cache at multiple levels: model weights in GPU memory (avoid reloading), KV cache for conversation history, query-result cache for frequent queries (Redis/Memcached), and embedding cache for common inputs. Implement cache invalidation based on model version changes. Use LRU eviction for memory-bounded caches. Monitor cache hit rates and adjust sizing accordingly.

Serverless inference (AWS Lambda, Google Cloud Functions, Azure Functions) auto-scales to zero when idle and scales up on demand. It eliminates infrastructure management and charges only for actual usage. Appropriate for low-traffic or bursty ML endpoints with small models. Not suitable for large models (memory limits), GPU workloads, or latency-sensitive applications due to cold start times.

Use transfer learning with a pre-trained CNN like EfficientNet fine-tuned on labelled crop disease images. The model should handle diverse Indian crops and work offline on low-end devices via model quantisation. Include data augmentation for varying lighting and camera quality. Deploy as a lightweight mobile app with results in local languages.

Interpretability means a model's internal logic is inherently understandable (e.g., decision trees), while explainability uses post-hoc methods (SHAP, LIME) to approximate why a complex model made a decision. In regulated Indian industries like banking (RBI guidelines) and insurance (IRDAI), explainability is often mandated for automated decisions affecting customers.

Reinforcement Learning from Human Feedback trains a reward model on human preferences about response quality, then uses PPO or similar algorithms to fine-tune the LLM to maximise that reward. It aligns model outputs with human values, making responses more helpful, harmless, and honest compared to pure supervised fine-tuning.

KV cache stores the key and value tensors computed for previously processed tokens so they do not need to be recomputed at each generation step. This reduces inference time from O(n^2) to O(n) per token. However, it requires significant GPU memory, especially for long sequences and large batch sizes.

In FP16, each parameter takes 2 bytes, so 70B parameters need approximately 140 GB just for model weights. Add KV cache memory which depends on context length and batch size. A rough estimate for serving would be 160-200 GB, requiring multiple A100 (80GB) or H100 GPUs with tensor parallelism. Quantisation to INT4 can reduce this to around 40 GB.

Emergent abilities are capabilities that appear only above certain model scale thresholds and are not present in smaller models. Examples include in-context learning, chain-of-thought reasoning, and few-shot arithmetic. These are debated in the research community, with some arguing they may be artefacts of evaluation metrics rather than true phase transitions.

Quantisation reduces model weights from higher precision (FP32/FP16) to lower precision (INT8/INT4), dramatically reducing memory usage and improving inference speed. Trade-offs include slight quality degradation, especially for complex reasoning tasks. Techniques like GPTQ, AWQ, and GGML enable 4-bit quantisation with minimal quality loss for many applications.

Speculative decoding uses a smaller draft model to generate candidate tokens quickly, then the larger target model verifies them in a single forward pass. If the draft tokens match what the target would have generated, multiple tokens are accepted at once. This can achieve 2-3x speedup without changing the output distribution.

Choose a model with strong multilingual capabilities (like GPT-4, Claude, or IndicBERT for embedding). Implement language detection for routing. Use RAG with a knowledge base containing Indian language content. Fine-tune on Indian language conversation data if quality is insufficient. Handle transliteration (Hindi in Roman script) and code-mixing (Hinglish) explicitly.

Distillation trains a smaller student model to mimic the outputs of a larger teacher model, resulting in a genuinely smaller architecture. Quantisation keeps the same architecture but reduces the numerical precision of weights. Distillation requires retraining but can achieve better compression ratios; quantisation is faster to apply but limited in compression.

Self-consistency generates multiple reasoning paths for the same question using sampling (higher temperature) and selects the most common answer via majority voting. It improves reliability on reasoning tasks by reducing the impact of individual reasoning errors. It increases cost proportionally to the number of samples but can significantly boost accuracy on complex tasks.

Break extraction into stages: first identify document type (Aadhaar, PAN, invoice), then extract specific fields per type. Use structured output format with field-level confidence scores. Handle multiple Indian scripts and transliterations. Provide examples of each document type. Include validation rules (PAN format: ABCDE1234F, Aadhaar: 12 digits). Chain extraction and validation prompts.

Tree-of-thought (ToT) explores multiple reasoning branches simultaneously, evaluates partial solutions, and backtracks from unpromising paths. Unlike linear chain-of-thought, ToT considers alternative approaches and self-evaluates. It is particularly useful for complex planning, puzzle-solving, and creative tasks where the first reasoning path may not be optimal.

Constitutional AI uses a set of principles (a constitution) to guide model behaviour through self-critique and revision. The model generates responses, critiques them against the constitution, and revises accordingly. In prompt engineering, similar principles can be embedded in system prompts to create self-correcting behaviour without explicit human feedback on each response.

Chain 1: Extract structured data (name, education, skills, experience, certifications). Chain 2: Score against job requirements (tech stack match, years of experience, education level). Chain 3: Generate a summary highlighting strengths and gaps. Include Indian education system context (IIT/NIT/IIIT tiers, GATE scores, Indian certifications). Handle varied resume formats and Indianised English.

Use the target language for examples and instructions when the model supports it. Handle code-mixing (Hinglish, Tanglish) explicitly. Test tokenisation efficiency for non-Latin scripts. Provide transliterated alternatives. Use language-specific formatting conventions. Test output quality in each language separately and adjust temperature. Consider language-specific evaluation metrics.

Route a percentage of traffic to each prompt variant. Define clear metrics (accuracy, user satisfaction, latency, cost). Ensure sufficient sample size for statistical significance. Log all inputs, outputs, and metadata. Use automated evaluation with LLM judges plus manual review of edge cases. Implement gradual rollout with rollback capability. Monitor for regression over time.

Meta-prompts are prompts designed to generate or optimise other prompts. You describe the task and constraints to an LLM and ask it to produce an effective prompt. This is useful for rapid prototyping, discovering non-obvious prompting strategies, and automating prompt optimisation. Tools like DSPy automate this meta-prompting process with optimisation algorithms.

Specify the legal domain (corporate, criminal, tax under Indian statutes). Instruct the model to preserve legal terminology and statutory references accurately. Include disclaimers that summaries are not legal advice. Structure output with key findings, relevant sections/acts, and actionable items. Handle references to Indian acts (IT Act, Companies Act) and court hierarchies. Test against verified legal summaries.

Ingest PDFs of policy documents with structure-aware parsing for Indian government formatting. Handle Hindi and English content with multilingual embeddings. Implement metadata filtering by ministry, date, and policy type. Use hybrid search for exact policy numbers and semantic queries. Include citation generation with specific section references. Handle amendments and superseded policies.

For tables, extract and store them as structured text or markdown. For images, generate text descriptions using vision models and index those descriptions. For charts, extract underlying data points. Use multi-modal embedding models that can embed both text and images into the same space. Store modality metadata for filtering and presentation.

Contextual retrieval enriches each chunk with additional context about its position and meaning within the broader document before embedding. For example, prepending a chunk with a brief summary of the document or section it came from. This reduces the ambiguity of isolated chunks and improves retrieval accuracy, especially for chunks that reference external context.

Store access control metadata (tenant, department, role) with each chunk. Apply metadata filters at query time to enforce permissions. Integrate with the enterprise's existing IAM (like Active Directory). Ensure vector database supports metadata filtering efficiently. Implement audit logging for compliance. Handle Indian data localisation requirements by keeping sensitive data in India-based storage.

Cache frequent queries and their results. Use approximate nearest neighbour indices for faster search. Implement query embedding caching. Batch vector lookups. Use streaming for LLM generation. Pre-compute and store embeddings for static content. Optimise chunk sizes to reduce the number of retrievals. Consider using smaller, faster models for retrieval stages.

Graph RAG combines knowledge graphs with vector-based retrieval. It constructs a graph of entities and relationships from documents, enabling multi-hop reasoning across connected information. When a query arrives, it retrieves both vector-similar chunks and graph-connected entities, providing richer context. It excels at questions requiring reasoning across multiple related documents.

HyDE generates a hypothetical answer to the query using an LLM, then embeds that hypothetical answer to search the vector store instead of embedding the query directly. Since the hypothetical answer is closer in semantic space to actual document chunks than the question is, it often retrieves more relevant results, especially for complex queries.

Agentic RAG adds an agent layer that can iteratively decide whether to retrieve more information, refine queries, switch data sources, or synthesise across multiple retrievals. Unlike standard RAG with a fixed retrieve-then-generate flow, agentic RAG adapts its retrieval strategy based on initial results, enabling complex multi-step information gathering.

Collect domain-specific query-document pairs from search logs or manually curated examples. Create training triplets (query, positive document, negative document) using hard negative mining. Fine-tune a pre-trained embedding model (like BGE or E5) using contrastive loss. Evaluate on a held-out domain-specific test set using retrieval metrics. This typically yields significant improvement over general embeddings.

Expose tools for checking transaction status, viewing payment history, and generating payment links. Use resources for account balance and recent transactions. Implement strict authentication with OAuth 2.0 and require human approval for any payment-related actions. Handle UPI-specific formats (VPA validation, transaction IDs). Comply with RBI guidelines for data handling and ensure audit logging.

Sampling allows MCP servers to request LLM completions through the client, enabling agentic behaviours within tools. The server sends a sampling/createMessage request to the client, which routes it to the LLM. This enables sophisticated patterns like tool chains where one tool's execution requires LLM reasoning as an intermediate step, while keeping the human in the loop.

Create separate MCP servers for each domain: HR (employee data, leave management), Finance (invoicing, expense reports), CRM (customer data), and IT (ticket management). Implement a gateway server for authentication and routing. Use SSE transport for shared remote servers. Standardise tool naming and error handling across servers. Document tools for both LLMs and developers.

MCP supports resource subscriptions where clients can subscribe to specific resource URIs. When the underlying data changes, the server sends a notifications/resources/updated notification. The client can then re-read the resource to get the latest data. This enables live dashboards, real-time monitoring, and collaborative features where AI assistants track changing data.

Use progress notifications to report incremental status updates to the client. Implement timeouts with configurable limits. For very long operations, return a task ID and provide a separate status-checking tool. Support cancellation through the MCP cancellation mechanism. Avoid blocking the server's event loop. Consider breaking long operations into smaller, resumable steps.

Map existing function definitions to MCP tool schemas. Create an MCP server that wraps the existing tool implementations. Update the host application to use an MCP client instead of direct function calling. Migrate incrementally, running both systems in parallel. Test tool discovery and invocation thoroughly. Update error handling to use MCP's error format. The tool logic itself stays the same.

Give the agent tools for data loading (CSV, SQL, APIs), Python code execution for analysis, chart generation, and report writing. Implement a sandboxed code execution environment. Include data profiling as a first step. Handle large datasets with sampling strategies. Add validation tools to verify statistical conclusions. Include guardrails against executing harmful code.

Implement input validation before tool execution. Set maximum iteration and cost limits. Use content filtering on outputs. Require human approval for high-impact actions (sending emails, modifying data). Implement allowlists for permitted tool actions. Add circuit breakers for repeated failures. Log all actions for audit trails. Monitor for anomalous behaviour patterns.

Equip the agent with tools for order lookup, return processing, refund status, and FAQ search. Support multilingual interactions (Hindi, English, and regional languages). Implement escalation to human agents for complex issues. Include tools for checking shipping status with Indian logistics providers (Delhivery, BlueDart). Handle Indian-specific payment issues (UPI failures, COD returns). Add sentiment detection for priority routing.

Use a persistent state store (database or Redis) for workflow state. Implement checkpointing at each step so workflows can resume after failures. Serialise state in a structured format with version tracking. Handle concurrent access with locking mechanisms. Implement state cleanup for abandoned workflows. Use event sourcing for full auditability of state changes.

Provide tools for file read/write, code execution in a sandbox, test running, and linting. Implement an iterative loop: generate code, run tests, analyse failures, fix issues. Use the LLM to reason about error messages and fix bugs. Include static analysis tools. Sandbox execution for security. Limit iterations and set timeouts. Log all generated code for review.

Create specialised agents: a classifier agent for categorising content type, a policy checker agent with access to moderation guidelines, a context analyser for understanding cultural nuances (important for Indian content), and a decision agent that synthesises verdicts. Implement escalation to human moderators for borderline cases. Use voting or consensus mechanisms across agents for reliability.

Stream the LLM's reasoning tokens as they are generated for transparency. Send tool execution status updates as events. Implement partial result streaming for long operations. Use WebSockets or SSE for the client connection. Display thinking indicators and progress bars. Allow users to interrupt and redirect the agent mid-execution. Buffer events for connection resilience.

Define a standardised task suite with varying complexity. Measure success rate, average steps to completion, cost, latency, and error recovery effectiveness. Test on edge cases and adversarial inputs. Compare single-agent vs multi-agent, different LLM backends, and various prompting strategies. Use frameworks like AgentBench or custom evaluation harnesses. Run multiple trials for statistical significance.

Agents face prompt injection through tool outputs, unintended tool execution that modifies systems, data exfiltration through crafted tool calls, privilege escalation by chaining permissions, and denial-of-service through infinite loops. Mitigate with strict input validation, least-privilege tool access, output sanitisation, action approval workflows, and comprehensive audit logging.

Equip the agent with tools for monitoring (Grafana/CloudWatch queries), deployment (kubectl, CI/CD triggers), log analysis, and alert management. Implement strict approval workflows for production changes. Start with read-only tools and gradually add write capabilities with human approval. Handle Indian cloud regions (Mumbai, Hyderabad). Include incident response runbooks as context. Monitor agent actions carefully during the initial rollout period.

QLoRA combines quantisation with LoRA. It loads the base model in 4-bit precision (NormalFloat4) and adds LoRA adapters in higher precision. This reduces memory requirements by approximately 4x compared to LoRA alone, enabling fine-tuning of 65B parameter models on a single 48GB GPU. It uses techniques like double quantisation and paged optimisers for memory efficiency.

Collect Indian legal documents (court judgments, statutes, legal opinions) and create instruction pairs for tasks like summarisation, clause extraction, and case law reference. Include Indian legal terminology and citation formats. Use QLoRA on a capable base model. Validate against expert-labelled outputs. Handle multiple Indian languages. Ensure the model correctly references Indian statutes and precedents.

Collect domain-specific query-passage pairs from search logs or expert annotation. Create training triplets with hard negatives (similar but irrelevant passages). Fine-tune using contrastive learning (InfoNCE or multiple negatives ranking loss). Evaluate with retrieval metrics on a held-out test set. Use a pre-trained model like BGE or E5 as the base. Typically 5K-50K pairs suffice.

Combine datasets from multiple tasks with task-specific prefixes or instruction formats. Balance dataset sizes to prevent dominant tasks from overwhelming others. Use weighted sampling or data mixing ratios. Monitor performance on each task separately. Consider using task-specific LoRA adapters that can be loaded independently. Evaluate for cross-task transfer and interference.

Use on-premise or private cloud GPU infrastructure. Apply differential privacy during training to prevent memorisation. Anonymise or pseudonymise PII in training data. Use federated fine-tuning when data cannot leave its source. Implement access controls on model artifacts. Audit the model for data leakage using membership inference tests. Comply with India's DPDP Act requirements.

Test with adversarial prompts designed to elicit harmful outputs. Evaluate fairness across demographic groups using disaggregated metrics. Test for common biases (gender, caste, regional in Indian context). Use red-teaming with diverse evaluators. Check for PII leakage. Evaluate content safety with toxicity classifiers. Test prompt injection resistance. Document findings in a model card.

Integrate eval scripts into the CI pipeline triggered by model or prompt changes. Store test sets and expected outputs in version control. Run automated metrics computation and LLM-as-judge evaluations. Set quality gates with minimum threshold scores. Generate evaluation reports comparing against baselines. Alert on regressions. Use tools like promptfoo, Braintrust, or custom evaluation frameworks.

Design multi-turn test scenarios with realistic conversation flows. Evaluate context retention (does the model remember earlier information?), coherence (are responses consistent?), and task completion over the full conversation. Measure conversation success rate, turns to resolution, and user satisfaction. Test with conversation branching and topic switches. Track per-turn and per-conversation metrics separately.

Create test cases with coding problems at various difficulty levels. Measure functional correctness (do generated solutions pass test cases?), code quality (readability, efficiency), and security (no vulnerabilities). Use pass@k metric for solution accuracy. Test across multiple languages and frameworks. Evaluate explanation quality alongside code. Benchmark against existing solutions and human performance.

Develop task-specific test sets covering major Indian languages (Hindi, Tamil, Telugu, Bengali, etc.). Include code-mixed data (Hinglish, Tanglish). Address script variations and transliterations. Recruit native speaker evaluators for each language. Account for dialectal variations. Create culturally relevant test cases. Measure per-language performance to identify disparities. Share benchmarks with the community.

Create a test suite of known injection attacks: direct instruction override, indirect injection through context, encoding-based attacks, and multi-step manipulation. Measure the rate at which the model follows injected instructions versus maintaining its original behaviour. Test regularly as new attack patterns emerge. Implement continuous monitoring in production for unusual output patterns.

Use GPU-optimised inference servers (vLLM, TGI, or Triton) with continuous batching. Quantise the model to INT8/INT4 for speed. Implement KV cache efficiently. Deploy in regions close to users (Indian cloud regions). Use load balancing with health checks. Implement request queuing and timeout handling. Pre-warm model caches. Monitor P99 latency and auto-scale based on queue depth.

vLLM is a high-throughput LLM serving engine that introduces PagedAttention for efficient KV cache memory management. Like virtual memory in operating systems, it pages KV cache blocks, reducing memory waste from 60-80% to near zero. This enables 2-4x higher throughput than naive implementations. It also supports continuous batching and speculative decoding.

Continuous batching (iteration-level batching) dynamically adds and removes requests from a batch at each generation step, unlike static batching which waits for all requests to complete. This prevents short requests from waiting for long ones, significantly improving GPU utilisation and throughput (2-10x). It is implemented in modern serving frameworks like vLLM and TGI.