Machine Learning Engineer

AI refers to systems that can perform tasks typically requiring human intelligence, such as learning, reasoning, and perception. Unlike traditional software that follows explicit rules, AI systems learn patterns from data and can generalise to new inputs.

AI is the broadest term covering any technique that enables machines to mimic human intelligence. ML is a subset of AI where models learn from data without explicit programming. Deep Learning is a subset of ML using neural networks with multiple layers to learn hierarchical representations.

Supervised learning uses labelled data (e.g., spam detection). Unsupervised learning finds patterns in unlabelled data (e.g., customer segmentation for Indian e-commerce platforms). Reinforcement learning learns via rewards (e.g., training a robot to navigate a warehouse).

A neural network is a computational model composed of layers of interconnected nodes (neurons) that process inputs through weighted connections. It is inspired by biological neurons but the analogy is loose. Each artificial neuron applies a weighted sum followed by a non-linear activation function.

Indian companies use AI extensively in fintech (credit scoring for underbanked populations), agriculture (crop disease detection), healthcare (radiology screening at scale), e-commerce (recommendation engines and vernacular search), and customer support (multilingual chatbots supporting Indian languages).

AI is like teaching a computer to learn from examples rather than following fixed rules. For instance, instead of writing rules to detect fraudulent UPI transactions, we show the system millions of past transactions labelled as fraud or legitimate, and it learns the patterns itself. It gets better with more data and can handle cases we never explicitly programmed.

Data is the foundation of AI. Models learn patterns from data, so poor quality data with noise, missing values, duplicates, or biases leads to unreliable models. The garbage-in-garbage-out principle applies strongly. Data cleaning, validation, and augmentation are critical preprocessing steps before model training.

Classification predicts discrete categorical labels (e.g., spam or not spam, disease type), while regression predicts continuous numerical values (e.g., house price, temperature). Classification uses metrics like accuracy, precision, and recall, while regression uses MSE, MAE, and R-squared.

An LLM is a neural network trained on massive text corpora to predict the next token in a sequence. It generates text autoregressively by sampling from a probability distribution over its vocabulary, one token at a time, using the previously generated tokens as context. Models like GPT-4 and Claude use the Transformer architecture.

Pre-training involves training a model on massive unlabelled text data with a self-supervised objective like next-token prediction. This gives the model general language understanding. Fine-tuning then adapts this pre-trained model to specific downstream tasks using smaller labelled datasets, transferring the knowledge learned during pre-training.

Python offers a rich ecosystem of ML libraries (NumPy, pandas, scikit-learn, PyTorch, TensorFlow), clean syntax that enables rapid prototyping, strong community support, and excellent interoperability with C/C++ for performance-critical components. Its interpreted nature allows interactive development through Jupyter notebooks, which is ideal for experimentation.

Python lists are dynamic, can hold mixed types, and support general-purpose operations. NumPy arrays are fixed-type, contiguous in memory, and support vectorised operations that are 10-100x faster for numerical computation. Use lists for general data structures and NumPy arrays for any numerical or matrix computation in ML pipelines.

Use NumPy: def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)). This computes the dot product of the vectors divided by the product of their magnitudes. The result ranges from -1 (opposite) to 1 (identical direction). Add epsilon to the denominator to prevent division by zero.

Use virtual environments (venv, conda, or poetry) to isolate project dependencies. Pin exact versions in requirements.txt or pyproject.toml for reproducibility. Use conda for complex native dependencies like CUDA. Docker containers ensure full environment reproducibility. Tools like pip-tools or poetry provide dependency resolution and lock files.

Type hints annotate function parameters and return types (e.g., def predict(input: np.ndarray) -> np.ndarray). They improve code readability, enable static analysis tools like mypy to catch bugs before runtime, and provide better IDE support. They are critical for ML production code where data type mismatches cause silent errors.

First understand the missing data pattern (MCAR, MAR, MNAR) using df.isnull().sum() and missingno library. Options include dropping rows/columns with too many missing values, imputing with mean/median/mode, using advanced imputation like KNN or iterative imputer, or using algorithms that handle NaN natively like XGBoost. Document your strategy for reproducibility.

A scalar is a single number (0D), a vector is a 1D array of numbers, a matrix is a 2D array, and a tensor is a generalisation to any number of dimensions. Each is a special case of a tensor. In ML, model weights are stored as tensors, and frameworks like PyTorch and NumPy operate on these n-dimensional arrays.

The dot product of two vectors is the sum of their element-wise products, producing a scalar. It measures similarity between vectors. In neural networks, each neuron computes a dot product between its weights and inputs, followed by a bias and activation function. Matrix multiplication, which powers layer computations, is built from dot products.

Gradient descent is an iterative optimisation algorithm that updates parameters in the direction opposite to the gradient of the loss function. The gradient points in the direction of steepest ascent, so moving against it reduces the loss. The learning rate controls step size. Variants include SGD, Adam, and AdaGrad, each handling learning rates differently.

Softmax converts a vector of real numbers (logits) into a probability distribution by exponentiating each element and normalising by the sum: softmax(z_i) = exp(z_i) / sum(exp(z_j)). It ensures outputs are positive and sum to 1, making them interpretable as class probabilities. It is used in the final layer of multi-class classification networks.

A probability distribution describes how probabilities are assigned to different outcomes. The Gaussian (normal) distribution is bell-shaped, defined by mean and variance, and arises naturally due to the Central Limit Theorem. The Bernoulli distribution models binary outcomes with probability p for success. These are foundational in ML for modelling data and defining loss functions.

Correlation means two variables tend to change together but does not imply one causes the other. Causation requires a direct mechanism. ML models primarily learn correlations, which can lead to spurious predictions when the underlying causal relationship changes. Understanding this distinction is critical for model robustness, fairness, and deployment in high-stakes domains.

Cosine similarity measures the angle between vectors (direction), ranging from -1 to 1. Euclidean distance measures the straight-line distance between points (magnitude matters). Use cosine similarity when only direction matters (text embeddings, recommendation systems). Use Euclidean distance when magnitude is meaningful (spatial data, anomaly detection based on deviation).

A deterministic model always produces the same output for the same input. A stochastic model incorporates randomness and may produce different outputs each time. Most ML training is stochastic (random initialisation, mini-batch sampling, dropout). Stochasticity often helps by preventing overfitting and escaping local minima during optimisation.

Prompt engineering is the practice of designing and refining inputs to LLMs to elicit desired outputs. It matters because LLMs are highly sensitive to how instructions are framed. Good prompts can dramatically improve output quality, accuracy, and relevance without any model changes, making it a cost-effective alternative to fine-tuning for many use cases.

Zero-shot gives the model a task description without examples. One-shot includes one example of the expected input-output pair. Few-shot provides multiple examples to establish the pattern. Few-shot typically improves performance on complex or ambiguous tasks by helping the model understand the desired format and reasoning style through demonstration.

The system prompt sets the AI's behaviour, persona, and constraints across the conversation. User prompts contain the human's messages and queries. Assistant prompts contain the AI's responses and can be pre-filled to guide output format. This separation enables consistent behaviour while allowing dynamic user interactions. Most API providers support this three-role structure.

Temperature controls output randomness. Use low temperature (0-0.3) for factual, deterministic tasks like classification and extraction. Use higher temperature (0.7-1.0) for creative tasks like brainstorming. Top-p and top-k further shape the probability distribution. These parameters interact with prompt design; well-constrained prompts can tolerate higher temperatures.

Delimiters (like triple backticks, XML tags, or separators) clearly mark boundaries between instructions, context, and user input. They prevent prompt injection by isolating user-provided content, improve parsing of structured prompts, and help the model distinguish between what to process and how to process it. Consistent delimiter usage improves output reliability.

RAG combines information retrieval with LLM generation. It retrieves relevant documents from a knowledge base and includes them in the prompt context, allowing the LLM to generate answers grounded in specific data. RAG reduces hallucination, enables access to private/current data without fine-tuning, and is more cost-effective than retraining models.

A RAG pipeline consists of: (1) document ingestion and chunking, (2) embedding generation for chunks, (3) vector storage in a database, (4) query embedding and similarity search at inference time, (5) context assembly with retrieved chunks, and (6) LLM generation using the augmented prompt. Each component can be optimised independently.

A vector database stores and indexes high-dimensional vector embeddings for efficient similarity search. Unlike traditional databases that use exact matching on structured fields, vector databases use approximate nearest neighbour (ANN) algorithms like HNSW or IVF for fast similarity search. Popular options include Pinecone, Weaviate, Qdrant, ChromaDB, and pgvector.

MCP is an open protocol that standardises how AI applications connect to external data sources and tools. It solves the M x N integration problem where each AI app would need custom integrations with each tool. With MCP, tools implement the protocol once and any MCP-compatible AI application can use them, similar to how USB standardised device connections.

Tool discovery is the process by which an MCP client learns what tools a server offers. The client calls the tools/list method, and the server responds with a list of available tools including their names, descriptions, and input schemas (JSON Schema). The host then presents these tools to the LLM so it can decide when and how to use them.

Official SDKs exist for TypeScript/JavaScript and Python, which are the most popular choices. Community SDKs are available for Kotlin, Go, Rust, C#, and other languages. The TypeScript SDK is the most mature with the best documentation. Choose based on your team's expertise and the ecosystem of the services you are integrating with.

An AI agent is an autonomous system that can perceive its environment, reason about goals, plan actions, and execute them using tools. Unlike a chatbot that responds to individual messages, an agent maintains state across interactions, can break complex tasks into sub-tasks, invoke external tools, and iterate on results until a goal is achieved.

Fine-tuning adapts a pre-trained model to a specific task by continuing training on task-specific data. Use it when prompt engineering alone is insufficient, you need consistent output formatting, domain-specific knowledge integration, or significant behaviour changes. It is more expensive than prompting but cheaper than training from scratch.

Evals are systematic methods for measuring AI system performance across quality, safety, and reliability dimensions. They are critical because they prevent deploying underperforming or harmful models, enable data-driven decision making between model versions, catch regressions early, and build stakeholder confidence. Without evals, AI development is essentially flying blind.

Offline evaluation tests models on static datasets before deployment using metrics like accuracy, F1, and perplexity. Online evaluation measures model performance in production with real users through A/B testing, user satisfaction, and business metrics. Offline catches obvious issues early; online captures real-world performance that test sets cannot fully represent. Both are essential.

Accuracy is the fraction of all predictions that are correct. Precision is the fraction of positive predictions that are correct (low false positive rate). Recall is the fraction of actual positives that are correctly identified (low false negative rate). F1 is the harmonic mean of precision and recall, useful when both matter equally. Use the metric that aligns with the cost of errors in your application.

Cross-validation splits data into k folds, trains on k-1 folds and evaluates on the remaining fold, rotating through all folds. It provides a more reliable performance estimate than a single train-test split, especially with limited data. Use k-fold (typically k=5 or 10) for model selection, stratified k-fold for imbalanced classes, and leave-one-out for very small datasets.

MLOps (Machine Learning Operations) applies DevOps principles to ML systems, covering the full lifecycle from data preparation through model deployment and monitoring. It is important because most ML models fail to reach production without proper engineering practices. MLOps ensures reproducibility, reliability, scalability, and continuous improvement of ML systems in production environments.

An ML pipeline includes data ingestion and validation, feature engineering and storage, model training and hyperparameter tuning, model evaluation and validation, model registry and versioning, deployment (serving infrastructure), monitoring and alerting, and feedback loops for retraining. Each component should be automated, reproducible, and version-controlled.

Batch inference processes large volumes of data periodically (hourly, daily) and stores results for later use. Real-time inference processes individual requests with low latency (milliseconds). Batch is simpler and cheaper for recommendations or scoring; real-time is needed for fraud detection, search ranking, and conversational AI. Many systems use both: batch for pre-computation and real-time for personalisation.

Experiment tracking records hyperparameters, metrics, code versions, data versions, and artifacts for each training run. It enables comparison across experiments and reproducibility. Popular tools include MLflow (open-source, self-hosted), Weights & Biases (cloud-based with rich visualisation), Neptune, and Comet. Choose based on team size, budget, and integration requirements.

Model deployment is the process of making a model available in a production environment, including infrastructure setup, configuration, and release management. Model serving is the runtime system that handles inference requests, including request batching, model loading, scaling, and response formatting. Deployment is a one-time event; serving is the ongoing operational system.

Docker containers package the model, code, dependencies, and runtime into a portable, reproducible unit. This eliminates 'works on my machine' issues, ensures consistent behaviour across environments, simplifies deployment, and enables scaling via orchestrators like Kubernetes. Use multi-stage builds to minimise image size and separate training from serving containers.

GPUs (NVIDIA A100, H100) are the dominant hardware for training and inference due to their massive parallelism. TPUs (Google's custom chips) are optimised for tensor operations. CPUs handle preprocessing and lightweight inference. Specialised accelerators like AWS Inferentia target inference cost-efficiency. FPGAs offer programmable acceleration for specific workloads.

CUDA is NVIDIA's parallel computing platform that enables general-purpose GPU programming. It provides APIs for launching parallel computations on GPU cores. Deep learning frameworks (PyTorch, TensorFlow) use CUDA internally for accelerated tensor operations. CUDA's ecosystem including cuBLAS, cuDNN, and NCCL is the primary reason NVIDIA dominates the AI hardware market.

Vertical scaling adds more resources (bigger GPU, more memory) to a single instance, limited by hardware maximums. Horizontal scaling adds more instances behind a load balancer, limited by coordination complexity. For ML serving, use vertical scaling for model size constraints (larger model needs bigger GPU) and horizontal scaling for throughput. Most production systems combine both approaches.

AI bias occurs when a model produces systematically unfair outcomes due to biased training data, flawed feature selection, or skewed labelling. This matters because biased models can discriminate against underrepresented groups, especially critical in applications like loan approvals, hiring, and criminal justice.

Transfer learning involves taking a model pre-trained on a large dataset and fine-tuning it on a smaller, task-specific dataset. It reduces training time and data requirements dramatically. Models like BERT and ResNet are commonly used as pre-trained bases for NLP and computer vision tasks.

Parametric models have a fixed number of parameters regardless of data size (e.g., linear regression, logistic regression). Non-parametric models grow in complexity with more data (e.g., k-NN, decision trees). Parametric models are faster but may underfit; non-parametric models are more flexible but can overfit.

Overfitting occurs when a model memorises training data and performs poorly on unseen data, detected by a large gap between training and validation accuracy. Underfitting means the model is too simple to capture patterns, shown by poor performance on both sets. Regularisation, dropout, and cross-validation address overfitting; increasing model complexity or features addresses underfitting.

The bias-variance tradeoff describes the tension between a model's ability to fit training data (low bias) and its ability to generalise to new data (low variance). High bias leads to underfitting while high variance leads to overfitting. The goal is to find the optimal complexity that minimises total error.

Start with exploratory data analysis and feature importance analysis. Reduce dimensionality using PCA or feature selection methods. Split data into train/validation/test sets. Begin with a simple baseline like logistic regression, then try gradient boosting (XGBoost/LightGBM) for structured data. Use cross-validation for hyperparameter tuning and evaluate with appropriate metrics.

Discriminative models learn the decision boundary between classes by modelling P(y|x) directly, such as logistic regression and SVMs. Generative models learn the joint probability P(x,y) or P(x|y) to understand how data is generated, such as Naive Bayes and GANs. Generative models can create new data samples while discriminative models typically cannot.

Embeddings are dense, low-dimensional vector representations of high-dimensional data like words, images, or users. They capture semantic relationships so that similar items have nearby vectors. They enable efficient similarity search, are used in recommendation systems, and form the foundation of modern NLP models.

An activation function determines whether a neuron should fire by applying a transformation to the weighted sum of inputs. Without non-linear activations, a multi-layer network would collapse to a single linear transformation, unable to learn complex patterns. Common non-linear activations include ReLU, sigmoid, and tanh.

Backpropagation is an algorithm that computes the gradient of the loss function with respect to each weight by applying the chain rule layer by layer from output to input. These gradients tell the optimiser how to adjust weights to minimise loss. It makes training deep networks computationally feasible.

Batch gradient descent computes gradients over the entire dataset per update, which is stable but slow. Stochastic gradient descent (SGD) updates weights after each sample, which is noisy but fast. Mini-batch gradient descent is the practical middle ground, computing gradients over small batches (32-512 samples), offering both speed and stability.

Key considerations include ensuring fairness across diverse Indian demographics (caste, language, region), respecting data privacy under India's DPDP Act, maintaining transparency in automated decision-making, preventing displacement of workers without reskilling programs, and ensuring AI systems work equitably for rural and urban populations.

Precision is the fraction of positive predictions that are correct; recall is the fraction of actual positives correctly identified. F1 score is their harmonic mean. Prioritise recall in medical diagnosis (missing a disease is costly), precision in spam filtering (false positives are annoying), and F1 when both matter equally.

Regularisation adds a penalty term to the loss function to prevent overfitting. L1 (Lasso) adds the absolute value of weights, encouraging sparsity and effective feature selection. L2 (Ridge) adds the squared weights, shrinking all weights toward zero without eliminating them. Elastic Net combines both approaches.

AutoML automates the process of selecting models, tuning hyperparameters, and engineering features. Tools like Google AutoML, H2O, and Auto-sklearn make ML accessible to non-experts and accelerate prototyping. However, domain expertise and understanding of results remain essential for production deployments.

The Transformer uses self-attention mechanisms to process all tokens in parallel rather than sequentially like RNNs. This enables better long-range dependency capture and massive parallelisation during training. Key components include multi-head attention, positional encoding, feed-forward layers, and layer normalisation.

Attention computes a weighted sum of Value vectors, where the weights are determined by the compatibility between Query and Key vectors. For each token, Q represents what it is looking for, K represents what it offers, and V represents the actual content. The attention score is computed as softmax(QK^T / sqrt(d_k)) * V.

Multi-head attention runs several attention operations in parallel, each with different learned projection matrices. This allows the model to attend to information from different representation subspaces simultaneously. For example, one head might capture syntactic relationships while another captures semantic ones. The outputs are concatenated and linearly projected.

Tokenisation splits text into units (tokens) the model can process. Byte Pair Encoding (BPE) starts with individual characters and iteratively merges the most frequent adjacent pairs to create a vocabulary of subword units. This handles rare words by breaking them into known subwords, balancing vocabulary size with representation quality.

Since Transformers process all tokens in parallel (unlike RNNs), they have no inherent sense of token order. Positional encodings add information about each token's position in the sequence. The original Transformer used sinusoidal functions, while modern models often use learned positional embeddings or rotary positional embeddings (RoPE).

Encoder-only models like BERT process entire input bidirectionally and are great for classification and embeddings. Decoder-only models like GPT generate text autoregressively using causal masking. Encoder-decoder models like T5 use a full encoder for input understanding and a decoder for generation, excelling at translation and summarisation.

Temperature controls randomness by scaling logits before softmax; lower values make output more deterministic. Top-k sampling restricts selection to the k most probable tokens. Top-p (nucleus sampling) selects from the smallest set of tokens whose cumulative probability exceeds p. These parameters trade off between creativity and coherence.

The context window is the maximum number of tokens an LLM can process in a single forward pass. Extending it is challenging because self-attention has O(n^2) computational complexity and memory usage. Techniques like FlashAttention, sparse attention, sliding window attention, and RoPE scaling help extend context windows to 100K+ tokens.

Hallucination occurs when LLMs generate plausible-sounding but factually incorrect information. It happens because LLMs are trained to predict probable next tokens, not verified facts. Mitigation strategies include RAG for grounding in verified sources, chain-of-thought prompting, constrained decoding, and implementing fact-checking pipelines.

Causal language modelling (used by GPT) predicts the next token using only previous tokens, enforced by a causal attention mask. Masked language modelling (used by BERT) randomly masks tokens and predicts them using bidirectional context. Causal LM is suited for generation, while masked LM is better for understanding tasks.

In-context learning is the ability of LLMs to learn new tasks from examples provided in the prompt without updating model weights. Fine-tuning permanently modifies model weights through gradient descent on task-specific data. In-context learning is faster and more flexible but limited by context window size and generally less reliable for complex tasks.

Evaluate on task-specific benchmarks with your actual data, not just public leaderboards. Compare latency, throughput, cost per token, context window size, and output quality. Consider factors like API reliability, data privacy policies, and hosting options (cloud vs self-hosted). Run A/B tests and collect human evaluation scores on representative queries.

Greedy decoding always selects the most probable next token, which is fast but can produce repetitive text. Beam search maintains multiple candidate sequences and selects the highest-probability one, better for translation. Sampling-based methods (top-k, top-p) introduce controlled randomness for more creative and diverse outputs.

Cross-entropy loss measures the average negative log probability of correct tokens. Perplexity is the exponential of cross-entropy and represents the effective number of equally probable token choices the model considers. Lower perplexity indicates better prediction. Perplexity is more interpretable but both metrics are mathematically equivalent for comparison.

Key risks include generating harmful content, producing convincing misinformation, leaking private training data, being manipulated through prompt injection, encoding and amplifying biases, and enabling malicious uses like phishing. Mitigations include RLHF alignment, content filtering, red-teaming, and responsible deployment practices.

Vocabulary size determines how text is segmented. Larger vocabularies produce fewer tokens per text (saving context window) but increase embedding layer size and memory usage. Smaller vocabularies are more efficient memory-wise but create longer sequences. Typical sizes range from 32K to 128K tokens. Multilingual models need larger vocabularies to cover diverse scripts.

Broadcasting is NumPy's mechanism for performing operations on arrays of different shapes. It automatically expands the smaller array to match the larger one without copying data. Rules require dimensions to be either equal or one of them to be 1, aligning from the trailing dimension. This enables concise and efficient vectorised code.

Use chunked reading with pd.read_csv(chunksize=...), specify dtypes to reduce memory, use category type for low-cardinality columns, and select only needed columns. For larger scale, consider Dask, Polars, or Vaex for out-of-core computation. Profile memory with df.info(memory_usage='deep') and optimise column types aggressively.

The Global Interpreter Lock (GIL) prevents multiple threads from executing Python bytecode simultaneously. For CPU-bound ML code, it limits true parallelism. However, most ML libraries (NumPy, PyTorch, TensorFlow) release the GIL during computation by running native C/C++/CUDA code. For true Python parallelism, use multiprocessing instead of threading.

PyTorch uses eager execution by default, making it intuitive for debugging and research with a Pythonic API. TensorFlow uses graph-based execution for better production optimisation and has a more mature serving ecosystem (TF Serving, TFLite). PyTorch dominates research and is increasingly popular in production; TensorFlow is still strong in mobile and edge deployment.

Decorators are functions that modify the behaviour of other functions, applied using the @decorator syntax. In ML codebases, they are used for logging (tracking experiments), timing (measuring function performance), caching (memoising expensive computations), and in frameworks like PyTorch for torch.no_grad() and custom autograd functions.

Generators are functions that yield items one at a time using the yield keyword, maintaining state between calls. They produce items lazily, consuming minimal memory regardless of dataset size. They are essential for data loading pipelines where the entire dataset cannot fit in memory, and PyTorch DataLoader uses generators internally.

Subclass torch.utils.data.Dataset, implement __len__ and __getitem__ methods. In __getitem__, load the image, apply transforms (resize, normalise, augment), and return the tensor and label. Wrap it in a DataLoader with batch_size, shuffle=True for training, num_workers for parallel loading, and pin_memory=True for GPU transfer speed.

Shallow copy creates a new object but references the same nested objects; deep copy recursively copies everything. In ML, this matters when modifying model configurations or data preprocessing pipelines. Accidentally modifying shared references can corrupt data or cause subtle training bugs. Use copy.deepcopy() for model state dicts and configuration objects.

Use cProfile or line_profiler to identify bottlenecks. Replace Python loops with vectorised NumPy/pandas operations. Use Cython or Numba for CPU-intensive functions. For GPU code, use torch.cuda.Event for timing and nvidia-smi for utilisation monitoring. Memory profiling with memory_profiler helps find leaks in data pipelines.

groupby splits a DataFrame by categorical columns, and apply runs a function on each group. For feature engineering, compute group-level statistics like mean, count, or std as features. Example: df.groupby('customer_id')['amount'].transform('mean') adds per-customer average amount as a feature. Use agg for multiple statistics at once.

Multithreading shares memory but is limited by the GIL for CPU tasks, suitable for IO-bound work like data downloading. Multiprocessing creates separate processes with their own memory, bypassing the GIL for true parallelism on CPU-bound tasks. For ML data preprocessing, multiprocessing is preferred; for async API calls, threading or asyncio suffices.

Test each transformation function with known inputs and expected outputs. Test edge cases like empty DataFrames, missing values, and unexpected types. Use pytest fixtures for reusable test data. Test the full pipeline end-to-end with a small sample. Verify output shapes, dtypes, and value ranges. Use hypothesis library for property-based testing of transformations.

asyncio provides an event loop for concurrent IO-bound operations using async/await syntax. In AI applications, use it for making concurrent API calls to LLM providers, handling multiple WebSocket connections for real-time inference, or orchestrating parallel data fetches in a RAG pipeline. It does not help with CPU-bound computation.

Polars is a DataFrame library written in Rust that offers lazy evaluation, multi-threaded execution, and memory-efficient processing. It can be 10-100x faster than pandas on large datasets. Unlike pandas, it uses Apache Arrow columnar format and supports lazy query optimisation. It is increasingly adopted in production ML pipelines where pandas becomes a bottleneck.

__init__ defines the layers and parameters of the module. forward defines the computation graph for the forward pass. __call__ is inherited from nn.Module and wraps forward with additional logic like hooks and autograd tracking. You should always call the module directly (model(x)) rather than model.forward(x) to ensure hooks execute properly.

Pickle serialises Python objects to byte streams for storage and loading. It is commonly used to save scikit-learn models and data. However, pickle can execute arbitrary code during deserialisation, making it a security risk for untrusted sources. Use safer alternatives like ONNX, safetensors, or joblib with known-source files. Never unpickle data from untrusted origins.

Sklearn Pipelines chain preprocessing and model steps so that transformations are fit only on training data and applied consistently to test data. Use Pipeline with ColumnTransformer for feature-specific preprocessing. This prevents leakage from fitting scalers or encoders on the full dataset. Always call pipeline.fit(X_train) and pipeline.predict(X_test) rather than transforming separately.

Context managers implement __enter__ and __exit__ methods and are used with the 'with' statement for resource management. In ML, they are used for managing GPU memory (torch.no_grad(), torch.cuda.amp.autocast()), file handles for large data loading, database connections, and experiment tracking contexts like mlflow.start_run().

Use FastAPI or Flask to create endpoints. Load the model at startup (not per request). Define Pydantic models for request/response validation. Implement async handlers for IO-bound operations. Add health checks, logging, and error handling. Use gunicorn/uvicorn with multiple workers for production. Containerise with Docker and add rate limiting.

Dataclasses provide a decorator to auto-generate __init__, __repr__, and comparison methods for data-holding classes. Pydantic models add runtime validation, serialisation, and type coercion. Use dataclasses for internal data structures like model configs. Use Pydantic for API request/response schemas and configuration files where validation of external inputs is critical.

Eigenvectors of a matrix are vectors whose direction remains unchanged when multiplied by that matrix; eigenvalues are the scaling factors. In ML, they are central to PCA for dimensionality reduction, spectral clustering, understanding covariance structures, and analysing the stability of dynamical systems and recurrent networks.

The chain rule states that the derivative of a composite function f(g(x)) is f'(g(x)) * g'(x). In backpropagation, each layer in a neural network is a composed function, and the chain rule enables computing how the loss changes with respect to each parameter by multiplying local gradients along the computation path.

Bayes' theorem states P(A|B) = P(B|A) * P(A) / P(B), relating posterior probability to likelihood, prior, and evidence. In ML, Naive Bayes classifiers use it for text classification. Bayesian inference updates beliefs about model parameters as new data arrives. It is also fundamental to probabilistic graphical models and Bayesian neural networks.

PCA finds the directions (principal components) of maximum variance in the data by computing the eigenvectors of the covariance matrix. Data is projected onto the top-k eigenvectors to reduce dimensionality while preserving the most variance. Mathematically, it is equivalent to SVD of the centered data matrix. It assumes linear relationships between features.

The L1 norm is the sum of absolute values of vector elements, while the L2 norm is the square root of the sum of squared elements. L1 regularisation (Lasso) promotes sparsity by driving weights to exactly zero, useful for feature selection. L2 regularisation (Ridge) distributes penalty evenly, shrinking all weights. L1 has diamond-shaped constraint regions; L2 has circular ones.

Cross-entropy loss measures the difference between predicted probability distribution and true distribution: -sum(y_i * log(p_i)). It is preferred over MSE for classification because it penalises confident wrong predictions heavily, has better gradient properties (avoids saturation with sigmoid/softmax), and is derived from maximum likelihood estimation for categorical distributions.

In CNNs, convolution slides a learnable kernel (filter) across the input, computing element-wise multiplication and summing at each position. Technically it is cross-correlation (no kernel flip). For a 2D input, output[i,j] = sum(input[i+m, j+n] * kernel[m,n]). This operation extracts local features like edges, textures, and patterns while sharing parameters across spatial positions.

Adam maintains two exponential moving averages: first moment (mean of gradients, like momentum) and second moment (mean of squared gradients, like RMSProp). It adapts per-parameter learning rates using these estimates with bias correction. The update rule is: theta -= lr * m_hat / (sqrt(v_hat) + epsilon). This combination makes it robust across different architectures.

The CLT states that the mean of many independent random variables tends toward a normal distribution, regardless of the original distribution. In ML, this justifies using Gaussian assumptions in many models, underpins statistical testing of model performance, explains why batch normalisation works, and is why many natural phenomena are approximately normally distributed.

A loss function quantifies the discrepancy between model predictions and true values. For regression, use MSE (sensitive to outliers) or MAE (robust). For classification, use cross-entropy. For ranking, use hinge loss or triplet loss. For imbalanced data, use focal loss or weighted cross-entropy. The choice depends on the task, data distribution, and what errors are most costly.

As dimensions increase, data becomes increasingly sparse, and distance metrics become less meaningful because points tend to be equidistant. This makes nearest-neighbour methods ineffective, increases the data needed for reliable estimation exponentially, and complicates optimisation. Dimensionality reduction techniques like PCA, t-SNE, or autoencoders help mitigate this problem.

Batch normalisation normalises each layer's inputs to zero mean and unit variance across the mini-batch, then applies learnable scale and shift parameters. Mathematically, it reduces internal covariate shift, smooths the loss landscape, and allows higher learning rates. It acts as a regulariser due to the noise introduced by per-batch statistics.

Entropy measures the uncertainty or randomness in a probability distribution: H = -sum(p_i * log(p_i)). Maximum entropy means maximum uncertainty (uniform distribution). In decision trees, information gain (reduction in entropy) is used to select the best feature to split on at each node, choosing the feature that most reduces uncertainty about the target variable.

The learning rate determines the step size in gradient descent: theta_new = theta_old - lr * gradient. Too high a learning rate causes oscillation or divergence (overshooting minima). Too low makes training prohibitively slow and can get stuck in poor local minima. The optimal learning rate depends on the loss curvature, which varies throughout training.

Chain-of-thought prompting instructs the model to show its reasoning step by step before giving the final answer. It is most effective for complex reasoning tasks like math, logic, multi-step analysis, and code debugging. Adding 'Let's think step by step' or providing worked examples significantly improves accuracy on tasks requiring multi-step reasoning.

Prompt injection occurs when user input manipulates the LLM into ignoring its system instructions or performing unintended actions. Defences include separating system and user prompts with clear delimiters, input sanitisation, output validation, using structured output formats, implementing allowlists for permitted actions, and layered defence with multiple validation steps.

Specify the exact schema in the prompt with examples. Use delimiters like code blocks to frame the output. Many APIs support function calling or structured output modes that constrain generation to valid JSON. Implement parsing with error handling and retry logic. For critical applications, validate against a JSON schema and use type-safe parsing libraries.

ReAct combines reasoning (thinking about what to do) and acting (taking actions like tool calls) in an interleaved fashion. The model reasons about the problem, decides on an action, observes the result, and repeats. This pattern powers most AI agent frameworks and is more reliable than pure reasoning because it grounds decisions in real observations.

Create a diverse test set of inputs with expected outputs. Measure quantitative metrics (accuracy, relevance scores, format compliance). Use LLM-as-judge for qualitative evaluation. Track prompt versions systematically. A/B test in production. Iterate by analysing failure cases and refining instructions. Tools like promptfoo, LangSmith, and Braintrust help automate this process.

Strategies include chunking the document and processing each chunk separately, using map-reduce patterns (summarise chunks then synthesise), implementing rolling summaries, using RAG to retrieve only relevant sections, and leveraging models with longer context windows. Choose based on whether the task needs global context (use map-reduce) or local information (use RAG).

Prompt engineering crafts inputs to guide a model's existing capabilities without changing weights. Instruction tuning fine-tunes the model on instruction-following data to make it inherently better at following prompts. Instruction-tuned models (ChatGPT, Claude) respond better to natural language prompts compared to base models that require more carefully engineered prompts.

Instruct the model to only use provided context and say 'I don't know' when uncertain. Use structured extraction from grounding documents rather than open generation. Ask for citations or evidence. Implement confidence scoring in the output format. Use chain-of-thought to make reasoning transparent. Combine with RAG for factual grounding.

Be specific about the role, capabilities, and limitations. Define the output format explicitly. Provide examples of good and bad responses. Set clear boundaries on what the model should and should not do. Use numbered steps for complex tasks. Place the most important instructions at the beginning and end (primacy/recency effect). Keep instructions concise and unambiguous.

Specify the programming language, framework, and version. Describe the function signature, inputs, outputs, and edge cases. Provide relevant context like existing code patterns and constraints. Ask for error handling and comments. Include test cases the code should pass. Use few-shot examples of similar functions. Request explanations of key design decisions.

Prompt chaining breaks a complex task into sequential simpler prompts where each output feeds the next. A single complex prompt tries to handle everything at once. Chaining is more reliable for multi-step tasks, easier to debug, and allows different models or parameters per step. Single prompts are faster and cheaper but may fail on complex reasoning.

Function calling is a structured way for LLMs to indicate they want to invoke a specific function with typed arguments, returning JSON matching a schema. Tool use is a broader concept where the model selects from available tools and generates appropriate calls. Function calling is a specific implementation of tool use. Both enable LLMs to interact with external systems reliably.

Fixed-size chunking splits by token/character count with overlap. Semantic chunking splits at natural boundaries (paragraphs, sections). Recursive chunking tries multiple separators hierarchically. Document-structure-based chunking uses headings and sections. Choose based on document type: structured documents benefit from semantic chunking, while unstructured text works with fixed-size plus overlap.

Evaluate retrieval quality using recall@k, precision@k, and MRR (Mean Reciprocal Rank). Evaluate generation quality using faithfulness (is the answer grounded in retrieved context?), relevancy (does it answer the question?), and correctness. Use frameworks like RAGAS for automated evaluation. Also measure end-to-end latency and user satisfaction.

Hybrid search combines dense vector search (semantic similarity) with sparse keyword search (BM25/TF-IDF). Dense search captures semantic meaning but can miss exact terms. Sparse search is excellent for specific keywords, names, and codes. Combining both with reciprocal rank fusion (RRF) or weighted scoring captures both semantic and lexical matches.

Reranking applies a more expensive cross-encoder model to rescore initially retrieved documents based on their relevance to the query. The initial retrieval uses fast bi-encoder embeddings to get candidate documents, then the reranker (like Cohere Rerank or a cross-encoder) scores each query-document pair jointly. This two-stage approach balances speed and accuracy.

Naive RAG follows a simple retrieve-then-generate pipeline. Advanced RAG adds pre-retrieval optimisation (query rewriting, expansion), retrieval optimisation (hybrid search, reranking, iterative retrieval), and post-retrieval processing (context compression, lost-in-the-middle mitigation). Advanced RAG also includes feedback loops and self-evaluation for continuous improvement.

Query decomposition breaks a complex question into simpler sub-questions, retrieves information for each, and synthesises the results. Use it when questions require information from multiple domains or documents. For example, 'Compare the AI policies of India and the US' decomposes into separate retrievals for each country's policy, then a comparison synthesis step.

LLMs tend to pay more attention to information at the beginning and end of the context, potentially ignoring relevant information in the middle. Mitigate by placing the most relevant chunks first and last, limiting the number of retrieved chunks, using reranking to prioritise relevance, or restructuring retrieved content into a focused summary before generation.

Consider the MTEB leaderboard for benchmark performance. Evaluate on your specific domain data using retrieval metrics. Compare dimensionality (trade-off between quality and storage), latency, and cost. Test multilingual capability if needed. Consider whether a domain-specific fine-tuned embedding outperforms general-purpose ones. Popular choices include OpenAI ada-002, Cohere embed, and open-source models like BGE and E5.

Dense retrieval uses learned vector embeddings from neural models to represent queries and documents as continuous vectors, capturing semantic similarity. Sparse retrieval uses traditional methods like BM25 with term frequency and inverse document frequency. Dense is better for semantic matching; sparse excels at exact term matching. Combining both often yields the best results.

Include source metadata (date, authority, reliability) with retrieved chunks. Instruct the LLM to identify and flag contradictions. Prioritise more recent or authoritative sources. Present multiple viewpoints when appropriate. Implement a confidence scoring mechanism. For critical applications, surface contradictions to users rather than silently choosing one version.

There is no universal optimal size; it depends on document type, query patterns, and embedding model. Typical ranges are 256-1024 tokens. Smaller chunks give more precise retrieval but may lack context. Larger chunks provide more context but may include irrelevant information. Experiment with different sizes and evaluate retrieval quality. Use overlap (10-20%) to maintain context.

Parent-child chunking stores documents at two granularity levels. Small child chunks are embedded and used for retrieval (precise matching). When a child is retrieved, its larger parent chunk is returned to the LLM (richer context). This combines the precision of small-chunk retrieval with the context richness of larger passages, improving both retrieval accuracy and generation quality.

Include source metadata (document title, page, URL) with each chunk. Instruct the LLM to reference specific sources using identifiers. Parse citations from the output and validate them against retrieved chunks. Display inline citations with links to source documents. For accuracy, use structured output to separate the answer from citations and validate each reference.

Use RAG when you need access to frequently updated data, require citations, or have a large knowledge base. Use fine-tuning when you need to change the model's behaviour, style, or reasoning patterns, or when knowledge is stable and fits in training data. RAG is cheaper and more flexible; fine-tuning gives better performance for specific formats. Many production systems combine both approaches.

MCP uses a client-server model where the AI application (host) contains an MCP client that connects to MCP servers. Each server exposes capabilities like tools, resources, and prompts. The host manages multiple client connections to different servers. Communication happens via JSON-RPC 2.0 over transports like stdio or SSE. The client initiates connections and the server responds to requests.

The three primitives are: Tools (model-controlled functions the LLM can invoke, like API calls or computations), Resources (application-controlled data the client can read, like files or database records), and Prompts (user-controlled templates for common interactions). Tools are called by the model, resources are read by the application, and prompts are selected by users.

Create an MCP server that exposes search and retrieval tools for the knowledge base. Implement tools like search_documents(query), get_document(id), and list_categories(). Use resources to expose document content. Add authentication via the transport layer. Handle pagination for large result sets. Include proper error handling and rate limiting. Test with multiple MCP clients.

Stdio transport communicates via standard input/output streams and is used for local server processes spawned by the client. SSE (Server-Sent Events) transport uses HTTP for remote servers, with SSE for server-to-client messages and HTTP POST for client-to-server messages. Stdio is simpler for local tools; SSE enables remote and shared server deployments.

Write clear, specific descriptions that explain what the tool does, when to use it, and what inputs it expects. Include parameter descriptions with examples. Specify constraints and valid values. Add negative instructions (when NOT to use the tool). Use consistent naming conventions. Test with actual LLMs to verify they invoke tools correctly in various scenarios.

Resources are read-only data sources exposed by MCP servers, identified by URIs (like file:///path or db://table/id). Unlike tools which are model-controlled and perform actions, resources are application-controlled and provide data. The client application decides when to read resources, often to provide context. Resources support subscriptions for real-time updates.

JSON-RPC 2.0 is the messaging format used for all MCP communication. It provides a standardised structure for requests (method + params), responses (result or error), and notifications (one-way messages). It supports batch requests and has a simple error handling model. MCP uses it as the application-level protocol over different transport layers.

Return structured error responses with clear error codes and human-readable messages. Implement isRetryable flags for transient errors. The host application should implement exponential backoff for retryable errors. Design tools to be idempotent where possible. Log errors for debugging. The LLM should receive informative error messages to adjust its approach rather than raw stack traces.

MCP prompts are pre-defined interaction templates exposed by servers that users can select to initiate specific workflows. They can include dynamic arguments and generate structured messages with roles. Unlike system prompts which set global behaviour, MCP prompts are task-specific templates that combine instructions, context, and tool usage patterns for common operations.

During the initialise handshake, client and server exchange their supported capabilities. The client declares what features it supports (like sampling or roots), and the server declares its capabilities (tools, resources, prompts, logging). This negotiation ensures both sides understand what features are available and prevents errors from using unsupported features.

Write unit tests for individual tool handlers with mock inputs. Use the MCP Inspector tool for interactive testing. Test with actual MCP clients like Claude Desktop. Verify schema validation for all inputs. Test error handling with invalid parameters. Load test for performance under concurrent connections. Test transport layer reliability with connection drops and reconnects.

An MCP server is specifically designed for AI consumption with structured tool descriptions, input schemas, and standardised discovery. A REST API requires custom integration code, prompt engineering to teach the LLM the API format, and manual error mapping. MCP provides a universal protocol so any AI host can use any MCP server without custom integration code.

Use JSON Schema with clear type definitions and descriptions for every parameter. Mark required vs optional parameters. Include validation constraints (min/max, patterns, enums). Provide default values where sensible. Keep schemas simple; break complex operations into multiple simpler tools. Include examples in descriptions. Validate inputs server-side regardless of schema enforcement.

MCP specifies that the host application should have the ability to require user approval before executing tool calls, especially for sensitive operations. This is crucial because LLMs may make incorrect tool calls or attempt unintended actions. The host acts as a gatekeeper, presenting tool calls to the user for confirmation, ensuring safety and maintaining user control.

MCP adds a layer of abstraction and latency that may be unnecessary for simple, single-tool integrations. It requires both client and server support. The ecosystem is still maturing with limited SDKs in some languages. For high-throughput, low-latency requirements, direct API integration may be more appropriate. If you only integrate with one or two tools, the protocol overhead may not be justified.

An AI agent typically consists of: (1) an LLM as the reasoning engine, (2) a planning module that breaks tasks into steps, (3) a memory system (short-term context and long-term storage), (4) tools/actions the agent can execute, and (5) an observation/feedback loop that processes tool results and decides next steps. The orchestrator ties these components together.

ReAct (Reasoning + Acting) interleaves thinking and doing in a loop: Thought (reason about the situation), Action (select and execute a tool), Observation (process the result). The agent continues this cycle until it reaches the goal or determines it cannot proceed. This pattern grounds reasoning in real observations and is the foundation of most production agent frameworks.

Define tools with clear names, descriptions, and parameter schemas. Present available tools to the LLM in the system prompt or through function calling APIs. Parse the LLM's tool selection and arguments. Execute the tool with validation and error handling. Feed the result back as an observation. Implement safeguards like confirmation prompts for destructive actions and rate limiting.

Single-agent systems use one LLM to handle all reasoning and actions. Multi-agent systems use multiple specialised agents that collaborate, each with different roles, tools, or expertise. Multi-agent systems handle complex tasks better through division of labour but add coordination complexity. Examples include manager-worker patterns, debate systems, and assembly line pipelines.

Implement short-term memory as the conversation context window. Use long-term memory with vector databases for past interactions and learned information. Implement working memory for the current task state. Summarise older context to stay within token limits. Use structured memory stores (key-value) for facts and episodic memory for past experiences. Implement memory retrieval based on relevance.

Common failures include infinite loops (agent keeps retrying failed actions), tool misuse (wrong tool or parameters), hallucinated tool calls, context window overflow, and goal drift. Handle with maximum iteration limits, tool call validation, structured error recovery, context summarisation, periodic goal re-evaluation, and human-in-the-loop checkpoints for critical decisions.

Planning is how agents decompose complex goals into executable steps. Approaches include: task decomposition (breaking into sub-tasks upfront), iterative planning (plan one step at a time based on observations), hierarchical planning (plans at multiple abstraction levels), and plan-and-execute (generate full plan, then execute with replanning on failure). Choice depends on task predictability.

Agent orchestration frameworks provide structure for building agent systems. LangGraph uses a graph-based approach where nodes are agent steps and edges define control flow, offering fine-grained control. CrewAI uses role-based agents in a crew metaphor with higher-level abstractions. LangGraph is more flexible but complex; CrewAI is easier to start with but less customisable.

Measure task completion rate, step efficiency (number of steps to complete a task), tool usage accuracy, cost per task, latency, and error recovery success. Create benchmark tasks with known solutions. Track trajectories to understand reasoning quality. Use human evaluation for subjective quality. Compare against non-agent baselines. Monitor for regressions when changing models or prompts.

Deterministic workflows follow predefined paths with fixed step sequences, like pipelines. Non-deterministic workflows let the LLM decide which tools to use and in what order based on the situation. Deterministic workflows are more predictable and debuggable; non-deterministic workflows are more flexible and can handle unexpected scenarios. Most production agents combine both approaches.

Function calling is the mechanism by which LLMs generate structured tool invocations. OpenAI uses a tools parameter with JSON schema definitions. Anthropic (Claude) uses a similar tool_use format. Google Gemini has function declarations. The LLM outputs a structured call with function name and arguments rather than free text, which the application executes and returns results.

In the supervisor pattern, a central agent (supervisor) receives tasks, delegates sub-tasks to specialised worker agents, collects their results, and synthesises the final output. The supervisor decides which worker to invoke based on the task requirements. This provides clear control flow, easier debugging, and the ability to use different models for different sub-tasks.

Reactive agents respond to stimuli (user messages, events) and take action. Proactive agents monitor conditions and initiate actions autonomously without explicit triggers, such as detecting anomalies and alerting users. Most current AI agents are reactive, but proactive capabilities are emerging through scheduled checks, event-driven triggers, and continuous monitoring patterns.

Reflection is when an agent critiques its own output or reasoning before finalising. The agent generates a response, then evaluates it against criteria like accuracy, completeness, and relevance. If the evaluation identifies issues, the agent revises its approach. This self-improvement loop can significantly enhance output quality at the cost of additional LLM calls.

Implement try-catch blocks around tool execution. Provide clear error messages back to the agent with context for recovery. Define fallback tools for common failures. Allow the agent to retry with modified parameters. Implement escalation paths when retries fail. Distinguish between retryable errors (network timeout) and permanent errors (invalid input). Track error patterns for tool improvement.

Computer use enables AI agents to interact with computer interfaces the same way humans do, through screenshots, mouse clicks, and keyboard input. The agent sees the screen, reasons about the interface, and performs actions like clicking buttons and typing text. This enables automation of tasks in any application without needing APIs, though it is slower and less reliable than tool-based approaches.

Agent chains execute steps in a fixed linear sequence, like a pipeline. Agent graphs define nodes (steps) and edges (transitions) that can include branching, parallel execution, loops, and conditional routing. Graphs are more expressive for complex workflows with dynamic control flow. LangGraph and similar frameworks implement graph-based agent orchestration.

Structured output (JSON, function calls) ensures agent decisions are parseable and actionable by the orchestration system. Without structure, the system must extract tool calls and parameters from free text, which is error-prone. Modern LLM APIs support constrained output modes that guarantee valid JSON. This reliability is essential for production agent systems.

Full fine-tuning updates all model parameters, requiring significant GPU memory and risking catastrophic forgetting. PEFT methods like LoRA, QLoRA, and adapters only modify a small subset of parameters, reducing compute and memory needs by 10-100x while achieving comparable performance. PEFT is preferred for most practical use cases due to its efficiency.

LoRA freezes the pre-trained weights and injects trainable low-rank decomposition matrices into each layer. Instead of updating a full weight matrix W (d x d), it learns two smaller matrices A (d x r) and B (r x d) where r is much smaller than d. This reduces trainable parameters by 100-1000x while maintaining performance. Multiple LoRA adapters can be swapped for different tasks.

Create instruction-response pairs in the model's expected format. Ensure diversity in instructions, response lengths, and complexity. Include edge cases and negative examples. Clean data for quality: remove duplicates, fix formatting, validate factual accuracy. Aim for 1K-100K high-quality examples. Use consistent formatting with clear instruction, input, and output fields.

Catastrophic forgetting occurs when fine-tuning causes the model to lose previously learned general capabilities. Prevent it by using a low learning rate, keeping training epochs low, mixing in general-purpose data, using PEFT methods like LoRA which preserve base weights, applying elastic weight consolidation, and validating on both the target task and general benchmarks.

Key hyperparameters include learning rate (typically 1e-5 to 5e-5), batch size, number of epochs (1-3 for LLMs), warmup ratio, weight decay, and for LoRA: rank (4-64), alpha (typically 2x rank), and target modules. Start with recommended defaults, use a validation set to monitor overfitting, and adjust learning rate first as it has the largest impact.

SFT trains the model to mimic demonstration data by minimising the difference between its outputs and the training examples. RLHF adds a preference optimisation step where a reward model learns human preferences and the LLM is optimised to maximise that reward using reinforcement learning. SFT teaches what to generate; RLHF teaches what humans prefer among options.

Compare against the base model on your task-specific evaluation set using relevant metrics. Check for regression on general capabilities with standard benchmarks. Compute perplexity on a held-out set. Use human evaluation for subjective quality. A/B test in production. Verify the model handles edge cases in your domain. Monitor both task performance and safety metrics.

High-quality data consistently outperforms large quantities of noisy data for fine-tuning. A few thousand expert-curated examples often beat tens of thousands of auto-generated ones. Focus on diverse, representative, and accurately labelled examples. Clean data aggressively. However, some quantity is needed for generalisation. The sweet spot is typically 1K-10K high-quality examples for most tasks.

Create training examples with consistent JSON output format. Include diverse examples covering all possible output structures. Add examples with edge cases (null values, empty arrays, special characters). Validate that training data contains only valid JSON. Fine-tune with a focus on format compliance. Evaluate using both format validity (parse success rate) and content accuracy metrics.

Continual pre-training extends the model's knowledge by training on domain-specific corpora using the same objective as pre-training (next-token prediction on raw text). Fine-tuning adapts the model's behaviour using instruction-response pairs. Continual pre-training adds knowledge; fine-tuning teaches how to use it. For domain adaptation, do continual pre-training first, then instruction fine-tuning.

Load the base model with appropriate quantisation config (BitsAndBytesConfig for QLoRA). Apply LoRA config specifying target modules, rank, and alpha. Prepare the dataset in the instruction format using a tokeniser. Use SFTTrainer from trl library with training arguments (learning rate, epochs, batch size). Monitor training loss and validation metrics. Save and merge LoRA weights after training.

The learning rate schedule controls how the learning rate changes during training. For fine-tuning, a warmup phase (5-10% of steps) gradually increases from zero to prevent large initial weight updates. Followed by cosine or linear decay to allow fine-grained convergence. The peak learning rate for fine-tuning is typically 10-100x lower than pre-training to preserve learned representations.

Use a stronger model (like GPT-4 or Claude) to generate instruction-response pairs. Provide diverse seed prompts covering your domain. Apply quality filters to remove low-quality generations. Validate with domain experts for a subset. Augment with paraphrasing and variation generation. Be careful about model distillation terms of service. Combine synthetic data with real examples for best results.

Overfitting is common with small fine-tuning datasets. Detect it by monitoring training loss vs validation loss divergence, checking for verbatim memorisation of training examples, and testing on held-out data. Prevent it by using fewer epochs (1-3), applying dropout, using weight decay, increasing dataset diversity, and employing early stopping based on validation metrics.

Consider: model size vs available compute, pre-existing knowledge in your domain, licensing restrictions (commercial vs research), language support (especially for Indian languages), community support and tooling, and baseline performance on your task. Start with the smallest model that meets quality requirements. Test 2-3 candidates on your evaluation set before committing to full fine-tuning.

Use the merge_and_unload() method from the PEFT library, which adds the LoRA weight deltas (A * B scaled by alpha/r) to the corresponding base model weights. The merged model has the same architecture as the base model with no inference overhead. Save the merged model for deployment. Verify that the merged model produces identical outputs to the LoRA model before deployment.

For a 7B model: QLoRA needs ~16GB VRAM (single consumer GPU), full fine-tuning needs ~112GB (2x A100). For 13B: QLoRA needs ~24GB, full needs ~208GB. For 70B: QLoRA needs ~48GB (single A100), full needs ~1TB+ (multi-node). Training time scales roughly linearly with data size. Use gradient checkpointing and DeepSpeed ZeRO to reduce memory at the cost of speed.

Fine-tuning updates model weights (all or a subset) using gradient descent on task data. Prompt tuning learns continuous prompt embeddings prepended to the input while keeping all model weights frozen. Prompt tuning uses far fewer parameters (often under 0.1% of model size) and can store multiple task adaptations cheaply, but generally underperforms LoRA for complex tasks.

Create a diverse test set covering common cases, edge cases, and adversarial inputs. Define metrics for each quality dimension: correctness, relevance, completeness, formatting, latency, and safety. Use automated metrics where possible, LLM-as-judge for subjective qualities, and human evaluation for ground truth. Track metrics across model versions and prompt changes. Automate evaluation in CI/CD.

LLM-as-judge uses a powerful LLM to evaluate the quality of another model's outputs by scoring or comparing responses. It scales better than human evaluation and correlates well with human judgments for many criteria. Limitations include self-preference bias, inconsistency across runs, sensitivity to evaluation prompt phrasing, and inability to catch factual errors without ground truth.

For retrieval: precision@k, recall@k, MRR, and NDCG to measure document relevance. For generation: faithfulness (is the answer grounded in context?), answer relevance (does it address the question?), and completeness. End-to-end: accuracy against ground truth, hallucination rate, latency, and cost per query. Use frameworks like RAGAS or custom evaluation scripts.

Collect diverse, representative examples from real usage patterns. Include edge cases, adversarial inputs, and failure modes. Have domain experts create ground truth answers. Ensure coverage across different input types, lengths, and topics. Version the test set and keep it separate from training data. Update periodically to reflect evolving usage. Aim for 100-1000 examples depending on task complexity.

Intrinsic evaluation measures the model's core capabilities in isolation using task-specific benchmarks (perplexity, BLEU, accuracy). Extrinsic evaluation measures how the model performs in the end application, affecting real-world outcomes (user satisfaction, task completion rate, business metrics). A model with better intrinsic scores may not always lead to better extrinsic performance.

AI regression testing verifies that model updates do not degrade performance on previously working cases. Unlike software regression testing with deterministic pass/fail, AI regression testing deals with probabilistic outputs and requires statistical comparison. Key differences include tolerance thresholds for quality changes, the need for representative test distributions, and the possibility of acceptable trade-offs between metrics.

Compare against known ground truth databases or verified sources. Use claim decomposition to break responses into individual facts and verify each. Implement automated fact-checking pipelines using knowledge bases. Use NLI (natural language inference) models to check consistency between claims and source documents. Track hallucination rate as a key metric. Human spot-checking remains essential for novel claims.

A/B testing compares two model versions with real users by randomly assigning traffic. Design: define a clear hypothesis and primary metric, calculate required sample size for statistical power, randomise at the user level to avoid cross-contamination, run for sufficient duration to capture temporal patterns, and use proper statistical tests (not just point estimates) for decision making.

MMLU (Massive Multitask Language Understanding) evaluates LLMs across 57 subjects from STEM, humanities, and social sciences at varying difficulty levels. It measures a model's breadth of knowledge and reasoning through multiple-choice questions. While widely used for model comparison, it has limitations including potential training data contamination and not measuring generation quality.

Pointwise evaluation scores each response independently on a scale (e.g., 1-5 for quality). Pairwise evaluation compares two responses and selects the better one. Pairwise is easier for human evaluators and LLM judges (comparing is easier than absolute scoring) but requires more evaluations to rank many models. Pairwise is preferred for model comparison; pointwise for tracking absolute quality.

Red teaming involves deliberately trying to make AI systems fail or produce harmful outputs through creative adversarial inputs. This includes prompt injection attempts, jailbreaking, edge case exploitation, and bias probing. It uncovers vulnerabilities that standard testing misses. Effective red teaming requires diverse evaluators with different backgrounds and perspectives to cover a wide attack surface.

Data contamination occurs when evaluation benchmark data appears in a model's training data, inflating performance scores without reflecting true capability. It is a major problem because contaminated benchmarks give misleading comparisons. Detect it by checking for memorisation of test examples, using held-out data created after training cutoff, and implementing novel evaluation tasks.

Measure P50, P95, and P99 latency at each pipeline stage (preprocessing, model inference, postprocessing). Profile with tools like NVIDIA Nsight for GPU utilisation. Optimise by reducing model size (quantisation, distillation), batching requests, using faster hardware, implementing caching, and optimising tokenisation. Set latency SLAs based on user experience requirements.

BLEU only measures n-gram overlap and misses semantic quality. Better approaches include BERTScore (embedding similarity), METEOR (semantic matching with synonyms), human evaluation along dimensions like fluency, coherence, and relevance, LLM-as-judge scoring, and task-specific metrics. For summarisation, use ROUGE and factual consistency checks. Combine multiple metrics for a comprehensive view.

An evaluation rubric defines criteria and scoring scales for assessing output quality. For LLM outputs, define dimensions like correctness, completeness, clarity, formatting, and safety. Create clear scoring guidelines with examples for each level (e.g., 1=incorrect, 3=partially correct, 5=fully correct). Use the rubric consistently across human evaluators and as guidance for LLM-as-judge prompts.

Run evaluations multiple times and report mean and variance. Use temperature 0 for reproducible baseline comparisons. For sampling-based evaluation, use enough samples for statistical significance. Set random seeds where possible. Use robust metrics that tolerate minor variations. Compare distributions of scores rather than individual outputs. Report confidence intervals alongside point estimates.

Evaluate task completion rate (does it achieve the goal?), step efficiency (how many steps?), tool usage accuracy (correct tool selection and parameters), cost (total tokens consumed), error recovery (can it handle failures?), safety (no harmful actions), and reasoning quality (is the thinking sound?). Also measure user experience metrics like response time and transparency of reasoning.

The Elo rating system, adapted from chess, assigns models a rating based on head-to-head comparisons. Chatbot Arena shows users two anonymous model responses and collects preferences. Models gain or lose rating points based on wins and losses against opponents of known strength. This crowdsourced approach provides robust rankings that account for human preferences and reduces individual evaluator bias.

A model card documents the model's intended use, training data, evaluation results across subgroups, limitations, and ethical considerations. Include performance metrics disaggregated by demographic groups. Document known failure modes and biases. Specify the evaluation methodology and datasets used. Include environmental impact of training. Follow templates from Google or Hugging Face for consistency.

Development evaluation uses curated test sets to make go/no-go decisions before deployment, focusing on model quality and safety. Production monitoring tracks live performance metrics, user feedback, latency, error rates, and data drift continuously. Production monitoring catches issues that test sets miss, like distribution shifts, real-world edge cases, and gradual performance degradation over time.

A feature store is a centralised repository for storing, managing, and serving features for ML models. It ensures consistency between training and serving features, enables feature reuse across teams, handles feature versioning, and provides both batch and real-time feature serving. Popular options include Feast, Tecton, and Hopsworks. It prevents training-serving skew.

CI includes automated testing of data pipeline code, feature engineering logic, and model training scripts. CD automates model deployment through staging and production environments. Include data validation checks, model quality gates (minimum accuracy thresholds), canary deployments, and automated rollback. Use tools like GitHub Actions, Jenkins, or Kubeflow Pipelines. Version both code and data artifacts.

Model versioning tracks different iterations of models including weights, hyperparameters, training data, and code. Use a model registry (MLflow, Weights & Biases, or cloud-native options) to store and manage versions. Track lineage (which data and code produced each model). Implement promotion stages (development, staging, production). Enable rollback to previous versions when issues arise.

Data drift occurs when the statistical properties of production data change from training data distribution. Detect it using statistical tests (KS test, PSI) on feature distributions, monitoring prediction confidence distributions, and tracking feature statistics over time. Handle it by triggering retraining pipelines, alerting data teams, and implementing adaptive models. Monitor continuously, not just at deployment.

Use a traffic splitting layer to route requests to different model versions. Randomise at the user level for consistency. Define clear success metrics and minimum sample sizes before starting. Run for sufficient duration to capture weekly patterns. Use proper statistical tests and correct for multiple comparisons. Implement guardrail metrics that trigger automatic rollback if violated.

Training-serving skew occurs when the feature computation during training differs from serving, leading to degraded model performance. Causes include different code paths, different libraries, or timing differences. Prevent it by using a feature store for consistent feature computation, sharing preprocessing code between training and serving, and implementing data validation checks.

Monitor prediction distributions for drift. Track model performance metrics against ground truth when available. Monitor input data quality and feature distributions. Set up alerts for anomalous prediction patterns. Track infrastructure metrics (latency, throughput, error rates, GPU utilisation). Implement dashboards for real-time visibility. Use tools like Evidently, WhyLabs, or Arize for ML-specific monitoring.

Level 0: Manual, ad-hoc ML with no automation. Level 1: ML pipeline automation with continuous training but manual deployment. Level 2: CI/CD for both ML pipelines and model deployment with automated testing. Level 3: Full automation with continuous monitoring, automated retraining triggered by drift detection, and self-healing systems. Most Indian companies are between levels 0 and 1.

Maintain versioned model artifacts in a model registry with deployment history. Implement blue-green or canary deployment strategies. Monitor canary metrics with automatic rollback triggers. Keep the previous model version warm for instant switching. Store rollback procedures as runbooks. Test rollback procedures regularly. Ensure the data pipeline supports the rolled-back model's feature requirements.

Pin all library versions in requirements files. Use Docker containers for complete environment reproducibility. Store training data versions with DVC or similar tools. Track random seeds and hardware specifications. Record exact Git commits for training code. Use deterministic training where possible. Document CUDA and driver versions for GPU training. Automate environment setup in CI/CD.

Canary deployment routes a small percentage of production traffic (e.g., 5%) to the new model while the majority uses the current model. If the canary model performs well on defined metrics, traffic is gradually increased. If it degrades, traffic is rolled back automatically. This minimises risk by limiting the blast radius of a bad model deployment.

Define data schemas with expected types, ranges, and constraints using tools like Great Expectations or TFX Data Validation. Check for missing values, outliers, and distribution changes. Validate referential integrity and business rules. Run validation at data ingestion, after feature engineering, and before training. Fail the pipeline early if validation fails. Generate data quality reports.

DVC tracks large files, datasets, and ML model artifacts alongside Git without storing them in the Git repository itself. It stores metadata in Git while the actual data lives in remote storage (S3, GCS, Azure Blob). DVC enables reproducible ML experiments by versioning data alongside code, supports pipeline definition, and enables data sharing across teams.

Model compression reduces model size and inference cost for production deployment. Techniques include quantisation (reducing precision from FP32 to INT8/INT4), pruning (removing unimportant weights or neurons), knowledge distillation (training a smaller model), and architecture search (finding efficient architectures). Choose based on acceptable quality loss and target deployment hardware.

Tag all ML resources for cost attribution. Use spot/preemptible instances for training (60-90% savings). Right-size GPU instances based on actual utilisation. Implement auto-scaling for serving. Use model optimisation (quantisation, distillation) to reduce inference costs. Schedule batch jobs during off-peak hours. Set up budget alerts and cost anomaly detection. Review costs weekly.

A model registry is a centralised catalog of trained models with their metadata. Store: model artifacts (weights, configs), training metrics, data lineage (dataset version, features used), code version (git commit), hyperparameters, evaluation results, deployment status, owner/team, creation date, and model card. Popular registries include MLflow Model Registry, Vertex AI Model Registry, and SageMaker Model Registry.

The H100 offers approximately 3x the performance of A100 for transformer training thanks to the Transformer Engine with FP8 support, higher memory bandwidth (3.35 TB/s vs 2 TB/s), and more CUDA cores. H100 has 80GB HBM3 memory vs A100's 80GB HBM2e. H100 also introduces NVLink 4.0 for faster multi-GPU communication. H100 is preferred for large model training and inference.

Cloud offers flexibility, scalability, and no upfront capital. On-premise provides lower long-term costs for sustained workloads and data sovereignty. Cloud is better for experimentation, burst capacity, and teams without infrastructure expertise. On-premise makes sense for consistent high utilisation (>70%), sensitive data requirements (Indian data localisation), or large-scale training where reserved instances cost more than owned hardware.

Profile GPU utilisation with nvidia-smi and PyTorch profiler. Increase batch size to fill GPU memory. Use mixed precision training (FP16/BF16) for higher throughput. Optimise data loading with multiple workers and prefetching. Enable gradient checkpointing to trade compute for memory. Use DeepSpeed or FSDP for multi-GPU efficiency. Reduce CPU-GPU data transfer bottlenecks.

ONNX (Open Neural Network Exchange) is an open format for representing ML models, enabling interoperability between frameworks. You can train in PyTorch and deploy with ONNX Runtime, TensorRT, or OpenVINO. ONNX Runtime provides optimised inference across CPU, GPU, and edge devices. This decouples training framework choice from deployment infrastructure, enabling hardware-specific optimisations.

Use PyTorch's DistributedDataParallel (DDP) with NCCL backend for inter-GPU communication. Set up process groups with one process per GPU. Use DistributedSampler for data partitioning. Launch with torchrun specifying nproc_per_node. Enable mixed precision with AMP. Ensure batch size is per-GPU and adjust learning rate for effective global batch size. Profile to verify GPU utilisation.

Edge deployment runs AI models on devices near the data source (phones, IoT devices, local servers) rather than in the cloud. It is appropriate when low latency is critical, network connectivity is unreliable (rural India), data privacy prevents cloud transmission, or cloud costs are prohibitive at scale. Requires model compression (quantisation, pruning) for device constraints.

Triton is an open-source inference serving platform that supports multiple frameworks (PyTorch, TensorFlow, ONNX, TensorRT) simultaneously. Key features include dynamic batching, model versioning, model ensembles, GPU scheduling across multiple models, metrics reporting, and HTTP/gRPC APIs. It is production-ready and widely used in enterprise deployments for serving multiple models efficiently.

Generative Adversarial Networks consist of two networks: a generator that creates synthetic data and a discriminator that distinguishes real from fake data. They are trained adversarially, with the generator improving at fooling the discriminator and the discriminator improving at detecting fakes. This process produces increasingly realistic synthetic data.

The vanishing gradient problem occurs when gradients become extremely small as they propagate through many layers, effectively stopping learning in earlier layers. It is addressed by using ReLU activation functions instead of sigmoid/tanh, batch normalisation, residual connections (skip connections), and proper weight initialisation techniques like He or Xavier initialisation.

Use transfer learning with a pre-trained CNN like EfficientNet fine-tuned on labelled crop disease images. The model should handle diverse Indian crops and work offline on low-end devices via model quantisation. Include data augmentation for varying lighting and camera quality. Deploy as a lightweight mobile app with results in local languages.

Use a combination of rule-based filters for known fraud patterns and ML models (gradient boosting or deep learning) for anomaly detection. Feature engineer from transaction metadata, user behavior, device info, and network graphs. Handle extreme class imbalance with SMOTE or focal loss. Deploy with real-time inference under 100ms latency and continuous model retraining.

Interpretability means a model's internal logic is inherently understandable (e.g., decision trees), while explainability uses post-hoc methods (SHAP, LIME) to approximate why a complex model made a decision. In regulated Indian industries like banking (RBI guidelines) and insurance (IRDAI), explainability is often mandated for automated decisions affecting customers.

Reinforcement Learning from Human Feedback trains a reward model on human preferences about response quality, then uses PPO or similar algorithms to fine-tune the LLM to maximise that reward. It aligns model outputs with human values, making responses more helpful, harmless, and honest compared to pure supervised fine-tuning.

KV cache stores the key and value tensors computed for previously processed tokens so they do not need to be recomputed at each generation step. This reduces inference time from O(n^2) to O(n) per token. However, it requires significant GPU memory, especially for long sequences and large batch sizes.

In FP16, each parameter takes 2 bytes, so 70B parameters need approximately 140 GB just for model weights. Add KV cache memory which depends on context length and batch size. A rough estimate for serving would be 160-200 GB, requiring multiple A100 (80GB) or H100 GPUs with tensor parallelism. Quantisation to INT4 can reduce this to around 40 GB.

Emergent abilities are capabilities that appear only above certain model scale thresholds and are not present in smaller models. Examples include in-context learning, chain-of-thought reasoning, and few-shot arithmetic. These are debated in the research community, with some arguing they may be artefacts of evaluation metrics rather than true phase transitions.

Quantisation reduces model weights from higher precision (FP32/FP16) to lower precision (INT8/INT4), dramatically reducing memory usage and improving inference speed. Trade-offs include slight quality degradation, especially for complex reasoning tasks. Techniques like GPTQ, AWQ, and GGML enable 4-bit quantisation with minimal quality loss for many applications.

Scaling laws (Kaplan et al., Chinchilla) describe how model performance improves predictably as a power law of model size, dataset size, and compute budget. Chinchilla showed that models should be trained on roughly 20 tokens per parameter for compute-optimal training. These laws help estimate the resources needed for a desired performance level.

FlashAttention is an IO-aware exact attention algorithm that reduces memory reads/writes by tiling the computation and keeping intermediate results in fast SRAM. It achieves 2-4x speedup in wall-clock time and reduces memory usage from O(n^2) to O(n) without approximating the attention computation. It has become standard in modern LLM training and inference.

Speculative decoding uses a smaller draft model to generate candidate tokens quickly, then the larger target model verifies them in a single forward pass. If the draft tokens match what the target would have generated, multiple tokens are accepted at once. This can achieve 2-3x speedup without changing the output distribution.

MoE models contain multiple expert sub-networks and a gating network that routes each token to a subset of experts. Only a fraction of parameters are activated per token, enabling much larger total parameter counts with manageable compute costs. Mixtral 8x7B, for example, has 47B total parameters but only activates 13B per token.

Choose a model with strong multilingual capabilities (like GPT-4, Claude, or IndicBERT for embedding). Implement language detection for routing. Use RAG with a knowledge base containing Indian language content. Fine-tune on Indian language conversation data if quality is insufficient. Handle transliteration (Hindi in Roman script) and code-mixing (Hinglish) explicitly.

Distillation trains a smaller student model to mimic the outputs of a larger teacher model, resulting in a genuinely smaller architecture. Quantisation keeps the same architecture but reduces the numerical precision of weights. Distillation requires retraining but can achieve better compression ratios; quantisation is faster to apply but limited in compression.

DPO directly optimises the LLM policy using preference pairs without training a separate reward model, simplifying the RLHF pipeline. It reformulates the reward-based objective as a classification loss on preference data. DPO is simpler to implement and more stable but may not perform as well as RLHF with a well-tuned reward model on complex alignment tasks.

Set accumulation_steps to the desired factor. In the loop, divide loss by accumulation_steps before backward(). Call optimizer.step() and optimizer.zero_grad() only every accumulation_steps iterations. This effectively increases batch size without requiring proportionally more GPU memory, useful when training large models on limited hardware.

Use Apache Kafka or Redis Streams for message ingestion, asyncio or Faust for stream processing, and a FastAPI/gRPC endpoint for inference. Implement windowed feature computation with a sliding buffer. Use connection pooling for database lookups and batch inference requests when possible. Monitor latency and throughput with Prometheus metrics.

Mixed precision uses FP16 for most operations and FP32 for numerically sensitive operations, reducing memory usage by nearly half and speeding up training on modern GPUs. In PyTorch, use torch.cuda.amp.autocast() for the forward pass and torch.cuda.amp.GradScaler for gradient scaling to prevent underflow. This typically yields 1.5-2x speedup.

Use torch.nn.parallel.DistributedDataParallel (DDP) for data parallelism. Initialise process groups with torch.distributed.init_process_group. Wrap the model in DDP and use DistributedSampler for the DataLoader. Launch with torchrun or torch.distributed.launch. For model parallelism across GPUs, use pipeline parallelism or tensor parallelism with libraries like DeepSpeed or FSDP.

In convex optimisation, any local minimum is the global minimum, guaranteeing convergence. Neural network loss landscapes are non-convex with many local minima and saddle points. This means gradient descent may get stuck. In practice, research shows that most local minima in high-dimensional spaces have similar loss values, and SGD's noise helps escape saddle points.

The Jacobian is a matrix of all first-order partial derivatives of a vector-valued function. In deep learning, it represents how each output dimension changes with respect to each input dimension. It is used in backpropagation through layers, normalising flows, and analysing the sensitivity of model outputs. The Jacobian determinant measures volume change under transformations.

KL divergence measures how one probability distribution Q differs from a reference distribution P: KL(P||Q) = sum(P(x) * log(P(x)/Q(x))). It is always non-negative and zero only when P=Q. Cross-entropy equals the entropy of P plus the KL divergence from P to Q. Minimising cross-entropy is equivalent to minimising KL divergence when P is fixed.

For binary cross-entropy L = -[y*log(sigma(z)) + (1-y)*log(1-sigma(z))], where sigma is sigmoid. Using the fact that sigma'(z) = sigma(z)(1-sigma(z)), the gradient dL/dz simplifies to sigma(z) - y, which is the predicted probability minus the true label. This elegant simplification is why sigmoid + BCE is commonly used.

SVD decomposes any matrix A into three matrices: A = U * Sigma * V^T, where U and V are orthogonal and Sigma is diagonal with singular values. In ML, it is used for PCA, matrix factorisation in recommendation systems, low-rank approximation for model compression, and latent semantic analysis in NLP.

MLE finds parameter values that maximise the probability of observing the training data. For a classification network with softmax output, maximising the likelihood of correct labels is equivalent to minimising cross-entropy loss. For regression with Gaussian noise, MLE is equivalent to minimising MSE. This probabilistic interpretation connects loss functions to statistical theory.

The Hessian is the matrix of all second-order partial derivatives of the loss function. It captures the curvature of the loss landscape and can inform optimal step sizes. For a network with n parameters, the Hessian has n^2 entries, making it infeasible to compute for large models. Approximations like Fisher information matrix, KFAC, or diagonal Hessian are used instead.

The reparametrisation trick transforms sampling from a learned distribution into a deterministic function of the parameters plus noise: z = mu + sigma * epsilon, where epsilon is from N(0,1). This makes the sampling operation differentiable, enabling backpropagation through the stochastic layer. Without it, VAEs cannot be trained end-to-end with gradient descent.

Self-consistency generates multiple reasoning paths for the same question using sampling (higher temperature) and selects the most common answer via majority voting. It improves reliability on reasoning tasks by reducing the impact of individual reasoning errors. It increases cost proportionally to the number of samples but can significantly boost accuracy on complex tasks.

For tables, extract and store them as structured text or markdown. For images, generate text descriptions using vision models and index those descriptions. For charts, extract underlying data points. Use multi-modal embedding models that can embed both text and images into the same space. Store modality metadata for filtering and presentation.

Contextual retrieval enriches each chunk with additional context about its position and meaning within the broader document before embedding. For example, prepending a chunk with a brief summary of the document or section it came from. This reduces the ambiguity of isolated chunks and improves retrieval accuracy, especially for chunks that reference external context.

Graph RAG combines knowledge graphs with vector-based retrieval. It constructs a graph of entities and relationships from documents, enabling multi-hop reasoning across connected information. When a query arrives, it retrieves both vector-similar chunks and graph-connected entities, providing richer context. It excels at questions requiring reasoning across multiple related documents.

HyDE generates a hypothetical answer to the query using an LLM, then embeds that hypothetical answer to search the vector store instead of embedding the query directly. Since the hypothetical answer is closer in semantic space to actual document chunks than the question is, it often retrieves more relevant results, especially for complex queries.

Agentic RAG adds an agent layer that can iteratively decide whether to retrieve more information, refine queries, switch data sources, or synthesise across multiple retrievals. Unlike standard RAG with a fixed retrieve-then-generate flow, agentic RAG adapts its retrieval strategy based on initial results, enabling complex multi-step information gathering.

Collect domain-specific query-document pairs from search logs or manually curated examples. Create training triplets (query, positive document, negative document) using hard negative mining. Fine-tune a pre-trained embedding model (like BGE or E5) using contrastive loss. Evaluate on a held-out domain-specific test set using retrieval metrics. This typically yields significant improvement over general embeddings.

Sampling allows MCP servers to request LLM completions through the client, enabling agentic behaviours within tools. The server sends a sampling/createMessage request to the client, which routes it to the LLM. This enables sophisticated patterns like tool chains where one tool's execution requires LLM reasoning as an intermediate step, while keeping the human in the loop.

MCP supports resource subscriptions where clients can subscribe to specific resource URIs. When the underlying data changes, the server sends a notifications/resources/updated notification. The client can then re-read the resource to get the latest data. This enables live dashboards, real-time monitoring, and collaborative features where AI assistants track changing data.

Use progress notifications to report incremental status updates to the client. Implement timeouts with configurable limits. For very long operations, return a task ID and provide a separate status-checking tool. Support cancellation through the MCP cancellation mechanism. Avoid blocking the server's event loop. Consider breaking long operations into smaller, resumable steps.

Map existing function definitions to MCP tool schemas. Create an MCP server that wraps the existing tool implementations. Update the host application to use an MCP client instead of direct function calling. Migrate incrementally, running both systems in parallel. Test tool discovery and invocation thoroughly. Update error handling to use MCP's error format. The tool logic itself stays the same.

Provide tools for file read/write, code execution in a sandbox, test running, and linting. Implement an iterative loop: generate code, run tests, analyse failures, fix issues. Use the LLM to reason about error messages and fix bugs. Include static analysis tools. Sandbox execution for security. Limit iterations and set timeouts. Log all generated code for review.

Create specialised agents: a classifier agent for categorising content type, a policy checker agent with access to moderation guidelines, a context analyser for understanding cultural nuances (important for Indian content), and a decision agent that synthesises verdicts. Implement escalation to human moderators for borderline cases. Use voting or consensus mechanisms across agents for reliability.

Stream the LLM's reasoning tokens as they are generated for transparency. Send tool execution status updates as events. Implement partial result streaming for long operations. Use WebSockets or SSE for the client connection. Display thinking indicators and progress bars. Allow users to interrupt and redirect the agent mid-execution. Buffer events for connection resilience.

Define a standardised task suite with varying complexity. Measure success rate, average steps to completion, cost, latency, and error recovery effectiveness. Test on edge cases and adversarial inputs. Compare single-agent vs multi-agent, different LLM backends, and various prompting strategies. Use frameworks like AgentBench or custom evaluation harnesses. Run multiple trials for statistical significance.

QLoRA combines quantisation with LoRA. It loads the base model in 4-bit precision (NormalFloat4) and adds LoRA adapters in higher precision. This reduces memory requirements by approximately 4x compared to LoRA alone, enabling fine-tuning of 65B parameter models on a single 48GB GPU. It uses techniques like double quantisation and paged optimisers for memory efficiency.

Collect Indian legal documents (court judgments, statutes, legal opinions) and create instruction pairs for tasks like summarisation, clause extraction, and case law reference. Include Indian legal terminology and citation formats. Use QLoRA on a capable base model. Validate against expert-labelled outputs. Handle multiple Indian languages. Ensure the model correctly references Indian statutes and precedents.

Collect domain-specific query-passage pairs from search logs or expert annotation. Create training triplets with hard negatives (similar but irrelevant passages). Fine-tune using contrastive learning (InfoNCE or multiple negatives ranking loss). Evaluate with retrieval metrics on a held-out test set. Use a pre-trained model like BGE or E5 as the base. Typically 5K-50K pairs suffice.

Adapter layers insert small bottleneck modules between existing layers of a frozen model. They consist of a down-projection, non-linearity, and up-projection. Compared to LoRA, adapters add sequential computation (slightly increasing latency) while LoRA modifies existing weights in parallel. LoRA has become more popular due to zero additional inference latency and easier merging with base weights.

Combine datasets from multiple tasks with task-specific prefixes or instruction formats. Balance dataset sizes to prevent dominant tasks from overwhelming others. Use weighted sampling or data mixing ratios. Monitor performance on each task separately. Consider using task-specific LoRA adapters that can be loaded independently. Evaluate for cross-task transfer and interference.

RLAIF (RL from AI Feedback) replaces human feedback with AI-generated feedback based on a constitution (set of principles). An AI model critiques and revises outputs according to these principles, generating preference data. This preference data trains a reward model or directly optimises the policy. It is more scalable than human feedback while maintaining alignment quality.

Knowledge distillation trains a smaller student model to mimic a larger teacher model's output distributions, not just hard labels. The student learns the teacher's soft probabilities, which contain richer information. It is used to create efficient production models. Combined with fine-tuning, you can distill a large fine-tuned model into a smaller, deployable one with minimal quality loss.

Use on-premise or private cloud GPU infrastructure. Apply differential privacy during training to prevent memorisation. Anonymise or pseudonymise PII in training data. Use federated fine-tuning when data cannot leave its source. Implement access controls on model artifacts. Audit the model for data leakage using membership inference tests. Comply with India's DPDP Act requirements.

DPO simplifies preference-based fine-tuning by eliminating the need for a separate reward model. It directly optimises the policy model using pairs of preferred and rejected responses. The loss function implicitly defines a reward and trains the model to increase the probability of preferred responses while decreasing rejected ones. It is simpler and more stable than PPO-based RLHF.

Prefix tuning prepends learnable continuous vectors (soft prompts) to the input at each layer while keeping model weights frozen. These prefixes steer the model's behaviour without modifying its parameters. Compared to LoRA, prefix tuning has fewer parameters but slightly higher inference latency due to added prefix tokens. LoRA generally achieves better performance and has become the preferred PEFT method.

Collect Indian documents (Aadhaar, PAN cards, invoices in multiple scripts) with annotated fields. Use a pre-trained VLM like LLaVA or PaLI as the base. Create instruction data for OCR, field extraction, and document classification. Handle multiple Indian scripts (Devanagari, Tamil, Bengali, etc.). Fine-tune with LoRA on vision and language components. Evaluate on diverse document formats and quality levels.

Test with adversarial prompts designed to elicit harmful outputs. Evaluate fairness across demographic groups using disaggregated metrics. Test for common biases (gender, caste, regional in Indian context). Use red-teaming with diverse evaluators. Check for PII leakage. Evaluate content safety with toxicity classifiers. Test prompt injection resistance. Document findings in a model card.

Integrate eval scripts into the CI pipeline triggered by model or prompt changes. Store test sets and expected outputs in version control. Run automated metrics computation and LLM-as-judge evaluations. Set quality gates with minimum threshold scores. Generate evaluation reports comparing against baselines. Alert on regressions. Use tools like promptfoo, Braintrust, or custom evaluation frameworks.

Create test cases with coding problems at various difficulty levels. Measure functional correctness (do generated solutions pass test cases?), code quality (readability, efficiency), and security (no vulnerabilities). Use pass@k metric for solution accuracy. Test across multiple languages and frameworks. Evaluate explanation quality alongside code. Benchmark against existing solutions and human performance.

Create a test suite of known injection attacks: direct instruction override, indirect injection through context, encoding-based attacks, and multi-step manipulation. Measure the rate at which the model follows injected instructions versus maintaining its original behaviour. Test regularly as new attack patterns emerge. Implement continuous monitoring in production for unusual output patterns.

Use a model serving framework (TorchServe, Triton, or TF Serving) behind a load balancer. Deploy in Indian cloud regions (AWS Mumbai, Azure Central India) for low latency. Implement auto-scaling based on traffic patterns (handle sale events like Diwali/Republic Day). Use model caching and batching for efficiency. Set up health checks, monitoring, and automatic failover. Consider edge deployment for latency-critical features.

Define triggers: scheduled (daily/weekly), data drift detection, or performance degradation below threshold. Automate data fetching, preprocessing, training, and evaluation. Implement quality gates that compare new model against current production model. Use automated deployment with canary rollout for approved models. Alert on failures and unusual metrics. Track retraining history for audit.

Route production traffic to both the current and new model simultaneously. The current model serves responses to users while the new model's predictions are logged but not shown. Compare predictions offline to detect discrepancies and evaluate the new model on real traffic. This is risk-free testing on live data. Implement using a proxy layer or traffic mirroring at the load balancer.

Start with managed services to move fast: GitHub for code, DVC for data versioning, MLflow for experiment tracking, AWS SageMaker or GCP Vertex AI for training and serving. Implement CI/CD with GitHub Actions. Use Docker and Kubernetes for serving. Set up monitoring with Prometheus and Grafana. Automate data validation and model evaluation. Plan for compliance with Indian data regulations. Scale infrastructure as the team grows.

Data parallelism replicates the model on each GPU and splits data batches. Tensor parallelism splits individual layers across GPUs (each GPU computes part of each layer). Pipeline parallelism splits layers across GPUs in stages. Data parallelism is simplest and scales well. Tensor parallelism is for layers too large for one GPU. Pipeline parallelism handles very deep models. Production systems often combine all three.

Use GPU-optimised inference servers (vLLM, TGI, or Triton) with continuous batching. Quantise the model to INT8/INT4 for speed. Implement KV cache efficiently. Deploy in regions close to users (Indian cloud regions). Use load balancing with health checks. Implement request queuing and timeout handling. Pre-warm model caches. Monitor P99 latency and auto-scale based on queue depth.

vLLM is a high-throughput LLM serving engine that introduces PagedAttention for efficient KV cache memory management. Like virtual memory in operating systems, it pages KV cache blocks, reducing memory waste from 60-80% to near zero. This enables 2-4x higher throughput than naive implementations. It also supports continuous batching and speculative decoding.

Use a streaming platform (Kafka, Kinesis) for event ingestion. Process events with a stream processor (Flink, Spark Streaming) for feature computation. Store computed features in a low-latency store (Redis, DynamoDB) for serving. Implement exactly-once processing semantics. Handle late-arriving data with watermarks. Monitor pipeline lag and throughput. Ensure consistency with batch-computed features.

DeepSpeed is a deep learning optimisation library from Microsoft that enables training models with billions of parameters. Its ZeRO (Zero Redundancy Optimiser) partitions model states across GPUs to reduce memory per GPU. It supports 3D parallelism (data, tensor, pipeline), mixed precision, gradient checkpointing, and offloading to CPU/NVMe. It can train trillion-parameter models on clusters of GPUs.

Use model quantisation (INT8/INT4) to reduce weight memory. Implement efficient KV cache management (paged attention). Use model parallelism to split across GPUs. Implement dynamic batching to optimise memory utilisation. Use memory-efficient attention implementations (FlashAttention). Offload inactive model layers to CPU when possible. Monitor GPU memory with CUDA memory management tools.

Consider spot/preemptible GPU instances for training (60-90% savings). Use reserved instances for steady inference workloads. Implement job scheduling (SLURM, Kubernetes) for GPU sharing. Use gradient accumulation to use smaller GPUs effectively. Consider Indian cloud providers (Jio Cloud, Yotta) alongside AWS/GCP/Azure Mumbai regions. Set up auto-shutdown for idle resources. Monitor utilisation aggressively.

TensorRT is NVIDIA's inference optimisation SDK that takes trained models and produces highly optimised runtime engines. It applies layer fusion, kernel auto-tuning, precision calibration (FP16/INT8), and dynamic tensor memory management. It typically provides 2-5x speedup over framework-native inference. It is hardware-specific and requires re-optimisation for different GPU architectures.

High-bandwidth, low-latency interconnects are critical. NVLink connects GPUs within a node (900 GB/s on H100). InfiniBand or RoCE connects nodes (200-400 Gbps). Standard Ethernet (25-100 Gbps) is insufficient for large-scale training. Communication patterns (all-reduce, all-gather) dominate training time at scale. Place training nodes in the same rack or availability zone to minimise network latency.

Continuous batching (iteration-level batching) dynamically adds and removes requests from a batch at each generation step, unlike static batching which waits for all requests to complete. This prevents short requests from waiting for long ones, significantly improving GPU utilisation and throughput (2-10x). It is implemented in modern serving frameworks like vLLM and TGI.

Use distributed file systems (HDFS, Lustre) or object storage (S3) for training data. Implement data sharding across storage nodes. Use streaming data loaders that prefetch and pipeline IO with computation. Cache frequently accessed data on local NVMe SSDs. Optimise data format (TFRecord, WebDataset, Parquet) for sequential read performance. Monitor IO bottlenecks with storage metrics.

Deploy Triton or a custom serving layer on Kubernetes with GPU node pools in Indian cloud regions. Implement model routing based on request type. Use shared GPU scheduling to maximise utilisation across models. Implement a model registry for version management. Set up centralised monitoring and cost attribution per team. Handle compliance with Indian data regulations. Implement rate limiting and authentication per consumer.