> ## Documentation Index > Fetch the complete documentation index at: https://sdk.observability.getonex.ai/llms.txt > Use this file to discover all available pages before exploring further. # Configuration Reference ## OneXMonitor parameters | Parameter | Type | Description | | ------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `endpoint` | string | Ingestion base URL. Determines which platform environment receives your logs (sandbox, development, production). Fetch environment-specific endpoints from the [OneX Observability Dashboard](https://dashboard.observability.getonex.ai). | | `api_key` | string | API key for authentication (from the same dashboard). | | `enable_logging` | bool | When `true`, enables SDK logging at INFO level (framework detection, signal export, etc.). Use for local debugging. Default: `false`. | | `sample_rate` | float | Fraction of requests to export (0.0–1.0). `1.0` = 100%, `0.1` = 10%. Reduces volume in high-traffic production. Default: `1.0`. | | `max_requests_per_minute` | int \| None | Cap on batch export requests per minute to avoid overloading the platform. `None` = no cap. Default: `None`. | ## Config options | Key | Type | Default | Description | | ------------------------------ | ----------- | ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `hidden_state_sample_tokens` | int | 4 | Number of tokens sampled per hidden-state payload. | | `hidden_state_sample_features` | int | 32 | Number of features sampled per token. | | `hidden_state_precision` | int | 6 | Decimal precision for rounded tensor samples (use -1 to disable rounding). | | `capture_full_hidden_state` | bool | false | Set `true` to include the entire hidden-state tensor (use cautiously). | | `payload_sample_items` | int | 5 | Max items serialized per list/dict when exporting request payloads. | | `payload_tensor_sample` | int | 32 | Flat element sample size for tensor summaries in request payloads. | | `payload_max_depth` | int | 2 | Maximum depth for nested request payload serialization. | | `request_metadata` | dict | `{}` | Extra metadata appended to every request payload/response event. | | `request_payload_endpoint` | string | derived | Override for the payload ingestion endpoint (defaults to `/api/requests/payload`). | | `request_response_endpoint` | string | derived | Override for the response ingestion endpoint (defaults to `/api/requests/response`). | | `capture_request_payload` | bool | true | When `false`, request payloads (model inputs and raw context) are not sent to the platform. Neural signals still go to `/api/signals/batch`. | | `capture_response_payload` | bool | true | When `false`, response payloads (model outputs and application responses) are not sent. Neural signals still go to `/api/signals/batch`. | | `enable_logging` | bool | false | When `true`, enables SDK logging at INFO level for debugging. Can also be passed as a top-level parameter to `OneXMonitor`. | | `sample_rate` | float | 1.0 | Fraction of requests to export (0.0–1.0). Per-request sampling; can also be passed as a top-level parameter. | | `max_requests_per_minute` | int \| None | None | Cap on batch export requests per minute. Can also be passed as a top-level parameter. | | `capture_logits` | bool | false | When `true`, model logits are extracted from the forward output and included in the response payload (shape + sample). For Uncertainty, Calibration, and Confidence assessments. | | `capture_probabilities` | bool | false | When `true`, probabilities (softmax of logits) are computed and included in the response payload (shape + sample). Requires logits to be present in the model output. | | `logits_sample_size` | int | 64 | Max number of values to include in the logits/probabilities sample (flattened). Limits payload size. | | `signal_queue_maxsize` | int | 10000 | Max neural signals queued in memory before the exporter drops new ones (`queue.Full`). LLMs emit many signals per token; increase if you see warnings while the batch API is healthy. Capped at 1,000,000. | | `request_queue_maxsize` | int | 2000 | Max queued request payload/response export events. Capped at 100,000. | | `capture_llm_extension` | bool | true | **PyTorch only.** When `true`, each exported signal includes an optional `llm_extension` object with extra LLM context (generation phase, output summaries, model metadata). Set `false` to reduce payload size. | | `llm_extension_cuda_memory` | bool | true | **PyTorch only.** When `true` and CUDA is available, `llm_extension` may include GPU memory usage summaries. | | `capture_llm_representation` | bool | true | **PyTorch.** One `representation.` signal per forward with an embedding vector for OOD-style assessments. | | `llm_representation_policy` | string | last\_token | `last_token`, `prompt_last` (second-to-last token when seq>1), or `mean_pool`. | | `llm_representation_max_dims` | int | 512 | Max dimensions exported in the representation embedding list. | | `capture_generation_context` | bool | true | Adds `phase` (prefill/decode), `sequence_length`, and related fields to `llm_extension` / representation. | | `capture_logits_extras` | bool | true | Adds margin, perplexity proxy, and top-1 probability summaries on output signals. | | `capture_final_norm` | bool | true | **Causal LMs.** Export summaries from the final normalization layer before the language-model head. | | `capture_kv_cache_meta` | bool | true | Include KV cache shape metadata in LLM output summaries (no raw tensor dumps). | ## LLM extension and export queues For **decoder / LLM** workloads, the PyTorch adapter can emit multiple signals per forward pass. The background exporter sends them in batches. If the HTTP path is slower than signal production, in-process queues can fill. * **`signal_queue_maxsize`** / **`request_queue_maxsize`**: Increase these when running large models or long generations and you observe exporter warnings about a full queue. * **`capture_llm_extension`**: Extra structured metadata for backends and debugging. Disable if you do not need it. * **`llm_extension_cuda_memory`**: Disable if you want to avoid reading CUDA memory statistics in `llm_extension`. Example: ```python theme={"dark"} monitor = OneXMonitor( api_key="your-api-key", endpoint="onex-ingestion-endpoint", config={ "signal_queue_maxsize": 50_000, "request_queue_maxsize": 5_000, "capture_llm_extension": True, "llm_extension_cuda_memory": True, }, ) ``` ## Sampling and throughput control * **Sample rate**: When `sample_rate` is less than `1.0`, the SDK exports only a fraction of requests. The decision is made once per request (by `request_id`), so either all signals for a request are exported or none. Use this in high-traffic production to reduce volume while keeping representative coverage (e.g. `sample_rate=0.1` for 10% of requests). * **Max requests per minute**: When set, the exporter limits how many batch HTTP requests it sends to the platform per 60-second sliding window. Batches are sent as soon as possible under this cap. Use this to avoid overloading the ingestion API when many models or processes export concurrently. Example: ```python theme={"dark"} monitor = OneXMonitor( api_key="your-api-key", endpoint="onex-ingestion-endpoint", sample_rate=0.1, # export 10% of requests max_requests_per_minute=120, # cap at 120 batch requests per minute ) ``` ## Logits and probabilities capture For **Uncertainty**, **Calibration**, and **Confidence** assessments, the platform can use model logits and/or probabilities. By default they are not captured (privacy and payload size). * **capture\_logits**: When `true`, the SDK extracts the `logits` tensor from the model forward output (e.g. HuggingFace `CausalLMOutputWithPast.logits`) and adds a `logits` field to the response payload with `shape`, `sample` (first N values), and `numel`. * **capture\_probabilities**: When `true`, the SDK computes `softmax(logits)` and adds a `probabilities` field with the same structure. Requires the model output to expose logits (e.g. `.logits` attribute or first element of a tuple). * **logits\_sample\_size**: Caps how many values are included in the sample (default 64). Use a higher value for small classification heads; keep lower for large vocabularies. When either option is enabled, the main `output` field in the response is serialized without the raw logits tensor so you avoid duplicating data and control size via the sampled fields. Example: ```python theme={"dark"} monitor = OneXMonitor( api_key="your-api-key", endpoint="onex-ingestion-endpoint", config={ "capture_logits": True, "capture_probabilities": True, "logits_sample_size": 128, }, ) ```