ClickHouse Cloud Executable UDFs GA: Native Code, Network Access, and Query-Level Metrics

LLM observability has a data problem hiding inside a measurement problem. Teams collect prompts, completions, model names, latency, and trace identifiers, but token-usage fields are not always present—particularly when responses are streamed, proxied, or generated by self-hosted models. Without reliable usage data, it becomes harder to explain model costs, compare workloads, and identify prompts that consume tokens without improving answer quality.

One traditional solution is to export traces, run tokenization or evaluation in a separate service, and load the results back into the analytics database. That works, but it creates another pipeline to operate and another place where data can become stale.

ClickHouse Cloud’s executable user-defined functions (UDFs) offer another approach: run custom code from SQL, close to the data being analyzed. In an October 5, 2026 announcement, ClickHouse made executable UDFs generally available across its AWS, Google Cloud, and Azure offerings. The release adds a Native runtime for compiled languages, broader outbound network access, infrastructure-as-code support, and query-level metrics for UDF execution. Source: ClickHouse

The accompanying example project demonstrates two complementary workloads: a compiled Rust function that counts LLM tokens and a network-enabled Python function that asks an external model to grade a sample of completions.

The architectural shift is useful, but the details matter. Compiled Native UDFs and network-enabled Python UDFs are different execution paths. Teams should not assume that every runtime has network access or that moving code into a database makes its resource consumption free.

Why it matters

LLM observability depends on more than the usage numbers returned by a model provider. Some responses omit token counts, and provider-reported totals may not reveal how much of the input came from system instructions, retrieved documents, user prompts, or generated output.

That breakdown can be valuable. If a retrieval pipeline suddenly doubles the amount of context it supplies, token costs may increase even when the number of model calls remains unchanged. If a system prompt grows over time, the cost may accumulate across thousands of requests.

ClickHouse’s example uses a synthetic dataset in which approximately 30% of spans lack usage information. This is a result from the company’s demonstration dataset, not a universal measurement of all production LLM traffic. The example then counts tokens from the recorded text and compares its estimates with provider-reported usage where available. Source: ClickHouse

The second problem is quality. Token counts explain part of the bill, but not whether a completion was useful. A separate model can grade a sample of responses, producing structured scores and verdicts that can be aggregated by application, model, or feature.

Putting these computations closer to the trace data allows engineers to connect usage, cost, and quality through familiar SQL queries. It also makes resource accounting more important: CPU-heavy tokenization and network-bound model evaluation have very different performance characteristics.

What ClickHouse announced

The general-availability release introduces several capabilities for executable UDFs in ClickHouse Cloud:

• Native runtime: Upload compiled binaries for supported languages, including Rust, Go, C++, and JavaScript compiled with Bun.
• Outbound network access: Network-enabled UDFs can call public endpoints, subject to the configured runtime and service capabilities.
• Azure availability: Executable UDFs are available across ClickHouse Cloud on AWS, Google Cloud, and Azure.
• Memory limits and deterministic functions: Configure per-process memory limits and mark functions deterministic when identical inputs are guaranteed to produce identical outputs.
• Cloud API and Terraform: Manage UDFs through the Cloud API and supported Terraform resources, including per-service version pinning.
• Execution metrics: ClickHouse 26.6 and later adds UDF-specific profile events and asynchronous metrics that help expose child-process costs alongside query telemetry.

The distinction between runtimes is important. The Native runtime is intended for compiled, self-contained code; the network-enabled example in ClickHouse’s article uses Python. The announcement describes outbound network access as generally available, but it does not mean a Native binary can automatically make external API calls. Check the current runtime documentation before choosing an implementation.

Another operational detail: executable UDFs are not billed as a separate line item in the described model. They run in the service’s pods and consume the same CPU and memory resources as queries. That makes per-query visibility especially valuable, because UDF work can affect the performance of other workloads sharing the service.

How the architecture works

The example project combines two functions with ordinary ClickHouse tables, materialized views, and SQL.

The count_tokens function is implemented in Rust using a tokenizer library. It receives a model identifier and text, then returns a token count. A materialized view calls it when new spans are inserted, so the counts can be stored alongside the trace data rather than recalculated for every dashboard query.

The judge_response function uses Python to send a prompt and completion to an external LLM API. The example constrains the response to a structured verdict, score, and explanation. A refreshable materialized view runs the evaluation periodically against a sample of recent spans.

The token counts can be aggregated by model, feature, or application. Evaluation results can be grouped by the same dimensions, then joined with token data to explore questions such as whether an expensive feature also has a high failure or low-quality rate.

This architecture keeps orchestration relatively small. ClickHouse stores the traces, runs the tokenization UDF, schedules the sample-evaluation query through a refreshable materialized view, and stores the results for analysis.

It does not eliminate the external model provider, nor does it make a network call deterministic. The quality-evaluation path still depends on API availability, latency, credentials, response handling, and the cost of the model being called.

Native runtime: compiled code next to the data

The Native runtime accepts a packaged executable rather than requiring the function to run as a Python script. ClickHouse’s example uses an executable_pool configuration so long-lived worker processes can be reused across data blocks and queries.

The repository’s Rust tokenizer illustrates the deployment requirements. The archive contains architecture-specific executables for amd64 and arm64, plus the data files needed by the function. The binary must be built for the expected environment and include the runtime dependencies it needs; ClickHouse does not install arbitrary application dependencies inside the uploaded executable.

The example uses RowBinary as the wire format between ClickHouse and the executable. This avoids the overhead of parsing and formatting text-based rows. ClickHouse reports roughly a 20% performance improvement from switching the example from TabSeparatedRaw to RowBinary, but this is a result from its specific test—not a guarantee for every UDF or dataset.

The configuration also uses a process pool of four workers. Pool size is a tuning parameter, not a universal recommendation. More workers may improve parallelism when the function is CPU-bound, but each process consumes resources, and excessive concurrency can compete with the queries themselves.

Why executable_pool matters

An executable UDF can start a process for each execution block or reuse long-lived processes through a pool. The pooled model avoids repeated process startup and can keep expensive-to-load resources in memory.

That is particularly relevant for tokenizers, parsers, and similar functions that load vocabulary files or initialize libraries before processing rows.

The trade-off is operational: pooled processes need sensible memory limits, execution limits, error handling, and version management. A bad binary or malformed input must fail visibly rather than return plausible but incorrect values.

Network-enabled UDFs: useful, but not free

The judge_response example shows a different pattern. It uses a Python UDF with outbound network access to call an external model and produce a structured evaluation.

The example sets a 30-second maximum command execution time and a pool size of four. It also uses a refreshable materialized view that evaluates a deterministic sample of recent spans every ten minutes. Those choices constrain how much work the evaluation loop attempts at each refresh, but they do not guarantee that every refresh finishes on schedule or that the external service will respond successfully.

Network access changes the failure model. A slow endpoint, API rate limit, authentication failure, or upstream outage can affect the UDF’s execution. Repeated calls may also incur external API charges even though the UDF itself does not have a separate ClickHouse billing line.

The example project also highlights a current secret-management limitation: its API key is supplied through configuration packaged with the UDF, rather than through a native UDF secret manager. That is an important caveat for production deployments. Do not copy this demonstration pattern into a sensitive production environment without reviewing the current supported secret-management options and applying your organization’s credential-handling requirements.

For network-enabled functions, teams should use sampling, timeouts, bounded concurrency, explicit failure handling, and appropriate controls on what data can be sent to an external service.

Per-query metrics make UDF costs visible

One of the most important parts of the release is observability. Previously, query-level telemetry could show the parent query’s resource consumption without fully explaining the work performed by its child UDF processes.

ClickHouse 26.6 adds eight ProfileEvents counters for executable UDFs. These appear in system.query_log and other system telemetry, alongside the query’s existing metrics.

Metric	What it helps measure
ExecutableUserDefinedFunctionInvocations	Number of UDF invocations
ExecutableUserDefinedFunctionElapsedMicroseconds	Time spent in UDF calls
ExecutableUserDefinedFunctionPoolWaitMicroseconds	Time waiting for a free pooled process
ExecutableUserDefinedFunctionUserTimeMicroseconds	CPU time in user mode
ExecutableUserDefinedFunctionSystemTimeMicroseconds	CPU time in kernel mode
ExecutableUserDefinedFunctionPeakMemoryByteSeconds	Memory consumption integrated over time
ExecutableUserDefinedFunctionInputBytes	Bytes sent to UDF processes
ExecutableUserDefinedFunctionOutputBytes	Bytes returned by UDF processes

Two asynchronous metrics also report the number of UDF processes currently alive and their aggregate resident memory.

Together, these measurements help answer questions that query duration alone cannot resolve:

• Is the UDF consuming most of the query’s CPU time?
• Are requests waiting for an available process?
• Is the function processing more data than expected?
• Is memory growing as the pool scales?
• Does a change in serialization or chunk size increase the number of invocations?

The practical benefit is that teams can investigate UDF overhead through the same system telemetry they already use to analyze queries. The metrics do not automatically assign a dollar cost to each function call, however. Translating resource usage into cost still requires context about service configuration, workload sharing, and the applicable pricing model.

What the token-counting benchmark tells us

ClickHouse’s example compares Rust, Python, and Go implementations of the token-counting workload. In its reported test, 100,000 synthetic spans containing approximately 351 MiB of prompt text were processed with four token-counting operations per span.

The reported query durations were approximately:

Implementation	Query duration
Rust Native UDF	5.2 seconds
Python UDF	7.2 seconds
Go UDF	15.0 seconds

These figures are measurements from ClickHouse’s benchmark setup, not general language-performance rankings. The Python tokenizer uses a library with native code underneath, while the Go implementation uses a pure-Go tokenizer. The library and implementation choices therefore matter at least as much as the language label.

The authors also note that the benchmark was run against open-source ClickHouse with an XML function configuration rather than reproduced directly on ClickHouse Cloud. Use it as an illustration of the potential trade-offs, not as a managed-service performance guarantee.

The broader engineering lesson is straightforward: moving a function into a compiled runtime does not automatically make it faster. Serialization, process boundaries, library implementation, batch size, and resource contention all affect throughput.

What data and AI engineers should do

1. Separate deterministic compute from network-dependent work

Tokenization is a good candidate for a deterministic UDF when the model and text fully determine the output. Network-based grading is different: the external model may change, requests may fail, and identical inputs are not necessarily guaranteed to produce identical results.

Keep these paths separate. Use deterministic caching only when the function’s behaviour genuinely satisfies that contract.

2. Measure before increasing the process pool

Start with a small pool and inspect UDF invocation counts, elapsed time, CPU, memory, and pool-wait metrics. Increase concurrency only when the measurements show that workers are a bottleneck and the service has sufficient headroom.

A larger pool can increase throughput, but it can also increase memory use and contention with normal analytical queries.

3. Keep network calls out of the hot path when possible

The Rust token-counting example processes spans during insertion, while the LLM judge evaluates a sample on a periodic schedule. That separation prevents a slow evaluation API from becoming a dependency for every incoming span.

For production observability, consider sampling, asynchronous evaluation, retry queues, and explicit backpressure rather than calling an external LLM for every record in a latency-sensitive ingestion path.

4. Version and deploy UDFs as software

The Cloud API and Terraform provider support managing UDF definitions and attaching versions to services. Pin versions for reproducibility, test updates against representative data, and roll out changes deliberately.

Treat executable bundles as production artifacts: build for the supported architectures, include required files, validate input and output formats, and fail loudly when a required model file or configuration is missing.

5. Treat external credentials and data as security boundaries

Network-enabled UDFs can send trace content outside the database. Review what fields are included in prompts, whether customer data or secrets could be exposed, which endpoints are allowed, and how credentials are stored and rotated.

The database’s access controls do not remove the need for outbound-data governance. An evaluation function should receive only the data it needs.

6. Account for the shared resource model

No separate UDF billing line does not mean no cost. UDF processes consume the service’s CPU and memory and can slow queries that share those resources.

Monitor UDF-specific metrics next to query latency and service health. Include the cost of external model calls in the overall workload model, and compare the operational expense of in-database execution with a separate service where appropriate.

Limitations and open questions

The release makes executable UDFs more practical, but several details remain important before adopting them for a critical workload:

• Runtime capabilities differ. The Native runtime supports compiled code, while the demonstrated network-enabled function uses Python. Verify current capabilities rather than assuming every runtime has the same network access.
• The benchmark is not a Cloud guarantee. The reported language comparison used open-source ClickHouse with an XML function configuration. Managed-service results can differ.
• UDFs compete for service resources. They consume the same CPU and memory budget as queries, so poor sizing can affect unrelated workloads.
• Secret management needs careful review. The example’s packaged API-key configuration is a demonstration limitation, not a pattern to adopt without security review.
• External API calls remain unreliable dependencies. Timeouts, rate limits, model changes, and upstream failures can all affect evaluation results.
• Operational limits still matter. Pool size, process memory, chunking, execution timeouts, and process failures should be tested against the actual workload.

The feature is most compelling when a custom function is difficult to express in SQL, needs a mature language library, or benefits from executing near the data. It is less compelling when the same operation can be performed efficiently with a built-in SQL function or when a network-dependent task would create an unacceptable dependency in the critical query path.

Key takeaways

1. Executable UDFs are generally available on ClickHouse Cloud across AWS, Google Cloud, and Azure. The release adds a Native runtime, network access for supported configurations, Terraform resources, and UDF-specific metrics.
2. Compiled code can bring libraries closer to analytical data. The Rust tokenizer demonstrates how token counts can be calculated during ingestion and stored for later analysis.
3. Network-enabled evaluation is a separate execution pattern. The example uses Python to call an external model periodically; do not assume the Native runtime supports the same network behaviour.
4. Per-query metrics improve troubleshooting. CPU, elapsed time, pool waits, memory, and pipe I/O help distinguish query work from UDF overhead.
5. No separate billing line does not mean zero cost. UDFs consume shared compute and memory, and external API calls can add their own charges.
6. Use the benchmark as evidence, not a promise. Actual performance depends on the tokenizer, runtime, serialization, pool configuration, and service workload.

ClickHouse’s executable UDF release narrows the gap between custom application logic and analytical SQL. For LLM observability, that can simplify token accounting and sampled quality evaluation by keeping the relevant computations near the trace data. The production challenge is to preserve that simplicity without losing control of performance, secrets, network dependencies, and failure handling.

Sources

1. ClickHouse Blog, Executable UDFs are now generally available on ClickHouse Cloud, October 5, 2026.
2. GitHub, ClickHouse/llm-token-udf: LLM observability token counting.
3. ClickHouse Blog, Executable UDFs are now in public beta on ClickHouse Cloud, May 30, 2026.