Real-Time Energy Trading on Databricks: One Governed Data Copy for the Desk, Risk and ML
Data Engineering

Real-Time Energy Trading on Databricks: One Governed Data Copy for the Desk, Risk and ML

Databricks streams market data with Real-Time Mode into Lakebase Postgres, serves the trading desk from it, and trains models and runs analytics on the same governed copy. Here is how the pieces fit, and how to wire an app to Lakebase.

Nov 13, 2026 · 5 min read
How IFCO Cut a 7-Hour dbt Job to 2 Hours 20 Minutes on Databricks
Data Engineering

How IFCO Cut a 7-Hour dbt Job to 2 Hours 20 Minutes on Databricks

IFCO reduced its core dbt job runtime by about 66% and daily compute costs by 58% by fixing incremental-write behaviour, inspecting actual execution plans, and splitting a large dbt graph into individual Databricks tasks—with model-level diagnostics and targeted recovery.

Oct 16, 2026 · 14 min read
Databricks SQL Serverless Cost Governance: Budgets, Alerts, and Query-Level Attribution
Data Engineering

Databricks SQL Serverless Cost Governance: Budgets, Alerts, and Query-Level Attribution

Databricks adds budgets, billing system tables and usage dashboards to make SQL Serverless spend easier to explain. But budgets are not spending caps, estimates are not invoices, and useful attribution depends on consistent tagging and operational controls.

Oct 10, 2026 · 14 min read
Databricks Runtime Introduces NEAREST BY: Native SQL Vector Search at Scale
Data Engineering

Databricks Runtime Introduces NEAREST BY: Native SQL Vector Search at Scale

Databricks brings batch vector search into SQL with NEAREST BY, a fused Photon GEMM operator and IVF indexes stored in liquid-clustered Delta tables, reducing the need for separate vector-serving infrastructure in large-scale analytical workloads.

Oct 09, 2026 · 13 min read
Databricks Lakehouse Platform for Real-Time, Agentic Data and AI Workloads
Data Engineering

Databricks Lakehouse Platform for Real-Time, Agentic Data and AI Workloads

Databricks is pushing the lakehouse beyond analytics: federated and streaming data, transactional serving, AI Search and agents can share one governed data foundation. What that changes for data and AI engineers, and where the limits are.

Oct 07, 2026 · 13 min read