A Targeted Approach: How Arcadia leverages AI to optimize energy management

In Part 1 of our Q&A with Udit Garg, Vice President of Product & Engineering, we explored why standard AI solutions often fall short in today's complex energy landscape and how Arcadia's experience shapes our approach to AI adoption.

Now, we delve into how Arcadia is strategically leveraging AI — alongside its leading utility data platform, deep industry expertise, and expansive partner network — to help businesses navigate this new era of energy management. Rather than applying AI broadly, we take a targeted approach that enhances specific elements of our platform while upholding the highest standards of data accuracy, security, and responsibility.

Can you describe how the different data layers within your AI stack work?

Our AI stack operates across three distinct but interconnected layers, each serving a crucial function in transforming raw energy data into actionable intelligence and meaningful impact.

The Foundation: Data Acquisition and Extraction

At the base level, we handle data acquisition and extraction — locating utility bills and interval data from thousands of sources, downloading this information, and transforming it into our structured data model. What makes our approach unique is how we selectively apply AI to complement our established systems.

Arcadia has spent over a decade building robust extraction infrastructure — more than a hundred million lines of Java code dedicated to processing energy data from 10,000+ utilities. While many startups are attempting to build AI-powered tools from scratch, these off-the-shelf large language models (LLMs) typically achieve only 90-95% accuracy in this domain, and for only a few summary data points on a bill. Our customers and high-value energy use cases require both historical data depth and 98-99% accuracy.

We strategically deploy AI to address edge cases where the AI will perform better than traditional automation or humans: low-frequency utilities, exceptionally complex bills, or specialized customer requirements beyond our standard capture parameters. Rather than replacing our proven infrastructure, we're weaving AI capabilities into our data acquisition processes to extend our reach while maintaining data integrity.

The Core Layer: Unification and Validation

The second layer focuses on data unification, standardization, and validation — ensuring what we collect is accurate, standardized in a universal data model, and reliable. Here, we've implemented nearly 100,000 automated "audits" or rule checks that verify data integrity. For instance, these audits confirm that all components of a bill add up correctly or that collected units fit within established parameters for electricity data.

We've enhanced this validation process by feeding these audits into multiple AI models that serve as layered checks on our data — an "AI-in-the-loop" framework where AI validates that existing systems work correctly. For accounts with historical data, we implement an anomaly detection model. Rather than relying solely on rigid rule-based approaches, our AI examines multiple vectors simultaneously, including the history of the data stream to generate anomaly scores and identify the nature of irregularities. 

For cases where we have only the bill itself with no additional information, we employ an ensemble-LLM approach to compare AI outputs against our system results. When these disagree (only 2-3% of cases) it signals a potential issue with either the AI or our existing system. We then route these flagged discrepancies to appropriate resolution channels. This approach gives us flexibility to incorporate new use cases and error types without rebuilding complex systems.

To enhance the capabilities of our AI data pipeline, we've implemented model context protocol (MCP) on top of our enterprise data. This protocol establishes a crucial connection between our AI models and our comprehensive historical context, encompassing modeling discussions, Jira tickets, technical documentation, and our codebase. By providing the LLMs with this rich contextual data, our AI models can navigate complex edge cases and understand specific, nuanced modeling decisions embedded within our system. 

For instance, without MCP, the AI model might generate outputs inconsistent with our established data standards. However, with MCP, the AI models gain the necessary contextual awareness to recognize unique utility-specific requirements, ensuring data outputs adheres to our precise and battle-tested (over 10 years) standards. This integration of historical context significantly elevates the intelligence and accuracy of our AI, allowing us to connect, extract, normalize, and analyze energy data with unparalleled precision. We believe this unique approach provides a substantial competitive advantage in leveraging AI for energy data management.

The Intelligence Layer: Insights and Optimization

The third layer leverages our validated data to surface actionable insights, including energy forecasting, optimization, and anomaly detection. Forecasting and anomaly detection are core to energy platforms, and our models are trained on millions of diverse customer data points to provide better accuracy and insight.

We have also been able to make significant advancements in our optimization work for energy storage and solar interventions, dramatically improving the economics of powering buildings.

With our highly accurate data foundation, we can simulate countless combinations of solar capacity, storage configurations, and charge-discharge schedules. Our system continuously calculates the economic impact of each scenario through our sophisticated tariff engine, ultimately identifying optimal solutions that maximize building energy efficiency and cost savings.

How does Arcadia ensure data security when working with third-party AI tools?

When partnering with third-party AI tools, we implement multiple layers of protection to ensure data security. First and foremost, all data is completely quarantined. We establish clear boundaries that prevent the LLM from using Arcadia’s data or customer information for additional training purposes.

Our security measures extend to contractual safeguards as well. Our agreements explicitly state that upon termination of any partnership, we can require the third party to delete all of our data completely. Alternatively, the data must be sequestered with guarantees that it will never be accessible or usable by anyone else.

What's next? How do you keep pace with innovation occurring among the AI-native companies and the new LLMs?

At Arcadia, we've developed a structured approach to staying current with AI advancements. We've established an experimentation and test framework built around carefully selected sample sets from our diverse data populations. This allows us to systematically evaluate new AI capabilities through controlled simulations.

Our framework is designed with flexibility in mind. Product developers can easily toggle between different available models to directly compare performance and capabilities. This hands-on testing process helps us make informed decisions about which technologies to adopt.

Through this methodical approach, we're committed to maintaining our position at the forefront of AI advancement—continuously testing emerging models and technologies to maintain our quality standards while exploring innovative use cases that deliver additional value to our customers.

Applying AI to today's complex energy landscape

Built on energy data expertise, Arcadia takes a focused approach to AI integration. By strategically implementing AI across our three-layered data stack while maintaining the highest standards of accuracy, security, and data protection, we deliver insights that help our customers navigate the complexities of modern energy management. Our commitment to responsible innovation ensures we maintain our 98-99% accuracy standards while continuously exploring new AI capabilities that deliver measurable value for our customers — ultimately unlocking a future of abundant, affordable clean power.