Beyond the LLM: Understanding the AI Stack for AR+IQ
Photo by Steve Johnson on Unsplash
Most teams building AI-powered products fall into the same trap. They plug a large language model (LLM) like GPT or Claude into a user interface, ship an app, and call it done. But for serious applied AI, especially for domains like AR+IQ, this is only the beginning.
At iSPARX™ and through the AR+IQ platform, we operate across the entire AI stack. Here’s a simple framework that guides our architectural thinking, with real-world examples from our work in immersive media, cultural storytelling, and augmented reality (AR) experiences.  .
⸻
1. LLM (Large Language Model) — The Base Layer
Think: GPT-4o, Claude 3, Mistral.
This is the general-purpose language generation layer. It’s where most AI builders start. LLMs excel at:
Summarising documents
Drafting text content
Powering FAQs
Supporting conversational interfaces
In AR+IQ:
We use LLMs to:
Automatically generate metadata for artworks in AR+artist™
Produce draft cultural narratives for AR+story™ modules
Create first-pass text content for AR+concierge™ pop-up installations 
Limitations:
No real-time awareness of current events or updates
No direct access to internal or enterprise data unless integrated further
Struggles with executing tasks or workflows
Lacks consistency in long-running interactions
Example failure mode:
If a user asks about the status of their AR+artist™ exhibit deployment, a pure LLM can’t query deployment data directly.
2. RAG (Retrieval-Augmented Generation) — The Knowledge Layer
Think: Vector databases + LLM.
RAG bridges internal data and AI generation. The model retrieves relevant context before answering.
In AR+IQ:
We embed RAG into:
The AR+concierge™ knowledge base for geospatial event data
Retrieval of cultural consultation documents (e.g. kaupapa Māori protocols, Indigenous IP frameworks) 
Querying system logs, location data, and user interaction telemetry 
RAG is good for:
Providing precise answers based on internal documentation
Building context-aware document copilots for internal staff
Serving structured content to artists, cultural partners, and event producers
Limitations:
Still passive — it cannot act or perform transactions
Complex multi-step workflows are still out of reach
Limited real-time interaction with live systems
Example:
When planning an AR+story™ guided experience at Ruatoki, RAG allows instant access to co-created cultural content libraries .
3. AI Agents — The Action Layer
Think: Orchestration models that can call APIs, monitor systems, and execute tasks.
In AR+IQ:
AI Agents are deployed to:
Automate AR+workshop scheduling across venues 
Manage participant onboarding and consent tracking (critical for ethics compliance) 
Update CMS content dynamically for changing installations 
Agents are good for:
Scheduling logistics across AR+concierge™ deployments
Generating real-time analytics reports for event partners 
Integrating ticketing, loyalty, and ecommerce platforms directly into AR+IQ pop-ups
Limitations:
Single-agent models struggle with complex multi-agent workflows
Long-term planning and adaptation require higher-order coordination
Example:
An agent automatically coordinates with venue management systems to update event calendars for GigBox.Ai installations .
4. Agentic AI — The System Orchestration Layer
Think: Teams of agents, each with specialised roles, operating with memory, coordination, and autonomy.
In AR+IQ:
We are piloting Agentic AI for:
Coordinated deployment of multi-location exhibitions with live telemetry feedback 
Dynamic adjustment of AR scene composition based on real-time crowd flow data 
Multi-agent content creation pipelines that balance artist input, cultural protocols, and technical deployment 
Agentic AI is good for:
Orchestrating complex AR+IQ deployments across multiple stakeholders
Managing long-running adaptive AR environments
Enabling self-healing AR experiences that adjust based on user behaviour
Limitations:
Complex to design and manage
Often unnecessary for simple use cases
Requires extensive domain knowledge to train and supervise
Example:
During the Pātaka Museum installation, agents collaborated to manage live updates, visitor engagement metrics, and narrative adjustments driven by participant feedback.
The Quick Guide: Choosing the Right AI Layer for AR+IQ
Problem >> Solution
You need customer Q&A over internal documents? >> Use RAG
You need automated workshop scheduling? >> Use Agents
You need end-to-end orchestration across multiple locations and teams? >> Use Agentic AI
The Reality: Most Teams Only Ship the First Layer
Many startups stop at LLM + UI. In iSPARX™, that approach would barely scratch the surface of what AR+IQ delivers. For true applied AI in immersive, cultural, geospatial experiences, the full stack is not optional — it’s foundational  .
We don’t build chatbots.
We build fully autonomous systems that understand context, execute real-world actions, and deliver culturally-anchored experiences.
In short:
LLM is the conversation.
RAG is the knowledge.
Agents are the hands.
Agentic AI is the brain.
At iSPARX™, we aren’t just “applying AI” — we are redefining how audiences interact with space, story, and culture. AR+IQ is built for 22nd-century creative industries.
Architecting AI for AR+IQ: Beyond the LLM Layer
When most AI startups say they’re “AI-powered,” they mean:
They called OpenAI’s API.
Wrapped it in a frontend.
Shipped.
For CTOs building scalable, production-grade platforms like AR+IQ, that approach is not even V0. It’s a prototype at best. Applied AI requires layered architectures — integrating multiple AI modalities, agents, orchestration logic, and deeply embedded domain knowledge.
At iSPARX™, AR+IQ is not a chatbot. It’s a multi-modal, agentic AI system designed to operate in highly contextualised, spatial, and culturally-sensitive environments across multiple sectors: arts, tourism, education, and commerce   .
Below is the simplified stack we’ve operationalised:
1. LLM Foundation Layer — Generative Core
Function:
Autoregressive language models trained on massive corpora.
Examples in stack: GPT-4o, Claude 3, Mistral 7B
Primary Roles in AR+IQ:
Drafting AR+story™ narrative scaffolds 
Summarising post-installation feedback reports
Generating dynamic textual overlays for AR+artist™ and AR+guide™
Assisting in metadata auto-generation for the CMS 
Architecture Considerations:
Hosted LLM endpoints (OpenAI/Azure OpenAI) via secured API layer
Wrapped with safety filters to enforce cultural protocol constraints 
Fine-tuned internal embeddings for domain-specific semantic precision
Weaknesses:
No system state awareness
Cannot call APIs or control spatial environments directly
Prone to hallucination without grounding
2. RAG Layer — Retrieval Augmented Knowledge Integration
Function:
Real-time grounding using proprietary indexed datasets and vector embeddings.
Examples in stack:
Azure Cognitive Search
Pinecone VectorDB
Custom Faiss deployments for embedded cultural knowledge graphs 
Use Cases in AR+IQ:
Indigenous knowledge retrieval for AR+concierge™ installations 
Regulatory compliance retrieval for cross-border deployments
Internal technical documentation retrieval for field techs
Architecture Considerations:
Embedding pipelines running on Azure Functions with daily refresh cycles
Vector databases with hybrid search (semantic + keyword + filtering)
Secure multi-tenant isolation to separate clients’ knowledge bases
Weaknesses:
Still passive — requires orchestration layer to act on retrieved data
Latency spikes under heavy complex queries
3. AI Agent Layer — Orchestrated Action Units
Function:
Encapsulated API-calling agents with single-domain responsibility.
Examples in stack:
Custom-built agents via LangChain + Azure OpenAI Orchestration
Workflow orchestrators using Temporal.io or Durable Functions
Integration to live operational systems (Venue CMS, CRM, logistics platforms)
Use Cases in AR+IQ:
Deployment automation for AR+IQ pop-up installations 
Real-time ethics compliance checks during participant onboarding 
Live workshop scheduling and resource allocation 
Dynamic telemetry monitoring for live spatial interactions 
Architecture Considerations:
Each agent operates with scoped permissions for auditability
Secure API token vaulting using Azure KeyVault
Real-time queue management for parallelised agent execution (Azure Service Bus)
Weaknesses:
Agents do not coordinate — orchestration logic must handle multi-agent state
Performance drops under complex emergent workflows
⸻
4. Agentic AI Layer — Distributed Autonomous Coordination
Function:
Multi-agent systems with persistent shared state, role-based task allocation, and dynamic reasoning.
Examples in stack:
Multi-agent coordination via Autogen/OpenDevin-like systems (internal prototype)
Memory layers via Redis Streams & event sourcing logs
Custom reasoning models for cultural protocols compliance (ethical constraints enforcer)
Use Cases in AR+IQ:
Coordinating real-time updates across multi-location AR+IQ deployments 
Managing cross-border asset approvals, cultural authority consultations, and ethics board reporting 
Negotiating optimal visitor flows based on real-time AR+concierge™ telemetry 
Architecture Considerations:
Temporal state machines with compensation strategies
Fine-grained task decomposition into micro-agents with context-limited reasoning
Persistent memory layers for longitudinal deployment state awareness
Weaknesses:
High operational complexity
Requires human-in-the-loop supervisory agents for governance
Limited generalisability — domain-specific engineering required
The Architectural Meta-Pattern
The Architectural Meta-Pattern
This is a layered system design approach for applied AI systems operating in real-world domains (AR, immersive media, multi-user environments, cultural storytelling, etc). Each layer builds capability beyond the previous one. You can think of it as stacked capabilities, each solving a class of problem:
Why We Can’t Just “Use GPT-4o”
The majority of real-world AR+IQ deployments involve:
Real-time geospatial coordination 
Multi-stakeholder cultural governance 
Physical environment state management (venue mapping, visitor flows, weather data) 
Long-running, stateful, inter-agent negotiation
These are non-trivial systems problems. They require distributed systems expertise, domain-specific ontologies, real-time sensor integration, and deeply ethical architecture considerations  .
CTO Takeaway
If your AI product simply calls GPT, you’re building a demo.
If your AI product runs RAG + Agents + Agentic AI across real-world, live spatial deployments like AR+IQ — you’re building operational AI.
The AR+IQ stack is built for:
Enterprise scale
Regulatory safety
Cultural safety
Multi-modal experiences across live physical spaces.
At iSPARX™, we are engineering 22nd century creative infrastructure, not chatbots.