This series is a complete 2027 revision of the "Enterprise Chatbot Strategy" originally written in 2021. Part 4 covers the two topics the original Part 3 and 4 addressed — LLM architecture and performance measurement — updated for the era of Transformer-based models, the EU AI Act, and enterprise governance.
LLM Architecture & AI Ethics (this article)
How LLMs Work — Transformer architecture in plain language; what "training" and "inference" actually mean
Why LLMs Fail — Hallucination, bias, prompt injection, context limits; causes and mitigations
Performance Metrics — Updated 2027 evaluation framework for enterprise AI
AI Ethics & Regulation — EU AI Act, Korean AI Basic Act, and what enterprise compliance requires
Prologue
The original 2021 Part 3 covered NLU engines — intent classification, slot filling, entity extraction. Part 4 covered chatbot performance measurement: comprehension rate, engagement rate, response speed, functionality coverage, interoperability, and scalability.
In 2027, those frameworks remain relevant for rule-based systems. But LLM-powered AI systems require a different understanding of both architecture and evaluation. The shift from "does this bot understand the right intent?" to "does this AI reason correctly, and is it safe to act on?" is the theme of this article.
1. What LLMs Actually Do
At its core, a Large Language Model is a statistical function trained to predict the next token (roughly: word or word-piece) given all previous tokens. GPT-4o, Claude 3.5, and Gemini 2.0 are all variations of the Transformer architecture, introduced by Google researchers in 2017.
Your text is split into tokens. "Shipbuilding" might become ["Ship", "building"]. GPT-4o uses roughly 100,000 unique tokens.
Each token is converted to a high-dimensional vector (thousands of numbers). Similar concepts cluster in this vector space — "vessel" and "ship" are nearby.
The "attention mechanism" lets each token weigh how much it should attend to every other token. This is how the model understands context: "bank" means different things near "river" versus "interest rate."
After many layers of attention, the model outputs a probability distribution over all possible next tokens. The highest-probability token is selected (or sampled). This process repeats until the model generates a complete response.
Training: Exposing the model to hundreds of billions of text tokens and adjusting billions of internal parameters to minimize prediction error. Costs tens of millions of dollars. Done once (or rarely) by model providers. Enterprises do not train frontier LLMs from scratch.
Fine-tuning: Additional training on a smaller, task-specific dataset to specialize the model's behavior. Feasible for enterprises with sufficient data and budget. Inference: Running a trained model to generate a response for a given input. This is what happens on every user query. Cost-per-query has dropped 97% since 2021.
2. The Four Core LLM Failure Modes
Enterprise AI programs that fail to understand these risks build systems that appear to work in demos but fail in production. Each failure mode requires specific mitigation strategies.
| Failure Mode | Description | Enterprise Mitigation |
|---|---|---|
| Hallucination | LLM generates plausible-sounding but factually wrong content. Most dangerous when it involves specific numbers, names, or regulations. | RAG with citation; human review layer; guardrails that restrict responses to retrieved content only |
| Bias | Models trained on internet text inherit cultural, gender, and racial biases of that text. Output may be systematically unfair for certain user groups. | Bias testing pre-deployment; diverse test set; ongoing monitoring; explicit fairness constraints in system prompt |
| Prompt Injection | Malicious user input overrides system instructions. Example: hidden text in a pasted document instructs the AI to reveal confidential data. The Samsung ChatGPT leak (2023) is an adjacent risk. | Input sanitization; instruction hardening in system prompt; no sensitive data in prompts; output filtering |
| Context Limit | LLMs have a maximum context window (tokens). Long conversations or large documents may exceed the window; early context gets truncated, causing coherence failures. | Context management strategy: summarize older turns; selective retrieval; track window usage in production |
3. From 2021 Metrics to 2027 Metrics
The 2021 series defined six chatbot performance metrics: comprehension rate, user engagement, speed, functionality coverage, interoperability, and scalability. These remain relevant — but the 2027 framework extends them significantly to account for LLM-specific behaviors.
| Metric | What to Measure | 2021 → 2027 |
|---|---|---|
| Accuracy / Groundedness | % of responses factually correct; hallucination rate in RAG systems | Extended from intent accuracy to factual accuracy |
| Task Completion Rate | % of sessions where the user's goal was fully achieved | Renamed from "comprehension" — now end-to-end |
| Latency (P95) | 95th percentile response time; streaming latency for first token | Now split into TTFT (time to first token) and total latency |
| Containment Rate | % of queries resolved without escalation to human agent | Same as 2021; more meaningful now with better AI capability |
| Safety Rate | % of responses free from harmful, biased, or inappropriate content | New in 2027; required for EU AI Act high-risk systems |
| Cost per Conversation | Total LLM API + infrastructure cost divided by conversation count | New in 2027; LLM token pricing is now a budget variable |
| Audit Coverage | % of AI decisions traceable to a source document or logic rule | New in 2027; required for regulated industries and AI Act compliance |
4. The EU AI Act — Risk Classification Framework
The EU AI Act, phased in from August 2026, is the world's first comprehensive AI regulation. It applies to any AI system used in the EU — regardless of where the vendor is headquartered. It classifies AI systems into four risk tiers.
AI systems that manipulate human behavior subconsciously, use real-time biometric surveillance in public spaces, or perform social scoring. These are banned entirely.
AI used in critical infrastructure, employment decisions, credit scoring, legal proceedings, medical devices, education assessments, and border control. Requires mandatory registration, risk assessment, human oversight, and high-quality training data documentation. Enterprise HR bots, insurance AI, and logistics decision systems typically fall here.
Chatbots, AI-generated content, emotion recognition systems. Must disclose AI nature to users ("you are interacting with an AI"). GPAI (General Purpose AI) models like GPT-4 and Claude must publish technical documentation and comply with copyright law.
Spam filters, AI in games, simple chatbots for non-critical tasks. Voluntary codes of conduct encouraged.
South Korea enacted its AI Basic Act in 2025, establishing a national AI governance framework with risk-tiered compliance similar to the EU AI Act. For Korean enterprises, this adds domestic compliance obligations on top of EU Act requirements for internationally operating companies. The Korea AI Safety Institute (KAISI) is the primary regulatory body.
Practical 2027 compliance steps: Audit every AI use case against the EU AI Act risk tiers. For High Risk systems, build technical documentation, risk registers, and human oversight mechanisms. All customer-facing AI must disclose its AI nature. Designate an AI compliance owner — in many enterprises this now sits in Legal, Risk, or the CTO's office.
Classification society AI: AI systems used in vessel classification decisions, safety assessments, or inspector recommendations are likely High Risk under the EU AI Act — consequential decisions affecting human safety. DNV and Bureau Veritas must document training data, testing procedures, and human oversight mechanisms.
SOLAS & ISM implications: The ISM Code's requirement for a Safety Management System (SMS) and documented risk assessments maps naturally onto AI Act compliance documentation. Companies with mature SMS processes have a structural advantage in AI governance — the frameworks are analogous.
IMO's AI guidance: The International Maritime Organization published its first AI guidelines for the maritime sector in 2024. While not yet mandatory, they signal the direction of future SOLAS amendments. Proactive compliance is advisable.
In 2021, the question "how does the AI decide that?" was often answered with "it's a black box." That answer is no longer acceptable — legally or organizationally. The EU AI Act, the Korean AI Basic Act, and the accountability standards of enterprise governance all demand explainability.
Understanding how LLMs work — not at a research level, but at a practitioner level — is what allows teams to build guardrails, measure performance honestly, and explain AI decisions to regulators, customers, and boards. Part 5 brings the full series to a close: given everything we now know about consensus, channels, RAG, agents, architecture, and ethics — how do we make the right enterprise AI choice?
LLM Architecture & AI Ethics (this article)
⚓ Join the ShipPaulJobs Community
Join →
Comments
Post a Comment