After this lesson you can explain the difference between Rule-Based AI, Machine Learning, Deep Learning, and Generative AI; describe the role of the Transformer architecture in enabling modern large language models; trace the evolutionary arc from expert systems to agentic AI; and articulate why this progression matters for maritime cyber-physical systems.
The word "AI" has been used since 1956, but what it describes today is radically different from what it described a decade ago — and almost unrecognisable compared to its 1980s incarnation. AI did not arrive as a single invention. It evolved through at least six distinct technological generations, each enabled by different ideas, different mathematics, and different computational resources.
Understanding that arc — from hand-written rules to self-improving systems that generate language, images, code, and decisions — is the foundation of this entire AI Learning Roadmap. Every subsequent lesson builds on concepts introduced here. Skip this lesson and the rest of the series is a set of disconnected facts. Master it and every technical concept that follows has a clear place in the story.
This lesson is also a course in intellectual humility: every generation of AI that looked "finished" turned out to be a precursor to something more capable. That pattern has not stopped.
1. The Idea Before the Machines — Rule-Based AI and Expert Systems
The first practical AI systems were built on a simple premise: if human experts know the rules of a domain, those rules can be encoded into a computer program. These Rule-Based AI (or Symbolic AI) systems used explicit if-then logic — "If patient has fever AND cough AND age > 60, then risk score = HIGH." The knowledge was in the rules; the machine's job was to evaluate them consistently at speed.
By the 1970s and 1980s, this became Expert Systems — programs like MYCIN (Stanford, 1972, medical diagnosis) and XCON (DEC, 1980, computer configuration) that encoded thousands of domain rules and could out-perform junior specialists in narrow tasks. These systems worked well when the rules were finite and stable. They failed catastrophically when the world was ambiguous, when exceptions were common, or when the domain was too large to hand-encode.
The fundamental insight that eventually replaced rule-based AI was deceptively simple: instead of telling the machine the rules, show it examples and let it figure out the rules itself. That insight is Machine Learning.
2. Machine Learning — When Data Replaces Rules
Machine Learning (ML) inverts the programming model. In classical programming, a developer writes rules (the algorithm) and the machine applies them to data to produce an output. In ML, a developer provides data and the desired output (labels), and the machine generates the algorithm — the learned model — that maps one to the other.
Classical Programming
Input: Rules + Data
Output: Answers
Developer encodes domain knowledge as logic. Machine applies it.
Machine Learning
Input: Data + Answers (labels)
Output: Rules (the model)
Machine discovers the pattern. Developer provides the examples.
The core techniques of classical ML — decision trees, support vector machines, logistic regression, random forests, gradient boosting — emerged through the 1990s and 2000s and still underlie much of the AI deployed in production systems today. They are interpretable, computationally efficient, and effective across structured data domains (tabular data, time series, classification tasks).
The limitation of classical ML is feature engineering: someone still has to decide which columns, transformations, and representations of the raw data to feed into the model. For structured data (a spreadsheet of vessel maintenance records), this is manageable. For unstructured data (a photograph, a sentence, a ship noise recording), it was nearly impossible to engineer features good enough for the model to work well. That limitation created the demand for Deep Learning.
3. Deep Learning — Neural Networks at Scale
Deep Learning is a subfield of Machine Learning that uses artificial neural networks with many layers (hence "deep") to learn representations of data directly from raw inputs. Rather than asking a human to engineer features, a deep neural network learns a hierarchy of increasingly abstract features automatically from the data itself.
The mathematical foundations — backpropagation, activation functions, gradient descent — were largely understood by the late 1980s (Rumelhart, Hinton, and Williams, 1986, Nature, "Learning representations by back-propagating errors"). What changed in 2012 was scale: more data (ImageNet — 1.2 million labelled images) and more computation (NVIDIA GPUs). AlexNet (Krizhevsky, Sutskever, and Hinton, NeurIPS 2012) used a deep convolutional network trained on ImageNet and reduced the image classification error rate by nearly 11 percentage points over the previous state of the art in a single year. The era of Deep Learning had begun.
A shallow network (1–2 layers) can approximate any function in theory, but requires exponentially more neurons to do so than a deep network. Depth enables the network to learn compositional representations: edges → textures → object parts → objects. This hierarchical feature learning is what makes deep learning so effective at images, speech, and text.
The 2010s were the decade of deep learning's expansion: CNNs for vision (Lesson 3), RNNs and LSTMs for sequences (Lesson 3), word embeddings (Word2Vec, 2013), and eventually models like ResNet (He et al., 2015) that trained networks 152 layers deep. But deep learning still operated one modality at a time. The architecture that unified everything came in 2017.
4. The Transformer — Attention Is All You Need (2017)
In June 2017, Vaswani et al. at Google Brain published "Attention Is All You Need" (arXiv:1706.03762, NeurIPS 2017). The paper introduced the Transformer architecture, which replaced recurrent layers entirely with a mechanism called self-attention: each position in a sequence attends directly to every other position, computing how relevant each token is to understanding the current token, in parallel rather than sequentially.
The immediate application was machine translation. But the implications went far beyond: because Transformers process entire sequences in parallel, they train dramatically faster on GPUs than RNNs. Because self-attention captures long-range dependencies without the vanishing gradient problem, they handle longer contexts than LSTMs. And because the architecture is general-purpose, it can be pre-trained on massive text corpora and then fine-tuned for downstream tasks.
Transformer 아키텍처 내부 구조, attention 메커니즘, 그리고 LLM 개발 과정에서 제기되는 AI 윤리 이슈를 상세히 다룹니다. 이번 레슨의 섹션 4를 더 깊이 탐구하고 싶다면 이 포스트를 참조하세요.
5. Generative AI — From Perception to Creation
The AI systems built in the 2010s were overwhelmingly discriminative: given an input (an image, a sentence, a sensor reading), they classified, detected, or predicted. They recognised cats in photos; they did not draw cats. Generative AI is AI that creates new artefacts — text, images, audio, code, 3D models — that did not exist in the training data.
The key developments enabling Generative AI were: Generative Adversarial Networks (GANs, Goodfellow et al., 2014 — covered in detail in Lesson 5), Variational Autoencoders (VAEs, Kingma and Welling, 2013), Diffusion Models (Ho et al., 2020, "Denoising Diffusion Probabilistic Models"), and crucially the application of large autoregressive Transformers (GPT-2, GPT-3) to text generation.
The practical significance: Generative AI shifted AI from a tool that answers questions to a tool that drafts, designs, summarises, translates, and creates. This changed the human-AI interaction model from "query and respond" to "collaborate and iterate" — and created the preconditions for Agentic AI.
6. Agentic AI — The Current Frontier
Agentic AI refers to AI systems that do not merely generate a single response but execute multi-step workflows autonomously — planning, using tools, making decisions, delegating subtasks, and adapting to intermediate results. The LLM becomes the reasoning engine at the centre of a larger system that can browse the web, write and execute code, call APIs, manage files, and coordinate with other agents.
The key architectural components of an AI agent (covered in detail across PART 4 and PART 5 of this roadmap) are: Memory (what has happened so far), Planning (what to do next), Tool Use (executing external actions), and Reflection (evaluating the output and deciding whether to continue). Frameworks like LangGraph, AutoGen, and CrewAI provide scaffolding for orchestrating these components across multi-agent systems.
Agentic AI가 왜 등장했는지, LLM이 어떻게 Agent의 핵심 엔진이 되는지를 5편 시리즈로 상세히 다룹니다. 이번 섹션 6을 심화 학습하려면 이 시리즈부터 시작하세요.
기업 현장에서 Agentic AI가 어떻게 배포되고, 어떤 조직적·기술적 조건이 필요한지를 다룹니다. GenAI에서 Agent AI로의 전환이 실무에 미치는 영향을 이해하는 데 도움이 됩니다.
The important framing distinction: earlier AI generations were tools that a human operates. Agentic AI is a system that operates on behalf of a human, deciding its own intermediate steps. That shift has profound implications for safety, accountability, and system design — particularly in safety-critical environments like maritime operations.
7. Maritime Connection — AI Across the Ship's Lifecycle
Each AI generation described in this lesson has a direct maritime application layer — and maritime is unusual as a domain because all six generations are simultaneously present in the industry today, often on the same vessel or in the same shipyard.
The cybersecurity implication is direct: the more autonomous an AI system becomes, the larger its attack surface. An agentic AI that can access vessel systems, write configuration files, and communicate with shore-based infrastructure is a target category that did not exist under rule-based automation. Understanding the AI progression is therefore inseparable from understanding the maritime threat landscape.
During E26 CRSI 프로젝트 수행, I regularly encounter a pattern I think of as the "AI label problem": every automated decision system gets called "AI" regardless of whether it is a rule-based alarm threshold or a trained neural network. This label conflation is not just semantically imprecise — it creates real risk.
When a vessel operator calls their PMS (Planned Maintenance System) "AI-powered", they often mean it uses rule-based triggers. When they plan to deploy "AI-assisted" anomaly detection, the type of ML model deployed determines the explainability, failure mode, and adversarial robustness of the system — categories that IACS UR E26 cyber resilience assessments are beginning to probe.
The practical takeaway: always ask "which generation of AI?" before evaluating risk. A rule-based system has no data poisoning attack surface. A deep learning model trained on historical AIS data does. The distinction matters for the cyber risk assessment, for the assurance evidence, and for the incident response plan.
- Rule-Based AI encodes human-written rules. It is transparent, auditable, and brittle — it cannot handle what it was not explicitly programmed for.
- Machine Learning inverts the programming model: data and labels in, model (rules) out. Feature engineering is still required for unstructured inputs.
- Deep Learning eliminates manual feature engineering by learning hierarchical representations from raw data. It requires large labelled datasets and GPU-scale computation.
- The Transformer (Vaswani et al., 2017) introduced self-attention, enabling parallel training over long sequences. It is the architecture underlying all major LLMs.
- Generative AI shifted AI from discriminative (classify/detect) to creative (generate/draft/design). This changed the human-AI interaction model fundamentally.
- Agentic AI adds planning, memory, and tool use to the LLM core, enabling autonomous multi-step workflows — and a correspondingly larger attack surface.
- All six AI generations coexist in maritime today. The cybersecurity risk profile of each is different and requires a different assessment approach.
Lesson 2 — How Machines Learn from Data takes the Machine Learning step and unpacks it: what does a training dataset actually look like, how does gradient descent adjust model parameters, what is the difference between training and inference, and what are the different learning paradigms (supervised, unsupervised, reinforcement)? These concepts are the shared vocabulary of every AI discussion that follows.
After Lesson 2, Lesson 3 dives into the two architectures — CNN and RNN — that dominated the deep learning era before the Transformer, explaining why each was necessary and what limitations each exposed that the Transformer ultimately resolved.
- Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). "Attention Is All You Need." NeurIPS 2017. arXiv:1706.03762.
- Krizhevsky, A., Sutskever, I., and Hinton, G. (2012). "ImageNet Classification with Deep Convolutional Neural Networks." NeurIPS 2012 (AlexNet paper).
- Rumelhart, D., Hinton, G., and Williams, R. (1986). "Learning representations by back-propagating errors." Nature, 323, 533–536.
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al. (2014). "Generative Adversarial Nets." NeurIPS 2014. arXiv:1406.2661.
- Ho, J., Jain, A., and Abbeel, P. (2020). "Denoising Diffusion Probabilistic Models." NeurIPS 2020. arXiv:2006.11239.
- Brown, T., Mann, B., Ryder, N., et al. (2020). "Language Models are Few-Shot Learners." NeurIPS 2020 (GPT-3 paper). arXiv:2005.14165.
- IACS UR E26 (Rev.3, 2024) — Cyber Resilience of Ships. iacs.org.uk
- NIST AI RMF 1.0 (January 2023) — AI Risk Management Framework. airc.nist.gov
⚓ Join the ShipPaulJobs Community
Join →
Comments
Post a Comment