Concept Lab · the granular primitives — transcript

1 / 24

Concept Lab原理实验室

Companion to Ithaca · Aeneas · LineageCompanion to Ithaca · Aeneas · Lineage

Ithaca · Aeneas · 谱系的配套Ithaca · Aeneas · 谱系的配套

The Concept Lab原理实验室

Sixteen granular primitives behind Ithaca & Aeneas — a string, a token, a label, an embedding — each made tangible with a real inscription and a thing you can touch and change.Ithaca 与 Aeneas 背后的十六个颗粒度原理 —— 字符串、词元、标签、嵌入 —— 每个都用一条真实铭文与一个你能触碰并改变的小工具变得具体。

▶ The full journey (merged deck)▶ The full journey (merged deck)

▶ 完整旅程（合并演示）▶ 完整旅程（合并演示）

Ithaca →Ithaca →

Aeneas →Aeneas →

Symbolon →Symbolon →

2 / 24

1 · String1 · 字符串

InputsWhat is a string?输入什么是字符串？

To a computer, this inscription is just an ordered sequence of characters — position 0, 1, 2… Hover any cell.对计算机而言，这条铭文只是有序的字符序列 —— 第 0、1、2…位。悬停任一格。

The substrateWatch for this row of numbered boxes — it’s the same string, reappearing across the demos: searched (#2), counted (#4), transformed (#5), tokenised (#8), embedded (#14).贯穿的基底留意这排带编号的方格 —— 它是同一个字符串，在各演示中反复出现：被检索（#2）、被统计（#4）、被转换（#5）、被词元化（#8）、被嵌入（#14）。

back to Ithaca · Aback to Ithaca · A

返回 Ithaca · A返回 Ithaca · A

3 / 24

2 · String search2 · 字符串检索

InputsString search — and why it fails输入字符串检索 —— 及其为何失败

Search #315181’s word εποιησε (“made it”). Records light up on an exact match — but the Delos signatures that wrote εποιει grey out, so literal search misses kin that mean the same. Assael 2022检索 #315181 的词 εποιησε（“制作”）。记录在精确匹配时点亮 —— 但写作 εποιει 的提洛署名变灰，故字面检索错过同义的同类。Assael 2022

back to Ithaca · Aback to Ithaca · A

返回 Ithaca · A返回 Ithaca · A

4 / 24

3 · Radiocarbon3 · 放射性碳

InputsWhy you can’t radiocarbon-date a stone输入为何不能用放射性碳给石头定年

Radiocarbon reads decaying ¹⁴C in once-living things. Stone and bronze have none — so a date must be read from the letters. Assael 2022放射性碳读取曾活之物中衰变的 ¹⁴C。石与铜没有 —— 故年代只能从字母读出。Assael 2022

back to Ithaca · Aback to Ithaca · A

返回 Ithaca · A返回 Ithaca · A

5 / 24

4 · Pattern-learning4 · 学规律

InputsLearning from examples, not rules输入从样本而非规则中学习

A pattern-learning model is shown millions of words and counts what follows. From real I.PHI words beginning μουσ-, a learned distribution emerges. Sommerschield 2023一个学规律的模型被展示数百万词并统计跟随者。从真实 I.PHI 中以 μουσ- 开头的词，浮现出一条学到的分布。Sommerschield 2023

Hand-off →These tallies are the model’s knowledge — no rules, just counts from data. But what machine does the counting, and how does it nudge itself to match? → Next: the neural network that learns this very μουσ- distribution.衔接 →这些计数就是模型的知识 —— 无规则，只有从数据来的计数。但什么机器在做这统计、又如何自我调整以吻合？→ 下一张：神经网络，它学的正是这条 μουσ- 分布。

back to Lineageback to Lineage

返回谱系返回谱系

6 / 24

· What is a neural network?· 什么是神经网络？

InputsWhat is a neural network?输入什么是神经网络？

A neural network is many simple units (neurons) in layers, joined by connections whose strengths are weights. A signal flows input → hidden → output; it learns by adjusting the weights from examples until the output is right.一个神经网络是许多简单单元（神经元）按层排列，由连接相连，其强度即权重。信号从输入 → 隐藏 → 输出流过；它通过从示例调整权重学习，直到输出正确。

← from the last slideThis is the machine behind the counting you just saw. Press “learn from an example” and feed again: it nudges its weights until its output matches the data’s μουσ- distribution — the wrong answer becomes right. The same engine sits inside Ithaca and Aeneas.← 承接上一张这正是你刚才所见“计数”背后的机器。点击“从示例学习”再馈入：它微调权重，直到输出吻合数据的 μουσ- 分布 —— 错误答案变正确。同一引擎也在 Ithaca 与 Aeneas 之中。

back to Lineageback to Lineage

返回谱系返回谱系

7 / 24

· Inside: the layers· 内部：各层

InputsWhat happens between input and output?输入输入与输出之间发生了什么？

The middle (hidden) layers each take the previous list of numbers and re-mix it a little — ×weights, +bias, squash. Step through and watch the vector change, and why depth is worth it.中间（隐藏）层各取上一层的数串、略加重混 —— ×权重、+偏置、压缩。逐步前进，看向量变化，及深度为何值得。

back to Lineageback to Lineage

返回谱系返回谱系

8 / 24

5 · Pipeline5 · 流水线

DataThe I.PHI pipeline, stage by stage数据逐级走过 I.PHI 流水线

Raw scholarly text can’t be fed to a model. Watch one real record — I.PHI #315181 (περγαμευς μουσαις) — flow through four stages.原始学术文本不能直接喂给模型。看一条真实记录 —— I.PHI #315181（περγαμευς μουσαις）—— 流过四级。

back to Ithaca · Bback to Ithaca · B

返回 Ithaca · B返回 Ithaca · B

9 / 24

6 · Clean6 · 清洗

DataCleaning a raw inscription数据对原始铭文清洗

I.PHI #315181’s real raw text — minuscule, polytonic, with final ς ([Φ]ιλέταιρος … Περγαμεὺς Μούσαις). Toggle each step; the result matches iphi.json exactly. Assael 2022I.PHI #315181 真实的原始文本 —— 小写、多调号、带词尾 ς（[Φ]ιλέταιρος … Περγαμεὺς Μούσαις）。逐项切换；结果与 iphi.json 完全吻合。Assael 2022

back to Ithaca · Bback to Ithaca · B

返回 Ithaca · B返回 Ithaca · B

10 / 24

7 · Final sigma7 · 词尾西格玛

DataOne rule, exactly: final sigma数据一条精确规则：词尾西格玛

Greek writes σ inside a word, ς at the end. Cleaning rewrites word-final σ → ς. Type a word and watch.希腊语词中作 σ、词尾作 ς。清洗把词尾 σ → ς。输入一个词看看。

back to Lineage · IIback to Lineage · II

返回谱系 · II返回谱系 · II

11 / 24

8 · Tokens8 · 词元

RepresentationTokens: letters → numbers表示字母 → 数字

Each character gets an integer id from Ithaca’s real table. The id is an address, not a quantity — it looks up a trainable vector the network can multiply & add. Type Greek text.每个字符从 Ithaca 真实表得到一个整数 id。id 是地址、不是数量 —— 它查出一个可训练向量，供网络乘与加。输入希腊文。

back to Ithaca · Cback to Ithaca · C

返回 Ithaca · C返回 Ithaca · C

12 / 24

9 · Labels9 · 标签

DataInput vs label数据输入 vs 标签

Every training example is a pair: the input (text) and the labels (place + date) it must learn to predict. Assael 2022每个训练样本是一对：输入（文本）与它须学会预测的标签（地点 + 年代）。Assael 2022

back to Ithaca · Bback to Ithaca · B

返回 Ithaca · B返回 Ithaca · B

13 / 24

10 · Tagging10 · 标注

DataTagging: messy metadata → clean fields数据标注：杂乱元数据 → 整洁字段

Free-text metadata must be parsed into structured fields. Ithaca could date only ~60% of PHI. Assael 2022自由文本元数据须解析为结构化字段。Ithaca 仅能为约 60% 的 PHI 定年。Assael 2022

back to Ithaca · Bback to Ithaca · B

返回 Ithaca · B返回 Ithaca · B

14 / 24

11 · Label smoothing11 · 标签平滑

LearningLabel smoothing学习标签平滑

A naive target says Attica = 100%. Label smoothing shaves a little off (Ithaca uses 10%) — humbler, better-calibrated. Assael 2022朴素目标说阿提卡 = 100%。标签平滑削去一点（Ithaca 用 10%）—— 更谦逊、更校准。Assael 2022

back to Ithaca · Bback to Ithaca · B

返回 Ithaca · B返回 Ithaca · B

15 / 24

12 · Weights12 · 权重

LearningWeights: the knobs that get learned学习被学习的旋钮

Every connection has a weight; learning is the search for good ones. Drag a neuron’s two weights and watch its decision flip.每条连接都有权重；学习就是寻找好权重。拖动神经元的两个权重，看其判断翻转。

back to Ithaca · Eback to Ithaca · E

返回 Ithaca · E返回 Ithaca · E

16 / 24

· Loss & descent· 损失与下降

LearningLoss, and rolling downhill学习损失，与滚下坡

Learning needs a number for how wrong each guess is — the loss — and a way to shrink it. Slide the confidence to feel cross-entropy; then step the weight downhill (gradient descent). Sommerschield 2023学习需要一个数来衡量每次猜测有多错 —— 损失 —— 以及缩小它的办法。拖动把握度感受交叉熵；再让权重走下坡（梯度下降）。Sommerschield 2023

back to Ithaca · Eback to Ithaca · E

返回 Ithaca · E返回 Ithaca · E

17 / 24

13 · Calibration13 · 校准

LearningCalibration: can you trust the 70%?学习校准：你能信那个 70% 吗？

A model is calibrated if its confidence matches reality — that is what lets a historian trust Ithaca’s ranked probabilities.模型若其置信度与现实吻合便是校准良好 —— 正是这让史学家能信赖 Ithaca 的排名概率。

back to Ithaca · Gback to Ithaca · G

返回 Ithaca · G返回 Ithaca · G

18 / 24

14 · Embedding (real streams)14 · 嵌入（真实流）

RepresentationThe embedding: real streams per position表示嵌入：每位置的真实流

The exact moment text becomes numbers. For every position of μουσαις (from #315181), two real streams concatenate — the learned character vector + the positional code. Hover one; toggle the word-stream Ithaca-2022 once added. Assael 2022文本变成数字的确切时刻。对 μουσαις（来自 #315181）的每个位置，两股真实流拼接 —— 学得的字符向量 + 位置编码。悬停一个；可切换 Ithaca-2022 曾加入的词流。Assael 2022

back to Ithaca · Cback to Ithaca · C

返回 Ithaca · C返回 Ithaca · C

19 / 24

· From vector to a dot· 从向量到一个点

RepresentationHow a long vector becomes an [x, y] dot表示一个长向量如何成为 [x, y] 点

A real Ithaca embedding has 384 numbers; to draw it as a 2-D dot, PCA reduces it to two — each a weighted blend of all 384. Watch #315181’s real vector project to its real coordinate, with the Delos signature #64942 beside it. Assael 2022真实 Ithaca 嵌入有 384 个数；为画作二维点，PCA 将其降到两个 —— 各为全部 384 个数的加权混合。看 #315181 的真实向量投影到其真实坐标，提洛署名 #64942 紧挨其旁。Assael 2022

back to Ithaca · Cback to Ithaca · C

返回 Ithaca · C返回 Ithaca · C

20 / 24

· Vectors in space· 空间中的向量

RepresentationA vector has direction & length表示向量有方向与长度

Real Ithaca embeddings of real inscriptions, projected by PCA. Click two to read the real cosine (in 384-d). #315181 clusters with the Delos “… made it” signatures (≈0.97); fragments sit far (≈0.03). Assael 2022真实 Ithaca 对真实铭文的嵌入，经 PCA 投影。点击两条读取真实余弦（在 384 维中）。#315181 与提洛“……制作”署名相聚（≈0.97）；残片远在他处（≈0.03）。Assael 2022

back to Ithaca · Cback to Ithaca · C

返回 Ithaca · C返回 Ithaca · C

21 / 24

· Positional embedding· 位置嵌入

RepresentationTelling position 3 from position 30表示区分第 3 位与第 30 位

Each slot 0…767 gets a smooth sinusoidal code — neighbours look alike (so “nearby” is built in) and it extends past lengths ever seen. Seeded, then trainable. Assael 2022每个槽位 0…767 得到平滑的正弦编码 —— 相邻相似（故“邻近”天生内建），并可超出曾见长度。先初始化、后可训练。Assael 2022

back to Ithaca · Cback to Ithaca · C

返回 Ithaca · C返回 Ithaca · C

22 / 24

· Ithaca vs Aeneas· Ithaca 对 Aeneas

RepresentationWhy Aeneas drops the word-stream表示Aeneas 为何弃用词流

Ithaca used three embedding streams; Aeneas turns the word-stream off (emb_word_disable=True) because the character stream already carries word-level signal. Assael 2025Ithaca 用三股嵌入流；Aeneas 关闭词流（emb_word_disable=True），因字符流已携带词级信号。Assael 2025

back to Aeneas · Dback to Aeneas · D

返回 Aeneas · D返回 Aeneas · D

23 / 24

15 · Non-autoregressive15 · 非自回归

DecodingNon-autoregressive restoration解码非自回归修复

Most models write strictly left-to-right. Ithaca fills the most confident position first — and its beam search "performed substantially better" this way. Assael 2022多数模型严格从左到右。Ithaca 先填最有把握的位置 —— 其集束搜索如此“显著更优”。Assael 2022

back to Ithaca · Aback to Ithaca · A

返回 Ithaca · A返回 Ithaca · A

24 / 24

16 · CER & beam16 · CER 与集束

DecodingScoring a guess: CER & beam search解码为猜测打分：CER 与集束搜索

CER counts the minimum single-character edits to fix a restoration ÷ length. Edit the prediction; watch beams stay alive. Assael 2022CER 统计修正一次修复所需的最少单字符编辑 ÷ 长度。编辑预测；看集束保持存活。Assael 2022

back to Ithaca · Fback to Ithaca · F

返回 Ithaca · F返回 Ithaca · F