[{"content":" In this blog post, we will explore the Grassmann.jl library written in Julia for working with Clifford algebras.\nFirst, we import the library and the GLMakie plotting library.\nusing Grassmann using GLMakie set_theme!(theme_light()) The following code constructs a positive definite 3-dimensional TensorBundle with an additional plane at infinity:\n@basis S\u0026#34;∞+++\u0026#34; (⟨∞111⟩, v, v∞, v₁, v₂, v₃, v∞₁, v∞₂, v∞₃, v₁₂, v₁₃, v₂₃, v∞₁₂, v∞₁₃, v∞₂₃, v₁₂₃, v∞₁₂₃) then plot the following vector field:\n$$\\exp\\left(\\frac{\\frac{\\pi}{2}(v_{12}+v_{\\infty3})}{2}\\right)$$ First we project it into the submanifold $\\langle\\infty++\\_\\rangle$:\nstreamplot( vectorfield( exp((π/2)*(v12+v∞3)/2), V(1,2,3),V(1,2,3) ),-1.5..1.5,-1.5..1.5,-1.5..1.5,gridsize=(12,12)) Then we project it into the submanifold $\\langle\\_+++\\rangle$:\nspeed = Observable{Float64}(0.0) vf = @lift(vectorfield( exp((π/2)*(v12+$speed*v∞3)/2), V(2,3,4),V(2,3,4) )) fig, ax, pl = streamplot(vf,-1.5..1.5,-1.5..1.5,-1.5..1.5,gridsize=(12,12)) framerate = 24 timestamps = range(0, 2, step=1/framerate) record(fig, \u0026#34;streamplot.mp4\u0026#34;, timestamps; framerate = framerate) do t speed[] = Float64(t/10.0) # ax.azimuth[] = 1.7pi + 0.3 * sin(2pi * t / 120) # pl.colorrange = (0, t) end \u0026quot;streamplot.mp4\u0026quot; basis\u0026#34;2\u0026#34; # Euclidean streamplot(vectorfield(v1*exp((π/4)*v12/2)),-1.5..1.5,-1.5..1.5) @basis S\u0026#34;+-\u0026#34; # Hyperbolic streamplot(vectorfield(v1*exp((π/4)*v12/2)),-1.5..1.5,-1.5..1.5) @basis S\u0026#34;∞+++\u0026#34; f(t) = (↓(exp(π*t*((3/7)*v12+v∞3))\u0026gt;\u0026gt;\u0026gt;↑(v1+v2+v3))) lines(V(2,3,4).(points(f))) Base.@kwdef mutable struct Lorenz dt::Float64 = 0.01 σ::Float64 = 10 ρ::Float64 = 28 β::Float64 = 8/3 x::Float64 = 1 y::Float64 = 1 z::Float64 = 1 end function step!(l::Lorenz) dx = l.σ * (l.y - l.x) dy = l.x * (l.ρ - l.z) - l.y dz = l.x * l.y - l.β * l.z l.x += l.dt * dx l.y += l.dt * dy l.z += l.dt * dz Point3f(l.x, l.y, l.z) end attractor = Lorenz() lorenz_points = Observable(Point3f[]) colors = Observable(Int[]) set_theme!(theme_black()) fig, ax, l = lines(lorenz_points, color = colors, colormap = :inferno, transparency = true, axis = (; type = Axis3, protrusions = (0, 0, 0, 0), viewmode = :fit, limits = (-30, 30, -30, 30, 0, 50))) record(fig, \u0026#34;lorenz.mp4\u0026#34;, 1:120) do frame for i in 1:50 push!(lorenz_points[], step!(attractor)) push!(colors[], frame) end ax.azimuth[] = 1.7pi + 0.3 * sin(2pi * frame / 120) notify(lorenz_points) notify(colors) l.colorrange = (0, frame) end \u0026quot;lorenz.mp4\u0026quot; ","permalink":"https://utensil.tngl.sh/blog/posts/ca-in-julia/","summary":"In this blog post, we will explore the Grassmann.jl library written in Julia for working with Clifford algebras.\nFirst, we import the library and the GLMakie plotting library.\nusing Grassmann using GLMakie set_theme!(theme_light()) The following code constructs a positive definite 3-dimensional TensorBundle with an additional plane at infinity:\n@basis S\u0026#34;∞+++\u0026#34; (⟨∞111⟩, v, v∞, v₁, v₂, v₃, v∞₁, v∞₂, v∞₃, v₁₂, v₁₃, v₂₃, v∞₁₂, v∞₁₃, v∞₂₃, v₁₂₃, v∞₁₂₃) then plot the following vector field:","title":"Clifford algebras in Julia"},{"content":"Here we test example Typst from various sources.\nThe following is adapted from the official example.\n📄 View as PDF The following is adapted from Typst Examples Book.\n📄 View as PDF ","permalink":"https://utensil.tngl.sh/blog/posts/typst-test/","summary":"\u003cp\u003eHere we test example Typst from various sources.\u003c/p\u003e","title":"Hello, Typst"},{"content":" The following are taken from official examples for now.\nMarkdown Basics superscript2 / subscript2\nverbatim code\nLine Block\nSpaces and newlines\nare preserved\nHere\u0026rsquo;s some raw inline HTML: html\nterm a long long long long long definition $$ \\def\\RR{{\\mathbb R}}$$ inline math: $E = mc^{2} \\in \\RR$\ndisplay math:\n$$E = mc^{2} \\in \\RR$$ Right Left Default Center 12 12 12 12 123 123 123 123 1 1 1 1 1 + 1 run.py\n1 + 1 library(tidyverse) library(palmerpenguins) penguins |\u0026gt; mutate( bill_ratio = bill_depth_mm / bill_length_mm, bill_area = bill_depth_mm * bill_length_mm ) Line 3 Take penguins, and then, Lines 4-7 add new columns for the bill ratio and bill area. (TODO: Highlight conflicted, code annotation JS disabled)\nTip 1\nNote that there are five types of callouts, including: note, tip, warning, caution, and important.\n(TODO: Callouts are not working for hugo-md, Bootstrap disabled too)\nTo print, press Shift-Ctrl-P. To open an existing new project, press Shift-Command-O (mac), Shift-Control-O (windows), Shift-Ctrl-L (linux).\n(TODO: shortcuts are not working for hugo-md, disabled)\nFigures \u0026#10; \u0026#10; Figure 1: Famous Elephants\nScholarly Writing Citations See (Baez, 2002).\n(TODO: citation linking and hover requires link-citations: true and extra hacking, see \u0026ldquo;PORTING NOTE\u0026rdquo; in layouts/partials/quarto.html)\nFootnotes Here is a footnote reference,1 and another note with multiple blocks.2\nThis paragraph won\u0026rsquo;t be part of the note, because it isn\u0026rsquo;t indented.\nHere is an inline note.3\nCross-References For a demonstration of a line plot on a polar axis, see Figure 2.\nFor tips on callouts, see Tip 1. (TODO: Not working for hugo-md)\nLiterate programming Python Code import numpy as np import matplotlib.pyplot as plt r = np.arange(0, 2, 0.01) theta = 2 * np.pi * r plt.plot([1,23,2,4]) plt.show() fig, ax = plt.subplots(subplot_kw={\u0026#39;projection\u0026#39;: \u0026#39;polar\u0026#39;}) ax.plot(theta, r) ax.set_rticks([0.5, 1, 1.5, 2]) ax.grid(True) plt.show() Figure 2: A line plot on a polar axis\nThe following example is based on 3DCGA: Intersections:\nCode from kingdon import Algebra alg = Algebra(4, 1) locals().update(alg.blades) ni = e4 + e5 no = 0.5 * (e5 - e4) nino = ni ^ no up = lambda x: no + x + 0.5 * x * x * ni p = up(0) # point S = (p - 0.5 * ni).dual() # main dual sphere around point (interactive) S2 = (up(-1.4*e1) - 0.125 * ni).dual() # left dual sphere C = (up(1.4*e1) - 0.125 * ni).dual() \u0026amp; e3.dual() # right circle L = up(0.9*e2) ^ up(0.9*e2 - 1*e1) ^ ni # top line P = (1*e2 - 0.9*ni).dual() # bottom dual plane P2 = (1*e1 + 1.7*ni).dual() # right dual plane C1 = S \u0026amp; P C2 = S \u0026amp; L C3 = S \u0026amp; S2 C4 = S \u0026amp; C C5 = C \u0026amp; P2 lp = up(nino.lc(P2 \u0026amp; (L ^ no))) print(\u0026#34;p=\u0026#34;, p) print(\u0026#34;lp=\u0026#34;, lp) print(\u0026#34;C1=\u0026#34;, C1) print(\u0026#34;C2=\u0026#34;, C2) print(\u0026#34;C3=\u0026#34;, C3) print(\u0026#34;C4=\u0026#34;, C4) print(\u0026#34;C5=\u0026#34;, C5) print(\u0026#34;L=\u0026#34;, L) print(\u0026#34;P=\u0026#34;, P) print(\u0026#34;S=\u0026#34;, S) print(\u0026#34;S2=\u0026#34;, S2) p= -0.5 𝐞₄ + 0.5 𝐞₅ lp= 0.9 𝐞₂ + -0.095 𝐞₄ + 0.905 𝐞₅ C1= -0.9 𝐞₁₂₃ + -1.0 𝐞₁₃₅ C2= 0.9 𝐞₁₂ + 1.0 𝐞₁₅ + -5.55e-17 𝐞₂₅ C3= 1.35 𝐞₁₂₃ + -1.4 𝐞₂₃₅ C4= -1.35 𝐞₁₂ + 1.4 𝐞₂₅ C5= 1.7 𝐞₁₂ + -1.02 𝐞₂₄ + -2.02 𝐞₂₅ L= 0.9 𝐞₁₂₄ + 0.9 𝐞₁₂₅ + -1.0 𝐞₁₄₅ + 5.55e-17 𝐞₂₄₅ P= -0.9 𝐞₁₂₃₄ + -0.9 𝐞₁₂₃₅ + 1 𝐞₁₃₄₅ S= -1.0 𝐞₁₂₃₅ S2= 1.35 𝐞₁₂₃₄ + 0.355 𝐞₁₂₃₅ + 1.4 𝐞₂₃₄₅ Code alg.graph( 0x00FF0000, p, \u0026#34;s1\u0026#34;, # point 0xFF00FF,lp,\u0026#34;l\u0026amp;p\u0026#34;, # line intersect plane 0x0000FF,C1,\u0026#34;s\u0026amp;p\u0026#34;, # sphere meet plane 0x888800,C2,\u0026#34;s\u0026amp;l\u0026#34;, # sphere meet line 0x0088FF,C3,\u0026#34;s\u0026amp;s\u0026#34;, # sphere meet sphere 0x008800,C4,\u0026#34;s\u0026amp;c\u0026#34;, # sphere meet circle 0x880000,C5,\u0026#34;c\u0026amp;p\u0026#34;, # circle meet sphere 0,L,0,C, # line and circle 0xE0008800, P, P2, # plane 0xE0FFFFFF, S, \u0026#34;s1\u0026#34;, S2, # spheres conformal=1, grid=1, gl=1, ) The following example is based on Introduction to using GAlgebra:\nCode import sympy from galgebra.ga import Ga from galgebra.printer import latex from IPython.display import Math # tell sympy to use our printing by default sympy.init_printing(latex_printer=latex, use_latex=\u0026#39;mathjax\u0026#39;) xyz = (x, y, z) = sympy.symbols(\u0026#39;x y z\u0026#39;, real=True) o3d = Ga(\u0026#39;e_x e_y e_z\u0026#39;, g=[1, 1, 1], coords=xyz) grad = o3d.grad a = o3d.mv(\u0026#39;a\u0026#39;,\u0026#39;vector\u0026#39;) b = o3d.mv(\u0026#39;b\u0026#39;,\u0026#39;vector\u0026#39;) Math(fr\u0026#39;\u0026#39;\u0026#39; \\begin{{align}} a \u0026amp;= {latex(a)} \\\\ b \u0026amp;= {latex(b)} \\\\ a+b \u0026amp;= {latex(a+b)} \\\\ a-b \u0026amp;= {latex(a-b)} \\\\ ab \u0026amp;= {latex(a*b)} \\\\ a\\cdot b \u0026amp;= {latex(a|b)} \\\\ a \\rfloor b \u0026amp;= {latex(a\u0026lt;b)} \\\\ a \\lfloor b \u0026amp;= {latex(a\u0026gt;b)} \\\\ a\\wedge b \u0026amp;= {latex(a^b)} \\end{{align}} \u0026#39;\u0026#39;\u0026#39;) $\\displaystyle \\begin{align} a \u0026= a^{x} \\boldsymbol{e}_{x} + a^{y} \\boldsymbol{e}_{y} + a^{z} \\boldsymbol{e}_{z} \\\\ b \u0026= b^{x} \\boldsymbol{e}_{x} + b^{y} \\boldsymbol{e}_{y} + b^{z} \\boldsymbol{e}_{z} \\\\ a+b \u0026= \\left ( a^{x} + b^{x}\\right ) \\boldsymbol{e}_{x} + \\left ( a^{y} + b^{y}\\right ) \\boldsymbol{e}_{y} + \\left ( a^{z} + b^{z}\\right ) \\boldsymbol{e}_{z} \\\\ a-b \u0026= \\left ( a^{x} - b^{x}\\right ) \\boldsymbol{e}_{x} + \\left ( a^{y} - b^{y}\\right ) \\boldsymbol{e}_{y} + \\left ( a^{z} - b^{z}\\right ) \\boldsymbol{e}_{z} \\\\ ab \u0026= \\left ( a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z}\\right ) + \\left ( a^{x} b^{y} - a^{y} b^{x}\\right ) \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{y} + \\left ( a^{x} b^{z} - a^{z} b^{x}\\right ) \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{z} + \\left ( a^{y} b^{z} - a^{z} b^{y}\\right ) \\boldsymbol{e}_{y}\\wedge \\boldsymbol{e}_{z} \\\\ a\\cdot b \u0026= a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z} \\\\ a \\rfloor b \u0026= a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z} \\\\ a \\lfloor b \u0026= a^{x} b^{x} + a^{y} b^{y} + a^{z} b^{z} \\\\ a\\wedge b \u0026= \\left ( a^{x} b^{y} - a^{y} b^{x}\\right ) \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{y} + \\left ( a^{x} b^{z} - a^{z} b^{x}\\right ) \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{z} + \\left ( a^{y} b^{z} - a^{z} b^{y}\\right ) \\boldsymbol{e}_{y}\\wedge \\boldsymbol{e}_{z} \\end{align}$\nCode B = o3d.mv(\u0026#39;B\u0026#39;,\u0026#39;bivector\u0026#39;) Math(fr\u0026#39;\u0026#39;\u0026#39; \\begin{{align}} B \u0026amp;= {latex(B)} \\\\ BB \u0026amp;= {latex(B*B)} \\\\ a+B \u0026amp;= {latex(a+B)} \\\\ a-B \u0026amp;= {latex(a-B)} \\\\ aB \u0026amp;= {latex(a*B)} \\\\ a\\cdot B \u0026amp;= {latex(a|B)} \\\\ a \\rfloor B \u0026amp;= {latex(a\u0026lt;B)} \\\\ a \\lfloor B \u0026amp;= {latex(a\u0026gt;B)} \\\\ a\\wedge B \u0026amp;= {latex(a^B)} \\\\ \\end{{align}} \u0026#39;\u0026#39;\u0026#39;) $\\displaystyle \\begin{align} B \u0026= B^{xy} \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{y} + B^{xz} \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{z} + B^{yz} \\boldsymbol{e}_{y}\\wedge \\boldsymbol{e}_{z} \\\\ BB \u0026= - {\\left ( B^{xy} \\right )}^{2} - {\\left ( B^{xz} \\right )}^{2} - {\\left ( B^{yz} \\right )}^{2} \\\\ a+B \u0026= a^{x} \\boldsymbol{e}_{x} + a^{y} \\boldsymbol{e}_{y} + a^{z} \\boldsymbol{e}_{z} + B^{xy} \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{y} + B^{xz} \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{z} + B^{yz} \\boldsymbol{e}_{y}\\wedge \\boldsymbol{e}_{z} \\\\ a-B \u0026= a^{x} \\boldsymbol{e}_{x} + a^{y} \\boldsymbol{e}_{y} + a^{z} \\boldsymbol{e}_{z} - B^{xy} \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{y} - B^{xz} \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{z} - B^{yz} \\boldsymbol{e}_{y}\\wedge \\boldsymbol{e}_{z} \\\\ aB \u0026= \\left ( - B^{xy} a^{y} - B^{xz} a^{z}\\right ) \\boldsymbol{e}_{x} + \\left ( B^{xy} a^{x} - B^{yz} a^{z}\\right ) \\boldsymbol{e}_{y} + \\left ( B^{xz} a^{x} + B^{yz} a^{y}\\right ) \\boldsymbol{e}_{z} + \\left ( B^{xy} a^{z} - B^{xz} a^{y} + B^{yz} a^{x}\\right ) \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{y}\\wedge \\boldsymbol{e}_{z} \\\\ a\\cdot B \u0026= \\left ( - B^{xy} a^{y} - B^{xz} a^{z}\\right ) \\boldsymbol{e}_{x} + \\left ( B^{xy} a^{x} - B^{yz} a^{z}\\right ) \\boldsymbol{e}_{y} + \\left ( B^{xz} a^{x} + B^{yz} a^{y}\\right ) \\boldsymbol{e}_{z} \\\\ a \\rfloor B \u0026= \\left ( - B^{xy} a^{y} - B^{xz} a^{z}\\right ) \\boldsymbol{e}_{x} + \\left ( B^{xy} a^{x} - B^{yz} a^{z}\\right ) \\boldsymbol{e}_{y} + \\left ( B^{xz} a^{x} + B^{yz} a^{y}\\right ) \\boldsymbol{e}_{z} \\\\ a \\lfloor B \u0026= 0 \\\\ a\\wedge B \u0026= \\left ( B^{xy} a^{z} - B^{xz} a^{y} + B^{yz} a^{x}\\right ) \\boldsymbol{e}_{x}\\wedge \\boldsymbol{e}_{y}\\wedge \\boldsymbol{e}_{z} \\\\ \\end{align}$\nObservable JS The following example is taken from Steven De Keninck\u0026rsquo;s Animated Orbits .\nCode Algebra = require(\u0026#39;ganja.js@1.0.151\u0026#39;) Code Algebra(3,0,1,()=\u0026gt;{ var t=1, E=Math.E, sin=Math.sin, cos=Math.cos, PI=Math.PI; var f=(x,y)=\u0026gt; E**(x*PI*1e12) *E**(.5e01+sin(t)*sin(x*PI*6)*.25e01) *E**((x*2+y)*PI*1e13) *E**(cos(t*1.31)*0.2e01+sin(t*1.31)*0.2e03); f.dx=400; f.dy=5; var camera=0e1+1, start=performance.now(); return (this.graph(()=\u0026gt;{ t=(performance.now()-start)/2234; camera.set((1+1e03)* (Math.cos(t/4) + Math.sin(t/4)*1e13 ) ); f.va=undefined; // force re-evaluate of geometry .. return [0xff0088,f]; },{camera,gl:1,animate:1,width:\u0026#34;100%\u0026#34;,height:\u0026#34;600px\u0026#34;})); }) The following example is taken from Steven De Keninck\u0026rsquo;s Klein Bottle:\nCode GAmphetamineModule = import(\u0026#39;https://enki.ws/GAM/src/GAmphetamine.js\u0026#39;) Code GAmphetamine = GAmphetamineModule.default Code kb = GAmphetamine(4,0,1,()=\u0026gt;{ const {E, sin, cos, PI} = Math; const [n1,n2] = [16, 128]; const [r1,r2] = [0.15, 0.6]; var circle = r1=\u0026gt;[...Array(n1+1).keys()] .map(x=\u0026gt;E**(x/n1 * PI * 1e12) * E**(r1*1e01) \u0026gt;\u0026gt;\u0026gt; !1e0) .map((x,i,a)=\u0026gt;[x,a[i+1]||a[0]]); var klein = [...Array(n2+1).keys()] .map(x=\u0026gt;E**(x/n2 * PI * 1e13) * E**(r2*1e01) * E**(x/n2 * PI * 1.5e14) \u0026gt;\u0026gt;\u0026gt; circle(r1+0.1*sin(8*PI*x/n2))); var edges = klein.map((t,ti)=\u0026gt;t.map((p,pi)=\u0026gt;[p[0],(klein[ti+1]||klein[0])[pi][0]])).slice(0,-1).flat() klein = E**(-0.78e23) * E**((.5e14 + .0e24)) \u0026gt;\u0026gt;\u0026gt; [...klein.flat(), ...edges]; var SVG=this.graph([0x009977, ...klein], {arrowSize:0, h:0, p:-0.25}); return SVG; }) Code svg`${kb}` References Baez, J. (2002). The octonions. Bulletin of the American Mathematical Society, 39(2), 145\u0026ndash;205. Retrieved from https://arxiv.org/pdf/math/0105155.pdf\nFigueroa-O\u0026rsquo;Farrill, J. (2010). Spin geometry. Lecture Notes. University of Edinburgh. Retrieved from http://mat.uab.cat/~rubio/csa2017/SpinNotes.pdf\nReynoso, A. (2023). Probing clifford algebras through spin groups: A standard model perspective. arXiv Preprint arXiv:2312.10071. Retrieved from https://arxiv.org/pdf/2312.10071\nHere is the footnote.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nHere\u0026rsquo;s one with multiple blocks.\nSubsequent paragraphs are indented to show that they belong to the previous footnote.\n{ some.code } The whole paragraph can be indented, or just the first line. In this way, multi-paragraph footnotes work like multi-paragraph list items.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nInlines notes are easier to write, since you don\u0026rsquo;t have to pick an identifier and move down to type the note.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://utensil.tngl.sh/blog/posts/quarto-test/","summary":"\u003cscript type=\"module\" src=\"index_files/libs/quarto-ojs/quarto-ojs-runtime.js\"\u003e\u003c/script\u003e\n\u003clink  href=\"index_files/libs/quarto-ojs/quarto-ojs.css\" rel=\"stylesheet\" /\u003e\n\u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js\" integrity=\"sha512-c3Nl8+7g4LMSTdrm621y7kf9v3SDPnhxLNhcjFJbKECVnmZHTdo+IRO05sNLTH/D3vA6u1X32ehoLC7WFVdheg==\" crossorigin=\"anonymous\"\u003e\u003c/script\u003e\n\u003cscript src=\"https://cdnjs.cloudflare.com/ajax/libs/jquery/3.5.1/jquery.min.js\" integrity=\"sha512-bLT0Qm9VnAYZDflyKcBaQ2gg0hSYNQrJ8RilYldYQ1FxQYoCLtUjuuRuZo+fjqhx/qtq/1itJ0C2ejDxltZVFg==\" crossorigin=\"anonymous\" data-relocate-top=\"true\"\u003e\u003c/script\u003e\n\u003cscript type=\"application/javascript\"\u003edefine('jquery', [],function() {return window.jQuery;})\u003c/script\u003e\n\u003cscript src=\"https://unpkg.com/@jupyter-widgets/html-manager@*/dist/embed-amd.js\" crossorigin=\"anonymous\"\u003e\u003c/script\u003e\n\u003cp\u003eThe following are taken from \u003ca href=\"https://quarto.org/docs/\"\u003eofficial examples\u003c/a\u003e for now.\u003c/p\u003e\n\u003c!-- ```{=html}\n\u003ca href=\"./index.pdf\" target=\"_blank\" style=\"float: right; font-size: 12px; text-decoration: none;\"\u003e📄 View as PDF\u003c/a\u003e\n``` --\u003e","title":"Hello, Quarto"},{"content":"I have just borrowed Clifford Algebras and Zeons: Geometry to Combinatorics and Beyond ( Citation: Staples,\u0026#32;2020 Staples,\u0026#32; G.\u0026#32; (2020). \u0026#32; Clifford algebras and zeons: Geometry to combinatorics and beyond. \u0026#32; World Scientific. ) from the library. This post should briefly walk through the contents of the book, highlight some key concepts, and provide further readings for each chapter of the book. For the complete and updated research work by George Stacey Staples, see his home page.\nThis note, however, is not intended to be a comprehensive review of the book. Instead, it is a personal note emphasizing the foundational part of the book, by viewing Zeon algebra as an extension and an application of Clifford algebra, to gain insights into the combinatorial properties hidden in Clifford algebra.\nPart I: The Essentials Algebra Linear algebra Gram-Schmidt orthogonalization (1) First, set $\\mathbf{u}_1:=\\mathbf{v}_1$. (2) For each $\\ell=2, \\ldots, k$, set $$\\mathbf{u}_{\\ell}:=\\mathbf{v}_{\\ell}-\\sum_{j=1}^{\\ell-1} \\frac{\\left\\langle\\mathbf{v}_{\\ell}, \\mathbf{u}_j\\right\\rangle}{\\left\\|\\mathbf{u}_j\\right\\|^2} \\mathbf{u}_j$$. (3) The collection $S^{\\prime}=\\left\\{\\mathbf{u}_1, \\ldots, \\mathbf{u}_k\\right\\}$ now satisfies $\\operatorname{span}(S)=\\operatorname{span}\\left(S^{\\prime}\\right)$. If unit vectors are required, normalize the collection $S^{\\prime}$ by dividing each element by its norm. Semigroup representation the theorem for determine the number of irreducible representations of a semigroup ( Citation: Staples,\u0026#32;2020 Staples,\u0026#32; G.\u0026#32; (2020). \u0026#32; Clifford algebras and zeons: Geometry to combinatorics and beyond. \u0026#32; World Scientific. ;\u0026#32; Citation: Serre\u0026#32;\u0026amp;\u0026#32;,\u0026#32;1977 Serre,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;\u0026#32; (1977). \u0026#32; Linear representations of finite groups. \u0026#32; Springer. ;\u0026#32; Citation: Clifford\u0026#32;\u0026amp;\u0026#32;Preston,\u0026#32;1961 Clifford,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Preston,\u0026#32; G. \u0026#32; (1961). \u0026#32;The algebraic theory of semigroups, vol. 1. Mathematical surveys,\u0026#32;7. ;\u0026#32; Citation: Rhodes\u0026#32;\u0026amp;\u0026#32;Zalcstein,\u0026#32;1991 Rhodes,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Zalcstein,\u0026#32; Y. \u0026#32; (1991). \u0026#32;Elementary representation and character theory of finite semigroups and its application. Monoids and semigroups with applications (Berkeley, CA, 1989).\u0026#32;334–367. ) Let $G_1, \\ldots, G_m$ be a choice of exactly one maximal subgroup from each regular $\\mathfrak{J}$-class of $S$. Then, letting $k_i$ denote the number of conjugacy classes of $G_i$, the number of irreducible representations of $S$ is $\\sum\\limits_{i=1}^m k_i$ Combinatorics Combinatorics associated with polynomials ( Citation: Cohen,\u0026#32;1978 Cohen,\u0026#32; D.\u0026#32; (1978). \u0026#32; Basic techniques of combinatorial theory. ;\u0026#32; Citation: Tucker,\u0026#32;1994 Tucker,\u0026#32; A.\u0026#32; (1994). \u0026#32; Applied combinatorics. \u0026#32; John Wiley \u0026amp; Sons, Inc.. ) Graph theory ( Citation: West\u0026#32;\u0026amp;\u0026#32;,\u0026#32;2001 West,\u0026#32; D.\u0026#32;\u0026amp;\u0026#32;\u0026#32; (2001). \u0026#32; Introduction to graph theory. \u0026#32; Prentice hall Upper Saddle River. ;\u0026#32; Citation: Wilson,\u0026#32;1979 Wilson,\u0026#32; R.\u0026#32; (1979). \u0026#32; Introduction to graph theory. \u0026#32; Pearson Education India. ) A $k$-walk $\\left\\{v_0, \\ldots, v_k\\right\\}$ in a graph $G$ is a sequence of vertices in $G$ with initial vertex $v_0$ and terminal vertex $v_k$ such that there exists an edge $\\left(v_j, v_{j+1}\\right) \\in E$ for each $0 \\leq j \\leq k-1$. Powers of the adjacency matrix of a graph provide a simple and convenient tool for counting walks ni finite graphs: Let $G$ be a graph on $n$ vertices with associated adjacency matrix $A$. Then for any positive integer $k$, the $(i, j)$ th entry of $A^k$ (i.e. the $k$-th power of $A$) is the number of $k$-walks $i \\rightarrow j$. In particular, the entries along the main diagonal of $A^k$ are the numbers of closed $k$-walks in $G$. hypercubes The $n$-dimensional cube, or hypercube $\\mathcal{Q}_n$, is the graph whose vertices are in one-to-one correspondence with the $n$-tuples of zeros and ones and whose edges are the pairs of $n$-tuples that differ in exactly one position. The structure of the hypercube allows one to construct a random walk on the hypercube by \u0026ldquo;flipping\u0026rdquo; a randomly selected digit from 0 to 1 or vice versa. Random walks on Clifford algebras have also been studied as random walks on directed hypercubes ( Citation: Schott\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2008 Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; S. \u0026#32; (2008). \u0026#32;Random walks on clifford algebras as directed hypercubes. Markov Processes and Related Fields,\u0026#32;14.\u0026#32;515–542. ) . By considering certain generalizations of hypercubes, combinatorial properties can be obtained for tackling a variety of problems in graph theory and combinatorics ( Citation: Schott\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2011 Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2011). \u0026#32;Nilpotent adjacency matrices and random graphs. Ars Combinatoria(98).\u0026#32;225–239. ;\u0026#32; Citation: Staples,\u0026#32;2005 Staples,\u0026#32; G. \u0026#32; (2005). \u0026#32;Clifford-algebraic random walks on the hypercube. Advances in Applied Clifford Algebras,\u0026#32;15.\u0026#32;213–232. ;\u0026#32; Citation: Staples,\u0026#32;2008 Staples,\u0026#32; G. \u0026#32; (2008). \u0026#32;A new adjacency matrix for finite graphs. Advances in Applied Clifford Algebras,\u0026#32;18(3-4).\u0026#32;979–991. ) . Part II: Geometric Algebra See also ( Citation: Ablamowicz,\u0026#32;Sobczyk \u0026amp; al.,\u0026#32;2004 Ablamowicz,\u0026#32; R.,\u0026#32; Sobczyk,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;\u0026#32; (2004). \u0026#32; Lectures on clifford (geometric) algebras and applications. \u0026#32; Springer. ;\u0026#32; Citation: Bayliss,\u0026#32;2019 Bayliss,\u0026#32; W.\u0026#32; (2019). \u0026#32; Geometric algebra workbook. \u0026#32;Retrieved from\u0026#32; http://web4.uwindsor.ca/users/b/baylis/main.nsf ;\u0026#32; Citation: Lounesto,\u0026#32;2001 Lounesto,\u0026#32; P.\u0026#32; (2001). \u0026#32; Clifford algebras and spinors. \u0026#32; Cambridge university press. ;\u0026#32; Citation: Porteous,\u0026#32;1995 Porteous,\u0026#32; I.\u0026#32; (1995). \u0026#32; Clifford algebras and the classical groups. \u0026#32; Cambridge University Press. ) .\nGeometry of the Complex Plane Quaternions $\\mathfrak{Q}$: a non-Albelian group of unit quaternions $\\mathbb{H} := \\mathbb{R} \\mathfrak{Q} = \\{ x_0 + x_1 \\mathbf{i} + x_2 \\mathbf{j} + x_3 \\mathbf{k} : x_0, x_1, x_2, x_3 \\in \\mathbb{R} \\}$, i.e. the group algerba of $\\mathfrak{Q}$ over $\\mathbb{R}$ Let $\\mathbf{u}$ be a unit column vector in $\\mathbb{R}^n$. The operator $\\ket{\\mathbf{u}}\\bra{\\mathbf{u}} = \\mathbf{u} \\mathbf{u}^{\\dagger}$ is a rank-one orthogonal projection onto the subspace spanned by $\\mathbf{u}$. Euclidean Clifford Algebras Gram-Schmidt orthogonalization Revisited Suppose $S=\\left\\{\\mathbf{v}_1, \\ldots, \\mathbf{v}_k\\right\\}$ is a linearly independent collection in $\\mathbb{R}^n$. Using properties of the geometric product, Gram-Schmidt orthogonalization is accomplished in $\\mathcal{C} \\ell_n$ as follows. (1) First, set $\\mathbf{u}_1:=\\mathbf{v}_1$. (2) For each $\\ell=2, \\ldots, k$, set $ \\mathbf{u}_{\\ell}:=\\mathbf{v}_{\\ell}-\\sum_{j=1}^{\\ell-1}\\left(\\frac{\\mathbf{v}_{\\ell} \\mathbf{u}_j+\\mathbf{u}_j \\mathbf{v}_{\\ell}}{2 \\mathbf{u}_j{ }^2}\\right) \\mathbf{u}_j $ (3) The collection $S^{\\prime}=\\left\\{\\mathbf{u}_1, \\ldots, \\mathbf{u}_k\\right\\}$ now satisfies $\\operatorname{span}(S)=\\operatorname{span}\\left(S^{\\prime}\\right)$. If unit vectors are required, normalize the collection $S^{\\prime}$ by dividing each element by its norm. Let $\\sharp \\mathfrak{u}$ denote the maximum grade among nonzero terms in the canonical basis blade expansion of $\\mathfrak{u}$. The grade of $\\mathfrak{u}$ is $\\sharp \\mathfrak{u}+1$. Decompositions in the Clifford Lipschitz Group Let $\\mathcal{C} \\ell_n{ }^*$ denote the multiplicative group of invertible Clifford elements. In particular, $$ \\mathcal{C} \\ell_n{ }^*=\\left\\{u \\in \\mathcal{C} \\ell_n: u \\tilde{u} \\in \\mathbb{R}^*\\right\\} . $$ The inverse of $u \\in \\mathcal{C} \\ell_n$ is then seen to be $u^{-1}=\\frac{\\tilde{u}}{u \\tilde{u}}$. The Clifford Lipschitz group, $\\Gamma_n$, is the subgroup of $\\mathcal{C} \\ell_n{ }^*$ whose elements $u \\in \\Gamma_n$ satisfy $u \\in \\mathcal{C} \\ell_n^{+} \\cup \\mathcal{C} \\ell_n^{-}$ for all $\\mathbf{x} \\in V, u \\mathbf{x} \\bar{u} \\in V$. Two important subgroups of the Clifford Lipschitz group are the pin and spin groups. The pin group $\\operatorname{Pin}(n)=\\left\\{u \\in \\mathcal{C} \\ell_n^{+} \\cup \\mathcal{C} \\ell_n^{-}: u \\tilde{u}= \\pm 1\\right\\}$ is a double covering of $O(n)$. The spin group $\\operatorname{Spin}(n)=\\left\\{u \\in \\mathcal{C} \\ell_n^{+} \\cup \\mathcal{C} \\ell_n^{-}: u \\tilde{u}=1\\right\\}$ is a double covering of $S O(n)$. The conformal orthogonal group $\\mathrm{CO}(n)$ is defined as the direct product of dilations and orthogonal transformations on $\\mathbb{R}^n$. An element $\\mathfrak{u} \\in \\mathcal{C} \\ell_n$ is said to be decomposable if $\\mathfrak{u} = \\mathbf{v}_1 \\cdots \\mathbf{v}_k$ for some linearly independent collection of vectors $\\left\\{\\mathbf{v}_1, \\ldots, \\mathbf{v}_k\\right\\}$ in $\\mathcal{C} \\ell_n$. Equivalently, $\\mathfrak{u}$ is decomposable if and only if it satisfies the following conditions: (1) $\\mathfrak{u} \\in \\mathcal{C} \\ell_n^{+} \\cup \\mathcal{C} \\ell_n^{-}$ (2) For all $\\mathrm{x} \\in V, \\mathfrak{u x} \\overline{\\mathfrak{u}} \\in V$. In fact, the decomposable elements of $\\mathcal{C} \\ell_n$ are precisely the elements of the Clifford Lipschitz group, $\\Gamma_n$. Further, one quickly sees that decomposable elements $\\mathfrak{u} \\in \\mathcal{C} \\ell_n^{+} \\cup \\mathcal{C} \\ell_n^{-}$satisfying $\\mathfrak{u} \\tilde{\\mathfrak{u}}=\\alpha \\neq 0$ provide a double covering of the conformal orthogonal group $\\mathrm{CO}(n)$. vector decomposition in definite signatures ( Citation: Aragón-González,\u0026#32;Aragón \u0026amp; al.,\u0026#32;2009 Aragón-González,\u0026#32; G.,\u0026#32; Aragón,\u0026#32; J.,\u0026#32; Rodriguez-Andrade,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32;Verde-Star,\u0026#32; L. \u0026#32; (2009). \u0026#32;Reflections, rotations, and pythagorean numbers. Advances in Applied Clifford Algebras,\u0026#32;19.\u0026#32;1–14. ) Clifford Algebras of Arbitrary Signature For positive integer $n$, let $V$ be an $n$-dimensional vector space over $\\mathbb{R}$ with orthonormal basis $\\beta=\\left\\{\\mathbf{e}_i\\right\\}_{1 \\leq i \\leq n}$. Suppose $p$ and $q$ are nonnegative integers such that $n=p+q$. Let $\\mathcal{B}$ be the multiplicative group generated by $\\{ \\pm 1\\} \\cup \\beta$ with multiplication defined by the following: $$\\mathbf{e}_i \\mathbf{e}_j= \\begin{cases}-\\mathbf{e}_j \\mathbf{e}_i \u0026 \\text { if } i \\neq j \\cr 1 \u0026 \\text { if } 1 \\leq i=j \\leq p \\cr -1 \u0026 \\text { if } p+1 \\leq i=j \\leq p+q\\end{cases}$$ Given $I \\subseteq\\{1,2, \\ldots, n\\}$, define the multi-index notation $\\mathbf{e}_I$ to denote the canonically ordered product $$ \\mathbf{e}_I=\\prod_{i \\in I} \\mathbf{e}_i . $$ The set $\\beta$ with the operation of multiplication as defined above generates a nonabelian group $\\mathcal{B}$ of order $2^{n+1}$. Elements of $\\mathcal{B}$ will be referred to as blades. Considering binary representations of subsets of the $n$-set, it becomes apparent that the Cayley graph of $\\mathcal{B} /\\{ \\pm 1\\}$ is isomorphic to the $n$-dimensional hypercube $\\mathcal{Q}_n$. Given positive integer $n$ and nonnegative integers $p, q$ such that $n=p+q$, the Clifford algebra of signature $(p, q)$ is the group algebra of $\\mathcal{B}$ over $\\mathbb{R}$, where $\\mathcal{B}$ is defined as above. $\\mathcal{B}$ can be written as a disjoint union of the form $\\mathcal{B}=\\bigcup_{k=0}^n \\mathcal{B}_k$, where $\\mathcal{B}_k=\\left\\{ \\pm \\mathbf{e}_I:|I|=k\\right\\}$. In light of this, $\\mathcal{C} \\ell_{p, q}$ is a graded algebra; i.e., it has the canonical decomposition $$ \\mathcal{C} \\ell_{p, q}=\\mathbb{R} \\oplus \\mathbb{R} \\mathcal{B}_1 \\oplus \\cdots \\oplus \\mathbb{R} \\mathcal{B}_{p+q} $$ Given $u \\in \\mathcal{C} \\ell_{p, q}$, define the grade-k part of $u$ by $$ \\langle u\\rangle_k:=\\sum_{\\{I:|I|=k\\}} u_I \\mathbf{e}_I $$ The even subalgebra of $\\mathcal{C} \\ell_{p, q}$ is defined by $$ \\mathcal{C} \\ell_{p, q}^{+}:=\\bigoplus_{k=0}^{\\lfloor n / 2\\rfloor} \\mathbb{R} \\mathcal{B}_{2 k}=\\mathbb{R} \\mathcal{B}^{+} $$ where $\\mathcal{B}^{+}:=\\left\\{ \\pm \\mathbf{e}_I:|I| \\text{is even} \\right\\}$ The geometric product of arbitrary blades $\\mathbf{e}_I, \\mathbf{e}_J \\in \\mathcal{C} \\ell_{p, q}$ is given by $$ \\mathbf{e}_I \\mathbf{e}_J=(-1)^{\\phi(I, J)+|I \\cap J \\cap Q|} \\mathbf{e}_{I \\triangle J} $$ where $I \\triangle J=(I \\cup J) \\backslash(I \\cap J), Q=\\{p+1, \\ldots, p+q\\}$, and $\\phi(I, J): 2^{[n]} \\times 2^{[n]}$ : $\\mathbb{Z}_{\\geq 0}$ is defined by $$ \\phi(I, J)=\\sum_{j \\in J}|\\{i \\in I: i\u003ej\\}|$$ Decompositions in $CO_Q(V)$ Beginning with a finite-dimensional vector space $V$ equipped with a nondegenerate quadratic form $Q$, we consider the decompositions of particular elements of the Clifford Lipschitz group $\\Gamma$ in the Clifford algebra $\\mathcal{C} \\ell_Q(V)$. These elements represent the conformal orthogonal group $\\mathrm{CO}_Q(V)$, defined as the direct product of the orthogonal group $O_Q(V)$ with dilations. The collection of all $Q$-orthogonal transformations on $V$ forms a group called the orthogonal group of $Q$, denoted $\\mathrm{O}_Q(V)$. Specifically, $T \\in \\mathrm{O}_Q(V)$ if and only if for every $\\mathbf{x} \\in V, Q(T(\\mathbf{x}))=Q(\\mathbf{x})$. The conformal orthogonal group, denoted $\\mathrm{CO}_Q(V)$, is the direct product of the orthogonal group with the group of dilations. More specifically, $\\tau \\in \\mathrm{CO}_Q(V)$ if and only if for every $\\mathbf{x} \\in V$, there exists a scalar $\\lambda$ such that $Q(\\tau(\\mathbf{x}))=\\lambda^2 Q(\\mathbf{x})$. Given a decomposable k-element $\\mathfrak{u}=\\mathbf{w}_1 \\cdots \\mathbf{w}_k \\in$ $\\mathcal{C} \\ell_Q(V)$, let $n=\\operatorname{dim} V$ and define $\\varphi_{\\mathfrak{u}} \\in \\mathrm{O}_Q(V)$ by $$ \\varphi_{\\mathfrak{u}}(\\mathbf{v})=\\mathfrak{u} \\mathbf{v} \\widehat{\\mathfrak{u}^{-1}} .$$ Then $\\varphi_{\\mathfrak{u}}$ has an eigenspace $\\mathcal{E}$ of dimension $n-k$ with corresponding eigenvalue 1 . Let $\\mathbf{x} \\in V$ be arbitrary. Then $\\mathbf{x}-\\varphi_{\\mathfrak{u}}(\\mathbf{x}) \\in V_{\\mathfrak{u}}$. In other words, the operator $\\pi_u:=\\mathbb{I}-\\varphi_{\\mathfrak{u}}$ is a projection into the subspace determined by $\\mathfrak{u}$. This observation alows one to define a $\\mathfrak{u}$-subspace projection by $\\pi_{\\mathfrak{u}}(\\mathbf{x}):=\\left(\\mathbf{x}-\\mathfrak{\\mathfrak{u} \\mathbf{x}} \\widehat{\\mathfrak{u}^{-1}}\\right)$ From Geometry to Combinatorics Hypercubes play an important role in Clifford algebras and their \u0026ldquo;combinatorially interesting\u0026rdquo; subalgebras. By constructing subalgebras with different multiplicative properties, different types of combinatorial computations can be performed： The Clifford algebra $\\mathcal{C} \\ell_{p, q, r}$. Generated by anticommutative generators $\\left\\{\\mathbf{e}_{\\{i\\}}: 1 \\leq i \\leq n=p+q+r\\right\\}$ and unit scalar 1 satisfying $$ \\mathbf{e}_{\\{i\\}}{ }^2= \\begin{cases}1 \u0026 1 \\leq i \\leq p, \\cr -1 \u0026 p+1 \\leq i \\leq p+q, \\cr 0 \u0026 p+q+1 \\leq i \\leq p+q+r .\\end{cases} $$ The \u0026ldquo;sym-Clifford\u0026rdquo; algebra $\\mathcal{C} \\ell_{p, q, r}{ }^{\\text {sym}}$. Generated by pairwise commutative $\\left\\{\\varsigma_{\\{i\\}}: 1 \\leq i \\leq p+q+r\\right\\}$ along with unit scalar 1 satisfying the squaring rules above. The $n$-particle zeon algebra $\\mathcal{C} \\ell_n{ }^{\\text {nil }}$. Generated by pairwise commutative $\\left\\{\\zeta_{\\{i\\}}: 1 \\leq i \\leq n\\right\\}$ along with unit scalar 1 subject to $\\zeta_{\\{i\\}}{ }^2=0$ for $i=1, \\ldots, n$. The \u0026ldquo;idem-Clifford\u0026rdquo; algebra $\\mathcal{C} \\ell_n{ }^{\\text {idem}}$. This algebra is generated by pairwise-commutative idempotent generators $\\left\\{\\varepsilon_{\\{i\\}}: 1 \\leq i \\leq n\\right\\}$ along with unit scalar 1 . In particular, $\\varepsilon_{\\{i\\}}{ }^2=\\varepsilon_{\\{i\\}}$ for $i=1, \\ldots, n$. Each of the algebras above can be viewed as the quotient of a group algebra or a semigroup algebra. In this chapter, we will focus on the particular groups underlying the Clifford algebras $\\mathcal{C} \\ell_{p, q}$ and the semigroups underlying $\\mathcal{C} \\ell_n{ }^{\\text {nil }}$. The Cayley graphs of these groups and semigroups are generalizations of hypercubes, and irreducible representations of the algebras can then be characterized by considering irreducible representations of the (semi)groups. A representation of a given group, $G$, is a homomorphism $\\rho: G \\rightarrow \\mathrm{GL}_n(\\mathbb{C})$ The degree of this representation is $n$, and the representation space is the space $\\mathbb{C}^n$ on which the elements of $\\mathrm{GL}_n(\\mathbb{C})$ act. Given a representation $\\rho(g)$ and a subspace $W$ of $\\mathbb{C}^n$, we say $W$ is $G$-invariant if $\\rho(g) W \\subseteq W$ for every $g \\in G$. If the only invariant spaces are ${0}$ and $V$, the representation is said to be irreducible. The character of a representation, $\\chi: G \\rightarrow \\mathbb{C}$, is defined by $\\chi(g)=\\operatorname{tr}(\\rho(\\mathrm{g}))$. A representation $\\rho$ with character $\\chi$ is irreducible if and only if $\\chi$ satisfies $$ (\\chi \\mid \\chi)=\\frac{1}{|G|} \\sum_{g \\in G} \\chi(g) \\overline{\\chi(g)}=1 . $$ ( Citation: Serre\u0026#32;\u0026amp;\u0026#32;,\u0026#32;1977 Serre,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;\u0026#32; (1977). \u0026#32; Linear representations of finite groups. \u0026#32; Springer. ) Two representations $\\rho$ and $r$ of a group $G$ are said to be isomorphic if there exists an invertible mapping $f: \\mathbb{C}^n \\rightarrow \\mathbb{C}^n$ such that $$ f \\circ \\rho=r \\circ f . $$ For the the classification of representations of irreducible complex representations of Clifford subalgebras, see ( Citation: Cassiday\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2015 Cassiday,\u0026#32; C.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2015). \u0026#32;On representations of semigroups having hypercube-like cayley graphs. ) . Let $B=\\left\\{\\mathbf{e}_1, \\ldots, \\mathbf{e}_n\\right\\}$, and let $p$ and $q$ be nonnegative integers such that $p+q=n$. Let $\\mathcal{B}_{p, q}$ be the multiplicative group generated by $B$ along with the elements $\\left\\{\\mathbf{e}_{\\varnothing}, \\mathbf{e}_\\alpha\\right\\}$, subject to the following generating relations: for all $x \\in B \\cup\\left\\{\\mathbf{e}_{\\varnothing}, \\mathbf{e}_\\alpha\\right\\}$, $$ \\begin{gathered} \\mathbf{e}_{\\varnothing} x=x \\mathbf{e}_{\\varnothing}=x \\cr \\mathbf{e}_\\alpha x=x \\mathbf{e}_\\alpha \\cr \\mathbf{e}_{\\varnothing}{ }^2=\\mathbf{e}_\\alpha{ }^2=\\mathbf{e}_{\\varnothing} \\end{gathered} $$ and $$ \\mathbf{e}_i \\mathbf{e}_j= \\begin{cases}\\mathbf{e}_\\alpha \\mathbf{e}_j \\mathbf{e}_i \u0026 \\text { if } i \\neq j \\cr \\mathbf{e}_{\\varnothing} \u0026 \\text { if } i=j \\leq p \\cr \\mathbf{e}_\\alpha \u0026 \\text { if } p+1 \\leq i=j \\leq n .\\end{cases} $$ The group $\\mathcal{B}_{p, q}$ is referred to herein as the blade group of signature $(p, q)$. Let $\\mathfrak{G}_n$ denote the null blade semigroup defined as the semigroup generated by the collection $G=\\left\\{\\gamma_i: 1 \\leq i \\leq n\\right\\}$ along with $\\left\\{\\gamma_{\\varnothing}, \\gamma_\\alpha, 0_\\gamma\\right\\}$ satisfying the following generating relations: for all $x \\in G \\cup$ $\\left\\{\\gamma_{\\varnothing}, \\gamma_\\alpha, 0_\\gamma\\right\\}$ $$ \\begin{gathered} \\gamma_{\\varnothing} x=x \\gamma_{\\varnothing}=x, \\cr \\gamma_\\alpha x=x \\gamma_\\alpha, \\cr 0_\\gamma x=x 0_\\gamma=0_\\gamma, \\cr \\gamma_{\\varnothing}{ }^2=\\gamma_\\alpha{ }^2=\\gamma_{\\varnothing}, \\end{gathered} $$ and $$ \\gamma_i \\gamma_j= \\begin{cases}0_\\gamma \u0026 \\text { if and only if } i=j \\cr \\gamma_\\alpha \\gamma_j \\gamma_i \u0026 i \\neq j .\\end{cases} $$ Let $\\mathfrak{Z}_n$ denote the zeon semigroup defined as the semigroup generated by the collection $C=\\left\\{\\zeta_i: 1 \\leq i \\leq n\\right\\}$ along with $\\left\\{\\zeta_{\\varnothing}, 0_\\zeta\\right\\}$ satisfying the following generating relations: for all $x \\in C \\cup\\left\\{\\zeta_{\\varnothing}, 0_\\zeta\\right\\}$, $$ \\begin{gathered} \\zeta_{\\varnothing} x=x \\zeta_{\\varnothing}=x, \\cr 0_\\zeta x=x 0_\\zeta=0_\\zeta, \\cr \\zeta_{\\varnothing}{ }^2=0_\\zeta, \\end{gathered} $$ and $$ \\zeta_i \\zeta_j= \\begin{cases}0_\\zeta \u0026 \\text { if and only if } i=j, \\cr \\zeta_j \\zeta_i \u0026 i \\neq j .\\end{cases} $$ The zeon semigroup is of particular interest, as its associated semigroup algebra is canonically isomorphic to the zeon algebra. The Clifford algebra $\\mathcal{C} \\ell_{p, q}(p+q\u003e1)$ is canonically isomorphic to the blade group quotient algebra $\\mathbb{R} \\mathcal{B}_{p, q} /\\left\\langle\\mathbf{e}_\\alpha+\\mathbf{e}_{\\varnothing}\\right\\rangle$. Considering the degree-1 representations, $\\rho_J\\left(\\mathbf{e}_{\\varnothing}\\right)=\\rho_J\\left(\\mathbf{e}_\\alpha\\right)=1$ for all $J \\in 2^{[p+q]}$. It then becomes clear that passing to the quotient has no effect on the number of irreducible representations. On the other hand, the higher-dimensional irreducible representations satisfy $\\tilde{\\tau}\\left(\\mathbf{e}_{\\varnothing}+\\mathbf{e}_\\alpha\\right)=$ 0 a priori, so that representations of the group algebra are precisely the representations of the quotient algebra. The Grassmann exterior algebra, $\\wedge \\mathbb{R}^n$, is canonically isomorphic to the null blade semigroup algebra $\\mathcal{B} \\ell_{\\wedge n}=\\mathbb{R} \\mathfrak{G}_n /\\left\\langle 0_\\gamma, \\gamma_\\alpha+\\gamma_{\\varnothing}\\right\\rangle$. This algebra is isomorphic to the algebra of fermion creation (or annihilation) operators. The $n$-particle zeon algebra $\\mathcal{C} \\ell_n{ }^{\\text {nil}}$ is canonically isomorphic to the Abelian null blade semigroup algebra $\\mathbb{R} \\mathfrak{Z}_n /\\left\\langle 0_\\zeta\\right\\rangle$. This algebra is isomorphic to an algebra of commuting lowering or raising (annihilation or creation) operators. Groups and Semigroups $\\mathcal{B}_{p, q}$ Generator Commutation: $\\mathbf{e}_i \\mathbf{e}_j=\\mathbf{e}_\\alpha \\mathbf{e}_j \\mathbf{e}_i$ Generator Squares: $\\{\\underbrace{\\mathbf{e}_{\\varnothing}, \\ldots, \\mathbf{e}_{\\varnothing}}_p, \\underbrace{\\mathbf{e}_\\alpha, \\ldots, \\mathbf{e}_\\alpha}_q\\}$ $\\mathfrak{G}_n$ Generator Commutation: $\\gamma_i \\gamma_j=\\gamma_\\alpha \\gamma_j \\gamma_i$ Generator Squares: $\\gamma_i{ }^2=0_\\gamma, i=1, \\ldots, n$ $\\mathfrak{Z}_n$ Generator Commutation: Abelian Generator Squares: $\\zeta_i{ }^2=0_\\zeta, i=1, \\ldots, n$ For the rest of the details worked out in this chapter, see the book itself, no other references can be found except the earlier book ( Citation: Schott\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2012 Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G.\u0026#32; (2012). \u0026#32; Operator calculus on graphs: Theory and applications in computer science. \u0026#32; World Scientific. ) . PART III: Algebraic Combinatorics \u0026amp; Zeons Algebraic and Combinatorial Properties of Zeons The $n$-particle zeon algebra, denoted $\\mathcal{C} \\ell_n{ }^{\\text {nil }}$, is defined as the real abelian algebra generated by the collection $\\left\\{\\zeta_i\\right\\}(1 \\leq i \\leq n)$ along with the scalar $1=\\zeta_\\varnothing$ subject to the following multiplication rules: $$ \\begin{gathered} \\zeta_i \\zeta_j=\\zeta_j \\zeta_i \\text { for } i \\neq j, \\text { and } \\cr \\zeta_i{ }^2=0 \\text { for } 1 \\leq i \\leq n . \\end{gathered} $$ In recent years, combinatorial properties and applications of zeons have been studied in numerous works, many of which are summarized in the monograph ( Citation: Schott\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2012 Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G.\u0026#32; (2012). \u0026#32; Operator calculus on graphs: Theory and applications in computer science. \u0026#32; World Scientific. ) . See also Chapter 3 Zeon algebras in the guide to the author\u0026rsquo;s CLIFFMATH package for Mathematica, available as a Mathematica Notebook. Zeon Polynomials See Section \bZeon polynomials of Chapter 3 Zeon algebras in the guide Norms and Inequalities in Zeon Algebras See Section \bZeon Norms of Chapter 3 Zeon algebras in the guide Zeon Matrices See Section \bZeon Matrices of Chapter 3 Zeon algebras in the guide Zeon Functions and Factorizations See Section \bZeon Function and Elementary factorizations of Chapter 3 Zeon algebras in the guide Zeon Differential Calculus See Section \bDerivitives and antiderivatives of Zeon functions of Chapter 3 Zeon algebras in the guide See also the dedicated paper ( Citation: Staples,\u0026#32;2019 Staples,\u0026#32; G. \u0026#32; (2019). \u0026#32;Differential calculus of zeon functions. Advances in Applied Clifford Algebras,\u0026#32;29(2).\u0026#32;25. ) Graph Enumeration Problems Examples of graph enumeration problems include counting structures like paths, trails, cycles, circuits, spanning trees, matchings, cliques, and independent sets in a given graph. The null-square property of zeon generators makes them especially convenient for symbolic computations associated with enumeration problems on finite graphs. Given a graph $G=(V, E)$ on $n=|V|$ vertices, let $\\mathcal{C} \\ell_V{ }^{\\text {nil }}$ be the zeon algebra of dimension $2^n$ whose generators are in one-to-one correspondence with the vertices of $G$. The nilpotent adjacency matrix associated with $G$ is defined by $$ \\Phi_{i j}=\\left\\{\\begin{array}{l} \\zeta_{v_j}, \\text { if }\\left\\{v_i, v_j\\right\\} \\in E(G) \\cr 0, \\text { otherwise. } \\end{array}\\right. $$ Most of contents in this chapter can be found in ( Citation: Staples,\u0026#32;2008 Staples,\u0026#32; G. \u0026#32; (2008). \u0026#32;A new adjacency matrix for finite graphs. Advances in Applied Clifford Algebras,\u0026#32;18(3-4).\u0026#32;979–991. ;\u0026#32; Citation: Schott\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2011 Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2011). \u0026#32;Nilpotent adjacency matrices and random graphs. Ars Combinatoria(98).\u0026#32;225–239. ) except 15.2 Matchings, Cliques, and Independent Sets (which can be found in ( Citation: Ewing\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2022 Ewing,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2022). \u0026#32;Zeon and idem-clifford formulations of hypergraph problems. Advances in Applied Clifford Algebras,\u0026#32;32(5).\u0026#32;61.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2201.05895.pdf ) ) and 15.3 Minimal Path Algorithms (which can be found in ( Citation: Schott\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2017 Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2017). \u0026#32;Generalized zeon algebras: Theory and application to multi-constrained path problems. Advances in Applied Clifford Algebras,\u0026#32;27.\u0026#32;45–57. ) ). Graph Colorings and Chromatic Structures The extension of nilpotent matrix methods to graph colorings allows one to count heterochromatic and monochromatic self-avoiding walks in colored graphs. Further, the zeon-algebraic formalism allows one to quickly verify whether a given graph coloring is proper, and it provides a convenient framework for implementing greedy coloring algorithms. See the dedicated paper ( Citation: Staples\u0026#32;\u0026amp;\u0026#32;Stellhorn,\u0026#32;2017 Staples,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;Stellhorn,\u0026#32; T. \u0026#32; (2017). \u0026#32;Zeons, orthozeons, and graph colorings. Advances in Applied Clifford Algebras,\u0026#32;27.\u0026#32;1825–1845. ) Boolean Satisfiability The Boolean satisfiability problem, or SAT, is the problem of determining whether the variables of a given Boolean formula can be consistently replaced by true or false in such a way that the formula evaluates to be true. See the dedicated paper ( Citation: Davis\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2019 Davis,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2019). \u0026#32;Zeon and idem-clifford formulations of boolean satisfiability. Advances in Applied Clifford Algebras,\u0026#32;29(4).\u0026#32;60. ) PART IV: Induced Operators Induced Operators and Kravchuk Polynomials The term induced operator will generally refer to an operator on a Clifford algebra $\\mathcal{C} \\ell_Q(V)$ obtained from an operator on the underlying vector space $V$ spanned by the algebra\u0026rsquo;s generators. The term reduced operator will generally refer to an operator on the paravector space $V_*:=\\mathbb{R} \\oplus V$ obtained from an operator on the Clifford algebra. A deduced operator will be an operator on $V$ obtained by restricting an operator on the full algebra, provided the operator on $V$ also induces the operator on the full algebra. Kravchuk polynomials appear as traces of conjugation operators in Clifford algebras and in Clifford Berezin integrals of Clifford polynomials. The three-term recurrence relation for the Kravchuk polynomials of order $n$ is as follows. Define $K_0(x ; n):=1$ and $K_1(x ; n):=x$. For $\\ell \\geq 2$, the $\\ell$ th Kravchuk polynomial is given by $$ K_{\\ell}(x ; n):=x K_{\\ell-1}(x ; n)+(\\ell-1)(n-\\ell+2) K_{\\ell-2}(x ; n) . $$ An explicit formula for the $n$th Kravchuk polynomial is given by$$ K_{\\ell}(x, n):=\\sum_{i=0}^n(-1)^i\\left(\\begin{array}{c} x \\cr i \\end{array}\\right)\\left(\\begin{array}{c} n-x \\cr \\ell-i \\end{array}\\right) $$ The $n$th Kravchuk matrix, $\\mathfrak{K}_n$, is the $(n+1) \\times(n+1)$ matrix defined via the Kravchuk polynomial generating function according to $$ (1+x)^{n-\\jmath}(1-x)^j=\\sum_{i=0}^n x^i\\left\\langle e_i\\left|\\mathfrak{K}_n\\right| e_j\\right\\rangle, $$ with $\\left\\langle e_i\\left|\\mathfrak{K}_n\\right| e_j\\right\\rangle=K_i(j ; n)$, the $i$ th Kravchuk polynomial evaluated at $j$. Kravchuk Classification of Clifford Algebras Fixing canonical unit vector basis $\\left\\{\\mathbf{e}_j: 0 \\leq j \\leq n\\right\\}$ for $\\mathbb{R}^{n+1}$, define the vector $\\mathfrak{h}_n \\in \\mathbb{R}^{n+1}$ by $$ \\mathfrak{h}_n=\\sum_{j=0}^n(-1)^{j(j-1) / 2} \\mathbf{e}_j . $$ Clifford algebras $\\mathcal{C} \\ell_{p, q}$ and $\\mathcal{C} \\ell_{p^{\\prime}, q^{\\prime}}$ are isomorphic if and only if $\\left\\langle\\mathfrak{h}_n\\left|\\mathfrak{K}_n\\right| \\mathbf{e}_q-\\mathbf{e}_{q^{\\prime}}\\right\\rangle=0$, where $n=p+q=p^{\\prime}+q^{\\prime}$. See the dedicated paper ( Citation: Staples,\u0026#32;2015 Staples,\u0026#32; G. \u0026#32; (2015). \u0026#32;Kravchuk polynomials and induced/reduced operators on clifford algebras. Complex Analysis and Operator Theory,\u0026#32;9.\u0026#32;445–478. ) Graph-Induced Operators Operators are induced on fermion and zeon algebras by the action of adjacency matrices and combinatorial Laplacians on the vector spaces spanned by the graph\u0026rsquo;s vertices. Properties of the algebras automatically give information about the graph\u0026rsquo;s spanning trees and vertex coverings by cycles \u0026amp; matchings. Combining the properties of operators induced on fermions and zeons gives a fermion-zeon convolution that recovers the number of Hamiltonian cycles in an arbitrary graph. See the dedicated paper ( Citation: Staples,\u0026#32;2017 Staples,\u0026#32; G. \u0026#32; (2017). \u0026#32;Hamiltonian cycle enumeration via fermion-zeon convolution. International Journal of Theoretical Physics,\u0026#32;56(12).\u0026#32;3923–3934. ) Solutions and Hints to Selected Exercises References Ablamowicz,\u0026#32; Sobczyk\u0026#32;\u0026amp;\u0026#32; (2004) Ablamowicz,\u0026#32; R.,\u0026#32; Sobczyk,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;\u0026#32; (2004). \u0026#32; Lectures on clifford (geometric) algebras and applications. \u0026#32; Springer. Aragón-González,\u0026#32; Aragón,\u0026#32; Rodriguez-Andrade\u0026#32;\u0026amp;\u0026#32;Verde-Star (2009) Aragón-González,\u0026#32; G.,\u0026#32; Aragón,\u0026#32; J.,\u0026#32; Rodriguez-Andrade,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32;Verde-Star,\u0026#32; L. \u0026#32; (2009). \u0026#32;Reflections, rotations, and pythagorean numbers. Advances in Applied Clifford Algebras,\u0026#32;19.\u0026#32;1–14. Bayliss (2019) Bayliss,\u0026#32; W.\u0026#32; (2019). \u0026#32; Geometric algebra workbook. \u0026#32;Retrieved from\u0026#32; http://web4.uwindsor.ca/users/b/baylis/main.nsf Cassiday\u0026#32;\u0026amp;\u0026#32;Staples (2015) Cassiday,\u0026#32; C.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2015). \u0026#32;On representations of semigroups having hypercube-like cayley graphs. Clifford\u0026#32;\u0026amp;\u0026#32;Preston (1961) Clifford,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Preston,\u0026#32; G. \u0026#32; (1961). \u0026#32;The algebraic theory of semigroups, vol. 1. Mathematical surveys,\u0026#32;7. Cohen (1978) Cohen,\u0026#32; D.\u0026#32; (1978). \u0026#32; Basic techniques of combinatorial theory. Davis\u0026#32;\u0026amp;\u0026#32;Staples (2019) Davis,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2019). \u0026#32;Zeon and idem-clifford formulations of boolean satisfiability. Advances in Applied Clifford Algebras,\u0026#32;29(4).\u0026#32;60. Ewing\u0026#32;\u0026amp;\u0026#32;Staples (2022) Ewing,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2022). \u0026#32;Zeon and idem-clifford formulations of hypergraph problems. Advances in Applied Clifford Algebras,\u0026#32;32(5).\u0026#32;61.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2201.05895.pdf Lounesto (2001) Lounesto,\u0026#32; P.\u0026#32; (2001). \u0026#32; Clifford algebras and spinors. \u0026#32; Cambridge university press. Porteous (1995) Porteous,\u0026#32; I.\u0026#32; (1995). \u0026#32; Clifford algebras and the classical groups. \u0026#32; Cambridge University Press. Rhodes\u0026#32;\u0026amp;\u0026#32;Zalcstein (1991) Rhodes,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Zalcstein,\u0026#32; Y. \u0026#32; (1991). \u0026#32;Elementary representation and character theory of finite semigroups and its application. Monoids and semigroups with applications (Berkeley, CA, 1989).\u0026#32;334–367. Schott\u0026#32;\u0026amp;\u0026#32;Staples (2008) Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; S. \u0026#32; (2008). \u0026#32;Random walks on clifford algebras as directed hypercubes. Markov Processes and Related Fields,\u0026#32;14.\u0026#32;515–542. Schott\u0026#32;\u0026amp;\u0026#32;Staples (2011) Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2011). \u0026#32;Nilpotent adjacency matrices and random graphs. Ars Combinatoria(98).\u0026#32;225–239. Schott\u0026#32;\u0026amp;\u0026#32;Staples (2012) Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G.\u0026#32; (2012). \u0026#32; Operator calculus on graphs: Theory and applications in computer science. \u0026#32; World Scientific. Schott\u0026#32;\u0026amp;\u0026#32;Staples (2017) Schott,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2017). \u0026#32;Generalized zeon algebras: Theory and application to multi-constrained path problems. Advances in Applied Clifford Algebras,\u0026#32;27.\u0026#32;45–57. Serre\u0026#32;\u0026amp;\u0026#32; (1977) Serre,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;\u0026#32; (1977). \u0026#32; Linear representations of finite groups. \u0026#32; Springer. Staples (2005) Staples,\u0026#32; G. \u0026#32; (2005). \u0026#32;Clifford-algebraic random walks on the hypercube. Advances in Applied Clifford Algebras,\u0026#32;15.\u0026#32;213–232. Staples (2008) Staples,\u0026#32; G. \u0026#32; (2008). \u0026#32;A new adjacency matrix for finite graphs. Advances in Applied Clifford Algebras,\u0026#32;18(3-4).\u0026#32;979–991. Staples (2015) Staples,\u0026#32; G. \u0026#32; (2015). \u0026#32;Kravchuk polynomials and induced/reduced operators on clifford algebras. Complex Analysis and Operator Theory,\u0026#32;9.\u0026#32;445–478. Staples (2017) Staples,\u0026#32; G. \u0026#32; (2017). \u0026#32;Hamiltonian cycle enumeration via fermion-zeon convolution. International Journal of Theoretical Physics,\u0026#32;56(12).\u0026#32;3923–3934. Staples\u0026#32;\u0026amp;\u0026#32;Stellhorn (2017) Staples,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;Stellhorn,\u0026#32; T. \u0026#32; (2017). \u0026#32;Zeons, orthozeons, and graph colorings. Advances in Applied Clifford Algebras,\u0026#32;27.\u0026#32;1825–1845. Staples (2019) Staples,\u0026#32; G. \u0026#32; (2019). \u0026#32;Differential calculus of zeon functions. Advances in Applied Clifford Algebras,\u0026#32;29(2).\u0026#32;25. Staples (2020) Staples,\u0026#32; G.\u0026#32; (2020). \u0026#32; Clifford algebras and zeons: Geometry to combinatorics and beyond. \u0026#32; World Scientific. Tucker (1994) Tucker,\u0026#32; A.\u0026#32; (1994). \u0026#32; Applied combinatorics. \u0026#32; John Wiley \u0026amp; Sons, Inc.. West\u0026#32;\u0026amp;\u0026#32; (2001) West,\u0026#32; D.\u0026#32;\u0026amp;\u0026#32;\u0026#32; (2001). \u0026#32; Introduction to graph theory. \u0026#32; Prentice hall Upper Saddle River. Wilson (1979) Wilson,\u0026#32; R.\u0026#32; (1979). \u0026#32; Introduction to graph theory. \u0026#32; Pearson Education India. ","permalink":"https://utensil.tngl.sh/blog/posts/zeon-algebra/","summary":"I have just borrowed Clifford Algebras and Zeons: Geometry to Combinatorics and Beyond ( Citation: Staples,\u0026#32;2020 Staples,\u0026#32; G.\u0026#32; (2020). \u0026#32; Clifford algebras and zeons: Geometry to combinatorics and beyond. \u0026#32; World Scientific. ) from the library. This post should briefly walk through the contents of the book, highlight some key concepts, and provide further readings for each chapter of the book. For the complete and updated research work by George Stacey Staples, see his home page.","title":"Notes on Zeon Algebra"},{"content":"This post studies group algebras with GAP, focusing on a few interested groups. See My math interests in 2024: Group Algebra for context.\nIt\u0026rsquo;s helpful to read GAP Manual and SO questions before using GAP.\nInstallation I\u0026rsquo;m using Mac, so I ran the following commands to install and start GAP:\nbrew install wget autoconf gmp readline wget https://github.com/gap-system/gap/releases/download/v4.13.0/gap-4.13.0.tar.gz tar xzvf gap-4.13.0.tar.gz cd gap-4.13.0 ./autogen.sh ./configure make -j4 V=1 all make check make install which gap gap -l . Then I can see something like:\n┌───────┐ GAP 4.13.0 of 2024-03-15 │ GAP │ https://www.gap-system.org └───────┘ Architecture: aarch64-apple-darwin23-default64-kv9 Configuration: gmp 6.3.0, GASMAN, readline Loading the library and packages ... Packages: AClib 1.3.2, Alnuth 3.2.1, AtlasRep 2.1.8, AutPGrp 1.11, CRISP 1.4.6, Cryst 4.1.27, CrystCat 1.1.10, CTblLib 1.3.9, FactInt 1.6.3, FGA 1.5.0, GAPDoc 1.6.7, IRREDSOL 1.4.4, LAGUNA 3.9.6, Polenta 1.3.10, Polycyclic 2.16, PrimGrp 3.4.4, RadiRoot 2.9, ResClasses 4.7.3, SmallGrp 1.5.3, Sophus 1.27, SpinSym 1.5.2, StandardFF 1.0, TomLib 1.2.11, TransGrp 3.6.5, utils 0.85 Try \u0026#39;??help\u0026#39; for help. See also \u0026#39;?copyright\u0026#39;, \u0026#39;?cite\u0026#39; and \u0026#39;?authors\u0026#39; gap\u0026gt; which is the GAP console.\nGreat, let\u0026rsquo;s run some GAP code! Note that the code can be copy-pasted into a GAP console, and gap\u0026gt; will be omitted by GAP, which is very convenient.\nCyclic groups $C_n$, the cyclic group of order $n$, has the following presentation:\n$$ C_n=\\left\\langle a \\mid a^n=e\\right\\rangle $$ Cyclic group can be constructed and examined in GAP by the corresponding functions:\ngap\u0026gt; C2 := CyclicGroup(2); \u0026lt;pc group of size 2 with 1 generator\u0026gt; gap\u0026gt; StructureDescription(C2); \u0026#34;C2\u0026#34; gap\u0026gt; IsCyclic(C2); true gap\u0026gt; MinimalGeneratingSet(C2); [ f1 ] gap\u0026gt; C2_ := AllGroups(2)[1]; \u0026lt;pc group of size 2 with 1 generator\u0026gt; gap\u0026gt; IsIsomorphicGroup(C2, C2_); gap\u0026gt; C2_ := SimplifiedFpGroup(Image(IsomorphismFpGroup(C2))); \u0026lt;fp group of size 2 on the generators [ F1 ]\u0026gt; gap\u0026gt; RelatorsOfFpGroup(C2_); [ F1^2 ] Let\u0026rsquo;s use the sonata package to check if the two groups are isomorphic:\ngap\u0026gt; LoadPackage(\u0026#34;sonata\u0026#34;); ___________________________________________________________________________ / ___ || / \\ /\\ Version 2.9.6 || || || |\\ | / \\ /\\ Erhard Aichinger \\___ || || |\\\\ | /____\\_____________/__\\ Franz Binder \\ || || | \\\\ | / \\ || / \\ Juergen Ecker || \\___/ | \\\\ | / \\ || / \\ Peter Mayr || | \\\\| / \\ || Christof Noebauer \\___/ | \\| || System Of Nearrings And Their Applications Info: https://gap-packages.github.io/sonata/ true gap\u0026gt; C2_ := AllGroups(2)[1]; \u0026lt;pc group of size 2 with 1 generator\u0026gt; gap\u0026gt; IsIsomorphicGroup(C2, C2_); true This section is inspired by SO question 1, .\nThe quaternion group The quaternion group $Q_8$ is a non-abelian group of order 8. It has the following presentation:\n$$ \\mathrm{Q}_8=\\left\\langle\\bar{e}, i, j, k \\mid \\bar{e}^2=e, i^2=j^2=k^2=i j k=\\bar{e}\\right\\rangle $$ Another (less intuitive) presentation of $Q_8$ is:\n$$ \\mathrm{Q}_8=\\left\\langle a, b \\mid a^4=e, a^2=b^2, b a=a^{-1} b\\right\\rangle $$ First, construct the group $Q_8$ by its presentation in GAP:\ngap\u0026gt; Q8_pre := FreeGroup( \u0026#34;n\u0026#34;, \u0026#34;i\u0026#34;, \u0026#34;j\u0026#34;, \u0026#34;k\u0026#34; ); \u0026lt;free group on the generators [ n, i, j, k ]\u0026gt; gap\u0026gt; AssignGeneratorVariables(Q8_pre); #I Assigned the global variables [ n, i, j, k ] gap\u0026gt; Q8_rel := ParseRelators(Q8_pre, \u0026#34;i^2 = j^2 = k^2 = i*j*k = n\u0026#34;); [ i^2*n^-1, j^2*n^-1, k^2*n^-1, i*j*k*n^-1 ] gap\u0026gt; Q8 := Q8_pre / Q8_rel; \u0026lt;fp group on the generators [ n, i, j, k ]\u0026gt; Let\u0026rsquo;s try to see if GAP recognizes $Q_8$:\ngap\u0026gt; StructureDescription(Q8); \u0026#34;Q8\u0026#34; Cool, now let\u0026rsquo;s learn more about $Q_8$:\ngap\u0026gt; Order(Q8); 8 gap\u0026gt; StructureDescription(Center(Q8)); \u0026#34;C2\u0026#34; gap\u0026gt; StructureDescription(DerivedSubgroup(Q8)); \u0026#34;C2\u0026#34; gap\u0026gt; hs := List(ConjugacyClassesSubgroups(Q8),Representative); [ Group([ ]), Group([ n ]), Group([ n, j ]), Group([ n, j*i^-1 ]), Group([ n, i ]), Q8 ] gap\u0026gt; g := Q8; Q8 gap\u0026gt; StructureDescription(g/Intersection(DerivedSubgroup(g),Center(g))); \u0026#34;C2 x C2\u0026#34; What\u0026rsquo;s the simplified presentation of $Q_8$?\ngap\u0026gt; MinimalGeneratingSet(Q8); [ j*k, j ] gap\u0026gt; Q8_ := SimplifiedFpGroup(Image(IsomorphismFpGroup(Q8))); \u0026lt;fp group of size 8 on the generators [ i, j ]\u0026gt; gap\u0026gt; RelatorsOfFpGroup(Q8_); [ j*i*j^-1*i, j^2*i^-2 ] $Q_8$ can also be constructed by the QuaternionGroup function, we can check the isomorphism:\ngap\u0026gt; GeneratorsOfGroup(Q8); [ n, i, j, k ] gap\u0026gt; RelatorsOfFpGroup(Image(IsomorphismFpGroup(Q8))); [ i^2*n^-1, j^2*n^-1, k^2*n^-1, i*j*k*n^-1 ] gap\u0026gt; Q8_ := QuaternionGroup(8); \u0026lt;pc group of size 8 with 3 generators\u0026gt; gap\u0026gt; GeneratorsOfGroup(Q8_); [ x, y, y2 ] gap\u0026gt; RelatorsOfFpGroup(Image(IsomorphismFpGroup(Q8_))); [ F1^2*F3^-1, F2^-1*F1^-1*F2*F1*F3^-1, F3^-1*F1^-1*F3*F1, F2^2*F3^-1, F3^-1*F2^-1*F3*F2, F3^2 ] gap\u0026gt; IsIsomorphicGroup(Q8, Q8_); true The better way (after GAP 4.5) is\ngap\u0026gt; Q8_ := QuaternionGroup(IsFpGroup, 8); \u0026lt;fp group of size 8 on the generators [ r, s ]\u0026gt; gap\u0026gt; GeneratorsOfGroup(Q8_); [ r, s ] gap\u0026gt; RelatorsOfFpGroup(Image(IsomorphismFpGroup(Q8_))); [ r^2*s^-2, s^4, r^-1*s*r*s ] gap\u0026gt; IsIsomorphicGroup(Q8, Q8_); true We can also obtain $Q_8$ using The Small Groups Library by:\ngap\u0026gt; Q8__ := SmallGroup(8,4); \u0026lt;pc group of size 8 with 3 generators\u0026gt; gap\u0026gt; IsIsomorphicGroup(Q8, Q8__); true This section is inspired by SO answers 1, 2, 3.\nThe group algebra of $Q_8$ We need to construct the group algebra of $Q_8$ over the reals, i.e. $\\mathbb{R}Q_8$, but GAP doesn\u0026rsquo;t support reals, so we use rationals instead, i.e. we construct $\\mathbb{Q}Q_8$:\ngap\u0026gt; QQ8 := GroupRing(Rationals,Q8_); \u0026lt;algebra-with-one over Rationals, with 2 generators\u0026gt; gap\u0026gt; IsGroupAlgebra(QQ8); true gap\u0026gt; RadicalOfAlgebra(QQ8); \u0026lt;algebra of dimension 0 over GF(5)\u0026gt; gap\u0026gt; WedderburnDecomposition(QQ8); [ Rationals, Rationals, Rationals, Rationals, \u0026lt;crossed product with center Rationals over GaussianRationals of a group of size 2\u0026gt; ] gap\u0026gt; WedderburnDecompositionInfo(QQ8); [ [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals, 4, [ 2, 3, 2 ] ] ] gap\u0026gt; WedderburnDecompositionWithDivAlgParts(QQ8); [ [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals ], [ 1, rec( Center := Rationals, DivAlg := true, LocalIndices := [ [ 2, 2 ], [ infinity, 2 ] ], SchurIndex := 2 ) ] ] gap\u0026gt; WedderburnDecompositionAsSCAlgebras(QQ8); [ Rationals, Rationals, Rationals, Rationals, \u0026lt;algebra of dimension 4 over Rationals\u0026gt; ] gap\u0026gt; WedderburnDecompositionByCharacterDescent(Rationals, Q8); [ [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals ], [ 1, Rationals, 4, [ 2, 3, 2 ] ] ] $\\mathbb{R} Q_8$ has a canonical (Wedderburn) decomposition as\n$$ \\mathbb{R} Q_8 \\cong 4 \\mathbb{R}+\\mathbb{H} $$ But we can\u0026rsquo;t identify the 5th part of the decomposition as $\\mathbb{H}$, i.e. the quaternion algebra (over rationals instead of reals), so we need to inspect the cross product.\nWe should have the quaternion algebra by both\ngap\u0026gt; HQ := WedderburnDecomposition(QQ8)[5]; \u0026lt;crossed product with center Rationals over GaussianRationals of a group of size 2\u0026gt; gap\u0026gt; HQ_ := QuaternionAlgebra(Rationals); \u0026lt;algebra-with-one of dimension 4 over Rationals\u0026gt; But they don\u0026rsquo;t look the same.\ngap\u0026gt; H := UnderlyingMagma( HQ ); \u0026lt;group of size 2 with 2 generators\u0026gt; gap\u0026gt; fam := ElementsFamily( FamilyObj( HQ ) ); \u0026lt;Family: \u0026#34;CrossedProductObjFamily\u0026#34;\u0026gt; gap\u0026gt; g := ElementOfCrossedProduct( fam, 0, [ 1, E(4) ], AsList(H) ); (ZmodnZObj( 1, 4 ))*(1)+(ZmodnZObj( 3, 4 ))*(E(4)) gap\u0026gt; SchurIndex(QQ8); \u0026#34;fail: Quaternion Algebra Over NonRational Field, use another method.\u0026#34; gap\u0026gt; SchurIndex(HQ); \u0026#34;fail: Quaternion Algebra Over NonRational Field, use another method.\u0026#34; gap\u0026gt; i:=First([1..Length(Irr(Q8))],i-\u0026gt;Size(KernelOfCharacter(Irr(Q8)[i]))=1);; gap\u0026gt; SchurIndexByCharacter(GaussianRationals,Q8,Irr(Q8)[i]); 1 But it doesn\u0026rsquo;t seem that any insight is gained in the process. On the other hand, what\u0026rsquo;s obtained by WedderburnDecompositionAsSCAlgebras has better luck:\ngap\u0026gt; WD := WedderburnDecompositionAsSCAlgebras(QQ8); [ Rationals, Rationals, Rationals, Rationals, \u0026lt;algebra of dimension 4 over Rationals\u0026gt; ] gap\u0026gt; HQ := WD[5]; \u0026lt;algebra of dimension 4 over Rationals\u0026gt; gap\u0026gt; HQ = HQ_; false gap\u0026gt; IsSimpleAlgebra(HQ); true gap\u0026gt; IsSimpleAlgebra(HQ_); true gap\u0026gt; GeneratorsOfAlgebra(HQ); [ v.1, v.2, v.3, v.4 ] gap\u0026gt; GeneratorsOfAlgebra(HQ_); [ e, i, j, k ] gap\u0026gt; BasisVectors( Basis( HQ ) ); [ v.1, v.2, v.3, v.4 ] gap\u0026gt; BasisVectors( Basis( HQ_ ) ); [ e, i, j, k ] gap\u0026gt; DirectSumDecomposition(HQ); [ \u0026lt;two-sided ideal in \u0026lt;algebra of dimension 4 over Rationals\u0026gt;, (1 generator)\u0026gt; ] gap\u0026gt; DirectSumDecomposition(HQ_); [ \u0026lt;two-sided ideal in \u0026lt;algebra-with-one of dimension 4 over Rationals\u0026gt;, (1 generator)\u0026gt; ] gap\u0026gt; AdjointBasis( Basis( HQ ) ); Basis( \u0026lt;vector space over Rationals, with 4 generators\u0026gt;, [ [ [ 0, 0, 1, 0 ], [ 0, 0, 0, -1 ], [ -1, 0, 0, 0 ], [ 0, 1, 0, 0 ] ], [ [ 0, 0, 0, 1 ], [ 0, 0, 1, 0 ], [ 0, -1, 0, 0 ], [ -1, 0, 0, 0 ] ], [ [ 1, 0, 0, 0 ], [ 0, 1, 0, 0 ], [ 0, 0, 1, 0 ], [ 0, 0, 0, 1 ] ], [ [ 0, -1, 0, 0 ], [ 1, 0, 0, 0 ], [ 0, 0, 0, -1 ], [ 0, 0, 1, 0 ] ] ] ) gap\u0026gt; AdjointBasis( Basis( HQ_ ) ); Basis( \u0026lt;vector space over Rationals, with 4 generators\u0026gt;, [ [ [ 1, 0, 0, 0 ], [ 0, 1, 0, 0 ], [ 0, 0, 1, 0 ], [ 0, 0, 0, 1 ] ], [ [ 0, -1, 0, 0 ], [ 1, 0, 0, 0 ], [ 0, 0, 0, -1 ], [ 0, 0, 1, 0 ] ], [ [ 0, 0, -1, 0 ], [ 0, 0, 0, 1 ], [ 1, 0, 0, 0 ], [ 0, -1, 0, 0 ] ], [ [ 0, 0, 0, -1 ], [ 0, 0, -1, 0 ], [ 0, 1, 0, 0 ], [ 1, 0, 0, 0 ] ] ] ) gap\u0026gt; HQ__ := AsAlgebraWithOne(Rationals, HQ); \u0026lt;algebra-with-one over Rationals, with 4 generators\u0026gt; gap\u0026gt; HQ_ = HQ__; false gap\u0026gt; IsomorphismFpAlgebra(HQ__); [ v.1, v.2, v.3, v.4, v.3 ] -\u0026gt; [ [(1)*x.1], [(1)*x.2], [(1)*x.3], [(1)*x.4], [(1)*\u0026lt;identity ...\u0026gt;] ] gap\u0026gt; IsomorphismFpAlgebra(HQ_); [ e, i, j, k, e ] -\u0026gt; [ [(1)*x.1], [(1)*x.2], [(1)*x.3], [(1)*x.4], [(1)*\u0026lt;identity ...\u0026gt;] ] gap\u0026gt; bA:= BasisVectors( Basis( HQ ) );; bB:= BasisVectors( Basis( HQ_ ) );; gap\u0026gt; f:= AlgebraHomomorphismByImages( HQ, HQ_, bA, bB ); fail gap\u0026gt; bA:= BasisVectors( Basis( HQ__ ) );; bB:= BasisVectors( Basis( HQ_ ) );; gap\u0026gt; f:= AlgebraHomomorphismByImages( HQ__, HQ_, bA, bB ); fail But they are still not so same.\nSome more explorations:\ngap\u0026gt; IsomorphismSCAlgebra(HQ); IdentityMapping( \u0026lt;algebra of dimension 4 over Rationals\u0026gt; ) gap\u0026gt; IsomorphismSCAlgebra(HQ_); IdentityMapping( \u0026lt;algebra-with-one of dimension 4 over Rationals\u0026gt; ) gap\u0026gt; IsomorphismSCAlgebra(HQ__); IdentityMapping( \u0026lt;algebra-with-one of dimension 4 over Rationals\u0026gt; ) gap\u0026gt; IsomorphismMatrixAlgebra(HQ); \u0026lt;op. hom. Algebra( Rationals, [ v.1, v.2, v.3, v.4 ] ) -\u0026gt; matrices of dim. 4\u0026gt; gap\u0026gt; IsomorphismMatrixAlgebra(HQ_); \u0026lt;op. hom. AlgebraWithOne( Rationals, [ e, i, j, k ] ) -\u0026gt; matrices of dim. 4\u0026gt; gap\u0026gt; IsomorphismMatrixAlgebra(HQ__); \u0026lt;op. hom. AlgebraWithOne( Rationals, [ v.1, v.2, v.3, v.4 ] ) -\u0026gt; matrices of dim. 4\u0026gt; This section is inspired by this SO question, the Abstract Algebra in GAP, and Wedderburn Decomposition of Group Algebras\n","permalink":"https://utensil.tngl.sh/blog/posts/group-algebra/","summary":"This post studies group algebras with GAP, focusing on a few interested groups. See My math interests in 2024: Group Algebra for context.\nIt\u0026rsquo;s helpful to read GAP Manual and SO questions before using GAP.\nInstallation I\u0026rsquo;m using Mac, so I ran the following commands to install and start GAP:\nbrew install wget autoconf gmp readline wget https://github.com/gap-system/gap/releases/download/v4.13.0/gap-4.13.0.tar.gz tar xzvf gap-4.13.0.tar.gz cd gap-4.13.0 ./autogen.sh ./configure make -j4 V=1 all make check make install which gap gap -l .","title":"Studying group algebras with GAP"},{"content":"I wish this post to be a continuously updated list of my math interests in 2024 with proper citations to literatures, as I keep wandering in the math wonderland and I don\u0026rsquo;t want to be lost in it without breadcrumbs.\nSome interests that have older origins will gradually moved to corresponding posts for earlier years.\nI also hope certain interests will be developed into research projects, and leaving only a brief summary and a link here.\nEach interest should have one or few central questions, and one or few references to literatures.\nFormalization This part of interests is about small-scale formalization of mathematical concepts and theorems, for learning and potential PRs to Lean\u0026rsquo;s Mathlib. Each should focus on one reference which is well organized and convenient to be converted into a blueprint.\nSpin groups The PR to Mathlib #9111 about Spin groups is ready to merge, but there are 2 open questions:\nwhat more lemmas about Spin groups are interesting to mathematians? what more should be formalized to formalize Versors and what’s in Section \u0026ldquo;The contents of a geometric algebra\u0026rdquo; in ( Citation: Chisolm,\u0026#32;2012 Chisolm,\u0026#32; E. \u0026#32; (2012). \u0026#32;Geometric algebra. arXiv preprint arXiv:1205.5935.\u0026#32;Retrieved from\u0026#32; http://arxiv.org/abs/1205.5935 ) , e.g. r-blades, r-vectors? For the former, I should take a closer look at ( Citation: Figueroa-O’Farrill,\u0026#32;2010 Figueroa-O’Farrill,\u0026#32; J. \u0026#32; (2010). \u0026#32;Spin geometry. Lecture Notes. University of Edinburgh.\u0026#32;Retrieved from\u0026#32; http://mat.uab.cat/~rubio/csa2017/SpinNotes.pdf ) and maybe ( Citation: Suarez,\u0026#32;2019 Suarez,\u0026#32; R. \u0026#32; (2019). \u0026#32;Expository paper on clifford algebras, representations, and the octonion algebra. arXiv preprint arXiv:1906.11460.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/1906.11460 ) , ( Citation: Reynoso,\u0026#32;2023 Reynoso,\u0026#32; A. \u0026#32; (2023). \u0026#32;Probing clifford algebras through spin groups: A standard model perspective. arXiv preprint arXiv:2312.10071.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2312.10071 ) .\nFor the latter, see the Z-filteration in lean-ga and the prototype.\nI also wish to include some latest results presented in ( Citation: Ruhe,\u0026#32;Brandstetter \u0026amp; al.,\u0026#32;2023 Ruhe,\u0026#32; D.,\u0026#32; Brandstetter,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Forré,\u0026#32; P. \u0026#32; (2023). \u0026#32;Clifford group equivariant neural networks. arXiv preprint arXiv:2305.11141. ) , with supplements from ( Citation: Brehmer,\u0026#32;De Haan \u0026amp; al.,\u0026#32;2023 Brehmer,\u0026#32; J.,\u0026#32; De Haan,\u0026#32; P.,\u0026#32; Behrends,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Cohen,\u0026#32; T. \u0026#32; (2023). \u0026#32;Geometric algebra transformers. arXiv preprint arXiv:2305.18415. ) (lately there is a new paper applying this in HEP ( Citation: Spinner,\u0026#32;Bresó \u0026amp; al.,\u0026#32;2024 Spinner,\u0026#32; J.,\u0026#32; Bresó,\u0026#32; V.,\u0026#32; Haan,\u0026#32; P.,\u0026#32; Plehn,\u0026#32; T.,\u0026#32; Thaler,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Brehmer,\u0026#32; J. \u0026#32; (2024). \u0026#32;Lorentz-equivariant geometric algebra transformers for high-energy physics. arXiv preprint arXiv:2405.14806. ) , in the same spirit, I should also read ( Citation: Berzins,\u0026#32;Radler \u0026amp; al.,\u0026#32;2024 Berzins,\u0026#32; A.,\u0026#32; Radler,\u0026#32; A.,\u0026#32; Sanokowski,\u0026#32; S.,\u0026#32; Hochreiter,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Brandstetter,\u0026#32; J. \u0026#32; (2024). \u0026#32;Geometry-informed neural networks. arXiv preprint arXiv:2402.14009. ) and possibly ( Citation: Raissi,\u0026#32;Perdikaris \u0026amp; al.,\u0026#32;2019 Raissi,\u0026#32; M.,\u0026#32; Perdikaris,\u0026#32; P.\u0026#32;\u0026amp;\u0026#32;Karniadakis,\u0026#32; G. \u0026#32; (2019). \u0026#32;Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics,\u0026#32;378.\u0026#32;686–707. ) )), in which some of the results are proven in ( Citation: Roelfs\u0026#32;\u0026amp;\u0026#32;De Keninck,\u0026#32;2023 Roelfs,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32;De Keninck,\u0026#32; S. \u0026#32; (2023). \u0026#32;Graded symmetry groups: Plane and simple. Advances in Applied Clifford Algebras,\u0026#32;33(3).\u0026#32;30. ) .\nSee also discussions in the lean-ga blueprint.\nI\u0026rsquo;ve started a Forester experiment about the definitions of Spin groups here. I also need to check citations of On some Lie groups in degenerate Clifford geometric algebras.\nMatrix The Matrix Cookbook (November 15, 2012) ( Citation: Petersen,\u0026#32;Pedersen \u0026amp; al.,\u0026#32;2008 Petersen,\u0026#32; K.,\u0026#32; Pedersen,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32; \u0026#32; (2008). \u0026#32;The matrix cookbook. Technical University of Denmark,\u0026#32;7(15).\u0026#32;510.\u0026#32;Retrieved from\u0026#32; https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf ) covers many useful results about matrices, and Eric Wieser\u0026rsquo;s project lean-matrix-cookbook aims to give one-liner proofs (with reference to the counter part in Mathlib) to all of them.\nThe project is far from complete and it would be great to claim a small portion of interested results and contribute to it. I also wish to figure out the GA counterpart of the same portion.\nPrevious interests about matrices rise from Steven De Keninck\u0026rsquo;s work on GALM ( Citation: De Keninck\u0026#32;\u0026amp;\u0026#32;Dorst,\u0026#32;2019 De Keninck,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Dorst,\u0026#32; L. \u0026#32; (2019). \u0026#32; Geometric algebra levenberg-marquardt. \u0026#32; Springer. ) , since the paper I have been interested in GA approaches that has practical advantages over traditional matrix-based methods. Notably the paper also discussed the link between degenerate Clifford algebras and dual numbers / automatic differentiation. A more recent inspiration might be his new work LookMaNoMatrices.\nTODO: decide which results are interesting and feasible to be formalized for me.\nI wish to pursue further on the topic of Matrix/Tensor, see ( Citation: Taylor,\u0026#32;2024 Taylor,\u0026#32; J. \u0026#32; (2024). \u0026#32;An introduction to graphical tensor notation for mechanistic interpretability. arXiv preprint arXiv:2402.01790. ) and ( Citation: S,\u0026#32;2023 S,\u0026#32; R. \u0026#32; (2023). \u0026#32; Matrix representations of clifford algebras. \u0026#32;Retrieved from\u0026#32; https://www.cphysics.org/article/86175.pdf ) . The former also led me to Einsums in C++. For the latter, I\u0026rsquo;m thinking of HepLean.SpaceTime.CliffordAlgebra.\nGroup Algebra In a sense, group algebras are the source of all you need to know about representation theory.\nThe primary reference is ( Citation: James\u0026#32;\u0026amp;\u0026#32;Liebeck,\u0026#32;2001 James,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;Liebeck,\u0026#32; M.\u0026#32; (2001). \u0026#32; Representations and characters of groups. \u0026#32; Cambridge university press. ) for understanding FG-module, Group algebra, the presentation of groups, Clifford theory (which is the standard method of constructing representations and characters of semi-direct products, see ( Citation: Woit,\u0026#32;Woit \u0026amp; al.,\u0026#32;2017 Woit,\u0026#32; P.,\u0026#32; Woit\u0026#32;\u0026amp;\u0026#32;Bartolini\u0026#32; (2017). \u0026#32; Quantum theory, groups and representations. \u0026#32; Springer. ) , and \u0026ldquo;3.6 Clifford theory\u0026rdquo; in ( Citation: Lux\u0026#32;\u0026amp;\u0026#32;Pahlings,\u0026#32;2010 Lux,\u0026#32; K.\u0026#32;\u0026amp;\u0026#32;Pahlings,\u0026#32; H.\u0026#32; (2010). \u0026#32; Representations of groups: A computational approach. \u0026#32; Cambridge University Press. ) ), Schur indices etc. We also need to check ( Citation: Lux\u0026#32;\u0026amp;\u0026#32;Pahlings,\u0026#32;2010 Lux,\u0026#32; K.\u0026#32;\u0026amp;\u0026#32;Pahlings,\u0026#32; H.\u0026#32; (2010). \u0026#32; Representations of groups: A computational approach. \u0026#32; Cambridge University Press. ) for its introduction to GAP, and we should pay close attention to the progress of GAP-LEAN. ( Citation: Sims,\u0026#32;1994 Sims,\u0026#32; C.\u0026#32; (1994). \u0026#32; Computation with finitely presented groups. \u0026#32; Cambridge University Press. ) might also be interesting in a similar manner as ( Citation: Lux\u0026#32;\u0026amp;\u0026#32;Pahlings,\u0026#32;2010 Lux,\u0026#32; K.\u0026#32;\u0026amp;\u0026#32;Pahlings,\u0026#32; H.\u0026#32; (2010). \u0026#32; Representations of groups: A computational approach. \u0026#32; Cambridge University Press. ) but with emphasis on the presentation of groups.\nSee also group algebra on nlab, particularly that \u0026ldquo;A group algebra is in particular a Hopf algebra and a $G$-graded algebra.\u0026rdquo;\nThe related Zulip thread is here, and I have preliminary explorations and experiments in Lean here.\nThis interest originates from reading Robert A. Wilson\u0026rsquo;s work ( Citation: Wilson,\u0026#32;2024 Wilson,\u0026#32; R. \u0026#32; (2024). \u0026#32;A discrete model for gell-mann matrices. arXiv preprint arXiv:2401.13000.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2401.13000 ) . The ultimate goal is to understand the group algebra of the binary tetrahedral group ($Q_8 \\rtimes Z_3$), then the three-dimensional complex reflection group ($G_{27} \\rtimes Q_8 \\rtimes Z_3$), a.k.a. the triple cover of the Hessian group, which can be interpreted as a finite analogue of the complete gauge group $U(1) \\times SU(2) \\times SU(3)$.\nType Theory Recently I have read some meta-level dependent type theory (Typst source). It might be time to re-read leantt paper, and start reading lean4lean paper/source.\nThe author ice1000 has strong interest in QIIT (Quotient Inductive-Inductive Types) and QIIR (Quotient Inductive-Inductive Recursion), he has implemented overlap in Aya with termination check and confluence check.\nAya has a philosophy that the kernel could include pattern matching, but at the cost of no generation and translation of eliminators at present. In principle this is feasible, without overlap, it could be implemented by \u0026ldquo;theory of signatures\u0026rdquo;, with overlap, it needs \u0026ldquo;Coherent and concurrent elimination for initial algebras\u0026rdquo; which I find fascinating, and have read ( Citation: Yeasin,\u0026#32;2011 Yeasin,\u0026#32; M. \u0026#32; (2011). \u0026#32;Initial algebra, final coalgebra and datatype. ) .\nThe author also has a great article on TT \u0026amp; Cat ( Citation: Zhang,\u0026#32;2021 Zhang,\u0026#32; T. \u0026#32; (2021). \u0026#32;Type theories in category theory. arXiv preprint arXiv:2107.13242. ) . His recommendation of ( Citation: Gundry\u0026#32;\u0026amp;\u0026#32;McBride,\u0026#32;2013 Gundry,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;McBride,\u0026#32; C. \u0026#32; (2013). \u0026#32;A tutorial implementation of dynamic pattern unification. Unpublished draft.\u0026#32;194. ) is also worth reading.\nIn the process of learning Topos, I wish to have a better understanding of Logic. Particularly, Curry–Howard–Lambek correspondance, Propositional truncation, and Paraconsistent logic are on the plate.\nAlthough remotely related, I\u0026rsquo;ll place Introduction to Formal Reasoning (COMP2065) in this section so I won\u0026rsquo;t lose track of it.\nPhilosophy Type theories, logic have their origins in philosophy. Lacan had drawn inspiration from the work of the mathematician and philosopher of science Georges Canguilhem, and the philosopher of mathematics Alain Badiou. If he had lived to see the rise of proof assistants, he would be interested in the formalization of his theories.\nThere is a project on Github called Lacan-Mathemes which visualized some core concepts of Lacan in TikZ. It would be interesting to start there and look for proper objects to formalize in Lean. Another interesting thing to do is to visualize them in Typst.\nHere I would like to quote a generated response from Claude 1.2 Instant, which is quite optimistic:\nHere are a few thoughts on formalizing aspects of Lacanian psychoanalytic theory using theorem provers:\nLacan\u0026rsquo;s structural theory of the psyche could potentially be formalized using logic. The tripartite structure of the Real, Symbolic, and Imaginary could be modeled as formal domains or ontologies with defined relationships between them. Key concepts like the mirror stage, the Name-of-the-Father, the objet petit a, etc. could be defined as logical predicates or functions operating within this structural framework. For example, one could define predicates like \u0026ldquo;inMirrorStage(subject)\u0026rdquo; or \u0026ldquo;desiresObjetPetitA(subject, object)\u0026rdquo;. Relations like the split between desire and drive, the tension between the Symbolic order and the Real, the misrecognition of the Imaginary could be expressed through logical rules and inferences between concepts. The dynamics of psychoanalytic concepts like repression, sublimation, transference could be modeled as state transitions or logical transformations within the structured system. Specific psychoanalytic theories like the stages of psychosexual development or the topology of the fantasy could be axiomatized and theorems deduced from the axioms. However, capturing the open-ended, non-deterministic nature of unconscious processes and the ambiguity/contingency of signification would be challenging and may require non-classical logics or probabilistic approaches.\nSo in summary, while difficult, certain aspects of Lacanian theory seem amenable to formalization using tools from logic, ontology, and knowledge representation. Careful design would be needed to address theory\u0026rsquo;s complexity.\nLately there is a paper formalizing Kant ( Citation: Achourioti\u0026#32;\u0026amp;\u0026#32;Van Lambalgen,\u0026#32;2011 Achourioti,\u0026#32; T.\u0026#32;\u0026amp;\u0026#32;Van Lambalgen,\u0026#32; M. \u0026#32; (2011). \u0026#32;A formalization of kant’s transcendental logic. The Review of Symbolic Logic,\u0026#32;4(2).\u0026#32;254–289. ) which might be interesting.\nGeneralized Clifford Algebra and related mathematical branches A Clifford algebra is a $Z_2$-graded algebra, and a Filtered algebra, the associated graded algebra is the exterior algebra.\nIt may be thought of as quantizations (cf. quantum group) of the exterior algebra, in the same way that the Weyl algebra is a quantization of the symmetric algebra. Specifically, for $V$ an inner product space, the symbol map constitutes an isomorphism of the underlying super vector spaces of the Clifford algebra with the exterior algebra on $V$, and one may understand the Clifford algebra as the quantization Grassmann algebra induced from the inner product regarded as an odd symplectic form.\nWeyl algebras and Clifford algebras admit a further structure of a *-algebra, and can be unified as even and odd terms of a superalgebra, as discussed in CCR and CAR algebras.\nA Clifford module is a representation of a Clifford algebra.\nA Generalized Clifford algebra (GCA) can also refer to associative algebras that are constructed using forms of higher degree instead of quadratic forms, e.g.\nFor $q_1, q_2, \\ldots, q_m \\in \\mathbb{k}^*$, the generalized Clifford algebra $C^{(n)}\\left(q_1, q_2, \\ldots, q_m\\right)$ is a unital associative algebra generated by $e_1, e_2, \\ldots, e_m$ subject to the relations $$ e_i^n=q_i \\mathbf{1}, \\quad e_i e_j=\\omega e_j e_i, \\quad \\forall j \\lt i . $$ It is easy to see that $C^{(n)}\\left(q_1, q_2, \\ldots, q_m\\right)$ is $\\mathbb{Z}_n$-graded where the degree of $e_i$ is $\\overline{1}$, the generator of $\\mathbb{Z}_n$. ( Citation: Cheng,\u0026#32;Li \u0026amp; al.,\u0026#32;2019 Cheng,\u0026#32; T.,\u0026#32; Li,\u0026#32; H.\u0026#32;\u0026amp;\u0026#32;Yang,\u0026#32; Y. \u0026#32; (2019). \u0026#32;A new view of generalized clifford algebras. Advances in Applied Clifford Algebras,\u0026#32;29.\u0026#32;1–11. ) In ( Citation: Cheng,\u0026#32;Li \u0026amp; al.,\u0026#32;2019 Cheng,\u0026#32; T.,\u0026#32; Li,\u0026#32; H.\u0026#32;\u0026amp;\u0026#32;Yang,\u0026#32; Y. \u0026#32; (2019). \u0026#32;A new view of generalized clifford algebras. Advances in Applied Clifford Algebras,\u0026#32;29.\u0026#32;1–11. ) , note also that \u0026ldquo;Clifford algebras are weak Hopf algebras in some symmetric tensor categories.\u0026rdquo; while \u0026ldquo;generalized Clifford algebras are weak Hopf algebras in some suitable braided linear categories of graded vector spaces.\u0026rdquo; as well as that \u0026ldquo;the Clifford process is a powerful technique to construct larger dimensional Clifford algebras from known ones.\u0026rdquo;\nTODO: add papers linking Hopf algebra and Clifford algebra together learned from the adjoint discord here.\nClifford algebras can be obtained by twisting of group algebras ( Citation: Albuquerque\u0026#32;\u0026amp;\u0026#32;Majid,\u0026#32;2002 Albuquerque,\u0026#32; H.\u0026#32;\u0026amp;\u0026#32;Majid,\u0026#32; S. \u0026#32; (2002). \u0026#32;Clifford algebras obtained by twisting of group algebras. Journal of Pure and Applied Algebra,\u0026#32;171(2-3).\u0026#32;133–148.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/math/0011040 ) , where twisted group algebras are studied in ( Citation: Conlon,\u0026#32;1964 Conlon,\u0026#32; S. \u0026#32; (1964). \u0026#32;Twisted group algebras and their representations. Journal of the Australian Mathematical Society,\u0026#32;4(2).\u0026#32;152–173. ) , ( Citation: Edwards\u0026#32;\u0026amp;\u0026#32;Lewis,\u0026#32;1969 Edwards,\u0026#32; C.\u0026#32;\u0026amp;\u0026#32;Lewis,\u0026#32; J. \u0026#32; (1969). \u0026#32;Twisted group algebras i. Communications in Mathematical Physics,\u0026#32;13.\u0026#32;119–130. ) , ( Citation: Edwards\u0026#32;\u0026amp;\u0026#32;Lewis,\u0026#32;1969 Edwards,\u0026#32; C.\u0026#32;\u0026amp;\u0026#32;Lewis,\u0026#32; J. \u0026#32; (1969). \u0026#32;Twisted group algebras II. Communications in Mathematical Physics,\u0026#32;13.\u0026#32;131–141. ) .\nThere exists isomorphisms between certain Clifford algebras and NDAs (Normed Division Algebras) over $\\mathbb{R}$.\nVariants of Clifford algebras whose generators are idempotent or nilpotent can be considered. Zeon algebras (\u0026ldquo;nil-Clifford algebras\u0026rdquo;) have proven to be useful in enumeration problems on graphs where certain configurations are forbidden, such as in the enumeration of matchings and self-avoiding walks. The idempotent property of the generators of idem-Clifford algebras can be used to avoid redundant information when enumerating certain graph and hypergraph structures. See ( Citation: Ewing\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32;2022 Ewing,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2022). \u0026#32;Zeon and idem-clifford formulations of hypergraph problems. Advances in Applied Clifford Algebras,\u0026#32;32(5).\u0026#32;61.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2201.05895.pdf ) .\nIt\u0026rsquo;s also closely related to universal enveloping algebra (see ( Citation: Figueroa-O’Farrill,\u0026#32;2010 Figueroa-O’Farrill,\u0026#32; J. \u0026#32; (2010). \u0026#32;Spin geometry. Lecture Notes. University of Edinburgh.\u0026#32;Retrieved from\u0026#32; http://mat.uab.cat/~rubio/csa2017/SpinNotes.pdf ) and \u0026ldquo;The universal enveloping algebra of a Lie algebra is the analogue of the usual group algebra of a group.\u0026rdquo; from group algebra on nlab).\nGreat discussions about the limitations and generalizations of Clifford algebras can be found in John C. Baez\u0026rsquo;s ( Citation: Baez,\u0026#32;2002 Baez,\u0026#32; J. \u0026#32; (2002). \u0026#32;The octonions. Bulletin of the american mathematical society,\u0026#32;39(2).\u0026#32;145–205.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/math/0105155.pdf ) . Particularly, note Cayley-Dickson construction, Bott periodicity, matrix algebra, triality, and $\\mathbb{R}$ as a real commutative associative nicely normed ∗-algebra. Also see Anthony Lasenby\u0026rsquo;s work on the embedding of octonions in the Clifford geometric algebra for space-time STA ( $\\mathop{\\mathcal{C}\\ell}(1, 3)$ ) ( Citation: Lasenby,\u0026#32;2024 Lasenby,\u0026#32; A. \u0026#32; (2024). \u0026#32;Some recent results for su (3) and octonions within the geometric algebra approach to the fundamental forces of nature. Mathematical Methods in the Applied Sciences,\u0026#32;47(3).\u0026#32;1471–1491.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2202.06733 ) .\nNote also Kingdon algebras: alternative Clifford-like algebras over vector spaces equipped with a symmetric bilinear form ( Citation: Depies,\u0026#32;Smith \u0026amp; al.,\u0026#32;2024 Depies,\u0026#32; C.,\u0026#32; Smith,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Ashburn,\u0026#32; M. \u0026#32; (2024). \u0026#32;Octonions as clifford-like algebras. Journal of Algebra,\u0026#32;644.\u0026#32;761–795.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2310.09972 ) .\nCategorified Clifford Algebra A categorical view of Clifford Algebra is discussed in ( Citation: Figueroa-O’Farrill,\u0026#32;2010 Figueroa-O’Farrill,\u0026#32; J. \u0026#32; (2010). \u0026#32;Spin geometry. Lecture Notes. University of Edinburgh.\u0026#32;Retrieved from\u0026#32; http://mat.uab.cat/~rubio/csa2017/SpinNotes.pdf ) .\nA Clifford Algebra can be categorified: \u0026ldquo;An Clifford algebra over a vector space is defined to be the Koszul dual to an abelian fully weak Lie-algebra\u0026rdquo; where \u0026ldquo;Fully weak Lie-algebras are Koszul dual to differential graded Clifford algebras.\u0026rdquo; See also Higher Clifford Algebras.\nSheaves of Clifford Algebras Sheaves of Clifford Algebras are studied in ( Citation: Yizengaw,\u0026#32;2015 Yizengaw,\u0026#32; B.\u0026#32; (2015). \u0026#32; On clifford a-algebras and a framework for localization of a-modules. \u0026#32; University of Pretoria (South Africa). ) and its references. See also ( Citation: Schapira,\u0026#32;2023 Schapira,\u0026#32; P. \u0026#32; (2023). \u0026#32; An introduction to categories and sheaves. \u0026#32;Retrieved from\u0026#32; https://webusers.imj-prg.fr/~pierre.schapira/LectNotes/CatShv.pdf ) , an elementary introduction to ( Citation: Kashiwara\u0026#32;\u0026amp;\u0026#32;Schapira,\u0026#32;2006 Kashiwara,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32;Schapira,\u0026#32; P.\u0026#32; (2006). \u0026#32; Categories and sheaves (). https://doi.org/10.1007/3-540-27950-4 ) , which presents categories, homological algebra and sheaves in a systematic and exhaustive manner.\nDKP Algebra Duffin-Kemmer-Petiau Algebra is $$\\frac{T(V)}{I(v\\otimes w\\otimes v - g(v,w)v)}$$ in the same way that Clifford Algebra is $$\\frac{T(V)}{I(v\\otimes v - g(v,v))}.$$ See ( Citation: Fernandes,\u0026#32;2022 Fernandes,\u0026#32; M. \u0026#32; (2022). \u0026#32;Clifford algebraic approach to the de donder–weyl hamiltonian theory. Advances in Applied Clifford Algebras,\u0026#32;32(2).\u0026#32;23.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2112.08483 ) which embeds DKP Algebra in Clifford Algebra with projectors.\nMisc TODO:\nRecovering Composition Algebras from 3D Geometric Algebras Eliminating topological errors in neural network rotation estimation using self-selecting ensembles - Sitao Xiang - SIGGRAPH 2021 OSMU24: Quantum Foundations, Particle Physics, and Unification of Forces https://web.archive.org/web/20201205215239/http://web4.uwindsor.ca/users/b/baylis/main.nsf Applied mathematics Topology I wish to render some pictures in ( Citation: Francis\u0026#32;\u0026amp;\u0026#32;Francis,\u0026#32;1987 Francis,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;Francis,\u0026#32; G.\u0026#32; (1987). \u0026#32; A topological picturebook. \u0026#32; Springer. ) with TikZ, GLSL shader, and by GA with ganja.js \u0026amp; GAmphetamine or their Lean version, inspired by Steven De Keninck\u0026rsquo;s notebooks, e.g. torus, orbit 1, orbit 2, origami, skinning, slicing etc.\nKnots See ( Citation: Manturov,\u0026#32;2018 Manturov,\u0026#32; V.\u0026#32; (2018). \u0026#32; Knot theory. \u0026#32; CRC press. ) and the tracking issue.\nFor interpreting knots in a sheaf-theoretic perspective, see ( Citation: Chrisman,\u0026#32;2023 Chrisman,\u0026#32; M. \u0026#32; (2023). \u0026#32;A geometric foundation of virtual knot theory. arXiv preprint arXiv:2301.10318. ) .\nSee also:\nA Table of Virtual Knots The homological arrow polynomial for virtual links (code) Origami See ( Citation: Hull,\u0026#32;2020 Hull,\u0026#32; T.\u0026#32; (2020). \u0026#32; Origametry: Mathematical methods in paper folding. \u0026#32; Cambridge University Press. ) and the tracking issue.\nDynamical Systems: Bifurcation Theory The preferred reference for me is ( Citation: Izhikevich,\u0026#32;2007 Izhikevich,\u0026#32; E.\u0026#32; (2007). \u0026#32; Dynamical systems in neuroscience. \u0026#32; MIT press.\u0026#32;Retrieved from\u0026#32; https://www.izhikevich.org/publications/dsn/ ) for its applications in neuroscience and various excellent diagrams. But it\u0026rsquo;s not a mathematically rigorous treatment of bifurcation theory.\nSheaves My first impression of sheaves is that they are useful to local-to-global applications \u0026ldquo;which ask for global solutions to problems whose hypotheses are local\u0026rdquo;.\nRoughly speaking, a sheaf requires some gluing conditions (axioms \u0026ldquo;Locality\u0026rdquo; and \u0026ldquo;Gluing\u0026rdquo;) so that local data can be collated compatibly into a global algebraic structure that varies continuously over local covering domains (\u0026ldquo;sections\u0026rdquo; of sheaves).\nTo do so, a sheaf in general, as defined in the category-theoretical language, needs\na topological space (or a site in general), denoted $X$ (or $\\mathcal{C}$ for a site) a category, sometimes denoted $\\mathcal{D}$, meaning \u0026ldquo;data category\u0026rdquo;, whose objects are algebraic structures and morphisms are structure-preserving maps and builds (gluing conditions) on a $\\mathcal{D}$-valued presheaf over $X$ (or $\\mathcal{C}$), denoted $\\mathcal{F}$ (as its French name is \u0026ldquo;faisceau\u0026rdquo;), which is essentially a contravariant functor $\\mathcal{F}: \\mathcal{C}^{op} \\to \\mathcal{D}$ but a concept with an attitude, and its morphisms are restriction maps between open sets in $X$ (or between objects that satisfy the pretopology $\\mathcal{J}$ in $C$, where $\\mathcal{J}$ is the pretopology on $\\operatorname{Open}(X)$, which specifies when a covering family of open sets exists).\nIts latest application to deep learning, Thomas Gebhart\u0026rsquo;s thesis ( Citation: Gebhart,\u0026#32;2023 Gebhart,\u0026#32; T. \u0026#32; (2023). \u0026#32; Sheaf representation learning \u0026nbsp;(PhD thesis).\u0026#32; University of Minnesota ) sees a sheaf over a topological space as a data structure \u0026ldquo;which defines rules for associating data to a space so that local agreements in data assignment align with a coherent global representation\u0026rdquo;, thus a generalization of both:\nrelational learning, which aims to \u0026ldquo;combine symbolic, potentially hand-engineered prior knowledge with the tabula rasa representational flexibility of deep learning to achieve a synthetic model family which can be defined with respect to symbolic knowledge priors about the data domain\u0026rdquo; geometric deep learning, which \u0026ldquo;provides a group-theoretic approach to reasoning about and encoding domain-symmetry invariance or equivariance within machine learning models\u0026rdquo;, \u0026ldquo;providing a mathematical framework for characterizing the interplay between the topological information embedded within a domain and the representations of data learned by machine learning models\u0026rdquo;.\nMy prior interest in geometric deep learning, particularly group-equivariant neural networks, and my believe in symbolism, are the background of my interest in sheaf representation learning.\nNotably, this thesis treats the discrete case of sheaves, a cellular sheaf, whose\ntopological space is a cell complex, which is \u0026ldquo;a topological generalization of a graph, with set inclusion and intersection given by the incidence relationships among cells in the complex\u0026rdquo;, thus \u0026ldquo;admitting a computable, linear-algebraic representation\u0026rdquo;. data category is $\\mathtt{FVect}$, the category of finite-dimensional vector spaces over a field $\\mathbb{F}$, which is a common choice for the data category in machine learning applications, a model-free approach with massive parameter space, flexible representational capacity, but inherits fundamental limitations, e.g. data inefficiency, generalization failure, and interpretability issues. For more details, see also Thomas Gebhart\u0026rsquo;s talk Sheaves for AI: Graph Representation Learning through Sheaf Theory.\nIts application to physics has the potential to formulate differential geometry in a more general setting, without assuming the existence of a locally Euclidean space as manifold did. It\u0026rsquo;s believe that this approach can overcome some difficulties in Quantum field theory even Quantum gravity, because locally there might be no concept of a metric space at all ( Citation: Mallios\u0026#32;\u0026amp;\u0026#32;Zafiris,\u0026#32;2015 Mallios,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Zafiris,\u0026#32; E.\u0026#32; (2015). \u0026#32; Differential sheaves and connections: A natural approach to physical geometry. \u0026#32; World Scientific. ) .\nNote that there are CAS systems that can do sheaf cohomology etc., e.g. Macaulay2, OSCAR.\nSynthetic Differential Geometry For SDG, ( Citation: Kock,\u0026#32;2006 Kock,\u0026#32; A.\u0026#32; (2006). \u0026#32; Synthetic differential geometry. \u0026#32; Cambridge University Press.\u0026#32;Retrieved from\u0026#32; https://users-math.au.dk/kock/sdg99.pdf ) is a classic. ( Citation: Schellingerhout\u0026#32;\u0026amp;\u0026#32;Portegies,\u0026#32;2023 Schellingerhout,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Portegies,\u0026#32; J. \u0026#32; (2023). \u0026#32;Circles in synthetic differential geometry.\u0026#32;Retrieved from\u0026#32; https://pure.tue.nl/ws/portalfiles/portal/308028780/Thesis_BTW_Rob_Schellingerhout_Report_002_.pdf ) is a concise bachelor thesis on the topic, and has interesting discussions on circles.\nRyszard Paweł Kostecki has very approachable notes on Topos ( Citation: Kostecki,\u0026#32;2011 Kostecki,\u0026#32; R. \u0026#32; (2011). \u0026#32;An introduction to topos theory. Technial Report.\u0026#32;Retrieved from\u0026#32; https://www.fuw.edu.pl/~kostecki/ittt.pdf ) and SDG ( Citation: Kostecki,\u0026#32;2009 Kostecki,\u0026#32; R. \u0026#32; (2009). \u0026#32; Differential geometry in toposes. \u0026#32; Citeseer.\u0026#32;Retrieved from\u0026#32; https://www.fuw.edu.pl/~kostecki/sdg.pdf ) .\nWe should also read ( Citation: Rosiak,\u0026#32;2022 Rosiak,\u0026#32; D.\u0026#32; (2022). \u0026#32; Sheaf theory through examples. \u0026#32; MIT Press. ) . It also has many diagrams in the way I imagined, for examples of sheaf.\nML I don\u0026rsquo;t want efforts in Transformers: from self-attention to performance optimizations to be discontinued, lately there is ( Citation: Ferrando,\u0026#32;Sarti \u0026amp; al.,\u0026#32;2024 Ferrando,\u0026#32; J.,\u0026#32; Sarti,\u0026#32; G.,\u0026#32; Bisazza,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Costa-jussà,\u0026#32; M. \u0026#32; (2024). \u0026#32;A primer on the inner workings of transformer-based language models. arXiv preprint arXiv:2405.00208. ) on this topic.\nI might need to follow on the latest development on the linear attention mechanism ( Citation: Peng,\u0026#32;Goldstein \u0026amp; al.,\u0026#32;2024 Peng,\u0026#32; B.,\u0026#32; Goldstein,\u0026#32; D.,\u0026#32; Anthony,\u0026#32; Q.,\u0026#32; Albalak,\u0026#32; A.,\u0026#32; Alcaide,\u0026#32; E.,\u0026#32; Biderman,\u0026#32; S.,\u0026#32; Cheah,\u0026#32; E.,\u0026#32; Ferdinan,\u0026#32; T.,\u0026#32; Hou,\u0026#32; H.,\u0026#32; Kazienko,\u0026#32; P.\u0026#32;\u0026amp;\u0026#32; \u0026#32; (2024). \u0026#32;Eagle and finch: RWKV with matrix-valued states and dynamic recurrence. arXiv preprint arXiv:2404.05892. ) .\nI have almost no understanding of diffusion models, so I should read ( Citation: Bao,\u0026#32;Nie \u0026amp; al.,\u0026#32;2023 Bao,\u0026#32; F.,\u0026#32; Nie,\u0026#32; S.,\u0026#32; Xue,\u0026#32; K.,\u0026#32; Cao,\u0026#32; Y.,\u0026#32; Li,\u0026#32; C.,\u0026#32; Su,\u0026#32; H.\u0026#32;\u0026amp;\u0026#32;Zhu,\u0026#32; J. \u0026#32; (2023). \u0026#32; All are worth words: A ViT backbone for diffusion models. ) and related papers.\nI should also read ( Citation: Mikuła,\u0026#32;Antoniak \u0026amp; al.,\u0026#32;2023 Mikuła,\u0026#32; M.,\u0026#32; Antoniak,\u0026#32; S.,\u0026#32; Tworkowski,\u0026#32; S.,\u0026#32; Jiang,\u0026#32; A.,\u0026#32; Zhou,\u0026#32; J.,\u0026#32; Szegedy,\u0026#32; C.,\u0026#32; Kuciński,\u0026#32; Ł.,\u0026#32; Miłoś,\u0026#32; P.\u0026#32;\u0026amp;\u0026#32;Wu,\u0026#32; Y. \u0026#32; (2023). \u0026#32;Magnushammer: A transformer-based approach to premise selection. arXiv preprint arXiv:2303.04488. ) and related papers.\nConsciousness Lately I became aware of the work on mathematical models of consciousness, namely Integrated information theory ( Citation: Albantakis,\u0026#32;Barbosa \u0026amp; al.,\u0026#32;2023 Albantakis,\u0026#32; L.,\u0026#32; Barbosa,\u0026#32; L.,\u0026#32; Findlay,\u0026#32; G.,\u0026#32; Grasso,\u0026#32; M.,\u0026#32; Haun,\u0026#32; A.,\u0026#32; Marshall,\u0026#32; W.,\u0026#32; Mayner,\u0026#32; W.,\u0026#32; Zaeemzadeh,\u0026#32; A.,\u0026#32; Boly,\u0026#32; M.,\u0026#32; Juel,\u0026#32; B.\u0026#32;\u0026amp;\u0026#32; \u0026#32; (2023). \u0026#32;Integrated information theory (IIT) 4.0: Formulating the properties of phenomenal existence in physical terms. PLoS Computational Biology,\u0026#32;19(10).\u0026#32;e1011465. ) and The information theory of individuality ( Citation: Krakauer,\u0026#32;Bertschinger \u0026amp; al.,\u0026#32;2020 Krakauer,\u0026#32; D.,\u0026#32; Bertschinger,\u0026#32; N.,\u0026#32; Olbrich,\u0026#32; E.,\u0026#32; Flack,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Ay,\u0026#32; N. \u0026#32; (2020). \u0026#32;The information theory of individuality. Theory in Biosciences,\u0026#32;139.\u0026#32;209–223. ) .\nPhysics Physics-Based Simulation is out.\nSo is ( Citation: Sen\u0026#32;\u0026amp;\u0026#32;Zwiebach,\u0026#32;2024 Sen,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Zwiebach,\u0026#32; B. \u0026#32; (2024). \u0026#32;String field theory: A review. arXiv preprint arXiv:2405.19421. ) on string theory.\nTODOs Incorporating the following interests into this post:\nThreads of thought in recent years (in Chinese) Interests logged here References Achourioti\u0026#32;\u0026amp;\u0026#32;Van Lambalgen (2011) Achourioti,\u0026#32; T.\u0026#32;\u0026amp;\u0026#32;Van Lambalgen,\u0026#32; M. \u0026#32; (2011). \u0026#32;A formalization of kant’s transcendental logic. The Review of Symbolic Logic,\u0026#32;4(2).\u0026#32;254–289. Albantakis,\u0026#32; Barbosa,\u0026#32; Findlay,\u0026#32; Grasso,\u0026#32; Haun,\u0026#32; Marshall,\u0026#32; Mayner,\u0026#32; Zaeemzadeh,\u0026#32; Boly,\u0026#32; Juel\u0026#32;\u0026amp;\u0026#32; (2023) Albantakis,\u0026#32; L.,\u0026#32; Barbosa,\u0026#32; L.,\u0026#32; Findlay,\u0026#32; G.,\u0026#32; Grasso,\u0026#32; M.,\u0026#32; Haun,\u0026#32; A.,\u0026#32; Marshall,\u0026#32; W.,\u0026#32; Mayner,\u0026#32; W.,\u0026#32; Zaeemzadeh,\u0026#32; A.,\u0026#32; Boly,\u0026#32; M.,\u0026#32; Juel,\u0026#32; B.\u0026#32;\u0026amp;\u0026#32; \u0026#32; (2023). \u0026#32;Integrated information theory (IIT) 4.0: Formulating the properties of phenomenal existence in physical terms. PLoS Computational Biology,\u0026#32;19(10).\u0026#32;e1011465. Albuquerque\u0026#32;\u0026amp;\u0026#32;Majid (2002) Albuquerque,\u0026#32; H.\u0026#32;\u0026amp;\u0026#32;Majid,\u0026#32; S. \u0026#32; (2002). \u0026#32;Clifford algebras obtained by twisting of group algebras. Journal of Pure and Applied Algebra,\u0026#32;171(2-3).\u0026#32;133–148.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/math/0011040 Baez (2002) Baez,\u0026#32; J. \u0026#32; (2002). \u0026#32;The octonions. Bulletin of the american mathematical society,\u0026#32;39(2).\u0026#32;145–205.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/math/0105155.pdf Bao,\u0026#32; Nie,\u0026#32; Xue,\u0026#32; Cao,\u0026#32; Li,\u0026#32; Su\u0026#32;\u0026amp;\u0026#32;Zhu (2023) Bao,\u0026#32; F.,\u0026#32; Nie,\u0026#32; S.,\u0026#32; Xue,\u0026#32; K.,\u0026#32; Cao,\u0026#32; Y.,\u0026#32; Li,\u0026#32; C.,\u0026#32; Su,\u0026#32; H.\u0026#32;\u0026amp;\u0026#32;Zhu,\u0026#32; J. \u0026#32; (2023). \u0026#32; All are worth words: A ViT backbone for diffusion models. Berzins,\u0026#32; Radler,\u0026#32; Sanokowski,\u0026#32; Hochreiter\u0026#32;\u0026amp;\u0026#32;Brandstetter (2024) Berzins,\u0026#32; A.,\u0026#32; Radler,\u0026#32; A.,\u0026#32; Sanokowski,\u0026#32; S.,\u0026#32; Hochreiter,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Brandstetter,\u0026#32; J. \u0026#32; (2024). \u0026#32;Geometry-informed neural networks. arXiv preprint arXiv:2402.14009. Brehmer,\u0026#32; De Haan,\u0026#32; Behrends\u0026#32;\u0026amp;\u0026#32;Cohen (2023) Brehmer,\u0026#32; J.,\u0026#32; De Haan,\u0026#32; P.,\u0026#32; Behrends,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Cohen,\u0026#32; T. \u0026#32; (2023). \u0026#32;Geometric algebra transformers. arXiv preprint arXiv:2305.18415. Cheng,\u0026#32; Li\u0026#32;\u0026amp;\u0026#32;Yang (2019) Cheng,\u0026#32; T.,\u0026#32; Li,\u0026#32; H.\u0026#32;\u0026amp;\u0026#32;Yang,\u0026#32; Y. \u0026#32; (2019). \u0026#32;A new view of generalized clifford algebras. Advances in Applied Clifford Algebras,\u0026#32;29.\u0026#32;1–11. Chisolm (2012) Chisolm,\u0026#32; E. \u0026#32; (2012). \u0026#32;Geometric algebra. arXiv preprint arXiv:1205.5935.\u0026#32;Retrieved from\u0026#32; http://arxiv.org/abs/1205.5935 Chrisman (2023) Chrisman,\u0026#32; M. \u0026#32; (2023). \u0026#32;A geometric foundation of virtual knot theory. arXiv preprint arXiv:2301.10318. Conlon (1964) Conlon,\u0026#32; S. \u0026#32; (1964). \u0026#32;Twisted group algebras and their representations. Journal of the Australian Mathematical Society,\u0026#32;4(2).\u0026#32;152–173. De Keninck\u0026#32;\u0026amp;\u0026#32;Dorst (2019) De Keninck,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Dorst,\u0026#32; L. \u0026#32; (2019). \u0026#32; Geometric algebra levenberg-marquardt. \u0026#32; Springer. Depies,\u0026#32; Smith\u0026#32;\u0026amp;\u0026#32;Ashburn (2024) Depies,\u0026#32; C.,\u0026#32; Smith,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Ashburn,\u0026#32; M. \u0026#32; (2024). \u0026#32;Octonions as clifford-like algebras. Journal of Algebra,\u0026#32;644.\u0026#32;761–795.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2310.09972 Edwards\u0026#32;\u0026amp;\u0026#32;Lewis (1969) Edwards,\u0026#32; C.\u0026#32;\u0026amp;\u0026#32;Lewis,\u0026#32; J. \u0026#32; (1969). \u0026#32;Twisted group algebras i. Communications in Mathematical Physics,\u0026#32;13.\u0026#32;119–130. Edwards\u0026#32;\u0026amp;\u0026#32;Lewis (1969) Edwards,\u0026#32; C.\u0026#32;\u0026amp;\u0026#32;Lewis,\u0026#32; J. \u0026#32; (1969). \u0026#32;Twisted group algebras II. Communications in Mathematical Physics,\u0026#32;13.\u0026#32;131–141. Ewing\u0026#32;\u0026amp;\u0026#32;Staples (2022) Ewing,\u0026#32; S.\u0026#32;\u0026amp;\u0026#32;Staples,\u0026#32; G. \u0026#32; (2022). \u0026#32;Zeon and idem-clifford formulations of hypergraph problems. Advances in Applied Clifford Algebras,\u0026#32;32(5).\u0026#32;61.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2201.05895.pdf Fernandes (2022) Fernandes,\u0026#32; M. \u0026#32; (2022). \u0026#32;Clifford algebraic approach to the de donder–weyl hamiltonian theory. Advances in Applied Clifford Algebras,\u0026#32;32(2).\u0026#32;23.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2112.08483 Ferrando,\u0026#32; Sarti,\u0026#32; Bisazza\u0026#32;\u0026amp;\u0026#32;Costa-jussà (2024) Ferrando,\u0026#32; J.,\u0026#32; Sarti,\u0026#32; G.,\u0026#32; Bisazza,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Costa-jussà,\u0026#32; M. \u0026#32; (2024). \u0026#32;A primer on the inner workings of transformer-based language models. arXiv preprint arXiv:2405.00208. Figueroa-O’Farrill (2010) Figueroa-O’Farrill,\u0026#32; J. \u0026#32; (2010). \u0026#32;Spin geometry. Lecture Notes. University of Edinburgh.\u0026#32;Retrieved from\u0026#32; http://mat.uab.cat/~rubio/csa2017/SpinNotes.pdf Francis\u0026#32;\u0026amp;\u0026#32;Francis (1987) Francis,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;Francis,\u0026#32; G.\u0026#32; (1987). \u0026#32; A topological picturebook. \u0026#32; Springer. Gebhart (2023) Gebhart,\u0026#32; T. \u0026#32; (2023). \u0026#32; Sheaf representation learning \u0026nbsp;(PhD thesis).\u0026#32; University of Minnesota Gundry\u0026#32;\u0026amp;\u0026#32;McBride (2013) Gundry,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;McBride,\u0026#32; C. \u0026#32; (2013). \u0026#32;A tutorial implementation of dynamic pattern unification. Unpublished draft.\u0026#32;194. Hull (2020) Hull,\u0026#32; T.\u0026#32; (2020). \u0026#32; Origametry: Mathematical methods in paper folding. \u0026#32; Cambridge University Press. Izhikevich (2007) Izhikevich,\u0026#32; E.\u0026#32; (2007). \u0026#32; Dynamical systems in neuroscience. \u0026#32; MIT press.\u0026#32;Retrieved from\u0026#32; https://www.izhikevich.org/publications/dsn/ James\u0026#32;\u0026amp;\u0026#32;Liebeck (2001) James,\u0026#32; G.\u0026#32;\u0026amp;\u0026#32;Liebeck,\u0026#32; M.\u0026#32; (2001). \u0026#32; Representations and characters of groups. \u0026#32; Cambridge university press. Kashiwara\u0026#32;\u0026amp;\u0026#32;Schapira (2006) Kashiwara,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32;Schapira,\u0026#32; P.\u0026#32; (2006). \u0026#32; Categories and sheaves (). https://doi.org/10.1007/3-540-27950-4 Kock (2006) Kock,\u0026#32; A.\u0026#32; (2006). \u0026#32; Synthetic differential geometry. \u0026#32; Cambridge University Press.\u0026#32;Retrieved from\u0026#32; https://users-math.au.dk/kock/sdg99.pdf Kostecki (2009) Kostecki,\u0026#32; R. \u0026#32; (2009). \u0026#32; Differential geometry in toposes. \u0026#32; Citeseer.\u0026#32;Retrieved from\u0026#32; https://www.fuw.edu.pl/~kostecki/sdg.pdf Kostecki (2011) Kostecki,\u0026#32; R. \u0026#32; (2011). \u0026#32;An introduction to topos theory. Technial Report.\u0026#32;Retrieved from\u0026#32; https://www.fuw.edu.pl/~kostecki/ittt.pdf Krakauer,\u0026#32; Bertschinger,\u0026#32; Olbrich,\u0026#32; Flack\u0026#32;\u0026amp;\u0026#32;Ay (2020) Krakauer,\u0026#32; D.,\u0026#32; Bertschinger,\u0026#32; N.,\u0026#32; Olbrich,\u0026#32; E.,\u0026#32; Flack,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Ay,\u0026#32; N. \u0026#32; (2020). \u0026#32;The information theory of individuality. Theory in Biosciences,\u0026#32;139.\u0026#32;209–223. Lasenby (2024) Lasenby,\u0026#32; A. \u0026#32; (2024). \u0026#32;Some recent results for su (3) and octonions within the geometric algebra approach to the fundamental forces of nature. Mathematical Methods in the Applied Sciences,\u0026#32;47(3).\u0026#32;1471–1491.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2202.06733 Lux\u0026#32;\u0026amp;\u0026#32;Pahlings (2010) Lux,\u0026#32; K.\u0026#32;\u0026amp;\u0026#32;Pahlings,\u0026#32; H.\u0026#32; (2010). \u0026#32; Representations of groups: A computational approach. \u0026#32; Cambridge University Press. Mallios\u0026#32;\u0026amp;\u0026#32;Zafiris (2015) Mallios,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Zafiris,\u0026#32; E.\u0026#32; (2015). \u0026#32; Differential sheaves and connections: A natural approach to physical geometry. \u0026#32; World Scientific. Manturov (2018) Manturov,\u0026#32; V.\u0026#32; (2018). \u0026#32; Knot theory. \u0026#32; CRC press. Mikuła,\u0026#32; Antoniak,\u0026#32; Tworkowski,\u0026#32; Jiang,\u0026#32; Zhou,\u0026#32; Szegedy,\u0026#32; Kuciński,\u0026#32; Miłoś\u0026#32;\u0026amp;\u0026#32;Wu (2023) Mikuła,\u0026#32; M.,\u0026#32; Antoniak,\u0026#32; S.,\u0026#32; Tworkowski,\u0026#32; S.,\u0026#32; Jiang,\u0026#32; A.,\u0026#32; Zhou,\u0026#32; J.,\u0026#32; Szegedy,\u0026#32; C.,\u0026#32; Kuciński,\u0026#32; Ł.,\u0026#32; Miłoś,\u0026#32; P.\u0026#32;\u0026amp;\u0026#32;Wu,\u0026#32; Y. \u0026#32; (2023). \u0026#32;Magnushammer: A transformer-based approach to premise selection. arXiv preprint arXiv:2303.04488. Peng,\u0026#32; Goldstein,\u0026#32; Anthony,\u0026#32; Albalak,\u0026#32; Alcaide,\u0026#32; Biderman,\u0026#32; Cheah,\u0026#32; Ferdinan,\u0026#32; Hou,\u0026#32; Kazienko\u0026#32;\u0026amp;\u0026#32; (2024) Peng,\u0026#32; B.,\u0026#32; Goldstein,\u0026#32; D.,\u0026#32; Anthony,\u0026#32; Q.,\u0026#32; Albalak,\u0026#32; A.,\u0026#32; Alcaide,\u0026#32; E.,\u0026#32; Biderman,\u0026#32; S.,\u0026#32; Cheah,\u0026#32; E.,\u0026#32; Ferdinan,\u0026#32; T.,\u0026#32; Hou,\u0026#32; H.,\u0026#32; Kazienko,\u0026#32; P.\u0026#32;\u0026amp;\u0026#32; \u0026#32; (2024). \u0026#32;Eagle and finch: RWKV with matrix-valued states and dynamic recurrence. arXiv preprint arXiv:2404.05892. Petersen,\u0026#32; Pedersen\u0026#32;\u0026amp;\u0026#32; (2008) Petersen,\u0026#32; K.,\u0026#32; Pedersen,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32; \u0026#32; (2008). \u0026#32;The matrix cookbook. Technical University of Denmark,\u0026#32;7(15).\u0026#32;510.\u0026#32;Retrieved from\u0026#32; https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf Raissi,\u0026#32; Perdikaris\u0026#32;\u0026amp;\u0026#32;Karniadakis (2019) Raissi,\u0026#32; M.,\u0026#32; Perdikaris,\u0026#32; P.\u0026#32;\u0026amp;\u0026#32;Karniadakis,\u0026#32; G. \u0026#32; (2019). \u0026#32;Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics,\u0026#32;378.\u0026#32;686–707. S (2023) S,\u0026#32; R. \u0026#32; (2023). \u0026#32; Matrix representations of clifford algebras. \u0026#32;Retrieved from\u0026#32; https://www.cphysics.org/article/86175.pdf Reynoso (2023) Reynoso,\u0026#32; A. \u0026#32; (2023). \u0026#32;Probing clifford algebras through spin groups: A standard model perspective. arXiv preprint arXiv:2312.10071.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2312.10071 Roelfs\u0026#32;\u0026amp;\u0026#32;De Keninck (2023) Roelfs,\u0026#32; M.\u0026#32;\u0026amp;\u0026#32;De Keninck,\u0026#32; S. \u0026#32; (2023). \u0026#32;Graded symmetry groups: Plane and simple. Advances in Applied Clifford Algebras,\u0026#32;33(3).\u0026#32;30. Rosiak (2022) Rosiak,\u0026#32; D.\u0026#32; (2022). \u0026#32; Sheaf theory through examples. \u0026#32; MIT Press. Ruhe,\u0026#32; Brandstetter\u0026#32;\u0026amp;\u0026#32;Forré (2023) Ruhe,\u0026#32; D.,\u0026#32; Brandstetter,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Forré,\u0026#32; P. \u0026#32; (2023). \u0026#32;Clifford group equivariant neural networks. arXiv preprint arXiv:2305.11141. Schapira (2023) Schapira,\u0026#32; P. \u0026#32; (2023). \u0026#32; An introduction to categories and sheaves. \u0026#32;Retrieved from\u0026#32; https://webusers.imj-prg.fr/~pierre.schapira/LectNotes/CatShv.pdf Schellingerhout\u0026#32;\u0026amp;\u0026#32;Portegies (2023) Schellingerhout,\u0026#32; R.\u0026#32;\u0026amp;\u0026#32;Portegies,\u0026#32; J. \u0026#32; (2023). \u0026#32;Circles in synthetic differential geometry.\u0026#32;Retrieved from\u0026#32; https://pure.tue.nl/ws/portalfiles/portal/308028780/Thesis_BTW_Rob_Schellingerhout_Report_002_.pdf Sen\u0026#32;\u0026amp;\u0026#32;Zwiebach (2024) Sen,\u0026#32; A.\u0026#32;\u0026amp;\u0026#32;Zwiebach,\u0026#32; B. \u0026#32; (2024). \u0026#32;String field theory: A review. arXiv preprint arXiv:2405.19421. Sims (1994) Sims,\u0026#32; C.\u0026#32; (1994). \u0026#32; Computation with finitely presented groups. \u0026#32; Cambridge University Press. Spinner,\u0026#32; Bresó,\u0026#32; Haan,\u0026#32; Plehn,\u0026#32; Thaler\u0026#32;\u0026amp;\u0026#32;Brehmer (2024) Spinner,\u0026#32; J.,\u0026#32; Bresó,\u0026#32; V.,\u0026#32; Haan,\u0026#32; P.,\u0026#32; Plehn,\u0026#32; T.,\u0026#32; Thaler,\u0026#32; J.\u0026#32;\u0026amp;\u0026#32;Brehmer,\u0026#32; J. \u0026#32; (2024). \u0026#32;Lorentz-equivariant geometric algebra transformers for high-energy physics. arXiv preprint arXiv:2405.14806. Suarez (2019) Suarez,\u0026#32; R. \u0026#32; (2019). \u0026#32;Expository paper on clifford algebras, representations, and the octonion algebra. arXiv preprint arXiv:1906.11460.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/1906.11460 Taylor (2024) Taylor,\u0026#32; J. \u0026#32; (2024). \u0026#32;An introduction to graphical tensor notation for mechanistic interpretability. arXiv preprint arXiv:2402.01790. Wilson (2024) Wilson,\u0026#32; R. \u0026#32; (2024). \u0026#32;A discrete model for gell-mann matrices. arXiv preprint arXiv:2401.13000.\u0026#32;Retrieved from\u0026#32; https://arxiv.org/pdf/2401.13000 Woit,\u0026#32; Woit\u0026#32;\u0026amp;\u0026#32;Bartolini (2017) Woit,\u0026#32; P.,\u0026#32; Woit\u0026#32;\u0026amp;\u0026#32;Bartolini\u0026#32; (2017). \u0026#32; Quantum theory, groups and representations. \u0026#32; Springer. Yeasin (2011) Yeasin,\u0026#32; M. \u0026#32; (2011). \u0026#32;Initial algebra, final coalgebra and datatype. Yizengaw (2015) Yizengaw,\u0026#32; B.\u0026#32; (2015). \u0026#32; On clifford a-algebras and a framework for localization of a-modules. \u0026#32; University of Pretoria (South Africa). Zhang (2021) Zhang,\u0026#32; T. \u0026#32; (2021). \u0026#32;Type theories in category theory. arXiv preprint arXiv:2107.13242. ","permalink":"https://utensil.tngl.sh/blog/posts/math-2024/","summary":"I wish this post to be a continuously updated list of my math interests in 2024 with proper citations to literatures, as I keep wandering in the math wonderland and I don\u0026rsquo;t want to be lost in it without breadcrumbs.\nSome interests that have older origins will gradually moved to corresponding posts for earlier years.\nI also hope certain interests will be developed into research projects, and leaving only a brief summary and a link here.","title":"My math interests in 2024"},{"content":"The purpose of this post is to understand what is under the hood and the performance factors involved when fine-tuning and running local Transformer models, keeping multi-modality in mind, with an emphasis on the decoder-only transformers (e.g. GPT series).\nTo accomplish this, we first present a brief account of the transformer architecture, including its design intuitions and the underlying mathematics, concretized by illustrative diagrams and code snippets. Then we aim to achieve a comprehensive understanding of the widely adopted performance optimizations for the original transformer architecture.\nThis post was initially inspired by An Introduction to Transformers (Turner, 2023) 1, which provides a mathematically precise, intuitive, and clean description of the transformer architecture. Many other sources are incorporated and improved upon for clarity and self-containedness.\nNotations Notations used in this post try to be consistent with \u0026ldquo;The Transformer Family Version 2.0\u0026rdquo; (Lilian, 2023)2 and latest papers on performance optimizations such as FlashAttention, GPTQ, etc. , which deviate from (Turner, 2023)1 and (Phuong et al., 2022)3 in many places, notably the shape of input/output matrices.\nTo simplify further, this post will use a simplified version of the concise and powerful \u0026ldquo;Named Tensor Notation\u0026rdquo; from (Chiang et al., 2021)4 which has many advantages similar to Einops (Rogozhnikov, 2022)5. It\u0026rsquo;s seldom adopted in the literatures but very intuitive and easy to recover the widely adopted notations visually. Essentially, it\u0026rsquo;s a notation trying to avoid concerning the readers with complicated indices and order of them, by naming indices and only specify the indices involved in the operation (e.g. summations,reductions, contractions, reshaping, etc.).\nThe original \u0026ldquo;Named Tensor Notation\u0026rdquo; paper has a dedicated appendix for the transformer architecture, notably, it uses different names for similar concepts:\nthe \u0026ldquo;features\u0026rdquo;(named feat) become input/out for inputs/outputs key/val for key/value matrices layer in attention layers that covers all heads hidden in FFN hidden layers \u0026ldquo;tokens\u0026rdquo; (the dimension is named seq in the view of the whole sequence) sometimes rename to seq' to distinguish the input/output and this causes quite some confusion. In this post, we will consistently use feat and seq, and let the context to indicate the precise meaning, hinted by subscripts if necessary.\n$$ \\newcommand{\\namedtensorstrut}{\\vphantom{fg}} % milder than \\mathstrut \\newcommand{\\name}[1]{\\mathsf{\\namedtensorstrut #1}} \\newcommand{\\nbin}[2]{\\mathbin{\\underset{\\substack{#1}}{\\namedtensorstrut #2}}} \\newcommand{\\ndot}[1]{\\nbin{#1}{\\odot}} \\newcommand{\\ncat}[1]{\\nbin{#1}{\\oplus}} \\newcommand{\\nsum}[1]{\\sum\\limits_{\\substack{#1}}} \\newcommand{\\nfun}[2]{\\mathop{\\underset{\\substack{#1}}{\\namedtensorstrut\\mathrm{#2}}}} \\newcommand{\\ndef}[2]{\\newcommand{#1}{\\name{#2}}} \\ndef{\\ax}{ax} \\ndef{\\dd}{d} \\ndef{\\layer}{layer} \\ndef{\\seq}{seq} \\ndef{\\subseq}{subseq} \\ndef{\\key}{key} \\ndef{\\val}{val} \\ndef{\\heads}{heads} \\ndef{\\batch}{batch} \\ndef{\\inp}{input} \\ndef{\\hidden}{hidden} \\ndef{\\out}{out} \\ndef{\\height}{height} \\ndef{\\width}{width} \\ndef{\\chans}{chans} \\ndef{\\kernel}{kernel} \\ndef{\\kh}{kh} \\ndef{\\kw}{kw} \\ndef{\\vocab}{vocab} \\ndef{\\classes}{classes} \\ndef{\\state}{state} \\ndef{\\emb}{emb} \\ndef{\\feat}{feat} \\def\\attnbox#1{\\colorbox{#1}{\\color{#1} O}} \\def\\smtxt#1{\\text{\\small{#1}}} $$ Symbol Type/Shape Meaning $X^{(0)}$ $\\mathbb{R}^{\\seq \\times \\feat}$ The tokenized input sequence of tokens, length $N$, each token has $D$ features $X^{(m)}$ $\\mathbb{R}^{\\seq \\times \\feat}$ The output of the $m^{th}$ transformer layer $x^{(m)}_n$ $\\mathbb{R}^{1 \\times \\feat}$ The (row) vector of features for the $n^{th}$ token of $X^{(m)}$ $A^{(m)}$ $\\mathbb{R}^{\\seq \\times \\seq}$ The $N \\times N$ attention matrix To convey the types/shapes and the definition of operations simontaneously, sometimes we use a syntax like this:\n$$ f = op(a, b) \\quad \\in A \\to B \\to C $$ where\n$f$ is an operation accepting two operands of type $A$ and $B$ respectively, and returns a value of type $C$ $f$ is defined as applying some operation $do$ to $a$ and $b$ where $op$ can be arbitrary operation like $a + b$ $∈$ reads \u0026ldquo;has the type/shape of\u0026rdquo;, the last $→$ reads \u0026ldquo;maps to\u0026rdquo; and the ones before simply read \u0026ldquo;and\u0026rdquo; When the types of operands are clear, we could also write the above as:\n$$ f = op(a, b) \\quad \\in C$$ $$ % NxD matrices in diargams are assumed to be 4x3, i.e. 4 rows and 3 columns % an example element would be placed on [2, 1], i.e. the 3rd row and the middle column % a row vector, size 3, with the symbol placed at the center \\def\\rvec#1{{\\fbox{$ \\rule{2em}{0pt} {#1} \\rule{2em}{0pt} $}}} % a wide row vector, size 4, with the symbol placed at the center \\def\\wrvec#1{{\\fbox{$ \\rule{3em}{0pt} {#1} \\rule{3em}{0pt} $}}} % a column vector, size 4, with the symbol placed at the center \\def\\cvec#1{\\fbox{$ {#1} \\rule[-3em]{0pt}{6.5em} $}} $$ The input/output of a transformer A transformer is capable of ingesting \u0026ldquo;a sequence of tokens\u0026rdquo; and generating \u0026ldquo;a representation of the sequence\u0026rdquo;.\nHere, a token refers to a $D$-dimensional vector that represents a small unit of data, e.g. a (sub-)word in a sentence, or an 8x8 patch of a 512x512 image. These units are not fixed and can be chosen based on some domain-specific insight into the underlying structure of the data type.\nRepresenting a token as a vector is commonly called an embedding, and usually is learned to better express the token\u0026rsquo;s features and the relationships among tokens. Each dimension of a embedding vector is called a feature but it\u0026rsquo;s not nessarily a feature in the traditional sense such as an attribute of an object.\nTurning data into \u0026ldquo;a sequence of tokens\u0026rdquo; is a process called tokenization (we\u0026rsquo;ll discuss it in the next section). After tokenization, the input becomes a sequence of $L$ tokens of dimension $D$, thus can be collected into a $N \\times D$ matrix:\n$$ \\def\\Xinput{\\begin{bmatrix} \\vdots \\cr \\vdots \\cr \\rvec{x_n^{(0)}} \\cr \\vdots \\cr \\end{bmatrix}} X^{(0)} = \\underbrace{\\Xinput}_{D \\text{ columns (features) }}\\left.\\vphantom{\\cvec{}}\\right\\}N \\text{ rows (tokens)} $$ where $X^{(0)}$ denotes the input of the transformer, and $x_n^{(0)}$ is the (row) vector of features for the $n^{th}$ token.\nThe output, \u0026ldquo;a representation of the sequence\u0026rdquo;, takes the same form as \u0026ldquo;a sequence of tokens\u0026rdquo;, and the $n^{th}$ \u0026ldquo;token\u0026rdquo; is a $D$-dimensional vector representing the sequence at the location of token $n$.\nVarious tasks can be achieved by designing an appropriate training objectives to make use of the representation, e.g. auto-regressive prediction of the next $(n+1)^{th}$ token, global classification of the entire sequence (by pooling across the whole representation), sequence-to-sequence or image-to-image prediction problems, etc.\nThe input/output of a transformer is a powerful and versatile abstraction so that it can be used for mixing data of different modalities (images, texts etc.) and mixed tasks of different types.\nTokenization Texts For texts, the tokenization process first needs to choose a vocabulary that could cover almost all the words in the language, plus some meta-tokens like\nbos_token representing the beginning of sequence eos_token representing the end of sequence mask_token for masked language modelling etc. Then it needs to learn the embedding of each vocabulary element, and it would be clear in a bit that it also needs to learn the embedding of the position of each token in the sequence, called positional embedding, which I presume to be a geometric embedding in general. Latest researches, such as ALiBi (Press et al., 2022 )6, suggest that positional information can be encoded in forms other than positional embedding.\n(TODO: expand on the tokenization process, provide formulas for vocabularies, embeddings, and positional embeddings)\n(TODO: refer to subword tokenization literatures such as Byte Pair Encoding used in GPT-2, and cover other commonly used tokenization methods such as SentencePiece, as well as various ways to do positional embedding.)\n(TODO: explain commonly used tokenizers in popular models, particularly the treatment for non-English languages, but better as an appendix)\n(TODO: expand on the tokenization process for images such as in ViT, and for other modalities, better as an appendix or a separate post)\n(TODO: latest research suggests that tokenization is not necessary for transformers, this is called \u0026ldquo;end-to-end\u0026rdquo; training.)\nThe transformer layer A transformer is composed of multiple transformer layers. The representation of the input sequence will be produced by iteratively applying a transformer layer\n$$ X^{(m)} = \\text{TransformerLayer}(X^{(m-1)}) $$ where $X^{(m)}$ denotes the output of the $m^{th}$ transformer layer, and recall that $X^{(0)}$ naturally denotes the input of the transformer.\nEvery transformer layer comprises two stages (or sub-layers):\nStage 1 refines each feature across the whole sequence Stage 2 refines each token across features By repeatedly applying the transformer layer the representation at token $n$ and feature $d$ can be shaped by information at token $n'$ and feature $d'$ . This gives the transformer the ability to model long-range dependencies between tokens and features. Such a completeness is a key advantage of the transformer over other architectures but also poses challenges for efficient implementation, especially for long sequences. The performance pros and cons can be succinctly summarized by a quote from (Liu et al., 2018)7:\nThe lack of recurrence enables greater within-training-example parallelization, at the cost of quadratic complexity in the input sequence length.\nIn order to apply transformers to longer sequences, 7 proposed a decoder-only transformer that\ndrops the encoder module (almost reducing model parameters by half for a given hyper-parameter set), combines the input and output sequences into a single \u0026ldquo;sentence\u0026rdquo;.\nFor simplity, we\u0026rsquo;ll\neschew the superscripts $(m)$ denoting the layer index use $X, Y$ for the input/output of each stage they all have the same type $\\mathbb{R}^{\\seq \\times \\feat}$ i.e. shape $N \\times D$ use dummy symbols $\\mathtt{X}, \\mathtt{Y}, \\dots$ for operands and results when defining operations/functions Stage 1: self-attention across the sequence Stage 1 is based on the self-attention mechanism, which is a special case of the attention mechanism. There\u0026rsquo;re many types of attentions, see \u0026ldquo;A Family of Attention Mechanisms\u0026rdquo; for a summary.\nAttention The key idea of the attention mechanism is to infer by focusing on a given set of data, hence the name \u0026ldquo;Attention\u0026rdquo;, as in the famous quote \u0026ldquo;Attention is all you need\u0026rdquo; from (Vaswani et al., 2017)8. It\u0026rsquo;s initially introduced by (Bahdanau et al., 2015)9 for machine translation where the attention mechanism is used to focus on some related parts of the source sentence when translating a target word.\nFor visual tasks, the attention mechanism is often used to focus on a small region or some closely related regions of an image. For text tasks, it could be used to focus on the relationship between words in one sentence or close context. For multi-modal tasks, it could relate within a modality or between modalities.\nThe amount of attention is quantified by learned weights given by a so-called attention matrix $A \\in \\mathbb{R}^{\\seq \\times \\seq}$.\nWe can think of the attention weights as probabilities and the input sequence as a set of random variables. The attention weights represent the probability of each token attending to other tokens in the sequence. In this view, the output is the expected value of the input sequence with respect to the attention distribution:\n$$ \\begin{split} Y \u0026= \\mathbb{E}[Y|X] \\cr\\cr \u0026= \\nsum{\\seq}{p(Y|X) X} \\end{split} $$ This is equivalent to taking the weighted average of the input sequence using the attention weights, compactly written as a matrix product:\n$$ \\begin{split} Y \u0026= \\sum_{\\seq} A \\odot X \\cr\\cr \u0026= A \\underset{\\seq}{\\odot} X \\cr\\cr \u0026= A X \\end{split} $$ which is essentially doing the following for each feature $d$:\n$$ \\def\\Ynd{\\begin{bmatrix} \u0026 \\vdots \u0026 \\cr \u0026 \\vdots \u0026 \\cr \\cdots \u0026 Y_{\\seq, \\feat} \u0026 \\cdots \\cr \u0026 \\vdots \u0026 \\end{bmatrix}} \\def\\Xod{\\begin{bmatrix} \\cdots \u0026 \\cvec{X_{:, \\feat}} \u0026 \\cdots \\end{bmatrix}} \\def\\Ano{\\begin{bmatrix} \\vdots \\cr \\vdots \\cr \\wrvec{A_{\\seq, :}} \\cr \\vdots \\end{bmatrix}} \\Ynd_{N \\times D} = \\Ano_{N \\times N} \\times \\Xod_{N \\times D} $$ Here $A$ normalizes over each column $\\sum\\limits_{\\seq} A =1$.\nSpecifically, $A_{\\seq, \\seq_X}$ will take a high value for locations in the output sequence \u0026ldquo;$\\seq$\u0026rdquo; which are of high relevance for location \u0026ldquo;$\\seq_X$\u0026rdquo; in the input slice $X_{:, \\feat}$.\nThe following example of attention matrix demonstrates translating from the English sentence \u0026ldquo;eating a green apple\u0026rdquo; to the French sentence \u0026ldquo;manger une pomme verte\u0026rdquo; assuming only 1 feature for simplicity.\nNote that the 3rd token (\u0026ldquo;pomme\u0026rdquo;, i.e. \u0026ldquo;apple\u0026rdquo;) in the French sentence pays the greatest attention (marked by a black box) to the 4th token \u0026ldquo;apple\u0026rdquo; in the English sentence, and also attend to the 1st token \u0026ldquo;eating\u0026rdquo; and the 3rd token \u0026ldquo;green\u0026rdquo; in the English sentence (marked by grey boxes) because they\u0026rsquo;re related context to \u0026ldquo;apple\u0026rdquo; but almost none to the 2nd token \u0026ldquo;a\u0026rdquo; (marked by a lighter grey box).\n$$ \\def\\Ynd{\\begin{bmatrix} \\text{manger} \\cr \\text{une} \\cr \\fbox{pomme} \\cr \\text{verte} \\end{bmatrix}} \\def\\Xod{\\begin{bmatrix} \u0026 \\fbox{ $ \\begin{array}{c} \\text{eating} \\newline \\text{a} \\newline \\text{green} \\newline \\text{apple} \\end{array} $ } \u0026 \\end{bmatrix}} \\def\\Ano{\\begin{bmatrix} \\vdots \\cr \\vdots \\cr \\fbox{ \\attnbox{gray} \\attnbox{darkgray} \\attnbox{gray} \\attnbox{black} } \\cr \\vdots \\end{bmatrix}} \\Ynd_{4 \\times 1} = \\Ano_{4 \\times 4} \\times \\Xod_{4 \\times 1} $$ Self-attention Self-attention, also known as intra-attention, first introduced by (Cheng et al., 2016)10, has been shown to be very useful in machine reading, abstractive summarization, image description generation, etc. These motivating tasks all require a decent understanding of the input sequence itself.\nIt\u0026rsquo;s just a special case of the attention mechanism where the attention matrix corelates different tokens of the same input sequence, that is, to infer for one part of the data using other parts of the observation about the same data.\nThe attention matrix in the example above becomes something like:\n$$ \\begin{array}{c}\\smtxt{eating} \\cr \\smtxt{a} \\cr \\smtxt{green} \\cr \\smtxt{apple} \\end{array} \\overset{\\begin{array}{cccc}\\smtxt{eating} \u0026 \\smtxt{a} \u0026 \\smtxt{green} \u0026 \\smtxt{apple} \\end{array}}{ \\begin{bmatrix} \\attnbox{black} \\attnbox{lightgray} \\attnbox{lightgray} \\attnbox{gray} \\cr \\attnbox{lightgray} \\attnbox{black} \\attnbox{darkgray} \\attnbox{lightgray} \\cr \\attnbox{lightgray} \\attnbox{lightgray} \\attnbox{black} \\attnbox{darkgray} \\cr \\attnbox{gray} \\attnbox{darkgray} \\attnbox{gray} \\attnbox{black} \\end{bmatrix} } $$ Note that this matrix is asymmetric:\nsyntactically related: \u0026ldquo;apple\u0026rdquo; pays unilateral attention to \u0026ldquo;a\u0026rdquo; for its quantity \u0026ldquo;a\u0026rdquo; pays unilateral attention to \u0026ldquo;green\u0026rdquo; for whether it should be \u0026ldquo;an\u0026rdquo; instead semantically related: \u0026ldquo;apple\u0026rdquo; pays more attentions to \u0026ldquo;green\u0026rdquo; (which is describing it) than the other way around because \u0026ldquo;green\u0026rdquo; is a general color regardless what it\u0026rsquo;s describing To quantify the amount of attention, we can use the softmax function to give a (naturally normalized) probability distribution that the probability of each token is proportional to proportional to the exponent of a score (which will be calculated later):\n$$ A = \\nfun{\\seq}{softmax}(\\nfun{}{score}(X)) $$ where\n$$ \\nfun{\\seq}{softmax} \\mathtt{X} =\\frac{\\exp \\mathtt{X}}{\\nsum{\\seq} \\exp \\mathtt{X}} $$ A simple way of calculating the score from the input would be to measure the similarity between two tokens by their features.\nMathematically, it can be given by the dot product between the features of those two tokens and then use a softmax function to handle the normalization i.e.\n$$ \\nfun{}{score}(X) = \\nfun{\\seq}{softmax}(X \\ndot{\\feat } X) $$ However, this naïve approach handles solely the content similarity between tokens, treats all features of the tokens equally, which hasn\u0026rsquo;t taken into account how the tokens are organized and related in the sequence, and their in-context semantics (e.g. the word \u0026ldquo;crane\u0026rdquo; could mean a machine or an animal depending on the context).\nAn improvement is to perform the same operation on a linear transformation of the sequence, $U \\ndot{\\seq} X$, so that\n$$ \\nfun{}{score}(X) = (U \\ndot{\\seq} X) \\ndot{\\feat} ( U \\ndot{\\seq} X ) $$ The linear transformation will selectively project out some of the features in the input sequence and focus on other features, thereby the attention matrix will have a better chance capturing the relations between tokens in the context of the given sequence.\nHowever, this attention matrix is symmetric unlike the asymmetric version seen above, which is essential for expressing asymmetric relations. The solution is to use two different linear transformations to compute the similarity, i.e.\n$$ \\nfun{}{score}(X) = Q \\ndot{\\feat} K = (W_q \\ndot{\\seq} X) \\ndot{\\feat} ( W_k \\ndot{\\seq} X ) $$ Here we start using notations closer to (Vaswani et al., 2017)8. $Q$ and $K$ are called query matrix and key matrix, generated by the linear transformations $W_q$ and $W_k$, which are the learnable weights for them, respectively.\nIn a similar spirit(TODO: why?), we will also need a third linear transformation $W_v$ applied to the $X$ in $Y= A X$ to generate the value matrix $V$.\nThe concepts query/key/value originate from retrieval systems. Using these concepts, the self-attention mechanism treats the input sequence as a database. Thus the query focuses on the question or the instruction embeded in the input sequence, the key is an index of the context embeded in the input sequence, and the value is what the self-attention mechanism retrieve by correlating the query and the key.\nTo summarize, the self-attention mechanism now becomes:\n$$ Y = \\nfun{}{Attention}(X) = \\nfun{}{Attention}(Q, K, V) = A \\ndot{\\seq} V = \\left( \\nfun{}{softmax} ( Q \\ndot{\\feat} K ) \\right) \\ndot{\\seq} V $$ To improve numerical stability, we can divide the dot product by the square root of the dimension of the feature space, denoted $|\\feat|$, i.e.\n$$ \\nfun{}{Attention}(Q, K, V) = \\left( \\nfun{}{softmax} \\frac{ Q \\ndot{\\feat} K }{\\sqrt{ |\\feat| }} \\right) \\ndot{\\seq} V $$ To address the need to apply a mask to ensure attention is only applied to valid keys (e.g. for causal language models, we need to masking out the future words in the sequence to prevent the model from cheating by looking ahead at the words it is supposed to predict), we can add a mask matrix $M$ to the dot product, i.e.\n$$ \\nfun{}{Attention}(Q, K, V, M) = \\left( \\nfun{}{softmax} \\left( \\frac{ Q \\ndot{\\feat} K }{\\sqrt{ |\\feat| }} + M \\right) \\right) \\ndot{\\seq} V $$ where\n$$ M_{ij} = \\begin{cases} 0 \u0026 \\text{if $i \\leq j$} \\cr -\\infty \u0026 \\text{otherwise} \\end{cases} \\quad \\in \\mathbb{R}^{\\seq_i \\times \\seq_j} $$ Multi-head self-attention (MHSA) In retrospect, the self-attention mechanism looks at the same data with 2 different perspectives (i.e. focuses) for 2 different purposes (query and context), build a holistic perspectives on these 2 perspectives, then have a final look at the data. How it allocates the attention in the perspectives is determined by the similarity between selected features. And how to select the features is what is learned during training.\nIt\u0026rsquo;s immediately obvious that the ways to select features is limited, there is only one attention matrix, a global attention. It would be useful for pairs of tokens to be similar in some features and different in others, so the self-attention attention matrix can capture multiple drastically different relations between tokens.\nA typical example is understanding jokes. Jokes make use of puns, irony, misdirection, satire etc. They all invovle multiple distinct relations between tokens, or reinterpretation of the same tokens.\nAnother consideration is performance bottleneck. A global attention matrix for a long sequence such as pages of documents is computationally expensive yet sparse in nature thus obviously unoptimized. It would be more efficient to have multiple attention matrices, each of which is local and dense. Then corelate the relavant attention matrices to capture long-range relations.\nBased on these ideas, Multi-head self-attention (MHSA) applies $H$ sets of self-attention in parallel (termed $H$ heads) and then linearly projects the results to $Y$.\nThis is summarized in (Vaswani et al., 2017)8 as:\nMulti-head attention allows the model to jointly attend to information from different representation subspaces at different positions. With a single attention head, averaging inhibits this.\nNot only do individual attention heads clearly learn to perform different tasks, many appear to exhibit behavior related to the syntactic and semantic structure of the sentences.\nOr put it in other words, MHSA saves the orginal self-attnetion mechanism from being \u0026ldquo;one-track minded\u0026rdquo; i.e. overly focused on one way and unable to consider other perspectives or ideas. Particularly, this one way is doomed to be mediocre, and with the diversity from MHSA, the output is more likely to excel.\nIf we lift $Q$/$K$/$V \\in \\mathbb{R}^{\\seq \\times \\feat_{Q/K/V}}$ to $\\mathbb{R}^{\\heads \\times \\seq \\times \\feat_{Q/K/V}}$, shrinking $|\\feat_{Q/K/V}|$ to $|\\feat| / |\\heads|$, then MHSA can be defined as:\n$$ \\nfun{}{MHSA}(X) = W_o \\ndot{\\heads \\cr \\feat_V} \\nfun{}{Attention}(Q, K, V) $$ where $W_o \\in \\mathbb{R}^{\\heads \\times \\feat_V \\times \\feat}$ is for linearly projecting the results of the heads to the output $Y$.\nThe computational cost of MHSA is usually dominated by the matrix multiplication involving the attention matrix and is therefore $\\mathcal{O}(H D N^2)$.\nStage 2: position-wise FFN across features Stage 1 of the transformer layer transformed tokens to representations using the self-attention mechanism, but we\u0026rsquo;ll keep calling the representations \u0026ldquo;tokens\u0026rdquo; which is also interchangeable with \u0026ldquo;locations\u0026rdquo; in this post.\nStage 2 of the transformer layer operates across features, refining each token using a non-linear transform.\nTo do this, we simply apply a fully connected feed-forward network(FFN) to the vector of features for each token in the sequence, which consists of two linear transformations with an activation in between:\n$$ Y = \\nfun{}{FFN}(X) = (\\sigma(X \\ndot{\\feat} W_1 + b_1)) \\ndot{\\feat} W_2 + b_2 $$ where the activation function $\\sigma$ is typically the rectified linear unit (ReLU) but can be any non-linear function:\n$$ \\sigma(\\mathtt{X}) = \\operatorname{ReLU}(\\mathtt{X}) = \\max(0, \\mathtt{X}) $$ Note that the parameters of the FFN are the same for each token, i.e. it is applied to each position separately and identically (hence termed \u0026ldquo;Position-wise Feed-Forward Networks\u0026rdquo; in (Vaswani et al., 2017)8).\nThe FFN used typically have hidden-layers with dimension equal to the number of features D (or larger). The computational cost of this stage is therefore $\\mathcal{O}(N D^2)$.\nWire stages up To have a more stable model that trains more easily, we need to employ two techniques before producing the final output:\nResidual connection (denoted $\\oplus$): it\u0026rsquo;s simply adding the input to the output of each stage: $$Y = X + \\operatorname{Stage}(X)$$ Layer normalization (denoted $\\operatorname{LayerNorm}$ or $\\operatorname{LN}$): it could be applied post-residual8: $$Y = \\operatorname{LN_post}(X + \\operatorname{Stage}(X))$$ pre-stage (dorminant in practice): $$Y = X + \\operatorname{Stage}(\\operatorname{LN_pre}(X))$$ both: $$ Y = \\operatorname{LN_post}(X + \\operatorname{Stage}(\\operatorname{LN_pre}(X))) $$ where $\\operatorname{Stage}$ is $\\operatorname{MHSA}$ and $\\operatorname{FFN}$ for the 1st and 2nd stages, respectively.\nThe formulas become a bit too verbose with the parentheses, we can rewrite the last formula using a pipe operator $\\triangleright$:\n$$ Y = X \\triangleright \\operatorname{LN_pre} \\triangleright \\operatorname{Stage} \\triangleright (X + \\cdot) \\triangleright \\operatorname{LN_post} $$ Now we are ready to recover a architecture diagram of the transformer layer like the ones in literatures (but using only $\\operatorname{LN_pre}$):\nResidual connection The use of residual connections make initialization simple, have a sensible inductive bias towards simple functions, and stabilize learning (Szegedy et al., 2017)11.\nWith residual connections, each stage doesn\u0026rsquo;t directly learn the weights $\\theta$ of a large transformation $\\operatorname{Trans}_\\theta$ like below:\n$$ Y = \\operatorname{Trans}_{\\theta}\\left(X\\right) $$ Instead it learns the difference between the transformed result and the identity function, called the residual $\\operatorname{Res}_{\\theta}$. This way each stage applies a mild non-linear transformation to the input:\n$$ Y = X + \\operatorname{Res}_{\\theta}\\left(X\\right) $$ Over many layers, these mild non-linear transformations compose to form large transformations.\nLayer normalization LayerNorm (Ba et al., 2016)12 acts on each feature dimension independently, removing the mean and dividing by the standard deviation.\nPre-LN addresses the gradient vanishing/exploding problem(as nonlinearities are repeatedly applied through stages and layers), and is dorminant in practice. But pre-LN could result in representation collapse that can be addressed by post-LN, see a discussion for details.\n$$ \\def\\mttX{\\mathtt{X}} \\nfun{}{LayerNorm}(\\mttX; \\gamma, \\beta) = \\gamma \\odot \\frac{\\mttX - \\nfun{\\feat}{mean}(\\mttX)}{\\sqrt{\\nfun{\\feat}{var}(\\mttX) + \\epsilon}} + \\beta $$ where\n$$ \\nfun{\\feat}{mean} \\mttX = \\frac{1}{|\\feat|} \\nsum{\\feat}{\\mttX} $$ $$ \\nfun{\\feat}{var} \\mttX = \\frac{1}{|\\feat|} \\nsum{\\feat}{(\\mttX - \\nfun{\\feat}{mean} \\mttX)^2} $$ $ \\epsilon $ is a small constant added to the variance to avoid division by zero, and $\\gamma$ and $\\beta$ are learned parameters that scale and shift the normalized value.\nReferences Richard E. Turner, \u0026ldquo;An Introduction to Transformers\u0026rdquo;, arXiv:2304.10557 (2023)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nLilian Weng, \u0026ldquo;The Transformer Family Version 2.0\u0026rdquo;, Lil’Log (Jan 2023)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nMary Phuong and Marcus Hutter, \u0026ldquo;Formal algorithms for transformers\u0026rdquo;, arXiv:2207.09238 (2022)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nChiang et al. \u0026ldquo;Named Tensor Notation\u0026rdquo;, arXiv:2102.13196 (2021)\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAlex Rogozhnikov, \u0026ldquo;Einops: Clear and Reliable Tensor Manipulations with Einstein-like Notation\u0026rdquo;, ICLR 2022.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nPress et al. \u0026ldquo;Train Short, Test Long: Attention With Linear Biases Enables Input Length Extrapolation.\u0026rdquo; ICLR 2022.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nPeter Liu et al. \u0026ldquo;Generating wikipedia by summarizing long sequences\u0026rdquo;, arXiv:1801.10198, 2018.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nAshish Vaswani, et al. \u0026ldquo;Attention is all you need.\u0026rdquo;, NIPS 2017.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nDzmitry Bahdanau, et al. \u0026ldquo;Neural machine translation by jointly learning to align and translate\u0026rdquo;, ICLR 2015.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nJianpeng Cheng et al. \u0026ldquo;Long short-term memory-networks for machine reading\u0026rdquo;, arXiv:1601.06733 EMNLP 2016.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nChristian Szegedy et al., \u0026ldquo;Inception-v4, inception-resnet and the impact of residual connections on learning\u0026rdquo;, AAAI 2017.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nBa et al., Layer normalization, arXiv:1607.06450, 2016.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","permalink":"https://utensil.tngl.sh/blog/posts/transformer/","summary":"The purpose of this post is to understand what is under the hood and the performance factors involved when fine-tuning and running local Transformer models, keeping multi-modality in mind, with an emphasis on the decoder-only transformers (e.g. GPT series).\nTo accomplish this, we first present a brief account of the transformer architecture, including its design intuitions and the underlying mathematics, concretized by illustrative diagrams and code snippets. Then we aim to achieve a comprehensive understanding of the widely adopted performance optimizations for the original transformer architecture.","title":"Transformers: from self-attention to performance optimizations"},{"content":" @cogmission: Super AIs will be at \u0026ldquo;Enlightenment\u0026rdquo;, and at that level, I humbly assert in my small mind that the only truth there is - is unification; that we are all one and of the same fabric and integrity, wholeness - love - is all there is because it is the fundamental ingredient. It is where we come from.\nLove is real when we are in it, it lasts for only the rare tiny moments. It does last longer, but only in its subsided form. Just as @cogmission points out, Enlightenment \u0026quot; is constantly re-acquired (in the moment)\u0026quot;. It\u0026rsquo;s an emerge of multiple sources of perceptual cognition, rational knowledge, emotional impulses, and precious memories.\nPersonally, I am strongly affected by Reductionism and Materialism and would have the tendency to break things down to their building blocks and consider the interactions and dynamics.\nFor example: Love can be viewed biologically as just the result of secreted chemicals which fade faster than pure spiritual form in our belief; Love can also be viewed psychologically as seeking similar elements to our self-projection in others or the resolution of our kink, and they cannot evolve covariantly over time.\nI would also break Ethics into some form of Utilitarianism, consider morality as some form of wisdom, consider EQ as some form of IQ, and the list goes on. It\u0026rsquo;s not that I don\u0026rsquo;t have faith or enthusiasm in the pure spiritual form of these humanitarian things, it\u0026rsquo;s just I feel the need of a more solid ground for preventing them to be invalidated by considerations from a realistic point of view.\nThis is the background why I won\u0026rsquo;t jump from rapid development of AI to the singularity theory, nor from a strong AI to a super and all-power AI, nor from mental power to love and kindness. The far ends are on the extended line of logic, but there are huge gaps of many logical steps and too many variables. There\u0026rsquo;s no logical inevitability but many alternate history branches.\nThat\u0026rsquo;s why I\u0026rsquo;m forced to settle on an optimistic view for AI which also takes all these real and potentially catastrophic dangers into account. And I\u0026rsquo;d like to end this post with the following quotes:\nBran thought about it. \u0026lsquo;Can a man still be brave if he\u0026rsquo;s afraid?\u0026rsquo; \u0026lsquo;That is the only time a man can be brave,\u0026rsquo; his father told him.\nGeorge R.R. Martin, A Game of Thrones ⧉\n","permalink":"https://utensil.tngl.sh/blog/posts/on-love-in-the-context-of-super-ai/","summary":"@cogmission: Super AIs will be at \u0026ldquo;Enlightenment\u0026rdquo;, and at that level, I humbly assert in my small mind that the only truth there is - is unification; that we are all one and of the same fabric and integrity, wholeness - love - is all there is because it is the fundamental ingredient. It is where we come from.\nLove is real when we are in it, it lasts for only the rare tiny moments.","title":"On Love in the Context of Super AIs"},{"content":"I believe the development of technology will eventually solve problems \u0026ldquo;introduced\u0026rdquo; by technology, one way or another, as long as we\u0026rsquo;re lucky to be in a timeline (in the context of multi-universe) that doesn\u0026rsquo;t include self-destruction. Let\u0026rsquo;s have the spirit of adventurers, and take our chances.\nNew technologies would always give humanity new ways to exercise our dark side, so the risk is always there. The risk feels high because its possible catastrophic consequences, but not the probability.\nIt might seem immoral to bind the fate of ordinary people to the ship of technology. On the contrary, it\u0026rsquo;s even more immoral to restrict the whole humanity just because of the concern of survival, especially for the generations to come. Put a constraint on the development of technology will not only take away the potential quality of their lives but also make them vulnerable and unprepared for the inevitable higher level of survival challenges to come.\nThe singularity is just another bottleneck in human history, just as the ones we left behind. It\u0026rsquo;s difficult for us who live in the ease that modern technologies provide to imagine the horrifying situation without them and the twisting and winding path lead us today.\nThe following quote from a Quora answer gives a historical perspective to this future issue:\n4.5 million tons of manure were being dropped on the streets of Manhattan in 1890, EVERY YEAR, by horses carrying people to work.\nThat was the big environmental problem of the day. \u0026ldquo;NYC will be buried in horse manure by 1950!\u0026rdquo; screamed the headlines.\nIt doesn\u0026rsquo;t matter what your opinion about this was. None of the people living in NY solved the problem despite the 1000s of opinions.\nPeople with passion for mechanics in Detroit made something called a car.\nProblem solved.\nDo what YOU love to do today. Surrender to the results. The more you surrender, the more results there will be.\nThe way you solve the world\u0026rsquo;s problems is to solve your problems. Then trust.\nThat\u0026rsquo;s also why we need to embrace new creations and advances as early as possible. The earlier we do so, the more prepared we can be. Also, the incautious damage in the process would also be minimalized.\nSometimes, \u0026ldquo;whether\u0026rdquo; is never the issue, \u0026ldquo;how\u0026rdquo; is.\n⧉\n","permalink":"https://utensil.tngl.sh/blog/posts/on-tech-dev-and-doomsday/","summary":"I believe the development of technology will eventually solve problems \u0026ldquo;introduced\u0026rdquo; by technology, one way or another, as long as we\u0026rsquo;re lucky to be in a timeline (in the context of multi-universe) that doesn\u0026rsquo;t include self-destruction. Let\u0026rsquo;s have the spirit of adventurers, and take our chances.\nNew technologies would always give humanity new ways to exercise our dark side, so the risk is always there. The risk feels high because its possible catastrophic consequences, but not the probability.","title":"On the Development of Technology and the Doomsday"}]