← All writing

Markdown element sample

This page contains a paragraph with strong text, emphasized text, struck text, inline code, an inline link, and inline math pθ(xt∣x<t)p_\theta(x_t \mid x_{<t}).NoteMargin notes are anchored to the exact phrase they clarify.

A second-level heading

A third-level heading

A fourth-level heading

A fifth-level heading
A sixth-level heading

A blockquote for quotations, excerpts, or a thought that deserves a distinct rhythm.

DefinitionAttention head

An attention head maps a query and a set of key-value pairs to a weighted sum of the values:

Attention⁡(Q,K,V)=softmax⁡ ⁣(QK⊤dk)V.\operatorname{Attention}(Q, K, V) = \operatorname{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V.
RemarkWhy keep a remark collapsible?

Remarks carry useful context that is not required for the main argument. Collapsing them preserves the article's reading rhythm while leaving the detail close at hand.

  • An unordered list item
  • A second item with emphasis
  • A completed task item
  • An incomplete task item
  1. A numbered step
  2. Another numbered step
  3. A final numbered step
python
def attention(query, key, value):    # A deliberately small example.    scores = query @ key.T    return softmax(scores) @ value

Display mathematics occupies its own line:

L(θ)=−∑t=1Tlog⁡pθ(xt∣x<t)\mathcal{L}(\theta) = -\sum_{t=1}^{T} \log p_\theta(x_t \mid x_{<t})
ElementExample use
Table rowComparing related ideas
Another rowOrganizing compact data

Image placeholder

The placeholder can carry a source citationCitationVaswani et al. Attention Is All You Need. 2017. arXiv without interrupting the article.

Progress markers can separate writing sessions without implying that the post is finished.