Skip to content
trendoviz

Formulas and thresholds

The technical reference behind the methodology: every number below is read from the default rules in the code.

Methodology 2.1 · score formula score-1.2 · lifecycle rules lifecycle-3 · attention/substance buzz-1.1 · updated 2026-11-21. Administrators may override values; overrides are recorded in the audit log, and every snapshot stores the formula version used.

Conventions

  • Time is UTC. Windows are rolling (measured back from the evaluation time), not calendar weeks; daily snapshots are keyed by UTC date.
  • A signal is dated by its publication time, or by its collection time when the source gives none. Duplicates and deleted signals never count; corrected or late signals are counted at their publication date on the next evaluation (evaluations run every 15 minutes and after each collection).
  • Null is not zero. A metric that cannot be measured is reported as unavailable with a reason (insufficient history, low volume, collection outage). Zero means “measured, and nothing happened”.
  • Every component is rounded to an integer between 0 and 100. Weights are normalised by their sum.

Trendoviz Score (score-1.2)

Score = Σ weight × component / Σ weights, then capped by confidence. Weights: velocity 30%, diversity 25%, recency 15%, persistence 15%, authority 10%, novelty 5%.

  • Velocity: r = signals in the last 7 days, p = the 7 days before. 0 when r = 0; otherwise clamp(50 + 25 × log2((r+1)/(p+1)) × v), where v = min(1, (r+p)/5) damps small numbers. The +1 avoids a zero denominator.
  • Source diversity: n = distinct publishers in the last 30 days; 100 × (1 − 0.6n). One publisher = 40, two = 64, three = 78, five = 92; it saturates towards 100.
  • Recency: 100 × 0.5days since the latest signal / 3.
  • Persistence: 50 × active days in the last 14 / 14 + 50 × active weeks in the last 4 / 4.
  • Source authority: best tier per publisher in the diversity window (official or tier 1 = 100, tier 2 = 70, tier 3 = 40); 0.6 × best + 0.4 × mean.
  • Novelty: 100 − title similarity with published trends (×100), −50 if the same project already has a published trend, −20 if first seen more than 60 days ago.
  • Confidence caps: at most 60 with 1 distinct piece of evidence (independence layer 5 below), 80 with 2. High activity cannot compensate for weak evidence.

Worked example. Five signals: an official one 12 hours ago, then 1, 2, 3 and 9 days ago, from three publishers, novelty 80. r = 4, p = 1, v = 1 → velocity 83; diversity 78; recency 89; persistence 50 × 5/14 + 50 × 2/4 = 43; authority 0.6 × 100 + 0.4 × 80 = 92; novelty 80. Score = 0.30×83 + 0.25×78 + 0.15×89 + 0.15×43 + 0.10×92 + 0.05×80 = 77. No cap applies (three publishers). This example is an executable test.

Cold start and missing history

  • Observation history = days since the first successful collection of the trend's earliest-observed source.
  • Velocity and the 7-day change need 14 days of observation; persistence needs 14. Before that they are shown as “Insufficient history”, never as 0.
  • The 7-day change is reported only when both weeks together hold at least 5 signals (low-volume protection); otherwise “Too few signals”.
  • A source that went more than max(24 h, 4 × its collection interval) without a successful run has an outage. If every source of a trend had one in the last two weeks, the change is shown as “Collection gap”: the catch-up spike after recovery is not momentum.
  • Backfill: items published more than 72 hours before collection, or undated items from a source's first run. They count as evidence; undated backfill never enters the velocity windows.
  • Assessment: full (everything measured), partial (one history metric unavailable) or provisional (neither velocity nor persistence measurable). The stored score keeps its score-1.2 formula; the assessment tells readers how complete it is, and provisional scores carry a warning wherever they appear.

Lifecycle stages (lifecycle-3)

Evaluated in this order; the first match is the day's candidate. The 7-day change is (r − p) / max(p, 1) × 100. “Growth guard” = the change is measurable (history, no blinding outage) and meaningful: r + p ≥ 5 and |r − p| ≥ 3.

  1. Dormant: no signal for 14 days. Applies at once.
  2. New: first seen less than 2 days ago.
  3. Breakout: growth guard, ≥ 3 distinct pieces of evidence, score ≥ 75 (stay ≥ 65) and change ≥ +50% (stay ≥ +20%).
  4. Rising: growth guard, r ≥ 3, change ≥ +25% (stay ≥ +10%).
  5. Cooling: growth guard, change ≤ -30%, and currently rising, breakout, established or cooling.
  6. Established: persistence measurable, age ≥ 21 days, persistence ≥ 50, ≥ 3 distinct pieces of evidence.
  7. Emerging: otherwise.

Hysteresis: a new stage is applied only after it has been the candidate for 2 consecutive daily evaluations, except dormant and leaving new, which apply at once. Provisional trends can only be new, emerging or dormant. Every applied change stores its metrics and rule version.

Example transitions. 1 → 3 signals (+200%) fails the growth guard: the trend stays emerging. 2 → 6 (+200%) with history passes: candidate rising, applied the next day if it holds. A rising trend at +9% (|Δ| ≥ 3) loses rising and becomes emerging or established.

Attention and substance (buzz-1.1)

  • Attention = 50% coverage + 30% growth + 20% breadth, over non-official signals. Coverage = 100 × (1 − 0.75recent); growth = 50 + 25 × log2((recent+1)/(prior+1)) × min(1, (recent+prior)/5) (0 without recent coverage); breadth = 100 × (1 − 0.6outlets). Window 7 days.
  • Substance = 60% weight + 40% breadth over 30 days. Points per official item: announcement 1, research 1, policy 1, release 0.6, tutorial 0.5, business 0.4, news 0.3, analysis 0.2, discussion 0; research and policy from other sources × 0.5. Within a publisher items are sorted by points, highest first, and each next one counts × 0.5. Weight = 100 × (1 − 0.6points), breadth = 100 × (1 − 0.6publishers). One publisher alone is capped at 45.
  • Each signal has one content type and one source type, so types do not overlap. A paper reported by media counts as attention and as (reduced) substance, by design.
  • Readings: attention ≥ 50 and substance ≥ 50 = high attention & substance (API name attention_and_substance); attention only = attention-led (attention_led); substance only = under the radar; neither = quiet. The thresholds are design choices, not validated cut-offs.

Worked example. Media coverage: 2 items this week from 2 outlets, 1 the week before; two releases from one GitHub organisation. Coverage 44, growth 59, breadth 64 → attention 53. Substance: 0.6 + 0.3 = 0.9 points → weight 37, breadth 40 → 38. Reading: attention-led. Tested in code.

Independence layers

  1. Signals: items.
  2. Feeds: configured sources.
  3. Publishers: the host an item belongs to (an aggregator item counts as the site it links to; GitHub releases count per organisation).
  4. Ownership groups: publishers under one owner or editorial control (configured, e.g. Condé Nast: Wired and Ars Technica).
  5. Underlying evidence: copies of one press release or newswire story, identical long headlines from different publishers (≥ 8 words), and stories that cite the same outlet are one piece of evidence.
  6. Genuine corroboration: evidence groups that do not copy the subject's own announcement and are not controlled by the subject.

Since score-1.2 and lifecycle-3, the trend confidence label (unconfirmed, corroborated, strong, broad), the score caps and the lifecycle thresholds count distinct evidence (layer 5), not publishers. The “Independent sources” figure still shows publishers; the trend page shows all six layers.

Grouping

  • Rules first (first match wins): exact duplicate, same software release, same canonical link, same project with an open trend, keyword similarity ≥ 0.34 with ≥ 2 shared keywords within 14 days.
  • Embeddings (service model recorded with every vector): duplicate ≥ 0.97, attach to a trend ≥ 0.86, new group ≥ 0.87, same source ≥ 0.9; between 0.8 and 0.86 the AI is asked. Thresholds were tuned on BAAI/bge-small-en-v1.5; vectors of another model are ignored and recomputed when the model changes.
  • AI proposals are recorded with their reason; uncertain ones go to the editorial queue. Editors merge, split, attach and detach signals; each action is audited.
  • Evaluation fixtures measure false merges, missed matches, precision and recall of the rule stage (validation status).