- 5 min read
- 8 key claims
- 1 source cited
- For Developers, Businesses
Overview
Together Storage Accelerator (TSA) is a distributed caching layer for Together GPU Clusters. It runs on every GPU node, caches objects on cluster NVMe after the first read, and exposes S3-compatible and Hugging Face Hub endpoints. Existing boto3 code, transformers calls, and command-line workflows can use these endpoints without a new SDK or rewritten data loaders, and users can enable TSA from the cluster Settings page [1].
TSA works with S3-compatible storage, including Amazon S3, Cloudflare R2, and Together AI's T4 managed object storage, without requiring data migration. Together AI says configured credentials are encrypted and stored in a secrets vault [1]. The company describes TSA as built on ASTRA, its distributed caching technology, but the scale and production reach claimed for ASTRA are not independently substantiated [1].
Why it matters
TSA addresses an infrastructure-layer problem rather than introducing a separate application product. By placing cached data on node NVMe and retaining familiar storage endpoints, it targets repeated downloads that can leave accelerators waiting for data without requiring teams to replace their storage layer or data pipelines [1].
For organizations, the feature makes data placement part of cluster configuration and could reduce repeated origin fetches when the same checkpoint or dataset is reused across nodes. Together AI's account team describes the 200-node case as one origin fetch with TSA instead of 200 without it, which would reduce duplicated transfer volume [1].
Opportunities
Potential uses include:
- Keeping frequently reused checkpoints and datasets available within a cluster through its local cache [1].
- Adding cached storage endpoints to existing data pipelines without replacing their object-storage layer or rewriting loaders [1].
- Prototyping systems that need to keep reading cached objects during temporary upstream outages or rate limits [1].
- Combining cluster-local caching with locally hosted object storage as T4 becomes more broadly available [1].
These are possible applications, not documented customer outcomes. The available evidence identifies no third-party products built on TSA.
Risks
Performance variability: Together AI says results depend on the origin, object size, network path, and region, and reports that one Hugging Face download ran five times faster from us-central than from ca-west [1].
Reliability and resilience: The claim that cached data continues serving during an upstream outage is vendor-reported. The evidence does not document what happens when the cache itself is unavailable, how failures propagate, or whether workload correctness is affected by stale cached objects.
Platform dependency: TSA is tied to Together GPU Clusters, while T4 remains in limited preview and is available through a sales or account team [1]. This may create an operational dependency on Together AI's infrastructure and roadmap.
Adoption and operating requirements: The evidence gives no information about cache eviction, retention, storage-capacity limits, or sustained cache-hit behavior. Together AI's suggestion that users can go from zero to accelerated in under ten minutes is not supported by timing data [1].
Security and legal exposure: The evidence states that configured credentials are encrypted and stored in a secrets vault, but it includes no audit, compliance, or vulnerability information [1]. The legal implications of caching third-party model or dataset artifacts, including any associated licensing or acceptable-use restrictions, are not addressed.
Context
AI training and inference clusters often fetch models, datasets, and checkpoints from remote object stores or the Hugging Face Hub. Repeated reads across nodes can add startup delays, duplicate origin traffic, and expose workloads to origin rate limits. Together AI says the same Hugging Face download ran five times faster from its us-central region than from ca-west, illustrating how origin distance and network path affect these workloads [1].
Together AI's T4, its managed S3-compatible object storage, remains in limited preview and is available through its sales or account team. The company positions it as a way to keep warm data close to GPUs with one configuration change [1].
Known, claimed, inferred
The launch and documented mechanics are directly verifiable in Together AI's announcement. The performance and savings figures are vendor claims rather than independently established results. Together AI reports roughly 20x faster warm reads, up to 3.4 GB/s per node, 60–80x lower latency in two small-file cases, a cold read within about 5% of a direct download, and one origin fetch instead of 200 for a shared checkpoint [1].
A reasonable inference is that TSA's greatest value is likely to appear in clusters with high reuse of the same objects and origins with slower or rate-limited access. This follows from the caching design and the vendor's own observation that the benefit depends on origin speed and network path; it is not a measured segmentation of customer workloads [1]. Another inference is that reducing redundant fetches could improve both pipeline efficiency and exposure to origin-transfer charges, but the evidence does not quantify either outcome beyond the reported tests [1].
Limitations & uncertainties
All performance figures come from Together AI's own measurements; no independent evaluation is present. The principal benchmark covered one H100 node in one us-central preview deployment, using single-stream curl and aws s3 cp commands and repeated reads of the same object verified by MD5 against direct download. Results for production deployments, other GPU types, other regions, multi-stream workloads, and write-heavy or cache-miss-heavy patterns are absent [1].
The 200-node origin-fetch and egress figures lack a separate methodology description or raw measurement record. No Together GPU Clusters pricing, total-cost-of-ownership analysis, or quantified egress-cost savings are given. Together AI notes that AWS egress charges can reach $0.09/GB, which provides context for origin-transfer costs rather than a measured user saving [1].
Cache eviction, retention, capacity limits, cache-hit-rate behavior over time, failure modes, and cache-unavailability behavior are not documented. The setup-time claim has no supporting timing data, and the scale and production reach attributed to ASTRA are not independently substantiated [1].
Evidence & sources
The key statements of this analysis, labeled by type, with the sources they rely on. Trendoviz does not run product tests.
- Primary-source or vendor statementTogether AI launched Together Storage Accelerator for Together GPU Clusters on November 19, 2026, and includes it at no additional cost.[1]
- Primary-source or vendor statementTSA runs on every GPU node, caches objects on cluster NVMe after the first read, and exposes S3-compatible and Hugging Face Hub endpoints without application changes, a new SDK, or rewritten data loaders.[1]
- Primary-source or vendor statementTogether AI reports roughly 20x faster warm reads than a cross-cloud origin, reaching up to 3.4 GB/s per node.[1]
- Primary-source or vendor statementIn Together AI's test, a 2 GiB Cloudflare R2 object took 12.7–15.0 seconds without TSA and 0.64–0.72 seconds with a warm cache.[1]
- Primary-source or vendor statementTogether AI reports that a 200-node cluster pulling the same 70 GB checkpoint made one origin fetch with TSA instead of 200 without it.[1]
- Primary-source or vendor statementTogether AI states that a cold read through TSA completed within about 5% of the time of a direct download.[1]
- Primary-source or vendor statementTogether AI says cached data continues to serve when the upstream origin is unavailable or rate-limited.[1]
- Primary-source or vendor statementThe principal measurements were taken on one H100 node in a us-central preview deployment using TSA chart 0.6.0.[1]
Sources
- 1Introducing Together Storage Accelerator: fast, free data caching for your GPU cluster●together.airetrieved Nov 19, 2026
Updates & corrections
- First published · published automatically after passing every check
Found an error? Report it.