Skip to content
trendoviz

crewAI adds headless eval and ChromaDB decoupling in six-release push

crewAI's 1.15.38–1.15.43 sequence adds headless evaluation, clearer failure signals, and storage decoupling from ChromaDB. The evidence is limited to official GitHub release notes, which name `gpt-6-luna` as the new default model without identifying its provider, availability, or pricing.

52 out of 100Emerging7D: Insufficient history ProvisionalPublished

Overview

Version 1.15.38 is the release with the most user-visible changes. It adds a --no-open flag for crewai eval so evaluations can run without opening a browser, and tags eval usage spans with a target and a per-evaluation ID. An agent's input can now reference a spreadsheet URL that tools read row by row; the notes do not describe automatic ingestion, row-to-task mapping, or behavior when a spreadsheet is unreachable [1].

Failure reporting is specified in two places: a crewai eval run that exits with an error is counted as run_failed, and crewai run exits non-zero when its project script fails [1]. Other fixes create the project .env file with owner-only permissions when eval turns tracing on, check an OpenAI-compatible provider's API key on the first call, and scope memoized crew methods to their instance so separate crews in one process do not share cached results [1]. The release also documents required_inputs on GET /inputs and which Flow fields count as inputs [1].

One change may affect existing projects directly: gpt-6-luna becomes crewAI's default model everywhere [1]. The release material does not identify a provider, model card, availability, pricing, or data-handling terms for this model, and no later note in the sequence mentions it [1].

Why it matters

The clearest practical change is the set of behaviors that only matter when nobody is watching. A browser-free evaluation path, a defined run_failed status, a non-zero process exit on a failed project script, first-call API key checks, and per-instance memoization all affect how a crew behaves inside a scheduler, a CI job, or a service that hosts more than one crew [1]. The release notes do not describe the design intent behind these changes, so this reading of their practical effect is inferred from the wording of the entries rather than measured.

The storage sequence matters for teams that have already committed to a vector backend. Removing chromadb from the embedding function layer and then making it an optional extra with local Qdrant as the Knowledge default reduces the number of components tied to one database, but the notes are silent on how existing ChromaDB installations should move [2][3]. A team considering the change has to work out that path itself.

The evaluation entries also have a data-handling dimension. Uploading a failed anonymous run that crewai eval started improves failure visibility, but it also means a failed run may leave the originating machine. The notes do not say what is uploaded, where it is stored, or how to disable it [1].

Opportunities

  • Decide what runs unattended. The headless eval path, explicit run_failed counting, and non-zero exit from crewai run are the pieces to evaluate if evaluations belong in a scheduled job rather than an interactive session [1]. Whether they meet a given team's reliability needs is untested here.
  • Choose a storage position before upgrading. Confirm whether existing ChromaDB deployments have a migration route, since the notes do not provide one [2][3].
  • Check the new default model before accepting it. Confirm provider, pricing, and data-handling terms; the release notes do not supply them, so existing projects inherit a model whose terms cannot be evaluated from this material [1].

Risks

The new default model is thinly documented. The release material names gpt-6-luna without identifying a provider, a model card, availability, or pricing. Teams running existing projects on the new default have insufficient information to evaluate the change from the release notes alone [1].

Failed runs may leave the machine. The upload of a failed anonymous crewai eval run is listed as a fix, which suggests prior failures were not visible; the same behavior means data from a failed run can be transmitted, with no documented opt-out, destination, or payload description [1].

The storage default changes the deployment footprint. Knowledge defaulting to local Qdrant and ChromaDB becoming an optional extra alters the dependency set for installed projects. The notes do not describe resource requirements or the cost of running the new default locally [2].

A dependency floor can conflict with pinned environments. Raising the soupsieve floor to 4.0 for CVE-2026-126190 is a security-motivated change that may not resolve in environments that pin an older version; the advisory's details are not given beyond the CVE identifier [2].

Private URL hosts widen the input surface. Allowing approved private URL hosts extends where agents can fetch data, and the notes do not describe how approval works or what access controls apply [4].

Context

crewAI describes itself as a framework for orchestrating role-playing, autonomous AI agents so that agents can work together on complex tasks — the project's own framing, not an independently evaluated description [1]. The six releases covered here run from 1.15.38 to 1.15.43 and are all signed with GitHub's verified signature under GPG key B5690EEEBB952194 [1][2][3][4][5][6]. The release notes do not provide publication dates, so the interval between versions is unknown.

The changes divide into two broad areas: making crews behave predictably when run unattended, and loosening the framework's dependence on one vector database. The storage work is sequential. Version 1.15.40 removes chromadb from the embedding function layer [3]. Version 1.15.42 then makes ChromaDB an optional extra and defaults Knowledge to local Qdrant while decoupling storage from ChromaDB [2]. The notes do not describe a migration path for existing ChromaDB deployments.

What changed in the follow-up releases

  • 1.15.39 is documentation-only: it describes the apps_not_connected error on /kickoff and /status [5].
  • 1.15.40 removes chromadb from the embedding function layer, adds exact historical state lookup by row ID, wires jsonl into RAG data type discovery, scaffolds standalone agents as JSON by default, and adds backend record lifecycle and pagination tests. Bug fixes run after_llm_call hooks on async calls and stop an agent whose apps CrewAI AMP cannot resolve [3].
  • 1.15.41 allows approved private URL hosts. Fixes move projects to PyMuPDF 2.x, retain earlier step outputs across a human-feedback resume, and strip the cache_breakpoint marker on the LiteLLM path [4].
  • 1.15.42 makes ChromaDB an optional extra, defaults Knowledge to local Qdrant, and lets standalone agents have tools picked and added later. Its fixes include honoring max_execution_time without waiting for the worker, giving each Flow its own ID from an initial_state instance, saving task outputs containing URLs and other Pydantic types, counting scoped LanceDB records exactly above 50,000 rows, normalizing flat Chroma metadata filters, and raising the soupsieve floor to 4.0 for CVE-2026-126190 [2].
  • 1.15.43 fixes detection of multimodal support for Claude 5 [6].

Known, claimed, inferred

Pattern in the releases. Across 1.15.38 to 1.15.43, the changes cluster around unattended operation and storage portability: headless evaluation, defined exit codes, per-evaluation usage attribution, per-instance memoization, and the staged removal of ChromaDB from the embedding layer and default Knowledge stack [1][2][3]. Reading this sequence as a reliability push is an inference from the release-note wording, not a claim made by the project.

A second pattern sits in the data-handling fixes. Several entries address format and boundary conditions rather than headline features: normalizing flat Chroma metadata filters, handling CSV record newlines without translating cell values, loading JSON crew and agent files saved with a UTF-8 BOM, and counting scoped LanceDB records exactly above 50,000 rows [2]. This suggests attention to data-formatting edge cases, but no performance or correctness measurements accompany any of these changes.

On evidence quality. None of the changes has been tested independently. There is no third-party test report or benchmark among these releases, so every statement about reduced friction or improved operability is inferred from how the notes are worded. The backend record lifecycle and pagination tests added in 1.15.40 are the project's own and report no results [3].

On provenance. The 1.15.42 and 1.15.43 pages both carry a "Latest" label. The retrieval snapshot may have captured that label at different times, and the pages offer no explanation. Because no dates are available, release order and cadence cannot be confirmed from this material [2][6].

Limitations & uncertainties

  • The evidence consists entirely of official GitHub release notes for versions 1.15.38 through 1.15.43. No independent testing, third-party audit, benchmark, or user report is available, so no change in this sequence has an independently verified effect [1].
  • No release dates or tag publication times are provided, so the cadence across these six versions is unknown [2][6].
  • Adoption is not measurable from this material: install counts, download figures, usage volume, dependent projects, and contributor counts beyond the handles listed per release are unknown.
  • The notes do not describe design rationale, scope, or intended impact for any change, and they do not state whether the gpt-6-luna default was later altered or reverted [1].
  • The retrieval snapshot may have captured the "Latest" label on the 1.15.42 and 1.15.43 pages at different times; no dates are available to confirm release order or cadence [2][6].

Evidence & sources

The key statements of this analysis, labeled by type, with the sources they rely on. Trendoviz does not run product tests.

  • Primary-source or vendor statementcrewAI 1.15.38 adds `--no-open` to `crewai eval` for headless use and tags eval usage spans with a target and a per-evaluation ID.[1]
  • Primary-source or vendor statementA `crewai eval` run that exits with an error is counted as `run_failed`, and `crewai run` exits non-zero when its project script fails.[1]
  • Primary-source or vendor statementVersion 1.15.40 removes `chromadb` from the embedding function layer, and 1.15.42 makes ChromaDB an optional extra and defaults Knowledge to local Qdrant while decoupling storage from it; the notes do not describe a migration path for existing ChromaDB deployments.[2][3]
  • Primary-source or vendor statement`gpt-6-luna` became crewAI's default model everywhere in 1.15.38, but the release material does not identify a provider, model card, availability, or pricing.[1]
  • Primary-source or vendor statementMemoized crew methods are now scoped to their instance, separating cached results between crews running in the same process.[1]
  • Primary-source or vendor statementUploading a failed anonymous `crewai eval` run improves failure visibility but may transmit data from the originating machine; the notes do not state what is uploaded, where it is stored, or how to disable it.[1]
  • HypothesisThe six releases cluster around unattended operation and storage portability, which is an inference from the wording of the release notes rather than a measured outcome.[1][2][3]
  • Primary-source or vendor statementNo change in this sequence has been tested independently, so all impact statements are inferred from the release-note wording.[1]

Sources

  1. 1
    crewAIInc/crewAI 1.15.38
    ●github.comretrieved Nov 18, 2026
  2. 2
    crewAIInc/crewAI 1.15.42
    ●github.comretrieved Nov 18, 2026
  3. 3
    crewAIInc/crewAI 1.15.40
    ●github.comretrieved Nov 18, 2026
  4. 4
    crewAIInc/crewAI 1.15.41
    ●github.comretrieved Nov 18, 2026
  5. 5
    crewAIInc/crewAI 1.15.39
    ●github.comretrieved Nov 18, 2026
  6. 6
    crewAIInc/crewAI 1.15.43
    ●github.comretrieved Nov 19, 2026

Updates & corrections

  1. First published · approved by an editor

Found an error? Report it.