v1.25.1 - Schema V2 Fixes, AWS modules Fixes, Google Gemini 1.5 models support, Overwriting an object which is locally deleted Fix

Breaking Changes

none

New Features

none

Fixes

• fix prevent empty segment generation by @jeroiraz in #4893
• raft: refactor db opening & logging by @moogacs in #4875
• raft: always apply local schema before db by @moogacs in #4876
• raft: open db on service.Open instead on apply by @moogacs in #4878
• raft: remove schemaOnly and rename lastAppliedIndexOnStart by @moogacs in #4882
• improve log messages to apply to also dump cmd related output by @reyreaud-l in #4895
• Ensure classCache is added to the context before validate objects call by @tsmith023 in #4902
• Fix batch delete proto Java option setting by @antas-marcin in #4903
• Octoai header fixes + image generation by @dirkkul in #4906
• chore: warn with older partially written compacted segments by @jeroiraz in #4908
• Re-add schemaOnly parameter when applying raft log entries and ensure we reloadDB on catching up by @reyreaud-l in #4897
• Support raft rf scale (part 2) by @aliszka in #4913
• RAFT: fix data race on registering call backs and atomic lastAppliedI… by @moogacs in #4914
• RAFT: support back RF scale increase by @moogacs in #4856
• Replace namedLocks with KeyLocker by @aliszka in #4919
• Fix generative-aws module, add support for all Bedrock models by @antas-marcin in #4918
• Migrate text2vec-huggingface module to batching by @dirkkul in #4894
• backup:scheduler: compare and error based on major versions only by @moogacs in #4925
• Improve error messages on failed get/init shard by @aliszka in #4923
• fix race in tombstone cleanup by @asdine in #4910
• fix: skipping deleted nodes when finding entry point for node by @jeroiraz in #4807
• Add support for Google Gemini 1.5 models by @antas-marcin in #4929
• Remove unsearchable properties by @donomii in #4930
• Fix cluster remote aggregate hybrid neartext by @donomii in #4926
• Update VoyageAI vectorizer available model names by @fzowl in #4933
• Optimize locking for HNSW flat search by @trengrj in #4938
• Improved error message when autotenant expected to work with single object insertion by @parkerduckworth in #4948
• fix: special case when overwriting object which is locally deleted by @jeroiraz in #4950
• Do not validate the model for openai with a different base url by @dirkkul in #4951
• RAFT: fix race between apply for classes in the schema and report ready only after db reloaded by @moogacs in #4946
• Fix govulnchecks by @antas-marcin in #4958
• Add retry on failure in CI for tests instability by @antas-marcin in #4961
• fix deadlock by avoiding recursive locking in meta_class by @reyreaud-l in #4959
• Add support for separate headers for Google Studio and Vertex AI by @antas-marcin in #4962
• add missing wait for schema version on vector.PutObject by @moogacs in #4963

Full Changelog: v1.25.0...v1.25.1

v1.24.14 - Overwriting an object which is locally deleted Fix, Google Studio and Vertex separate headers support

Breaking Changes

none

New Features

none

Fixes

• Update VoyageAI vectorizer available model names by @fzowl in #4933
• fix: special case when overwriting object which is locally deleted by @jeroiraz in #4950
• Fix govulnchecks by @antas-marcin in #4958
• Add support for separate headers for Google Studio and Vertex AI by @antas-marcin in #4962

Full Changelog: v1.24.13...v1.24.14

v1.23.15 - Overwriting an object which is locally deleted Fix

Breaking Changes

none

New Features

none

Fixes

• fix: special case when overwriting object which is locally deleted by @jeroiraz in #4950
• Fix govulnchecks by @antas-marcin in #4958

Full Changelog: v1.23.14...v1.23.15

v1.24.13 - Empty segment generation Fix, Bedrock generative and Gemini 1.5 models support

Breaking Changes

none

New Features

none

Fixes

• remove debug logs by @etiennedi in #4889
• fix: max length field validation during object marshalling by @jeroiraz in #4877
• fix prevent empty segment generation by @jeroiraz in #4893
• chore: warn with older partially written compacted segments by @jeroiraz in #4908
• Fix generative-aws module, add support for all Bedrock models by @antas-marcin in #4918
• backup:scheduler: compare and error based on major versions only by @moogacs in #4925
• fix: skipping deleted nodes when finding entry point for node by @jeroiraz in #4807
• Add support for Google Gemini 1.5 models by @antas-marcin in #4929

Full Changelog: v1.24.12...v1.24.13

v1.25.0 - RAFT-based Schema, Batch Vectorization, Hybrid Search Improvements, Implicit Tenant Creation, Dynamic Index Switching

Breaking Changes

none

New Features

RAFT-based Schema

We're excited to announce the release of our RAFT-based schema! With this, Weaviate now supports concurrent schema updates, eliminating bottlenecks and significantly improving performance in large-scale and dynamic settings.

• Update schema manager interface to expose schema version (Part 1) by @reyreaud-l in #4659
• Update schema manager interface to expose schema version (Part 2) by @reyreaud-l in #4660
• Implement schema updates using RAFT consensus by @redouan-rhazouani in #3944
• Rebuild GQL just once when reloading local db by @redouan-rhazouani in #4670
• Update local_dev script RAFT_JOIN config to bootstrap cluster correctly by @moogacs in #4676
• Refactor consensus dev script on nodes name weaviate-* by @moogacs in #4677
• Update object interface to include version by @reyreaud-l in #4675
• RAFT store Apply tests calling mocks expectations assertion by @moogacs in #4684
• Save copy of the current RAFT schema into the old format by @moogacs in #4679
• Trigger schema callbacks on AddProperty only on success by @moogacs in #4685
• Check error on apply to avoid panics by @moogacs in #4687
• Add schema version to batch write client and server by @reyreaud-l in #4681
• Added store-specific statistics to existing RAFT statistics by @nathanwilk7 in #4689
• Client rpc reuse the same conn by @moogacs in #4686
• Update batch delete with schema version by @reyreaud-l in #4690
• Schema v2 props upsert by @aliszka in #4680
• Autoschema optimizations - cached class schema by @aliszka in #4662
• RAFT store.Service.Ready is flaky in cluster/store/store_test.go, poll for 2s instead of 1s wait to improve robustness by @nathanwilk7 in #4706
• Implement schema querying based on a specific version for enhanced version control by @redouan-rhazouani in #4693
• Update put, merge and add ref with schema version by @reyreaud-l in #4704
• Update UpdateShardStatus to use schema version by @reyreaud-l in #4713
• Fix nil ptr panic when closing RAFT leader client by @reyreaud-l in #4723
• RAFT GRPC: tweak the client service config by @moogacs in #4715
• Add query shard tenant and update usages by @reyreaud-l in #4727
• Enable cancelation for HTTP replication requests and reduce timeout for faster node failure detection by @redouan-rhazouani in #4730
• Schema v2 idempotent tenants by @aliszka in #4699
• Make batch operation retrieve and pass schema version to client by @reyreaud-l in #4738
• Update auto schema to return the schema version by @reyreaud-l in #4742
• Refactor usage of index.getOrInitLocalShard() to the minimum by @redouan-rhazouani in #4744
• Schema V2 writes with schema version by @aliszka in #4745
• Allow migration from RAFT to take as much time as necessary by @redouan-rhazouani in #4710
• Add missing CloudService to ClusterService rename by @aliszka in #4747
• Update RAFT subsystem to use the same log format by @redouan-rhazouani in #4746
• Convert TenantExists to handle eventual consistency by @tsmith023 in #4750
• Add an external gRPC method for getting tenant information by @tsmith023 in #4741
• update object endpoints to fetch and propagate schema version by @reyreaud-l in #4752
• Replace slog with logrus for logs consistency by @moogacs in #4753
• Add action logrus field to RAFT logs by @moogacs in #4755
• Autoschema fix to correctly return max schemaVersion by @aliszka in #4757
• Make default CLUSTER_HOSTNAME consistent between memberlist and RAFT by @moogacs in #4758
• Cluster log formatting by @nathanwilk7 in #4759
• Handle possible nil classes in autoSchema by @moogacs in #4761
• Refresh class's schema when props added by autoschema by @aliszka in #4762
• Specialize voter nodes for metadata storage by @redouan-rhazouani in #4734
• Update the schema version parsing to not always return 0 by @reyreaud-l in #4767
• Refactor leader error distinguish between election and network issues by @moogacs in #4784
• Update query shard tenant to get multiple shards at once by @moogacs in #4731
• Avoid schema updating race with DB update by @reyreaud-l in #4769
• Backup restore: return err class exists if it does by @moogacs in #4792
• Make coordinator node wait for schema version if changed on validate step by @reyreaud-l in #4794
• Add RAFT_GRPC_MESSAGE_MAX_SIZE Environment Variable to set maximum GRPC message size for RAFT by @nathanwilk7 in #4799
• Only involve leader for tenant status when necessary by @etiennedi in #4803
• Shift add tenant to the RAFT leader by @reyreaud-l in #4801
• Fix RAFT shutdown, force it for dependencies and convert to enterrors.GoWrapper by @moogacs in #4810
• Allow Leader QueryShardingState type by @moogacs in #4817
• Update replica usecase to wait for EC on writes with schema version by @reyreaud-l in #4814
• Schema v2 prevent shutdown by @aliszka in #4821
• Aggregate querying the leader for classes with variadic by @moogacs in #4787
• Observe durations of schema reads and writes for local and leader reads by @etiennedi in #4844
• Reserve RAFT (all casing permutations) as a class name by @moogacs in #4874
• Make bootstrap exit early on RAFT store reporting ready by @reyreaud-l in #4871

Batch Vectorization

Avoid vectorization API rate-limiting and enjoy faster insertion.

• Batch vectorization openai by @dirkkul in #4384
• Factor out batch vectorization and add Cohere support by @dirkkul in #4515
• Factor out batching in modules by @dirkkul in #4564
• Vectorizer batching for VoyageAI by @dirkkul in #4578

Dynamic Index Switching

Automatically switch vector index types to achieve peak performance and efficiency.

• Dynamic vector index type by @abdelr in #4350
• Fix merge conflict on schema validation of dynamic index by @trengrj in #4829
• Adding more tests around the dynamic index and fixing an existing bug by @abdelr in #4836

Implicit Tenant Creation

Create tenants on the fly by simply including the tenant name during batch inserts.

• Create nonexistent tenants in batch by default by @parkerduckworth in #3741
• Support auto-tenant toggling on MT-enabled classes by @parkerduckworth in #4813

Hybrid Search Improvements

• Add nearvector and neartext to hybrid search by @donomii in #4462
• Batch vectorization with custom rate limits by @dirkkul in #4546
• Add groupby to hybrid search and bm25f, add moveto/movefrom etc to aggregate hybrid search by @donomii in #4477
• Add tests, improve parameter extraction in hybrid aggregate by @donomii in #4809

Other

• Add endpoint for checking if tenant exists by @litlig in #4155
• Allow for the go profiler to be hosted on ports other than 6060 by @bennycortese in #4289
• Custom linter to report all files before exit by @moogacs in #4485
• Migrate co...

v1.24.12 - LSM Segment Size limit setting, text2vec-aws module Bedrock models support Fix

Breaking Changes

none

New Features

none

Fixes

• fix: vector validation in batch operation by @jeroiraz in #4858
• Introduce optional LSM Segment Size limit by @etiennedi in #4872
• Fix text2vec-aws module Bedrock support by @antas-marcin in #4881
• Allow setting HNSW Max log size ("condensing limit") dynamically by @etiennedi in #4884
• Add support for Amazon Titan Text Embeddings V2 model by @antas-marcin in #4886

Full Changelog: v1.24.11...v1.24.12

v1.24.11 - Vector search with named vectors Fix, improved validation when using nearObject and nearVector searches

Breaking Changes

none

New Features

none

Fixes

• Fix macOS monitor tests by @antas-marcin in #4708
• Add latest tag to docker image only when it's a tag build by @antas-marcin in #4732
• Remove unintended debug logs by @etiennedi in #4766
• Improve input validation of ID in nearObject and vector in nearVector by @tsmith023 in #4782
• Fix named vector groupby non-search aggregate queries by @tsmith023 in #4783
• Add recovery wrappers for GRPC APIs by @dirkkul in #4780
• index queue: add logs by @asdine in #4842
• Revert "index queue: add logs" by @asdine in #4850
• Fix an issue introduced in v1.24.9 where HNSW commit logs are no longer optimized by @etiennedi in #4853
• Fix vector search with named vectors by @antas-marcin in #4857

Full Changelog: v1.24.10...v1.24.11

v1.25.0-rc.0 - RAFT-based Schema, Batch Vectorization, Hybrid Search Improvements, Implicit Tenant Creation, Dynamic Index Switching

This is a release candidate for the upcoming v1.25.0 release
A release candidate (RC) means the release is considered feature complete and has finished beta-testing. Any issues discovered during the RC phase will lead to new rc releases. The final rc release becomes the stable release. We're happy for your feedback about this pre-release.

This pre-release contains:

• RAFT-based schema
• Batch vectorization
• Hybrid search support for moveTo/moveAway, distance/certainty filtering, property selection for Aggregation queries, and groupBy
• Implicit tenant creation
• Dynamic index switching
• New modules: text2vec-ollama, generative-ollama, texte2vec-octoai, generative-octoai

v1.24.10 - HNSW performance regression Fix

Breaking Changes

none

New Features

none

Fixes

• fix:hnsw: data race on findNewLocalEntrypoint by @moogacs in #4701
• Fix HNSW performance regression (introduced in v1.24.9) by @etiennedi in #4721

Full Changelog: v1.24.9...v1.24.10

v1.24.9 - Added support for new Google models and Cohere V3 reranking models, Added memory guardrails, MTTR improvement Fix

Breaking Changes

none

New Features

none

Fixes

• Fix stuck transaction after tx commit failure by @etiennedi in #4638
• Significantly improve MTTR when number of tenants is massive by @parkerduckworth in #4636
• Bump testcontainers-go version to v0.30.0 by @antas-marcin in #4635
• Additional entrypoint error handling by @trengrj in #4626
• Improved memory guardrails by @etiennedi in #4643
• Update VoyageAI vectorizer available model names by @fzowl in #4657
• Add Reranker 3 to reranker-cohere by @CShorten in #4655
• Add support for all Vertex AI chat bison models by @antas-marcin in #4661
• Fix schema repair: include tenant activity status by @parkerduckworth in #4644
• Memory guardrails part2 by @dirkkul in #4646
• make replication broadcast use timeout-able context by @reyreaud-l in #4669
• Fix multi-node classifications by @parkerduckworth in #4694
• Memory mapping guardrails by @dirkkul in #4667
• 🔒 enable and address govulnchecks by @moogacs in #4698
• Add support for new Google AI models by @antas-marcin in #4702

Full Changelog: v1.24.8...v1.24.9