Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support IVF_HNSW_SQ #1284

Merged
merged 7 commits into from
May 16, 2024
Merged

feat: support IVF_HNSW_SQ #1284

merged 7 commits into from
May 16, 2024

Conversation

BubbleCal
Copy link
Contributor

No description provided.

Signed-off-by: BubbleCal <bubble-cal@outlook.com>
@BubbleCal
Copy link
Contributor Author

leave docs and python to future PRs, or it's too large

Signed-off-by: BubbleCal <bubble-cal@outlook.com>
@BubbleCal BubbleCal requested a review from westonpace May 9, 2024 11:41
@BubbleCal BubbleCal marked this pull request as ready for review May 10, 2024 06:16
@BubbleCal BubbleCal requested a review from wjones127 May 11, 2024 15:26
Copy link
Contributor

@westonpace westonpace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor suggestions. I think we should decide on usize / u64 before merging (as changing that would be a breaking change) but the other suggestion about the comments we can figure out later.

Comment on lines 312 to 324
/// The number of neighbors to select for each vector in the HNSW graph.
/// The default value is 20.
pub fn m(mut self, m: usize) -> Self {
self.m = m;
self
}

/// The number of candidates to evaluate during the construction of the HNSW graph.
/// The default value is 300.
pub fn ef_construction(mut self, ef_construction: usize) -> Self {
self.ef_construction = ef_construction;
self
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add this in a future PR (when we extend this to node and python perhaps) but we will want these parameters to include:

  • What happens if this is set incorrectly?
  • When should the user adjust the default?

Comment on lines 227 to 228
pub(crate) m: usize,
pub(crate) ef_construction: usize,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use u64 instead of usize? We can convert to usize internally. I'd rather avoid usize in our public API since it is slightly more confusing to think about.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, let me change it to u32 cause we don't need such great value of u64

Signed-off-by: BubbleCal <bubble-cal@outlook.com>
@github-actions github-actions bot added enhancement New feature or request Rust Rust related issues labels May 16, 2024
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
Signed-off-by: BubbleCal <bubble-cal@outlook.com>
@BubbleCal BubbleCal merged commit 5e01810 into lancedb:main May 16, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Rust Rust related issues
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants