Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(build): disable vendored openssl #16761

Merged
merged 5 commits into from
May 23, 2024
Merged

Conversation

wangrunji0408
Copy link
Contributor

@wangrunji0408 wangrunji0408 commented May 15, 2024

Signed-off-by: Runji Wang wangrunji0408@163.comI hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

330529616-877333d4-7e41-45ec-8a09-bf33c08f03e3

Building OpenSSL from source is a bottleneck of compilation. This PR disables "vendored" feature of openssl, requires a system openssl3 and statically links it on release.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Signed-off-by: Runji Wang

Signed-off-by: Runji Wang <wangrunji0408@163.com>
Signed-off-by: Runji Wang <wangrunji0408@163.com>
Signed-off-by: Runji Wang <wangrunji0408@163.com>
Copy link
Member

@xxchan xxchan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So by default we will dynamic link openssl, and static link if OPENSSL_STATIC=1 right? This solution is generally acceptable to me, but some points:

  1. Dockerfile might also need to be updated (or not?). And please run the pipeline to test docker & binary release.
  2. Homebrew formula might need to be updated.
  3. We met compile issue with openssl several times (and developers are not good at solving it!) e.g., cannot build rdkakfa-sys with M1 Mac according to developer guide #5739 and compile error when openssl@3 is installed in the system #11467. But I'm OK to try it again..
  4. Why not always static link? e.g., OPENSSL_STATIC=1 in build.rs. So that we don't need to manage many scripts.

@xxchan
Copy link
Member

xxchan commented May 15, 2024

  1. We met compile issue with openssl several times

openssl@3 in homebrew is not keg-only any more. So perhaps the experience will be better.

@wangrunji0408
Copy link
Contributor Author

wangrunji0408 commented May 15, 2024

  1. Homebrew formula might need to be updated.

I see the openssl3 dependency in the formula. So I guess it's just fine?

https://github.com/risingwavelabs/homebrew-risingwave/blob/1cf04a5a3aef86e1f63912048578c2821816af48/Formula/risingwave%401.7-standalone.rb#L20

  1. Why not always static link? e.g., OPENSSL_STATIC=1 in build.rs. So that we don't need to manage many scripts.

I tried to set this env variable in build.rs in workspace-config, but it didn't work.

Signed-off-by: Runji Wang <wangrunji0408@163.com>
@wangrunji0408 wangrunji0408 added this pull request to the merge queue May 15, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 15, 2024
@wangrunji0408 wangrunji0408 added this pull request to the merge queue May 15, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 15, 2024
@wangrunji0408 wangrunji0408 added this pull request to the merge queue May 15, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 15, 2024
@xxchan xxchan added this pull request to the merge queue May 17, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 17, 2024
@wangrunji0408
Copy link
Contributor Author

No idea why the e2e test is always timed out. 🤪

@xxchan
Copy link
Member

xxchan commented May 20, 2024

BTW, I'm thinking zstd-sys's pkg-config feature is also static link, instead of dynamic link:

https://github.com/gyscos/zstd-rs/blob/e470f0021d25cd5f79431acba0b7456301c00b82/zstd-safe/zstd-sys/build.rs#L42-L49

BTW, the timeout in this PR is quite strange.

From the trending, not sure whether we should increase timeout https://rw-buildkite-stats.streamlit.app/

image

Signed-off-by: Runji Wang <wangrunji0408@163.com>
@wangrunji0408 wangrunji0408 added this pull request to the merge queue May 23, 2024
Merged via the queue into main with commit eb6ca34 May 23, 2024
27 of 29 checks passed
@wangrunji0408 wangrunji0408 deleted the wrj/openssl-no-vendered branch May 23, 2024 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants