Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misiu/os advance may10 #826

Open
wants to merge 72 commits into
base: master
Choose a base branch
from
Open

Misiu/os advance may10 #826

wants to merge 72 commits into from

Conversation

misiugodfrey
Copy link
Collaborator

Bump by 100 commits.

mattpulver and others added 30 commits May 10, 2024 13:24
…#7519)

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…ata(). (#7521)

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Fixup window func translator logic

* Add test

* Re-enable the test query

* Refactor window function translation logic

* Address comments #1

* Address comment

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Add extra version info to dependencies and copy dependency
version file to build

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Refactor cpu memory status logging

* Allow loggig for batch insert

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Add table page size validation

* Add table string options type validation

* Add table options validation tests

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Add missing lock

* Add test

* Add string column query to check the concurrency under TSAN

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…ng column (#7530)

* Avoid redundant i64 vector copy

* Parallel resultset copy

* Address comments

* Templatize getLike function

* Refactor is_like impl selection logic

* Refactor parallelization logic

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Disables `CREATE OR REPLACE` for all database objects except `MODEL`

Co-authored-by: Matt Gara <matt.gara@omnisci.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…d string and none-encoded string (#7546)

* Improve string cmp codegen logic

* Add test

* Address comments

* Throw test queries on aggregator

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Throw an exception

* Add test

* Address comment

* Address comments

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Fixup resultset metadata prep logic

* Add test

* Fixup cache invaliadtor logic for chunk metadata cache per table

* Address comments

* Address comments v2

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Return text col as is

* Add tests

* Address comment

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Fixup crash by adding an artificial literal to a child projection node

* Add tests

* Address comment against test structure

* Use << when calling toString for SqlWindowFunctionKind enum class value

* Apply MattP's suggestion related to toString for window_func_kind

* Address comments v2

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…to increase the compiled kernel cache hit ratio (#7548)

* Hoisting literals used to call bit_is_set function

* Add tests

* Address comments

* Address comments v2

* Address comments v3

* Address comments v4

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Fix issue: \detect parquet fails in `heavysql` for array types.

* Correct a bug where arrays were incorrectly counted in detect.

* Parquet detect now uses 0 for the value of comp param to be compatible
with `heavysql` expectations.

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Add query hint

* Disable table reordering per query hint

* Add test

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…ity. (#7549)

Co-authored-by: Paul Aiyedun <paul.aiyedun@heavy.ai>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…porary table returns a wrong result (#7576)

* Add missing checking condition

* Remove redundant union query check

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Reorder tables for polygon-point join to exploit geo hash join

* Add tests

* Add debug log

* Add more logs

* Address comments

* Address comments v2

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Fixup null checking logic for st_distance

* Add test

* Address comment

* Address comment

* Remove mistakenly added CHECK stmt.

* Remove shadowed variable

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Add test

* Add index bound checking logic

* Address comment

* Address comments

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
yoonminnam and others added 25 commits May 10, 2024 13:39
…ng from subquery's resultset (#7603)

* Use dictionary key instead of column_key to get a catalog instance

* Use controllable knobs instead of hard-coded values for watchdog

* Add test

* Address comments

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…602)

* Separate geo_size codegen logic

* Handle negative index case

* Add test

* Address comment

* Address comments v2

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…ction (#7619)

* Change regex to find a string literal: skip empty string literal

* Add exception check logic

* Add test from repro w/ randomization

* Disable dist test: it's enough to check this on single-node env b/c parsing logic is the same

* Address comments

* Address comments v2

* Address comments v3

* Remove unnecessary includes

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Throw an exception for ST_Intersects hash join queries that result in
over the maximum supported number of bounding box overlaps.

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Co-authored-by: Chris Matzenbach <chris.matzenbach@omnisci.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Bump google benchmark to 1.8.3

* Add docs

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Add Abseil to deps

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* The script is intended to be run manually during the preparation of a new minimal Vulkan deps bundle

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
… (#7644)

* Reset max_group_buffer_entry_guess iff out-of-slot error was occurred

* Address comment

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…#7645)

* Introduce a server conf. to control ndv-group-estimator

* check whether the value is valid

* Change knob name

* Add query hint

* Address comments

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
… to decorrelate (#7639)

* Throw readable exception msg

* Add tests

* Address comments

* Address comments v2

* Address comments v3

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…tadataCopy

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
…#7632)

* Prepare DramArena in the RowSetMemOwner after determining # kernels

* Fixup test failures

* remove prepareAllocator for shared allocator

* Init shared allocator by default

* Misc cleanup

* Address comments

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
The max slab size configuration parameter currently determines both the
maximum possible and default slab allocation size. This update
introduces a configuration parameter that takes on the latter
responsibility of determining the default slab allocation size and
allows allocations above this size up to the configured max slab size.

* Add server configuration parameters for default CPU and GPU slab sizes.

* Set the default slab size to the same value as the max slab size when
it is not configured.

* Add new test cases for default slab size and update existing tests to
set this size.

* Update buffer pool and slab size related variable names to conform to
style guide.

* Rename current_max_slab_page_size_

* Add max/min CPU/GPU slab size validation

* Remove redundant parenthesis

* Use default_slab_size_ in DramArena constructor

* Move slab size calculation to a shared function

* Add slab count assertion to assertExpectedBufferMgrAttributes()

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Throw an exception instead of crash

* Add tests

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Enable the use of CPU buffer manager for output buffer allocations behind a flag.

* Add a CpuMgrArenaAllocator class that provides an Arena interface on top of CPU buffer manager.

* Use fast allocators for count distinct buffer allocations.

* Add g_use_cpu_mem_pool_for_output_buffers parameterized test.

* Set max cpu slab size based on g_use_cpu_mem_pool_for_output_buffers.

* Check for g_use_cpu_mem_pool_size_for_max_cpu_slab_size when validating against max_cpu_slab_size.

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Find # pts in WKTString

* Add tests

* Address comment

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
* Add new CopyParam COPY FROM WITH option " bool raster_drop_if_all_null"
* Add a test (manually-edited uncompressed GeoTIFF with last pixel NULL)

If set to true, any raster pixel where ALL the data bands are NULL will be dropped on import
Note that this requires the file to have a valid NoData value for all its data band types

Signed-off-by: Misiu Godfrey <misiu.godfrey@kraken.mapd.com>
@CLAassistant
Copy link

CLAassistant commented May 10, 2024

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
8 out of 14 committers have signed the CLA.

✅ mattpulver
✅ yoonminnam
✅ jack-mapd
✅ mattgara
✅ paul-aiyedun
✅ misiugodfrey
✅ tmostak
✅ pearu
❌ AndrewVDo
❌ Israel Vicars
❌ simoneves
❌ brenocfg
❌ cmatzenbach
❌ steveblackmon-mapd


Israel Vicars seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@mapd-bot-os
Copy link
Contributor

clang-format failed

@mapd-bot-os
Copy link
Contributor

clang-format failed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet