Skip to content

fix: task cleanup in ExchangeClientTest to prevent CI timeout (#16168) #1433

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
prestodb-ci wants to merge 34 commits into
base: oss-baseline
Choose a base branch
from
Open

fix: task cleanup in ExchangeClientTest to prevent CI timeout (#16168) #1433

prestodb-ci wants to merge 34 commits into from

Conversation

@prestodb-ci
Copy link
Collaborator

@prestodb-ci prestodb-ci commented Nov 25, 2025 •
edited
Loading

Test PR for branch staging-b3aa142e8-pr with head b3aa142

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Nov 26, 2025

Test passed for commit 1777c7523, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/394/display/redirect for details

@prestodb-ci prestodb-ci deleted the staging-1777c7523-pr branch November 26, 2025 01:29
@zhouyuan @FelixYBW
fix: Enable CI on IBM repo
e962632 
Signed-off-by: Yuan <[email protected]>

Set ccache maximum size to 1G

Remove sed command from Gluten workflow

Removed a sed command that replaces 'oap-project' with 'IBM' in the get-velox.sh script.

Modify get-velox.sh to change 'ibm' to 'ibm-xxx'

Update the get-velox.sh script to replace 'ibm' with 'ibm-xxx'.

Update sed command to be case-insensitive

Update gluten.yml

fix iceberg unit test

Signed-off-by: Yuan <[email protected]>

Update gluten.yml

Enable enhanced features in gluten build script

Update cache keys for Gluten workflow
@FelixYBW FelixYBW restored the staging-1777c7523-pr branch November 27, 2025 09:14
@FelixYBW FelixYBW reopened this Nov 27, 2025
@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Nov 27, 2025

❌ Test commit 2a50cd131 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/PR-2057/2/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Nov 28, 2025

❌ Test commit 0007f37c6 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/PR-2057/4/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 1, 2025

❌ Test commit 0007f37c6 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/PR-2057/5/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 1, 2025

❌ Test commit 0007f37c6 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/PR-2057/6/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 2, 2025

❌ Test commit 0007f37c6 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/PR-2057/7/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 3, 2025

❌ Test commit 0007f37c6 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/PR-2057/8/display/redirect for details

@FelixYBW FelixYBW changed the title fix: Avoid TSAN data race during cache entry initialization (#15623) refactor: Extract common BaseSerializedPage API (#15626) Dec 3, 2025
@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 3, 2025

❌ Test commit 0007f37c6 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/PR-2057/9/display/redirect for details

@prestodb-ci prestodb-ci changed the title refactor: Extract common BaseSerializedPage API (#15626) fix(build): Ambiguity caused by long literal (#15670) Dec 3, 2025
@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 3, 2025

❌ Test commit a017fca78 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/427/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 3, 2025

❌ Test commit a236cf5b3 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/428/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 3, 2025

❌ Test commit 0f21ff954 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/429/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 3, 2025

❌ Test commit e01ca2187 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/430/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 3, 2025

❌ Test commit 4f39eef54 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/431/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 4, 2025

❌ Test commit eba05296b failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/437/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 4, 2025

❌ Test commit bf9e9fd6d failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/440/display/redirect for details

@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Dec 5, 2025

❌ Test commit 4ff28c6bf failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/449/display/redirect for details

zhouyuan and others added 27 commits January 30, 2026 20:27
@zhouyuan
[OAP][11771]fix: Fix smj result mismatch issue in semi, anit and full…
ba122ef 
… outer join

Signed-off-by: Yuan <[email protected]>

Alchemy-item: (ID = 882) [OAP] [11771] Fix smj result mismatch issue commit 1/1 - ada7dd2
@PingLiuPing
Revert "refactor: Replace as<> with asChecked<> in IcebergDataSink (f…
3d0477f 
…acebookincubator#16035)"

This reverts commit 9f98432.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 1/21 - 4830b15
@PingLiuPing
Revert "feat: Add iceberg column handle (facebookincubator#15607)"
75ef4f3 
This reverts commit 536cc78.

# Conflicts:
#	velox/connectors/hive/iceberg/IcebergColumnHandle.cpp
#	velox/connectors/hive/iceberg/IcebergColumnHandle.h

Alchemy-item: (ID = 1066) Iceberg staging hub commit 2/21 - f69af99
@PingLiuPing
Revert "feat: Support iceberg partition transform (facebookincubator#…
1fa5286 
…15509)"

This reverts commit d791d84.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 3/21 - c5a7bb5
@PingLiuPing
Revert "refactor: Add Iceberg connector (facebookincubator#15581)"
8bd0eec 
This reverts commit fd0682b.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 4/21 - c5884a5
@PingLiuPing
Revert "feat: Add Iceberg partition name generator (facebookincubator…
2a62552 
…#15461)"

This reverts commit 7576f4e.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 5/21 - f8e07ba
@PingLiuPing
Revert "feat: Add support for evaluating Iceberg partition transforms (…
fc8b27d 
…facebookincubator#15477)"

This reverts commit 1895711.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 6/21 - 29d5860
@PingLiuPing
Revert "feat: Add Iceberg partition name generation utility (facebook…
9cd2090 
…incubator#15443)"

This reverts commit 51d4a94.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 7/21 - 2efb143
@PingLiuPing
Revert "feat: Add iceberg partition specification (facebookincubator#…
91a3569 
…15423)"

This reverts commit 600524b.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 8/21 - ca32e0e
@yingsu00
refactor: Move toValues from InPredicate.cpp to Filter.h
0a35d4c 
The function toValues removes duplicated values from the vector and
return them in a std::vector. It was used to build an InPredicate. It
will be needed for building NOT IN filters for Iceberg equality delete
read as well, therefore moving it from velox/functions/prestosql/InPred
icate.cpp to velox/type/Filter.h. This commit also renames it to
deDuplicateValues to make it easier to understand.

Alchemy-item: (ID = 1066) Iceberg staging hub commit 9/21 - 59e7116
@yingsu00
feat(connector): Support reading Iceberg split with equality deletes
393a303 
This commit introduces EqualityDeleteFileReader, which is used to read
Iceberg splits with equality delete files. The equality delete files
are read to construct domain filters or filter functions, which then
would be evaluated in the base file readers.

When there is only one equality delete field, and when that field is
an Iceberg identifier field, i.e. non-floating point primitive types,
the values would be converted to a list as a NOT IN domain filter,
with the NULL treated separately. This domain filter would then be
pushed to the ColumnReaders to filter our unwanted rows before they
are read into Velox vectors. When the equality delete column is a
nested column, e.g. a sub-column in a struct, or the key in a map,
such column may not be in the base file ScanSpec. We need to add/remove
these subfields to/from the SchemaWithId and ScanSpec recursively if
they were not in the ScanSpec already. A test is also added for such
case.

If there are more than one equality delete field, or the field is not
an Iceberg identifier field, the values would be converted to a typed
expression in the conjunct of disconjunts form. This expression would
be evaluated as the remaining filter function after the rows are read
into the Velox vectors. Note that this only works for Presto now as
the "neq" function is not registered by Spark. See https://github.com/
facebookincubator/issues/12667

Note that this commit only supports integral types. VARCHAR and
VARBINARY need to be supported in future commits (see
facebookincubator#12664).

Co-authored-by: Naveen Kumar Mahadevuni <[email protected]>

# Conflicts:
#	velox/connectors/hive/iceberg/tests/IcebergReadTest.cpp

# Conflicts:
#	velox/dwio/common/ScanSpec.h

# Conflicts:
#	velox/type/Filter.h

# Conflicts:
#	velox/connectors/hive/HiveDataSource.cpp
#	velox/connectors/hive/SplitReader.cpp
#	velox/connectors/hive/SplitReader.h

Alchemy-item: (ID = 1066) Iceberg staging hub commit 10/21 - cf94216
@PingLiuPing
Support insert data into iceberg table.
718a8ae 
# Conflicts:
#	velox/connectors/hive/HiveConnectorUtil.cpp

Alchemy-item: (ID = 1066) Iceberg staging hub commit 11/21 - a59bca9
@PingLiuPing
Add iceberg partition transforms.
a73f48b 
Co-authored-by: Chengcheng Jin <[email protected]>

Alchemy-item: (ID = 1066) Iceberg staging hub commit 12/21 - e45fb49
@PingLiuPing
Add NaN statistics to parquet writer.
65cd2fa 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 13/21 - e7d93fe
@PingLiuPing
Collect Iceberg data file statistics in dwio.
220fc2a 
# Conflicts:
#	velox/dwio/parquet/writer/Writer.cpp
#	velox/dwio/parquet/writer/Writer.h
#	velox/dwio/parquet/writer/arrow/ArrowSchema.cpp
#	velox/dwio/parquet/writer/arrow/ArrowSchema.h
#	velox/dwio/parquet/writer/arrow/Metadata.cpp

Alchemy-item: (ID = 1066) Iceberg staging hub commit 14/21 - 56c55ca
@PingLiuPing
Integrate Iceberg data file statistics and adding unit test.
9d030f2 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 15/21 - c661c4e
@PingLiuPing
Support write field_id to parquet metadata SchemaElement.
89ad926 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 16/21 - bbb7671
@PingLiuPing
Implement iceberg sort order
24a4fcf 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 17/21 - c82ee88
@PingLiuPing
Add clustered Iceberg writer mode.
d8f4819 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 18/21 - 9db788b
@PingLiuPing
Fix parquet writer ut
3f5b6b0 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 19/21 - dd6148f
@PingLiuPing
Add IcebergConnector
f12561b 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 20/21 - 57e0283
@PingLiuPing
Fix unittest error
cb40878 
Alchemy-item: (ID = 1066) Iceberg staging hub commit 21/21 - a3275b7
@zhouyuan
adding daily tests
d669ec4 
Signed-off-by: Yuan <[email protected]>

Alchemy-item: (ID = 906) fix: Adding daily tests commit 1/2 - e2eb2c6
@FelixYBW
remote gluten daily build
3fc23d7 
we can cache ccache on every build even on failure, since ibm/velox is
always incremental build

Alchemy-item: (ID = 906) fix: Adding daily tests commit 2/2 - 0899ddc
@Zouxxyy
1
c0245fd 
Alchemy-item: (ID = 988) Add fileNameGenerator to the constructor of IcebergInsertTableHandle commit 1/1 - a5f7e46
@FelixYBW
remote website folder
fc7879b 
Alchemy-item: (ID = 1046) Remove website folder commit 1/1 - 1787292
@wanglinsong
Merge branch 'ci-fix-pr' into staging-b3aa142e8-pr
054c862 
Signed-off-by: Linsong Wang <[email protected]>
@prestodb-ci prestodb-ci changed the title perf: Avoid allocating memory for lookup and output vectors in IndexLookupJoin (#16155) fix: task cleanup in ExchangeClientTest to prevent CI timeout (#16168) Jan 30, 2026
@prestodb-ci
Copy link
Collaborator Author

prestodb-ci commented Jan 30, 2026

❌ Test commit b3aa142e8 failed, open https://ci.ibm.prestodb.dev/job/presto-performance/job/presto-performance/job/pipeline-rebase-ibm-velox/771/display/redirect for details

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@majetideepak majetideepak Awaiting requested review from majetideepak majetideepak is a code owner

@jinchengchenghh jinchengchenghh Awaiting requested review from jinchengchenghh

@rui-mo rui-mo Awaiting requested review from rui-mo

Assignees

No one assigned

Labels

do_not_review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.