Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unifying column names validation for both row and column tables #14722 #15453

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

Emgariko
Copy link
Collaborator

@Emgariko Emgariko commented Mar 7, 2025

Changelog entry

Unifying column names validation for both row and column tables #14722

Changelog category

  • Bugfix

Description for reviewers

Added name validation call used in row tables to column tables(link). Covered with tests - testing Create/Alter TABLE/ColumnStore behavior.

@github-actions github-actions bot added the bugfix label Mar 7, 2025
Copy link

github-actions bot commented Mar 7, 2025

🟢 2025-03-10 07:00:18 UTC The validation of the Pull Request description is successful.

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 10:18:00 UTC Pre-commit check linux-x86_64-release-asan for c2d1e31 has started.
2025-03-07 10:18:32 UTC Artifacts will be uploaded here
2025-03-07 10:22:03 UTC ya make is running...
🟡 2025-03-07 11:30:53 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11786 11546 0 187 18 35

2025-03-07 11:32:23 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-07 11:50:52 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
291 (only retried tests) 199 0 58 3 31

2025-03-07 11:51:04 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-07 12:03:42 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
129 (only retried tests) 98 0 0 0 31

🟢 2025-03-07 12:03:53 UTC Build successful.
🟡 2025-03-07 12:04:28 UTC ydbd size 3.7 GiB changed* by +329.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: a790a6d merge: c2d1e31 diff diff %
ydbd size 3 994 271 672 Bytes 3 994 609 424 Bytes +329.8 KiB +0.008%
ydbd stripped size 1 388 750 024 Bytes 1 388 829 544 Bytes +77.7 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 10:19:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for c2d1e31 has started.
2025-03-07 10:19:29 UTC Artifacts will be uploaded here
2025-03-07 10:22:21 UTC ya make is running...
🟡 2025-03-07 11:11:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19252 17919 0 2 1218 113

2025-03-07 11:14:04 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-07 11:25:43 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
160 (only retried tests) 55 0 0 0 105

🟢 2025-03-07 11:25:54 UTC Build successful.
🟢 2025-03-07 11:26:19 UTC ydbd size 2.1 GiB changed* by +8.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 1237379 merge: c2d1e31 diff diff %
ydbd size 2 293 645 624 Bytes 2 293 654 312 Bytes +8.5 KiB +0.000%
ydbd stripped size 480 484 512 Bytes 480 484 704 Bytes +192 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@Emgariko Emgariko self-assigned this Mar 7, 2025
TString tableDescr = R"(
Name: "TestTable"
Schema {
Columns {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add Column with name containing all of allowed symbols

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 12:20:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1ea1fe7 has started.
2025-03-07 12:20:33 UTC Artifacts will be uploaded here
2025-03-07 12:23:14 UTC ya make is running...
2025-03-07 13:25:24 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 12:21:46 UTC Pre-commit check linux-x86_64-release-asan for 1ea1fe7 has started.
2025-03-07 12:22:10 UTC Artifacts will be uploaded here
2025-03-07 12:25:06 UTC ya make is running...
2025-03-07 13:25:23 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 13:28:06 UTC Pre-commit check linux-x86_64-relwithdebinfo for ce14c50 has started.
2025-03-07 13:28:24 UTC Artifacts will be uploaded here
2025-03-07 13:31:05 UTC ya make is running...
🟢 2025-03-07 14:19:12 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19262 17916 0 0 1229 117

🟢 2025-03-07 14:21:14 UTC Build successful.
🟢 2025-03-07 14:21:33 UTC ydbd size 2.1 GiB changed* by +8.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: e4821cc merge: ce14c50 diff diff %
ydbd size 2 294 621 968 Bytes 2 294 630 656 Bytes +8.5 KiB +0.000%
ydbd stripped size 480 593 312 Bytes 480 593 504 Bytes +192 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 13:29:30 UTC Pre-commit check linux-x86_64-release-asan for ce14c50 has started.
2025-03-07 13:29:57 UTC Artifacts will be uploaded here
2025-03-07 13:33:06 UTC ya make is running...
🟡 2025-03-07 14:40:30 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11790 11579 0 154 21 36

2025-03-07 14:41:57 UTC ya make is running... (failed tests rerun, try 2)


status = annoyingClient.CreateColumnTable("/Root/OlapStore", R"(
Name: "Test"
ColumnShardCount : 4
Copy link
Collaborator Author

@Emgariko Emgariko Mar 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Без явного указания запрос падал с ошибкой -

2025-03-07T12:05:20.003260Z node 1 :TX_PROXY ERROR: Actor# [1:7479037519220164375:2896] txid# 281474976710659, issues: { message: "Cannot create table with 64 column shards, only 4 are available" severity: 1 } 
Error 128: Cannot create table with 64 column shards, only 4 are available

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 14:53:58 UTC Pre-commit check linux-x86_64-relwithdebinfo for fee3ec2 has started.
2025-03-07 14:54:13 UTC Artifacts will be uploaded here
2025-03-07 14:57:03 UTC ya make is running...

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 14:55:42 UTC Pre-commit check linux-x86_64-release-asan for fee3ec2 has started.
2025-03-07 14:55:58 UTC Artifacts will be uploaded here
2025-03-07 14:58:54 UTC ya make is running...

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 15:19:07 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3b2a09d has started.
2025-03-07 15:19:22 UTC Artifacts will be uploaded here
2025-03-07 15:22:13 UTC ya make is running...
2025-03-07 15:22:59 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 15:19:15 UTC Pre-commit check linux-x86_64-release-asan for 3b2a09d has started.
2025-03-07 15:19:29 UTC Artifacts will be uploaded here
2025-03-07 15:22:10 UTC ya make is running...
2025-03-07 15:22:59 UTC Check cancelled

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 15:25:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8de7e73 has started.
2025-03-07 15:25:33 UTC Artifacts will be uploaded here
2025-03-07 15:28:13 UTC ya make is running...
🟡 2025-03-07 16:15:29 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19277 17939 0 2 1221 115

2025-03-07 16:17:11 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-03-07 16:17:42 UTC ydbd size 2.1 GiB changed* by +8.5 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9f78607 merge: 8de7e73 diff diff %
ydbd size 2 294 622 192 Bytes 2 294 630 880 Bytes +8.5 KiB +0.000%
ydbd stripped size 480 593 376 Bytes 480 593 568 Bytes +192 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Mar 7, 2025

2025-03-07 15:25:32 UTC Pre-commit check linux-x86_64-release-asan for 8de7e73 has started.
2025-03-07 15:25:47 UTC Artifacts will be uploaded here
2025-03-07 15:28:37 UTC ya make is running...
🟡 2025-03-07 16:36:23 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
11803 11609 0 126 30 38

2025-03-07 16:37:23 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-03-07 17:00:15 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
267 (only retried tests) 231 0 6 1 29

2025-03-07 17:00:29 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-03-07 17:13:47 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
67 (only retried tests) 39 0 0 0 28

🟢 2025-03-07 17:13:59 UTC Build successful.
🟢 2025-03-07 17:14:33 UTC ydbd size 3.7 GiB changed* by +11.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 9f78607 merge: 8de7e73 diff diff %
ydbd size 3 996 097 656 Bytes 3 996 108 872 Bytes +11.0 KiB +0.000%
ydbd stripped size 1 389 147 400 Bytes 1 389 147 720 Bytes +320 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@Emgariko Emgariko marked this pull request as ready for review March 7, 2025 15:32
@Emgariko Emgariko requested a review from a team as a code owner March 7, 2025 15:32
@Emgariko Emgariko requested a review from a team as a code owner March 7, 2025 15:32
@Emgariko Emgariko linked an issue Mar 7, 2025 that may be closed by this pull request
@github-actions github-actions bot added bugfix and removed bugfix labels Mar 7, 2025
@dorooleg dorooleg requested a review from swalrus1 March 7, 2025 16:23
@dorooleg
Copy link
Collaborator

dorooleg commented Mar 7, 2025

Напиши пожалуйста заголовок понятнее. По аналогии с changelog entry

@@ -176,6 +177,10 @@ bool TOlapColumnsDescription::ValidateForStore(const NKikimrSchemeOp::TColumnTab
return false;
}
const TString& colName = colProto.GetName();
if (!IsValidColumnName(colName, false)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Зачем эта валидация? Кажется, она здесь не обязательна, потому что все колонки из tableSchema должны присутствовать в схеме стора (следующая проверка), а их имена валидируются в TOlapColumnBase::ParseFromRequest.

Я обратил внимание, потому что одинаковая валидация описана в двух местах, что потом может привести к ошибкам (например, кто-то добавит другую валидацию только в одном месте и из-за этого не покроет часть сценариев).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Тесты схемных операций можно писать без KQP слоя, как в schemeshard/ut_olap/.... Лучше написать такой тест, чтобы, когда он падал, было очевидно, что проблема в схемшарде.

@Emgariko Emgariko changed the title Issue 14722 fix Unifying column names validation for both row and column tables #14722 Mar 10, 2025
@github-actions github-actions bot added bugfix and removed bugfix labels Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Spaces are supported in column names for COLUMN tables
3 participants