-
Notifications
You must be signed in to change notification settings - Fork 33
Refactor TableSpan creation to use NewTableSpan constructor #3461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Refactors the creation of `heartbeatpb.TableSpan` instances across various packages to consistently use the `heartbeatpb.NewTableSpan` constructor. This change aims to: - Improve code readability and maintainability by centralizing span creation logic. - Ensure consistent initialization of `TableSpan` fields. - Reduce the likelihood of errors due to missing field initializations. The following packages and files have been updated: - `downstreamadapter/dispatcher/event_dispatcher_test.go` - `downstreamadapter/dispatchermanager/dispatcher_manager_helper.go` - `downstreamadapter/dispatchermanager/dispatcher_manager_test.go` - `downstreamadapter/eventcollector/dispatcher_stat_test.go` - `downstreamadapter/eventcollector/event_collector_test.go` - `heartbeatpb/table_span.go` - `logservice/eventstore/event_store_test.go` - `logservice/logpuller/priority_queue_test.go` - `logservice/logpuller/region_event_handler_test.go` - `logservice/logpuller/region_req_cache_test.go` - `logservice/logpuller/regionlock/region_range_lock.go` - `logservice/logpuller/regionlock/region_range_lock_test.go` - `logservice/logpuller/regionlock/util_test.go` - `logservice/logpuller/subscription_client.go` - `logservice/logpuller/subscription_client_test.go` - `logservice/schemastore/ddl_job_fetcher.go` - `maintainer/barrier_test.go` - `maintainer/maintainer_controller_bootstrap.go` - `maintainer/maintainer_controller_helper.go` - `maintainer/maintainer_controller_test.go` - `maintainer/operator/operator_add_test.go` - `maintainer/operator/operator_merge.go` - `maintainer/operator/operator_merge_test.go` - `maintainer/operator/operator_split_test.go` - `maintainer/replica/replication_span.go` - `maintainer/replica/replication_span_test.go` - `maintainer/replica/split_span_checker_test.go` - `maintainer/span/span_controller.go` - `maintainer/split/region_count_splitter.go` - `maintainer/split/region_count_splitter_test.go` - `maintainer/split/splitter_test.go` - `maintainer/split/write_bytes_splitter.go` - `maintainer/split/write_bytes_splitter_test.go` - `maintainer/testutil/test_util.go` - `pkg/common/format_test.go` - `pkg/common/span_hash_map.go` - `pkg/common/span_op.go` - `pkg/common/span_test.go` - `pkg/common/table_span.go` - `pkg/common/table_span_test.go` - `pkg/eventservice/event_scanner_test.go` - `pkg/eventservice/event_service_test.go` - `pkg/pdutil/api_client_test.go` - `pkg/spanz/btree_map.go` - `pkg/spanz/btree_map_test.go` - `pkg/spanz/convert.go` - `pkg/spanz/convert_test.go` Signed-off-by: tenfyzhong <[email protected]>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Summary of ChangesHello @tenfyzhong, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly refactors the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This is a great refactoring that improves code robustness by ensuring TableSpan objects are consistently initialized. The changes are applied thoroughly across the codebase.
I've identified a potential issue in pkg/common/span_hash_map.go and pkg/spanz/convert.go where KeyspaceID is not considered when creating a hashable span, which could lead to bugs when multiple keyspaces are in use.
Additionally, it seems there is some code duplication between pkg/common/span_hash_map.go and pkg/spanz/convert.go regarding hashableSpan and its helper functions. This could be a good opportunity for future cleanup to centralize this logic.
| StartKey: unsafeStringToBytes(h.StartKey), | ||
| EndKey: unsafeStringToBytes(h.EndKey), | ||
| } | ||
| span := heartbeatpb.NewTableSpan(h.TableID, unsafeStringToBytes(h.StartKey), unsafeStringToBytes(h.EndKey), 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
KeyspaceID is hardcoded to 0. This can lead to incorrect behavior when dealing with multiple keyspaces, as spans that are identical except for their TableID, StartKey, and EndKey will be treated as the same, causing hash collisions. The heartbeatpb.TableSpan.Equal method considers KeyspaceID, so the hashable representation should as well.
To fix this, hashableSpan struct should be updated to include KeyspaceID, and toHashableSpan and toSpan functions should be updated to handle it.
For example:
// hashableSpan is a hashable span, which can be used as a map key.
type hashableSpan struct {
TableID int64
StartKey string
EndKey string
KeyspaceID uint32
}
// toHashableSpan converts a Span to a hashable span.
func toHashableSpan(span heartbeatpb.TableSpan) hashableSpan {
return hashableSpan{
TableID: span.TableID,
StartKey: unsafeBytesToString(span.StartKey),
EndKey: unsafeBytesToString(span.EndKey),
KeyspaceID: span.KeyspaceID,
}
}
// toSpan converts to Span.
func (h hashableSpan) toSpan() heartbeatpb.TableSpan {
span := heartbeatpb.NewTableSpan(h.TableID, unsafeStringToBytes(h.StartKey), unsafeStringToBytes(h.EndKey), h.KeyspaceID)
return *span
}| StartKey: unsafeStringToBytes(h.StartKey), | ||
| EndKey: unsafeStringToBytes(h.EndKey), | ||
| } | ||
| span := heartbeatpb.NewTableSpan(h.TableID, unsafeStringToBytes(h.StartKey), unsafeStringToBytes(h.EndKey), 0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
KeyspaceID is hardcoded to 0. This can lead to incorrect behavior when dealing with multiple keyspaces, as spans that are identical except for their TableID, StartKey, and EndKey will be treated as the same, causing hash collisions. The heartbeatpb.TableSpan.Equal method considers KeyspaceID, so the hashable representation should as well.
To fix this, hashableSpan struct should be updated to include KeyspaceID, and toHashableSpan and toSpan functions should be updated to handle it.
For example:
// hashableSpan is a hashable span, which can be used as a map key.
type hashableSpan struct {
TableID int64
StartKey string
EndKey string
KeyspaceID uint32
}
// toHashableSpan converts a Span to a hashable span.
func toHashableSpan(span heartbeatpb.TableSpan) hashableSpan {
return hashableSpan{
TableID: span.TableID,
StartKey: unsafeBytesToString(span.StartKey),
EndKey: unsafeBytesToString(span.EndKey),
KeyspaceID: span.KeyspaceID,
}
}
// toSpan converts to Span.
func (h hashableSpan) toSpan() heartbeatpb.TableSpan {
span := heartbeatpb.NewTableSpan(h.TableID, unsafeStringToBytes(h.StartKey), unsafeStringToBytes(h.EndKey), h.KeyspaceID)
return *span
}
What problem does this PR solve?
This PR refactors the
heartbeatpb.TableSpanstruct by introducing aNewTableSpanconstructor. This constructor ensures that all fields ofTableSpanare properly initialized, preventing potential issues arising from uninitialized fields. This change aims to improve code robustness and maintainability by enforcing consistent initialization patterns forTableSpanobjects across the codebase.Issue Number: close #2689
What is changed and how it works?
The primary change is the addition of the
NewTableSpanfunction inheartbeatpb/table_span.go. This function serves as a factory for creatingTableSpaninstances, ensuring thatTableID,StartKey,EndKey, andKeyspaceIDare all explicitly set.Throughout the codebase, instances of
heartbeatpb.TableSpanthat were previously created using struct literals (e.g.,&heartbeatpb.TableSpan{...}) are now replaced with calls toheartbeatpb.NewTableSpan(...). This systematic replacement ensures that allTableSpanobjects are created using the new, safer constructor.The
NewTableSpanconstructor is designed to be flexible and can be used to createTableSpanobjects with any combination of fields populated, while still ensuring that all fields have a defined zero value if not explicitly provided.Check List
Tests
Questions
Will it cause performance regression or break compatibility?
No, this change should not cause performance regressions. The
NewTableSpanconstructor is a simple initialization function, and replacing struct literals with constructor calls is a common refactoring that does not impact performance.This change is backward-compatible as it only affects how
TableSpanobjects are created internally. Existing code that readsTableSpanobjects will continue to function correctly.Do you need to update user documentation, design documentation or monitoring documentation?
No, this change is an internal refactoring and does not require updates to user-facing documentation, design documents, or monitoring configurations.
Release note