Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added separated rate limits for gateway #10525

Closed

Conversation

imbeacon
Copy link
Member

@imbeacon imbeacon commented Apr 10, 2024

Pull Request description

PE PR - https://github.com/thingsboard/thingsboard-pe/pull/2497

Such as a gateway can send some necessary logs and it may trigger rate limits, we need to have a different rate limit for gateway devices.
In this PR I have added separated rate limits, for gateway devices, they can be configured in tenant profile configuration and they affects only devices that marked as a gateway.

General checklist

  • You have reviewed the guidelines document.
  • Labels that classify your pull request have been added.
  • The milestone is specified and corresponds to fix version.
  • Description references specific issue.
  • Description contains human-readable scope of changes.
  • Description contains brief notes about what needs to be added to the documentation.
  • No merge conflicts, commented blocks of code, code formatting issues.
  • Changes are backward compatible or upgrade script is provided.
  • Similar PR is opened for PE version to simplify merge. Crosslinks between PRs added. Required for internal contributors only.

Front-End feature checklist

  • Screenshots with affected component(s) are added. The best option is to provide 2 screens: before and after changes;
  • If you change the widget or other API, ensure it is backward-compatible or upgrade script is present.
  • Ensure new API is documented here

Back-End feature checklist

  • Added corresponding unit and/or integration test(s). Provide written explanation in the PR description if you have failed to add tests.
  • If new dependency was added: the dependency tree is checked for conflicts.
  • If new service was added: the service is marked with corresponding @TbCoreComponent, @TbRuleEngineComponent, @TbTransportComponent, etc.
  • If new REST API was added: the RestClient.java was updated, issue for Python REST client is created.
  • If new yml property was added: make sure a description is added (above or near the property).

@imbeacon imbeacon added Core Changes to Core Transport Changes to Transport labels Apr 10, 2024
@imbeacon imbeacon added this to the 3.7 milestone Apr 10, 2024
Comment on lines +162 to +163
WHEN trigger_config::jsonb->'apis' ? 'TRANSPORT_MESSAGES_PER_GATEWAY' THEN trigger_config
ELSE jsonb_set(trigger_config::jsonb, '{apis}', (trigger_config::jsonb->'apis') || '"TRANSPORT_MESSAGES_PER_GATEWAY"', false)::text
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check how does notification look like (with default notification rule) when per-gateway rate limits are exceeded

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like regular rate limits exceeded notification, just with different content - "transport messages per gateway" instead of "transport messages per device"

Comment on lines +174 to +179
<dependency>
<groupId>org.thingsboard</groupId>
<artifactId>application</artifactId>
<version>${project.version}</version>
<scope>test</scope>
</dependency>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because without this dependency we have no access for necessary entities like EntityDataUpdate, UnreadNotificationsUpdate, AlarmCountUpdate etc. I think it will be better to have this dependency instead of having two similar entities in different places.

@@ -134,3 +134,33 @@ DELETE FROM asset WHERE type='TbServiceQueue';
DELETE FROM asset_profile WHERE name ='TbServiceQueue';

-- QUEUE STATS UPDATE END
-- TENANT PROFILE UPDATE START

UPDATE tenant_profile tp
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please rewrite this to 3 separate statements (one per configuration item, e.g. transportDeviceMsgRateLimit ) and also check that the field you are using is set. For example, that the transportDeviceMsgRateLimit is set.

@ashvayka
Copy link
Member

ashvayka commented May 7, 2024

  1. Create PR from scratch once we finish the review because it has 27 commits and 24 files.
  2. Avoid any changes to the MSA package. We don't need to run black-box tests to check the rate limits. Such logic needs to be validated with the unit/integration tests on the transport level.

@@ -25,5 +25,8 @@ public class EntityTransportRateLimits {
private TransportRateLimit regularMsgRateLimit;
private TransportRateLimit telemetryMsgRateLimit;
private TransportRateLimit telemetryDataPointsRateLimit;
private TransportRateLimit gatewayMsgRateLimit;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is completely redundant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Core Changes to Core Transport Changes to Transport
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants