Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using buffered channel in the aggregation process #267

Open
srikartati opened this issue Oct 22, 2021 · 0 comments
Open

Using buffered channel in the aggregation process #267

srikartati opened this issue Oct 22, 2021 · 0 comments

Comments

@srikartati
Copy link
Contributor

Is your feature request related to a problem? Please describe.
We have multiple senders in the collection process with goroutines for each client, and multiple receivers with workers in the aggregation process. With the current implementation using the unbuffered channel, senders will wait for the worker to finish the processing of the message to move forward. In addition, other workers are not being run in parallel.

sched trace will show the usage of logical cores and this will let us show the extent of parallelism.
In performance tests, we should check the above. Because of this, with UDP clients sending to the collector process, we may see record loss. Moreover, with TCP clients, we won't be getting good performance in terms of bandwidth between the client and server in the collection process.

The data needs to be looked but it is mostly the case according to the golang concepts
Till now we looked at memory aspects of the aggregation process in the usage with Antrea. CPU time needs to be looked at.

Describe the solution you'd like
Move to the buffered channel.

Additional context
Anything else which is relevant and may help us understand the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant