Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory #80

Open
emmaccode opened this issue Aug 5, 2023 · 2 comments
Open

Memory #80

emmaccode opened this issue Aug 5, 2023 · 2 comments
Labels
enhancement New feature or request performance
Milestone

Comments

@emmaccode
Copy link
Member

Olive has a pretty low base memory usage, but as mentioned this has not been a key focus for the project yet. The biggest problem currently is that Olive stacks memory, only waiting for ToolipsSession timeouts to completely clear events.
Why is this a problem? Session timeouts only happen when the session user is inactive. In other words, the function pipeline still holds data within events. There are a few ways we could facilitate this change in Olive, but there will also be some upcoming improvements to ToolipsSession to also improve upon an automatic clearing interface. I feel this is an apt improvement -- the next version step for that package also has a lot of really neat stuff in it -- and a lot of radical improvements! Anywho, here I will outline what needs to change about Olive. If you would like to read more about the changes to ToolipsSession, I opened an issue on that repository.


olive improvements

So something I want to change is that these functions also tend to store an outrageous amount of project data. At least for the event references, we do not want to be passing the actual Vector{Cell} itself. This of course stores it in the function pipeline. That isn't good, and sometimes this is done with Projects, as well. So a better approach is to have all event functions only hold references to get from the Connection, there is no sense passing it if it is in the Connection. We will need to index it, but this is the price we pay for greatness.

That really is about it other than #49 which is on the roster for 0.1.0. I just needed to get this thought here so I can assign it. cool stuff real soon ttyl <3

@emmaccode emmaccode added the enhancement New feature or request label Aug 5, 2023
@emmaccode emmaccode added this to the 0.1.0 Release milestone Aug 5, 2023
@emmaccode
Copy link
Member Author

0.0.91 has been the first update assessing memory in Olive. A huge problem we have here is simply julia -- the language is too high-level in that no garbage collection can be done by myself aside from gc and simply removing values with delete!, setting them to nothing, and deleteat!.

All of our actual memory things are getting refined to use as little as possible, the shortcoming yields in Julia removing previous Olive projects from memory, which I have certainly facilitated a little better with this 0.0.91 release. It might be worth looking into how Pluto does this. Either way, 0.0.91 is a really nice build and I am wrapping it up -- we will see!

@emmaccode
Copy link
Member Author

New version of Toolips has made GC and multi-threading both automatic, and this is going to be a substantial gain in terms of performance; for memory, processing time, and garbage collection. Everything is generally lighter, more functional, and performs a bit better as a result of the improvements to Session and Toolips. This will require a brief update to get working with Olive -- the main challenge is going to be that multi-threading does not work with closure callbacks.... Updating all of Olive to facilitate that will be a challenge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request performance
Projects
None yet
Development

No branches or pull requests

1 participant