Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Godot 4] Better numpy support for image / np.arrays #370

Open
TheBricktop opened this issue Sep 8, 2022 · 2 comments
Open

[Godot 4] Better numpy support for image / np.arrays #370

TheBricktop opened this issue Sep 8, 2022 · 2 comments

Comments

@TheBricktop
Copy link

As @touilleMan himself wrote in one of the issues, there is a considerable amount of developers that are interested in using python in godot for image processing / computer vision and AI or machine learning.
Now the communication between numpy standard library and Godots way of handling large byte arrays are not so optimal so there is a room for improvement.
This issue is made to collect the ideas, discuss possibilities and usecases for numpy in godot.

@TheBricktop
Copy link
Author

My input:

  • More performant communication from opencv / numpy.arrays to godot types
  • functions that encapsulate creating images or streaming audio files generated in python.

@kb173
Copy link

kb173 commented Nov 1, 2022

I know you commented on my related issue for this (#305) but for the sake of completeness, this is how one can currently go about efficiently converting a Matplotlib canvas (a Numpy Array) to a Godot PoolByteArray:

def canvas_to_godot_image(self, canvas):
	image = Image()
	
	width = canvas.get_width_height()[1]
	height = canvas.get_width_height()[0]
	
	size = width * height * 3
	
	pool_array = PoolByteArray()
	pool_array.resize(size)
	
	with pool_array.raw_access() as ptr:
		numpy_array = np.ctypeslib.as_array(ctypes.cast(ptr.get_address(), ctypes.POINTER(ctypes.c_uint8)), (size,))
		numpy_array[:] = np.frombuffer(canvas.tostring_rgb(), dtype=np.uint8).reshape(canvas.get_width_height()[::-1] + (3,)).flatten()
	
	image.create_from_data(height, width, False, Image.FORMAT_RGB8, pool_array)
	return image

As you wrote, it would be great if godot-python could provide this kind of functionality out of the box.

I think there are two ways in which this could be done:

  1. Define a PoolByteArray constructor specifically to construct from a Numpy Array, or
  2. Provide a Numpy-specific util class with a function such as NumpyUtil.numpy_array_to_pool_byte_array(array)

I think the first one would be more comfortable for users since they don't need to know that Numpy Arrays are a special case with a performant conversion method. However, the second one would be better for the godot-python codebase since it doesn't pollute the core codebase with Numpy-specific functionality.

Alternatively (or in addition), I think both options are realizable with compile options specifying whether Numpy-specific functionality should be compiled in or not - that would be another way of keeping things a bit separate.

@touilleMan what are your thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants