提问人:Aleksandar Kondić 提问时间:11/15/2023 更新时间:11/15/2023 访问量:40
如何在vispy中曲面图的背景中显示图像?
How to show an image in the background of a surface plot in vispy?
问:
我想在 vispy 中有一个曲面图,但也要显示一个 2D 图像来代替白色背景。
我使用 vispy 的 Draw a SurfacePlot 示例作为基础。我尝试的第一件事就是向视图添加一个对象,但最终将图像渲染为 3D 场景中的对象之一:scene.visuals.Image
canvas = scene.SceneCanvas(keys='interactive', bgcolor='w')
background_texture = scene.visuals.Image(np.random.rand(512, 512, 3).astype(np.float32))
view = canvas.central_widget.add_view()
view.camera = scene.TurntableCamera(up='z', fov=60)
view.add(background_texture)
...
由于图像需要在背景中渲染,我认为它需要自己的 2D 视图,在此基础上渲染 3D 场景,因此我为图像创建了一个新视图:SurfacePlot
canvas = scene.SceneCanvas(keys='interactive', bgcolor='w')
background_texture = scene.visuals.Image(np.random.rand(512, 512, 3).astype(np.float32))
viewbg = canvas.central_widget.add_view()
viewbg.add(background_texture)
view = canvas.central_widget.add_view()
view.camera = scene.TurntableCamera(up='z', fov=60)
...
但是,图像呈现在曲面图的顶部,但轴标签除外,轴标签呈现在图像顶部。重新排序和代码 so 首先创建不会有什么区别;这只会让事情变得更糟,因为那样我也会失去与表面图交互的能力。view
viewbg
view
有没有办法在图像上渲染整个 3D 场景?
答:
多亏了@djhoese的指导,我才能够想出一些解决方案。首先,背景图像必须位于场景图之外,以免它成为视图显示的 3D 空间的一部分。之后是实际显示 3D 场景后面图像的问题。view
TurntableCamera
解决方案 1:在绘制图像后和绘制 3D 场景之前清除 OpenGL 深度缓冲区
最干净的方法是先绘制图像,然后在绘制 3D 场景之前清除 OpenGL 深度缓冲区。这将确保场景绘制在图像的顶部。最简单的方法是在绘制后对 进行子类化并添加调用:scene.Image
gloo.clear(color=False, depth=True)
class BackgroundImage(scene.Image):
def draw(self):
super().draw()
gloo.clear(color=False, depth=True)
确保先于其他任何内容进行渲染。完整代码如下:BackgroundImage
import sys
import numpy as np
from vispy import app, scene, gloo
from vispy.util.filter import gaussian_filter
class BackgroundImage(scene.Image):
def draw(self):
super().draw()
gloo.clear(color=False, depth=True)
canvas = scene.SceneCanvas(keys='interactive', bgcolor='w')
view_bg = canvas.central_widget.add_view(camera=scene.PanZoomCamera())
view_bg.order = float("-inf") # For good measure
background_image = BackgroundImage(np.random.rand(512, 512, 3).astype(np.float32), parent=view_bg.scene)
view_bg.camera.set_range(margin=0)
view = canvas.central_widget.add_view()
view.camera = scene.TurntableCamera(up='z', fov=60)
# Simple surface plot example
# x, y values are not specified, so assumed to be 0:50
z = np.random.normal(size=(250, 250), scale=200)
z[100, 100] += 50000
z = gaussian_filter(z, (10, 10))
p1 = scene.visuals.SurfacePlot(z=z, color=(0.3, 0.3, 1, 1))
p1.transform = scene.transforms.MatrixTransform()
p1.transform.scale([1/249., 1/249., 1/249.])
p1.transform.translate([-0.5, -0.5, 0])
view.add(p1)
# p1._update_data() # cheating.
# cf = scene.filters.ZColormapFilter('fire', zrange=(z.max(), z.min()))
# p1.attach(cf)
xax = scene.Axis(pos=[[-0.5, -0.5], [0.5, -0.5]], tick_direction=(0, -1),
font_size=16, axis_color='k', tick_color='k', text_color='k',
parent=view.scene)
xax.transform = scene.STTransform(translate=(0, 0, -0.2))
yax = scene.Axis(pos=[[-0.5, -0.5], [-0.5, 0.5]], tick_direction=(-1, 0),
font_size=16, axis_color='k', tick_color='k', text_color='k',
parent=view.scene)
yax.transform = scene.STTransform(translate=(0, 0, -0.2))
# Add a 3D axis to keep us oriented
axis = scene.visuals.XYZAxis(parent=view.scene)
if __name__ == '__main__':
canvas.show()
if sys.flags.interactive == 0:
app.run()
A 用于确保缩放图像以适合整个画布。您将无法与该相机进行交互,因为带有 的视图绘制在顶部,因此它会进行对焦。PanZoomCamera
background_image
TurntableCamera
泛化解决方案以在背景中绘制任何场景
如果要绘制任何场景,而不仅仅是在另一个场景的背景中绘制,则可以实现一个子类,该子类在绘制其子类后清除深度缓冲区。最好从中继承,因为它提供了最多的功能:Image
Node
Widget
class Background(scene.Widget):
"""Node that clears the depth buffer after drawing its children."""
ClearDepth = type("ClearDepthBuffer", (scene.Node,), {'draw': lambda self: gloo.clear(color=False, depth=True)})
def __init__(self, *args, **kwargs):
self._last = None
super().__init__(*args, **kwargs)
self._last = self.ClearDepth(parent=self)
self._last.order = float("inf")
assert self._children.pop() == self._last
@property
def children(self):
return super().children + [self._last] if self._last is not None else super().children
确保它实际上是在绘制其他任何内容之前绘制的。使用这种方法的全曲面图代码如下:Background
import sys
import numpy as np
from vispy import app, scene, gloo
from vispy.util.filter import gaussian_filter
class Background(scene.Widget):
"""Node that clears the depth buffer after drawing its children."""
ClearDepth = type("ClearDepthBuffer", (scene.Node,), {'draw': lambda self: gloo.clear(color=False, depth=True)})
def __init__(self, *args, **kwargs):
self._last = None
super().__init__(*args, **kwargs)
self._last = self.ClearDepth(parent=self)
self._last.order = float("inf")
assert self._children.pop() == self._last
@property
def children(self):
return super().children + [self._last] if self._last is not None else super().children
canvas = scene.SceneCanvas(keys='interactive', bgcolor='w')
background = canvas.central_widget.add_widget(Background())
background.order = float("-inf") # For good measure
view_bg = background.add_view(camera=scene.PanZoomCamera())
background_image = scene.Image(np.random.rand(512, 512, 3).astype(np.float32), parent=view_bg.scene)
view_bg.camera.set_range(margin=0)
view = canvas.central_widget.add_view()
view.camera = scene.TurntableCamera(up='z', fov=60)
print(canvas.scene.describe_tree())
# Simple surface plot example
# x, y values are not specified, so assumed to be 0:50
z = np.random.normal(size=(250, 250), scale=200)
z[100, 100] += 50000
z = gaussian_filter(z, (10, 10))
p1 = scene.visuals.SurfacePlot(z=z, color=(0.3, 0.3, 1, 1))
p1.transform = scene.transforms.MatrixTransform()
p1.transform.scale([1/249., 1/249., 1/249.])
p1.transform.translate([-0.5, -0.5, 0])
view.add(p1)
# p1._update_data() # cheating.
# cf = scene.filters.ZColormapFilter('fire', zrange=(z.max(), z.min()))
# p1.attach(cf)
xax = scene.Axis(pos=[[-0.5, -0.5], [0.5, -0.5]], tick_direction=(0, -1),
font_size=16, axis_color='k', tick_color='k', text_color='k',
parent=view.scene)
xax.transform = scene.STTransform(translate=(0, 0, -0.2))
yax = scene.Axis(pos=[[-0.5, -0.5], [-0.5, 0.5]], tick_direction=(-1, 0),
font_size=16, axis_color='k', tick_color='k', text_color='k',
parent=view.scene)
yax.transform = scene.STTransform(translate=(0, 0, -0.2))
# Add a 3D axis to keep us oriented
axis = scene.visuals.XYZAxis(parent=view.scene)
if __name__ == '__main__':
canvas.show()
if sys.flags.interactive == 0:
app.run()
请确保在使用 或 的子类时使用 和方法,以便正确处理画布事件。这种方法的巧妙之处在于,您可以将多个实例渲染成一行,每个实例都渲染在前一个实例之上,从而为您提供多层背景,其中每个背景可以是任意的 2D 或 3D 场景。add_widget
add_view
Widget
Widget
Background
解决方案 2:对视图外部的图像使用平移和缩放转换
清除 OpenGL 深度缓冲区的替代方法是平移 ,使其坐标尽可能接近,但实际上不等于 。但是,这种方法不适用于 ,因此您还必须处理图像的缩放,以便在调整画布大小时它适合整个画布。该事件也会在初始画布设置期间调用。完整代码如下:Image
z
1.0
1.0
PanZoomCamera
resize
import sys
import numpy as np
from vispy import app, scene
from vispy.util.filter import gaussian_filter
from vispy.visuals.transforms import STTransform
canvas = scene.SceneCanvas(keys='interactive', bgcolor='w')
background_image = scene.Image(np.random.rand(512, 512, 3).astype(np.float32), parent=canvas.central_widget)
canvas.events.resize.connect(lambda _: background_image.__setattr__('transform', STTransform(
translate=(0, 0, 0.999999970197677556793536268742172978818416595458984374),
scale=tuple(c / b for c, b in zip(canvas.size, background_image.size))
)))
view = canvas.central_widget.add_view()
view.camera = scene.TurntableCamera(up='z', fov=60)
# Simple surface plot example
# x, y values are not specified, so assumed to be 0:50
z = np.random.normal(size=(250, 250), scale=200)
z[100, 100] += 50000
z = gaussian_filter(z, (10, 10))
p1 = scene.visuals.SurfacePlot(z=z, color=(0.3, 0.3, 1, 1))
p1.transform = scene.transforms.MatrixTransform()
p1.transform.scale([1/249., 1/249., 1/249.])
p1.transform.translate([-0.5, -0.5, 0])
view.add(p1)
# p1._update_data() # cheating.
# cf = scene.filters.ZColormapFilter('fire', zrange=(z.max(), z.min()))
# p1.attach(cf)
xax = scene.Axis(pos=[[-0.5, -0.5], [0.5, -0.5]], tick_direction=(0, -1),
font_size=16, axis_color='k', tick_color='k', text_color='k',
parent=view.scene)
xax.transform = scene.STTransform(translate=(0, 0, -0.2))
yax = scene.Axis(pos=[[-0.5, -0.5], [-0.5, 0.5]], tick_direction=(-1, 0),
font_size=16, axis_color='k', tick_color='k', text_color='k',
parent=view.scene)
yax.transform = scene.STTransform(translate=(0, 0, -0.2))
# Add a 3D axis to keep us oriented
axis = scene.visuals.XYZAxis(parent=view.scene)
if __name__ == '__main__':
canvas.show()
if sys.flags.interactive == 0:
app.run()
使用这种方法,您不必担心绘制 3D 场景之前。任何绘图顺序都有效。如果图像未显示,则其坐标可能被舍入为 ,因此请尝试删除 参数中的几个小数点。的值 和 up 应该足够好用。我只是去找到了我可以放入坐标中的最大可能值,该坐标将为我显示图像。Image
z
1.0
translate
STTransform
0.9999
z
评论
Image
parent=canvas.scene
Image
parent=canvas.scene
SceneCanvas
image.transform = STTransform(translate=(0, 0, -100))
scale=(x, y, z)