Volume rendering + mesh have occlusion problems

[haykh]

Hi! First of all, thanks for the awesome tool, this has been quite an upgrade compared to lots of other 3D tools that support python scripting. I hope to stick to using this for my research in the near future!

I was having some trouble doing a volume rendering with an included mesh in the scene. Here's what the snapshot looks like.

There clearly is a problem with the mesh being occluded by the volume rendering while in reality it should have covered the back side. Here's a view from top.

Paraview/Mayavi for comparison, while being infinitely many times less attractive to use, don't have these issues. I was wondering if there is an easy way to address this issue, or is there a fundamental problem with the engine which will make this problem unsolvable.

For the reference, here's the code that produces this plot.

# loading data to the `density` array
cmap = k3d.colormaps.matplotlib_color_maps.jet

volume = k3d.volume(
    density.astype(np.float32),
    alpha_coef=100,
    shadow='dynamic',
    samples=600,
    shadow_res=128,
    shadow_delay=50,
    color_range=[50, 1500],
    color_map=(np.array(cmap).reshape(-1,4)).astype(np.float32)
)
pnts = k3d.points(positions=[[0,0,0]], point_size = 2*RADIUS)

plot = k3d.plot(camera_auto_fit=False, height=800)

plot += volume
plot += pnts
plot.lighting = 2
plot.display()

Thank you in advance for your response!

[artur-trzesiok]

Nice example! Its looks like kind of vortex - could you provide more details about that?

That's a common problem in computer graphics called OIT - Order-independent transparency. Even when we talking only mesh to mesh interactions (with opacity) this can be tricky. I have in development stage a implementation of depth peeling. I will test that long time because that affect whole render pipeline in K3D.

Proper implementation of volume-mesh interaction with opacity is harder but can be done. Paraview have that options for a while but still expose this feature separately:

They have reason for that :).

K3D works on top of WebGL that works on top of Opengl ES. That limits significantly a capabilities of GPU that we can use (compare to Paraview). My initial PoC shows that i found way how to implement that in WebGL but I meet with many problems on mobile devices (this is quite unique feature of K3D that you can run snapshot of some research directly from your inbox on mobile phone).

Workaround:
You can voxelize w mesh and add it to "denisity" variable with high number of mask - so you will have only volume object.

Alternative:
I'm working on OIT but I cannot tell when that will be ready because this is not a highest priority for K3D at this moment.

Plan for OIT in K3D:

1. Deliver mesh to mesh interaction with OIT on depth peeling that will works only on limited amount of devices
2. Deliver mesh to mesh interaction with OIT on depth peeling that will recognize when capabilites allow to use it and have fallback strategy
3. Deliver volume to mesh interaction with OIT that utilies depth peeling middle stage data that will works only on limited amount of devices

[haykh]

Thanks for a quick and detailed response! Yes, indeed I've seen these problems occur in matplotlib 3d projection as well (with just simple meshes), so I figured this must be something fundamental.

You can voxelize w mesh and add it to "denisity" variable with high number of mask - so you will have only volume object.

This is certainly a solution, the problems here are:

a. my data is typically ~lowish resolution up to maybe 1500^3 (otherwise it won't fit the GPU memory), meaning the mesh in the middle itself is maybe 40x40 "cells" or so; that resolution is pretty bad for a sphere I think;

b. even more important is the fact that in our science we often want to overplot the fieldlines on top of the volume rendering (similar to what's shown here); that won't work with this rasterization.

So I'll be looking forward to seeing OIT implemented whenever that happens. I think this might and will be an important addition that could potentially attract people in our area who're stuck with old and clunky VisIt-s or mayavi-s. Otherwise, as I mentioned, the frontend of k3d looks flawless!

This is, for instance, the same snapshot reproduced in mayavi. Main problems with mayavi (among billions of others) is that it's not really maintained, the documentation is bad, and configuring it to do what you want is quite a headache (I basically had to hack the tvtk.ctf to make it look this way). As you might imagine, having a tool like k3d where you simply do plot += volume in jupyter and it looks awesome out of the box is immensely valuable!

Nice example! Its looks like kind of vortex - could you provide more details about that?

This is data from one of our plasma simulations of magnetospheres of spinning neutron stars. That sphere in the middle is the neutron star (which essentially just provides boundary conditions for our equations), and the color around is the density of plasma (electrons and positrons). What's important for us in terms of visualization (and actually also for people doing black-hole simulations) is to draw this together with magnetic fieldlines that thread the simulation box, because these fieldlines are the ones controlling the whole dynamics. Something similar to what is shown in the image below.

PS. Feel free to either close or label this issue at your convenience.