Addition of Metalness Property for IBL and non-IBL.

[perminder-17]

Increasing Access

I would like to propose the addition of a new feature to the lighting system, specifically the inclusion of an "Image Lighting" functionality that introduces a metalness property to materials. This enhancement aims to make materials look more realistic, resembling actual metal surfaces.

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build Process
• Unit Testing
• Internalization
• Friendly Errors
• Other (specify if possible)

Feature request details

The implementation involves introducing a function, let's call it metalness(value, roughness), where:

1. The first parameter (value) controls the metallic property of the material.
   Increasing this parameter enhances the metallic appearance.
2. The second parameter (roughness) dictates the roughness of the material.
   Increasing this parameter results in a decrease in shininess.

This feature would significantly enhance the versatility of the lighting system, allowing developers to achieve a more authentic representation of metallic materials within their scenes. It provides a simple and intuitive way to control metalness and roughness.

[perminder-17]

It would be great if anyone here share's there thoughts on the implemntation or any futher changes to it. :)

[davepagurek]

Thanks for bringing this up! Before deciding to add something like this, I think we should answer a few questions about how it fits in with existing p5 functionality to see what the best approach would be.

According to https://learnopengl.com/PBR/Theory:

Metallic surfaces follow the same principles of reflection and refraction, but all refracted light gets directly absorbed without scattering. This means metallic surfaces only leave reflected or specular light; metallic surfaces show no diffuse colors.

Later on, they also say:

We preserve this energy conserving relation by first calculating the specular fraction that amounts the percentage the incoming light's energy is reflected. The fraction of refracted light is then directly calculated from the specular fraction as:

float kS = calculateSpecularComponent(...); // reflection/specular fraction
float kD = 1.0 - kS; // refraction/diffuse fraction

This way we know both the amount the incoming light reflects and the amount the incoming light refracts, while adhering to the energy conservation principle. Given this approach, it is impossible for both the refracted/diffuse and reflected/specular contribution to exceed 1.0, thus ensuring the sum of their energy never exceeds the incoming light energy. Something we did not take into account in the previous lighting chapters.

This leads me to two questions:

1. Are we able to manually create metals currently by setting the fill to black and setting the specularMaterial to the albedo color? How does that look right now?
2. They mention they balance specular and diffuse: if the specular component is high, less is used from the diffuse component. Currently, we just add the two together with no balancing. Does it still look OK if we do the things in (1) without the balancing? If we do need that balancing, what would that involve for non-image light in p5?

I think the answer to (1) can help guide discussion of accessibility, like if this is to make something already possible easier, or to enable something not currently possible.

[perminder-17]

I think I understand why we set the fill to black and use the albedo color for specular material in the first point. It's possible that we aim to eliminate diffuse light by setting the fill to black, and we only display the specular reflections by using the albedo color for the specularMaterial and also adding some shininess to it.

By applying this technique with imageLight , we achieve a result similar to this.

Please correct me if I'm mistaken @davepagurek . And wanted to know your further thoughts on this?

Although I'm not entirely certain about my thoughts, what I believe is that by eliminating diffuse lights and setting only specular lights, we may not be able to achieve a metallic texture. I suppose we need to explore enabling something that may not be currently possible like doing balancing stuffs or something else. Waiting for your thoughts :)

[davepagurek]

I guess we have two code paths to consider: one for image light, and one for all the other lights (point, directional, and spot lights.)

It looks like for image light, we definitely will need to adjust the way we balance between diffuse and specular highlights -- currently, since we are adding one on top of the other, we use a pow function to, essentially, threshold just the bright parts to appear in the specular highlights, which results in the material looking extra dark if there is no diffuse light.

p5.js/src/webgl/shaders/lighting.glsl

Lines 131 to 133 in fc660dd

	// this is to make the darker sections more dark
	// png and jpg usually flatten the brightness so it is to reverse that
	return pow(outColor.xyz, vec3(10.0));

To address it, it seems like we have two options:

• Introduce a new metalness uniform into the lighting shader that we can read when calculating these values to determine what proportion of specular and diffuse light to use, and to reduce the exponent of the pow when there will be mostly just specular light
• See if we can infer the metalness based on the difference in brightness between the specular material color and the fill color. We could potentially still add a metalness() helper function for users, but it would just set the fill and specular colors like in your test, and the resulting visuals would be the same regardless of whether you do it with the helper or manually.

The other thing to consider is how it would address the other light sources. How do those look right now with a black fill and non-black specular material? Do you think we could introduce the same balancing between diffuse and specular here? It could be worth experimenting with hardcoding some numbers in the shader to see what the effects of that balancing would be, to see if it looks the way we want.

p5.js/src/webgl/shaders/lighting.glsl

Lines 75 to 78 in fc660dd

	LightResult lr;
	if (uSpecular)
	lr.specular = _phongSpecular(lightDir, viewDirection, normal, uShininess);
	lr.diffuse = _lambertDiffuse(lightDir, normal);