Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Solves issue #7059 #7113

Merged
merged 18 commits into from
Nov 2, 2024
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions src/core/environment.js
Original file line number Diff line number Diff line change
Expand Up @@ -1146,6 +1146,128 @@ function exitFullscreen() {
}
}


/**
* Converts 3D world coordinates to 2D screen coordinates.
*
* This function takes a 3D vector and converts its coordinates
* from the world space to screen space. This is useful for placing
* 2D elements in a 3D scene or for determining the screen position
* of 3D objects.
*
* @method worldToScreen
* @param {p5.Vector} worldPosition The 3D coordinates in the world space.
* @return {p5.Vector} A vector containing the 2D screen coordinates.
* @example
* <div>
* <code>
* // Example 1: Convert 2D world coordinates of a rotating square to screen coordinates
* function setup() {
* createCanvas(100, 100);
*
* let vertices = [
* createVector(-5, -5),
* createVector(5, -5),
* createVector(5, 5),
* createVector(-5, 5)
* ];
*
* push(); // Start a new drawing state
* translate(50, 50);
* rotate(PI / 4);
*
* // Convert each vertex to screen coordinates
* let screenPos = vertices.map(v => worldToScreen(v));
* pop(); // Restore original drawing state
*
* background(200);
*
* screenPos.forEach((pos, i) => {
* text(`(${pos.x.toFixed(1)}, ${pos.y.toFixed(1)})`, pos.x, pos.y);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

* });
* }
* </code>
* </div>
* @example
* <div>
* <code>
* // Example 2: Convert 3D world coordinates of a rotating cube to 2D screen coordinates
* let vertices;
*
* function setup() {
* createCanvas(100, 100, WEBGL);
* vertices = [
* createVector(-25, -25, -25),
* createVector(25, -25, -25),
* createVector(25, 25, -25),
* createVector(-25, 25, -25),
* createVector(-25, -25, 25),
* createVector(25, -25, 25),
* createVector(25, 25, 25),
* createVector(-25, 25, 25)
* ];
* }
*
* function draw() {
* background(200);
*
* // Animate rotation
* let rotationX = millis() / 1000;
* let rotationY = millis() / 1200;
*
* push();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you feel about moving this to draw() and making the rotation change slowly over time?

Copy link
Member Author

@Garima3110 Garima3110 Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A great idea , will do this in a while.

*
* rotateX(rotationX);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want to make sure the text itself isn't rotated, I think we might need to wrap the rotation + worldToScreen calls in a push/pop like you've got in the 2D mode example.

* rotateY(rotationY);
*
* // Convert world coordinates to screen coordinates
* let screenPos = vertices.map(v => worldToScreen(v));
*
* pop();
*
* // Display screen coordinates
* screenPos.forEach((pos, i) => {
*
* let screenX = pos.x - width / 2;
* let screenY = pos.y - height / 2;
* fill(255);
* noStroke();
* ellipse(screenX, screenY, 3, 3); // Draw points as small ellipses
* fill(0);
* text(`(${screenX.toFixed(1)}, ${screenY.toFixed(1)})`, screenX, screenY);
* });
* }
* </code>
* </div>
*
*/

p5.prototype.worldToScreen = function(worldPosition) {
const renderer = this._renderer;
if (renderer.drawingContext instanceof CanvasRenderingContext2D) {
// Handle 2D context
const transformMatrix = new DOMMatrix()
.scale(1 / renderer._pInst.pixelDensity())
.multiply(renderer.drawingContext.getTransform());
const screenCoordinates = transformMatrix.transformPoint(
new DOMPoint(worldPosition.x, worldPosition.y)
);
return new p5.Vector(screenCoordinates.x, screenCoordinates.y);
} else {
// Handle WebGL context (3D)
const modelViewMatrix = renderer.calculateCombinedMatrix();
const cameraCoordinates = modelViewMatrix.multiplyPoint(worldPosition);
const normalizedDeviceCoordinates =
renderer.uPMatrix.multiplyAndNormalizePoint(cameraCoordinates);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this one is causing the test failures now:
image

I think this one should be in states too:

Suggested change
renderer.uPMatrix.multiplyAndNormalizePoint(cameraCoordinates);
renderer.states.uPMatrix.multiplyAndNormalizePoint(cameraCoordinates);

const screenX = (0.5 + 0.5 * normalizedDeviceCoordinates.x) * this.width;
const screenY = (0.5 - 0.5 * normalizedDeviceCoordinates.y) * this.height;
const screenZ = 0.5 + 0.5 * normalizedDeviceCoordinates.z;
return new p5.Vector(screenX, screenY, screenZ);
}
};



/**
* Returns the sketch's current
* <a href="https://developer.mozilla.org/en-US/docs/Learn/Common_questions/Web_mechanics/What_is_a_URL" target="_blank">URL</a>
Expand Down
9 changes: 9 additions & 0 deletions src/webgl/p5.RendererGL.js
Original file line number Diff line number Diff line change
Expand Up @@ -935,6 +935,15 @@
this.clear(_r, _g, _b, _a);
}

// Combines the model and view matrices to get the uMVMatrix
// This method will be reusable wherever you need to update the combined matrix.
calculateCombinedMatrix() {
const modelMatrix = this.uModelMatrix;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this and the line below would need to access states now:

Suggested change
const modelMatrix = this.uModelMatrix;
const modelMatrix = this.states.uModelMatrix;

const viewMatrix = this.uViewMatrix;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const viewMatrix = this.uViewMatrix;
const viewMatrix = this.states.uViewMatrix;

return modelMatrix.copy().mult(viewMatrix);

Check failure on line 943 in src/webgl/p5.RendererGL.js

View workflow job for this annotation

GitHub Actions / test

test/unit/core/environment.js > Environment > 3D context test > worldToScreen for 3D context

TypeError: Cannot read properties of undefined (reading 'copy') ❯ RendererGL.calculateCombinedMatrix src/webgl/p5.RendererGL.js:943:24 ❯ p5.worldToScreen src/core/environment.js:1258:42 ❯ test/unit/core/environment.js:271:28

Check failure on line 943 in src/webgl/p5.RendererGL.js

View workflow job for this annotation

GitHub Actions / test

test/unit/core/environment.js > Environment > 3D context test > worldToScreen with rotation in 3D around Y-axis

TypeError: Cannot read properties of undefined (reading 'copy') ❯ RendererGL.calculateCombinedMatrix src/webgl/p5.RendererGL.js:943:24 ❯ p5.worldToScreen src/core/environment.js:1258:42 ❯ test/unit/core/environment.js:280:28

Check failure on line 943 in src/webgl/p5.RendererGL.js

View workflow job for this annotation

GitHub Actions / test

test/unit/core/environment.js > Environment > 3D context test > worldToScreen with rotation in 3D around Z-axis

TypeError: Cannot read properties of undefined (reading 'copy') ❯ RendererGL.calculateCombinedMatrix src/webgl/p5.RendererGL.js:943:24 ❯ p5.worldToScreen src/core/environment.js:1258:42 ❯ test/unit/core/environment.js:290:28
}


//////////////////////////////////////////////
// COLOR
//////////////////////////////////////////////
Expand Down
2 changes: 1 addition & 1 deletion src/webgl/p5.Shader.js
Original file line number Diff line number Diff line change
Expand Up @@ -937,7 +937,7 @@
const viewMatrix = this._renderer.states.uViewMatrix;
const projectionMatrix = this._renderer.states.uPMatrix;
const modelViewMatrix = (modelMatrix.copy()).mult(viewMatrix);
this._renderer.states.uMVMatrix = modelViewMatrix;
this._renderer.states.uMVMatrix = this._renderer.states.calculateCombinedMatrix();

Check failure on line 940 in src/webgl/p5.Shader.js

View workflow job for this annotation

GitHub Actions / test

test/unit/webgl/p5.Camera.js > p5.Camera > Camera attributes after resizing > Camera position is the same

TypeError: this._renderer.states.calculateCombinedMatrix is not a function ❯ Shader._setMatrixUniforms src/webgl/p5.Shader.js:940:63 ❯ Shader.bindShader src/webgl/p5.Shader.js:883:14 ❯ RendererGL._setFillUniforms src/webgl/p5.RendererGL.js:2145:16 ❯ p5.RendererGL.drawBuffers src/webgl/p5.RendererGL.Retained.js:143:10 ❯ p5.RendererGL.rect src/webgl/3d_primitives.js:2685:14 ❯ p5._renderRect src/core/shape/2d_primitives.js:1359:20 ❯ p5.rect src/core/shape/2d_primitives.js:1220:15 ❯ testShape test/unit/webgl/p5.Camera.js:1141:14 ❯ test/unit/webgl/p5.Camera.js:1144:7

Check failure on line 940 in src/webgl/p5.Shader.js

View workflow job for this annotation

GitHub Actions / test

test/unit/webgl/p5.Camera.js > p5.Camera > Camera attributes after resizing > Camera rotation is the same

TypeError: this._renderer.states.calculateCombinedMatrix is not a function ❯ Shader._setMatrixUniforms src/webgl/p5.Shader.js:940:63 ❯ Shader.bindShader src/webgl/p5.Shader.js:883:14 ❯ RendererGL._setFillUniforms src/webgl/p5.RendererGL.js:2145:16 ❯ p5.RendererGL.drawBuffers src/webgl/p5.RendererGL.Retained.js:143:10 ❯ p5.RendererGL.rect src/webgl/3d_primitives.js:2685:14 ❯ p5._renderRect src/core/shape/2d_primitives.js:1359:20 ❯ p5.rect src/core/shape/2d_primitives.js:1220:15 ❯ testShape test/unit/webgl/p5.Camera.js:1172:14 ❯ test/unit/webgl/p5.Camera.js:1175:7

Check failure on line 940 in src/webgl/p5.Shader.js

View workflow job for this annotation

GitHub Actions / test

test/unit/webgl/p5.Framebuffer.js > p5.Framebuffer > formats and channels > framebuffers work with WebGL 1, unsigned-byte rgba unsigned-int depth antialiased

TypeError: this._renderer.states.calculateCombinedMatrix is not a function ❯ Shader._setMatrixUniforms src/webgl/p5.Shader.js:940:63 ❯ Shader.bindShader src/webgl/p5.Shader.js:883:14 ❯ RendererGL._setFillUniforms src/webgl/p5.RendererGL.js:2145:16 ❯ p5.RendererGL.drawBuffers src/webgl/p5.RendererGL.Retained.js:143:10 ❯ p5.RendererGL.drawBuffersScaled src/webgl/p5.RendererGL.Retained.js:228:10 ❯ fn.box src/webgl/3d_primitives.js:1221:20 ❯ test/unit/webgl/p5.Framebuffer.js:44:18 ❯ Framebuffer.draw src/webgl/p5.Framebuffer.js:1310:7 ❯ test/unit/webgl/p5.Framebuffer.js:40:15

Check failure on line 940 in src/webgl/p5.Shader.js

View workflow job for this annotation

GitHub Actions / test

test/unit/webgl/p5.Framebuffer.js > p5.Framebuffer > formats and channels > framebuffers work with WebGL 1, unsigned-byte rgba float depth antialiased

TypeError: this._renderer.states.calculateCombinedMatrix is not a function ❯ Shader._setMatrixUniforms src/webgl/p5.Shader.js:940:63 ❯ Shader.bindShader src/webgl/p5.Shader.js:883:14 ❯ RendererGL._setFillUniforms src/webgl/p5.RendererGL.js:2145:16 ❯ p5.RendererGL.drawBuffers src/webgl/p5.RendererGL.Retained.js:143:10 ❯ p5.RendererGL.drawBuffersScaled src/webgl/p5.RendererGL.Retained.js:228:10 ❯ fn.box src/webgl/3d_primitives.js:1221:20 ❯ test/unit/webgl/p5.Framebuffer.js:44:18 ❯ Framebuffer.draw src/webgl/p5.Framebuffer.js:1310:7 ❯ test/unit/webgl/p5.Framebuffer.js:40:15

Check failure on line 940 in src/webgl/p5.Shader.js

View workflow job for this annotation

GitHub Actions / test

test/unit/webgl/p5.Framebuffer.js > p5.Framebuffer > formats and channels > framebuffers work with WebGL 1, unsigned-byte rgba no depth antialiased

TypeError: this._renderer.states.calculateCombinedMatrix is not a function ❯ Shader._setMatrixUniforms src/webgl/p5.Shader.js:940:63 ❯ Shader.bindShader src/webgl/p5.Shader.js:883:14 ❯ RendererGL._setFillUniforms src/webgl/p5.RendererGL.js:2145:16 ❯ p5.RendererGL.drawBuffers src/webgl/p5.RendererGL.Retained.js:143:10 ❯ p5.RendererGL.drawBuffersScaled src/webgl/p5.RendererGL.Retained.js:228:10 ❯ fn.box src/webgl/3d_primitives.js:1221:20 ❯ test/unit/webgl/p5.Framebuffer.js:44:18 ❯ Framebuffer.draw src/webgl/p5.Framebuffer.js:1310:7 ❯ test/unit/webgl/p5.Framebuffer.js:40:15

Check failure on line 940 in src/webgl/p5.Shader.js

View workflow job for this annotation

GitHub Actions / test

test/unit/webgl/p5.Framebuffer.js > p5.Framebuffer > formats and channels > framebuffers work with WebGL 1, unsigned-byte rgba unsigned-int depth

TypeError: this._renderer.states.calculateCombinedMatrix is not a function ❯ Shader._setMatrixUniforms src/webgl/p5.Shader.js:940:63 ❯ Shader.bindShader src/webgl/p5.Shader.js:883:14 ❯ RendererGL._setFillUniforms src/webgl/p5.RendererGL.js:2145:16 ❯ p5.RendererGL.drawBuffers src/webgl/p5.RendererGL.Retained.js:143:10 ❯ p5.RendererGL.drawBuffersScaled src/webgl/p5.RendererGL.Retained.js:228:10 ❯ fn.box src/webgl/3d_primitives.js:1221:20 ❯ test/unit/webgl/p5.Framebuffer.js:44:18 ❯ Framebuffer.draw src/webgl/p5.Framebuffer.js:1310:7 ❯ test/unit/webgl/p5.Framebuffer.js:40:15

Check failure on line 940 in src/webgl/p5.Shader.js

View workflow job for this annotation

GitHub Actions / test

test/unit/webgl/p5.Framebuffer.js > p5.Framebuffer > formats and channels > framebuffers work with WebGL 1, unsigned-byte rgba float depth

TypeError: this._renderer.states.calculateCombinedMatrix is not a function ❯ Shader._setMatrixUniforms src/webgl/p5.Shader.js:940:63 ❯ Shader.bindShader src/webgl/p5.Shader.js:883:14 ❯ RendererGL._setFillUniforms src/webgl/p5.RendererGL.js:2145:16 ❯ p5.RendererGL.drawBuffers src/webgl/p5.RendererGL.Retained.js:143:10 ❯ p5.RendererGL.drawBuffersScaled src/webgl/p5.RendererGL.Retained.js:228:10 ❯ fn.box src/webgl/3d_primitives.js:1221:20 ❯ test/unit/webgl/p5.Framebuffer.js:44:18 ❯ Framebuffer.draw src/webgl/p5.Framebuffer.js:1310:7 ❯ test/unit/webgl/p5.Framebuffer.js:40:15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method needs to be on the renderer and not the states:

Suggested change
this._renderer.states.uMVMatrix = this._renderer.states.calculateCombinedMatrix();
this._renderer.states.uMVMatrix = this._renderer.calculateCombinedMatrix();


const modelViewProjectionMatrix = modelViewMatrix.copy();
modelViewProjectionMatrix.mult(projectionMatrix);
Expand Down
57 changes: 57 additions & 0 deletions test/unit/core/environment.js
Original file line number Diff line number Diff line change
Expand Up @@ -236,4 +236,61 @@ suite('Environment', function() {
assert.isNumber(myp5.displayDensity(), pd);
});
});

suite('2D context test', function() {
beforeEach(function() {
myp5.createCanvas(100, 100);
});

test('worldToScreen for 2D context', function() {
let worldPos = myp5.createVector(50, 50);
let screenPos = myp5.worldToScreen(worldPos);
assert.closeTo(screenPos.x, 50, 0.1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is great! Can we add a test of rotation in 2D too?

assert.closeTo(screenPos.y, 50, 0.1);
});

test('worldToScreen with rotation in 2D', function() {
myp5.push();
myp5.translate(50, 50);
myp5.rotate(myp5.PI / 2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think since we translate before we rotate, the rotation doesn't end up actually affecting the position. If we rotate first, we should see an effect (I would expect (x, y) to become (y, -x) I think?)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, actually, ignore this -- it's only true if you're using (0,0) as your local coordinate. I see the offset by 10 does actually switch axes here!

let worldPos = myp5.createVector(10, 0);
let screenPos = myp5.worldToScreen(worldPos);
myp5.pop();
assert.closeTo(screenPos.x, 50, 0.1);
assert.closeTo(screenPos.y, 60, 0.1);
});
});

suite('3D context test', function() {
beforeEach(function() {
myp5.createCanvas(100, 100, myp5.WEBGL);
});

test('worldToScreen for 3D context', function() {
let worldPos = myp5.createVector(0, 0, 0);
let screenPos = myp5.worldToScreen(worldPos);
assert.closeTo(screenPos.x, 50, 0.1);
assert.closeTo(screenPos.y, 50, 0.1);
});

test('worldToScreen with rotation in 3D around Y-axis', function() {
myp5.push();
myp5.rotateY(myp5.PI / 2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar comment here: just rotating won't affect the coordinates, so could we do a translation after this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, ignore that, as mentioned above, it should be ok as long as you're converting a nonzero coordinate. But it feels odd that the resulting coordinate would still be 50,50 since the untransformed example above also ends up at that. should this value be something else?

Copy link
Member Author

@Garima3110 Garima3110 Aug 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, ignore that, as mentioned above, it should be ok as long as you're converting a nonzero coordinate. But it feels odd that the resulting coordinate would still be 50,50 since the untransformed example above also ends up at that. should this value be something else?

Maybe it should be 200,200
The default camera position in p5.js is at (0, 0, 800), looking towards the origin (0, 0, 0) along the negative Z-axis. The camera's view is centered on the Z-axis, so any point on this axis is projected to the center of the screen.
rotateY(myp5.PI / 2) rotates the point (50, 0, 0) 90 degrees around the Y-axis.
This rotation transforms the point (50, 0, 0) into (0, 0, -50). After rotation, the point is 50 units in front of the origin along the negative Z-axis. After the rotation, the point (0, 0, -50) is at a distance of 850 units from the camera (800 - (-50)). The screen coordinates are calculated based on the projection of this point into the 2D view of the camera.
Since the rotated point lies directly on the Z-axis, its X and Y screen coordinates should map to the center of the canvas, so --> 200,200

What are your thoughts on this @davepagurek please let me know?!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, I definitely didn't catch that this was a rotateY -- I read it as just rotate, which would have been around the z axis. That now explains why the local offset doesn't update the resulting screen x and y values. The 50,50 result now makes sense, since I think the canvas is only 100x100:

myp5.createCanvas(100, 100);

So I think this result does actually make sense, and maybe we should just add one more test of rotation about the Z axis so that we can confirm that an example like the one you use in the 2D tests also works in WebGL.

let worldPos = myp5.createVector(50, 0, 0);
let screenPos = myp5.worldToScreen(worldPos);
myp5.pop();
assert.closeTo(screenPos.x, 50, 0.1);
assert.closeTo(screenPos.y, 50, 0.1);
});

test('worldToScreen with rotation in 3D around Z-axis', function() {
myp5.push();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one doesn't assert anything so it might not work super well as a unit test, but this would be a cool example to have in the reference for this item! Maybe we could move it into the reference comments instead? (Same for the 2D case, having those as two examples would be great!)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@limzykenneth in the 2.0 branch, is there an easy way to get a preview of how the docs will look, similar to grunt yui:dev on the main branch?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not at the moment. The offline reference (which I also forked out here) can potentially help but it is not setup to work automatically yet.

myp5.rotateZ(myp5.PI / 2);
let worldPos = myp5.createVector(10, 0, 0);
let screenPos = myp5.worldToScreen(worldPos);
myp5.pop();
assert.closeTo(screenPos.x, 50, 0.1);
assert.closeTo(screenPos.y, 40, 0.1);
});
});
});
Loading