From c3b722ffb0a6e1bc453c51fac87939c801bd73b2 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Thu, 22 Aug 2024 09:29:39 -0700 Subject: [PATCH 1/8] Adding USD RFC sans images Signed-off-by: AMZN-Gene --- ...RFC Feature - USD in the Scene Pipeline.md | 636 ++++++++++++++++++ 1 file changed, 636 insertions(+) create mode 100644 rfcs/RFC Feature - USD in the Scene Pipeline.md diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md new file mode 100644 index 0000000..c7ecb7c --- /dev/null +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -0,0 +1,636 @@ +* 1[Opportunity & Background Info](#USDintheScenePipeline-Opportunity&BackgroundInfo) + * 1.1[Opportunity](#USDintheScenePipeline-Opportunity) + * 1.1.1[What are scene files?](#USDintheScenePipeline-Whatarescenefiles?) + * 1.1.2[What makes USD interesting, why use USD instead of another format like FBX, GLTF, STL, Collada, STEP, etc?](#USDintheScenePipeline-WhatmakesUSDinteresting,whyuseUSDinsteadofanotherformatlikeFBX,GLTF,STL,Collada,STEP,etc?) + * 1.2[Required O3DE Knowledge](#USDintheScenePipeline-RequiredO3DEKnowledge) + * 1.3[USD Terminology](#USDintheScenePipeline-USDTerminology) + * 1.4[USD Basics](#USDintheScenePipeline-USDBasics) + * 1.4.1[Basic USD file](#USDintheScenePipeline-BasicUSDfile) + * 1.4.2[Basic USD file references](#USDintheScenePipeline-BasicUSDfilereferences) + * 1.4.3[Beyond the Basics](#USDintheScenePipeline-BeyondtheBasics) +* 2[Recommended Solution](#USDintheScenePipeline-RecommendedSolution) + * 2.1[Solution Overview](#USDintheScenePipeline-SolutionOverview) + * 2.1.1[Why these three phases?](#USDintheScenePipeline-Whythesethreephases?) + * 2.1.2[Simple Example](#USDintheScenePipeline-SimpleExample) + * 2.1.3[Example with References](#USDintheScenePipeline-ExamplewithReferences) + * 2.2[Solution in Detail](#USDintheScenePipeline-SolutioninDetail) + * 2.2.1[Phase 1 - Migrate to latest Assimp and treat USD like other scene formats](#USDintheScenePipeline-Phase1-MigratetolatestAssimpandtreatUSDlikeothersceneformats) + * 2.2.2[Phase 2 - Update Assimp to support flattened USD with references](#USDintheScenePipeline-Phase2-UpdateAssimptosupportflattenedUSDwithreferences) + * 2.2.2.1[The Current Setup](#USDintheScenePipeline-TheCurrentSetup) + * 2.2.2.2[After Phase 2, USD files can reference other USD or OBJ meshes](#USDintheScenePipeline-AfterPhase2,USDfilescanreferenceotherUSDorOBJmeshes) + * 2.2.3[Phase 3 - Prefab output mimicking USD relationship graph](#USDintheScenePipeline-Phase3-PrefaboutputmimickingUSDrelationshipgraph) +* 3[Alternate Solutions](#USDintheScenePipeline-AlternateSolutions) + * 3.1[Integrate Pixar USD Library Instead of Using Assimp](#USDintheScenePipeline-IntegratePixarUSDLibraryInsteadofUsingAssimp) +* 4[Open Questions](#USDintheScenePipeline-OpenQuestions) + * 4.1[Should we move Assimp to a Gem instead of using O3DE 3rdParty system?](#USDintheScenePipeline-ShouldwemoveAssimptoaGeminsteadofusingO3DE3rdPartysystem?) + * 4.2[Should we fail a job, or emit an error if a reference to another USD file can't be resolved?](#USDintheScenePipeline-Shouldwefailajob,oremitanerrorifareferencetoanotherUSDfilecan'tberesolved?) + * 4.3[Do we need to emit references to other USD files as job dependencies, or can they just be product dependencies?](#USDintheScenePipeline-DoweneedtoemitreferencestootherUSDfilesasjobdependencies,orcantheyjustbeproductdependencies?) + * 4.3.1[Example](#USDintheScenePipeline-Example) + * 4.4[Why not create a new USD builder upstream of the Scene Builder, and use intermediate assets as a middle step?](#USDintheScenePipeline-WhynotcreateanewUSDbuilderupstreamoftheSceneBuilder,anduseintermediateassetsasamiddlestep?) + * 4.5[Can we put most of this logic into a Gem, instead of directly updating the scene API?](#USDintheScenePipeline-CanweputmostofthislogicintoaGem,insteadofdirectlyupdatingthesceneAPI?) + * 4.6[Do we need to update the scene API to support loading multiple scenes from the same file?](#USDintheScenePipeline-DoweneedtoupdatethesceneAPItosupportloadingmultiplescenesfromthesamefile?) + * 4.7[How configurable can we make the prefab generation?](#USDintheScenePipeline-Howconfigurablecanwemaketheprefabgeneration?) + * 4.8[What do we need from procedural prefabs for USD to be fully functional?](#USDintheScenePipeline-WhatdoweneedfromproceduralprefabsforUSDtobefullyfunctional?) + * 4.8.1[New Entity Context?](#USDintheScenePipeline-NewEntityContext?) + * 4.8.2[Abstract Prefabs](#USDintheScenePipeline-AbstractPrefabs) + * 4.9[USD files referencing non-USD file?](#USDintheScenePipeline-USDfilesreferencingnon-USDfile?) + * 4.10[Do all USD overrides work with prefabs, or will we have to special case some?](#USDintheScenePipeline-DoallUSDoverridesworkwithprefabs,orwillwehavetospecialcasesome?) + * 4.11[Would it make sense to do this in phases, flattening in the first part?](#USDintheScenePipeline-Woulditmakesensetodothisinphases,flatteninginthefirstpart?) + * 4.12[When should we do this?](#USDintheScenePipeline-Whenshouldwedothis?) +* 5[Out of Scope / Long Term Work](#USDintheScenePipeline-OutofScope/LongTermWork) + * 5.1[Advanced shaders and materials](#USDintheScenePipeline-Advancedshadersandmaterials) + * 5.2[Convert O3DE prefabs to USD files](#USDintheScenePipeline-ConvertO3DEprefabstoUSDfiles) +* 6[FAQ](#USDintheScenePipeline-FAQ) + * 6.1[Why not stop after phase 2: just treat USD like other formats, flatten everything, and then generate product assets?](#USDintheScenePipeline-Whynotstopafterphase2:justtreatUSDlikeotherformats,flatteneverything,andthengenerateproductassets?) + * 6.2[Why do we need both source dependencies and job dependencies for references?](#USDintheScenePipeline-Whydoweneedbothsourcedependenciesandjobdependenciesforreferences?) + * 6.3[Why build USD support into the Scene Builder, why not create a new USD builder?](#USDintheScenePipeline-WhybuildUSDsupportintotheSceneBuilder,whynotcreateanewUSDbuilder?) + * 6.4[Why use an existing library for loading USD files, instead of rolling our own solution?](#USDintheScenePipeline-WhyuseanexistinglibraryforloadingUSDfiles,insteadofrollingourownsolution?) + * 6.5[Are there other reasons for customers to want to use USD over our existing, supported scene file formats?](#USDintheScenePipeline-ArethereotherreasonsforcustomerstowanttouseUSDoverourexisting,supportedscenefileformats?) + * 6.5.1[FBX versus USD](#USDintheScenePipeline-FBXversusUSD) + * 6.5.2[FBX versus GLTF](#USDintheScenePipeline-FBXversusGLTF) + * 6.6[Would we drop support for other scene file formats with USD support? Why wouldn't we want to only support USD?](#USDintheScenePipeline-WouldwedropsupportforotherscenefileformatswithUSDsupport?Whywouldn'twewanttoonlysupportUSD?) + * 6.7[Are there other benefits to supporting USD this way?](#USDintheScenePipeline-ArethereotherbenefitstosupportingUSDthisway?) + +Opportunity & Background Info +============================= + +Opportunity +----------- + +The [Universal Scene Description](https://graphics.pixar.com/usd/release/index.html) (USD) file format is frequently requested to be a supported scene format, as an addition to the FBX, GTLF, and STL that we already support. + +### What are scene files? + +Scene files describe 3D content: Geometry, materials, animations. They also include transforms and hierarchy information, as well as generic nodes, and user defined properties. + +### What makes USD interesting, why use USD instead of another format like FBX, GLTF, STL, Collada, STEP, etc? + +USD is built around a composition feature, the ability to easily reference other files. It's a lot like our prefab system. + +![](@todo_image) + +Required O3DE Knowledge +----------------------- + +* Familiarity with the O3DE scene pipeline. + * [https://www.o3de.org/docs/user-guide/assets/scene-pipeline/](https://www.o3de.org/docs/user-guide/assets/scene-pipeline/) +* Familiarity with the O3DE asset pipeline. + * [https://www.o3de.org/docs/user-guide/assets/pipeline/](https://www.o3de.org/docs/user-guide/assets/pipeline/) +* Familiarity with O3DE Assimp integration. + * [o3de/Code/Tools/SceneAPI/SDKWrapper/AssImpSceneWrapper.cpp at bbc8fa583c34186a9c3ede51b2d6254701c4afca · o3de/o3de (github.com)](https://github.com/o3de/o3de/blob/bbc8fa583c34186a9c3ede51b2d6254701c4afca/Code/Tools/SceneAPI/SDKWrapper/AssImpSceneWrapper.cpp#L91) + * Note: You can debug AssetBuilder.exe by enabling --debug and providing USD asset path. (Must have AssetProcessor.exe running) + + [?](#) + + `--project-path=``"D:/prj/o3de/AutomatedTesting"` `--debug` `"D:\prj\o3de\AutomatedTesting\Assets\asset.usd"` `--platform pc --tags dx12 --project-name AutomatedTesting --project-cache-path` `"D:\prj\o3de\AutomatedTesting\Cache"` + + +USD Terminology +--------------- + +* USD - Universal Scene Description - The scene file format under discussion here. + * [https://graphics.pixar.com/usd/release/index.html](https://graphics.pixar.com/usd/release/index.html) +* USD specific terms and concepts + * [https://graphics.pixar.com/usd/release/glossary.html](https://graphics.pixar.com/usd/release/glossary.html) + * At minimum, you should be familiar with: + * Prim -The basic building block of USD content. + * [https://graphics.pixar.com/usd/release/glossary.html#usdglossary-prim](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-prim) + * Asset - USD's system for referencing other file. + * [https://graphics.pixar.com/usd/release/glossary.html#asset](https://graphics.pixar.com/usd/release/glossary.html#asset) + * Composition Arcs & Composition - USD's system for resolving references, similar to our nested prefab system. + * [https://graphics.pixar.com/usd/release/glossary.html#usdglossary-compositionarcs](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-compositionarcs) + * Stage - A composed USD file. + * [https://graphics.pixar.com/usd/release/glossary.html#usdglossary-stage](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-stage) +* USDA - File extension - Text based USD files. +* USDC - File extension - Binary USD files. +* USDZ - File extension - Zipped container of multiple files, intended to group USD content together. + +USD Basics +---------- + +### Basic USD file + +This usda file represents a stage that defines two prims: A transform named "hello" and a sphere child of that transform named "world". + +[?](#) + +`#usda 1.0` + +`def Xform` `"hello"` + +`{` + +`def Sphere` `"world"` + +`{` + +`}` + +`}` + +Visual representation of this file: + +![](@todo_image) + +### Basic USD file references + +This pair of usda files, from [https://graphics.pixar.com/usd/release/tut\_referencing\_layers.html](https://graphics.pixar.com/usd/release/tut_referencing_layers.html), demonstrates how references work between USD files. + +_Below: RefExample.usda references HelloWorld.usda and overrides world's color._ + +HelloWorld.usda + +[?](#) + +`#usda 1.0` + +`(` + +`defaultPrim =` `"hello"` + +`)` + +`def Xform` `"hello"` + +`{` + +`double3 xformOp:translate = (4, 5, 6)` + +`uniform token[] xformOpOrder = [``"xformOp:translate"``]` + +`def Sphere` `"world"` + +`{` + +`float3[] extent = [(-2, -2, -2), (2, 2, 2)]` + +`color3f[] primvars:displayColor = [(0, 0, 1)]` + +`double` `radius = 2` + +`}` + +`}` + +``` +RefExample.usda +``` + +[?](#) + +`#usda 1.0` + +`over` `"refSphere2"` `(` + +`prepend references = @./HelloWorld.usda@` + +`)` + +`{` + +`over` `"world"` + +`{` + +`color3f[] primvars:displayColor = [(1, 0, 0)]` + +`}` + +`}` + + + +### Beyond the Basics + +I recommend reading through the USD tutorials here [https://graphics.pixar.com/usd/release/tut\_usd\_tutorials.html](https://graphics.pixar.com/usd/release/tut_usd_tutorials.html), and experimenting in Blender and Nvidia USD Composer with creating or importing scenes and exporting to usda format files. + +Note: Blender does not support references: +_    "Like the USD exporter, the importer does not yet handle more advanced USD concepts, such as layers and references." [Universal Scene Description - Blender 4.2 Manual](https://docs.blender.org/manual/en/latest/files/import_export/usd.html)_ + +Digital content creation (DCC) tools which takes advantage of USD references are Nvidia Isaac Sim, Nvidia USD Composer, and Pixar tooling.   + +Recommended Solution +==================== + +Solution Overview +----------------- + +Build support for USD into the scene pipeline. + +Do this in three phases: + +1. Upgrade O3DE to use Assimp's latest implementation + 1. Input: single USD geometry files that don't use layers or references + 2. Output: azmodel containing the USDs geometry  +2. Contribute back to Assimp and update scene builder to load USD files with references. Stages are flattened by adding in reference geometry. O3DE then processes USD like FBX and other scene file formats. + 1. Input: USD files which contains a hierarchy of references to other geometry inside of other USD files  + 2. Output: Flattened azmodel containing the geometry from the multiple USD files inside one azmodel file +3. Contribute back to Assimp giving API option to flatten stages or keep reference geom separate. Update O3DE to create a procedural prefab that mimics the relationship graph of USD files by matching models and transforms. This could be expanded later to allow O3DE developers to translate USD properties into O3DE component properties as well as add advanced shaders/materials. + 1. Input: USD files which contains a hierarchy of references to other geometry inside of other USD files + 2. Output: Procedural prefabs generated per USD. Root prefab preserves the hierarchy of the original USD. + +### Why these three phases? + +* Phase 1 is extremely fast. +* The work for phase 2 is used in phase 3, so there won't be much wasted effort here. +* Once phase 2 is completed, there's great benefits to using USD: content creators can start splitting up content on their end. +* Benefits of phase: + * Performance - asset processing time, and hard drive space used by product assets: We won't need to re-create product assets, such as models, that are referenced by multiple other USD files. + * Workflow - additional procedural prefabs will be available for content creators to use in the Editor. + +### Simple Example + +![](@todo_image) + +### Example with References + +![](@todo_image) + +Solution in Detail +------------------ + +### Phase 1 - Migrate to latest Assimp and treat USD like other scene formats + +[AssImp now supports USD](https://github.com/assimp/assimp/issues/5547) without composition (no references) meaning that single asset usd, usda, usdc, and usdz files can be loaded. + +1. Update O3DE 3rdParty to use Assimp's latest library + 1. There's no work required in O3DE code, just updating 3rdParty to use the latest Assimp release. + 2. Note: The current Assimp release 5.4.2 from July 2024 doesn't contain USD. O3DE 3rdParty package builder [relies on git release tags](https://github.com/aws-lumberyard-dev/3p-package-source/blob/main/package-system/assimp/build_config.json#L3). If we want USD support now without waiting for the next release, the packaging script will need to be updated to _git pull_ this [specific commit](https://github.com/assimp/assimp/commit/0cb169368956fba4508fbd3af361cf37c8060e8e). + 3. Note: [AssImp latest has compiler error](https://github.com/assimp/assimp/pull/5693) +2. Add USD File Types to Asset Importer Settings + 1. Edit _Registry/sceneassetimporter.setreg_ to process _usd, usda, usdz, usdc_ file types. + 2. _![](@todo_image)_ +3. Run Asset Processor + 1. Allow asset processor to digest USD files, such as the toy\_biplane.usd file here. + 2. [![](@todo_image)toy\_biplane\_idle.usdz](@todo_image) + 3. ![](@todo_image) + +At this point, USDs containing just geometry will load, however AssetProcessor will fail to create a product for a simple USD which references that very same geom file. + +For example, the _suzanne.usdc_ geometry file will be processed correctly and generate a _cache/pc/suzanne.azmodel_, but _usdObj-001.usda_ won't produce anything in the cache folder. + +_usdObj-001.usda which references a separate suzanne.usd geometry file._ + +[?](#) + +`def Mesh` `"suzanne"` `(` + +`prepend references = @suzanne.usdc@` + +`)` + +`{}` + + + +[![](@todo_image)suzanne.usdc](@todo_image) [![](@todo_image)usdObj-001.usda](@todo_image) + +![](@todo_image) + +[?](#) + +`Warning | The builder (Scene Builder) has not indicated it handled outputting product dependencies` `for` `file Assets/usdObj-001.usda.  This is a programmer error.` + +Phase 2 enables referencing... + +### Phase 2 - Update Assimp to support flattened USD with references + +Update Assimp to read USDs, composite all sub-layers and references and flatten all the meshes into one Assimp scene. O3DE will treat the root USD as one big mesh. + +#### The Current Setup + +Currently, scene files such as FBX store all their sub-meshes in a single FBX. + +The Assimp scene builder returns the sub-meshes and O3D breaks them out into separate entities.  + +Current: FBX Scene File to O3DE Entity + +[ + +](@todo_image)[ + +](@todo_image) + +[ + +Embed Link Embed Link to Hipchat, Jira, Confluence + +](#) + +#### After Phase 2, USD files can reference other USD or OBJ meshes + +While ultimately the same O3DE entity hierarchy is produced, notice how unlike before, the car asset references external files (_carbody.usd_ and _tire.obj_) allowing multiple content creators to work at the same time. + +_Car.usd_ + +[?](#) + +`def Xform` `"Xform"` + +`{` + +`def Mesh` `"carbody"` `(` + +`prepend references = @carbody.usdc@){}` + +`def Mesh` `"tire_1"` `(` + +`prepend references = @tire.obj@){` + +`double3 xformOp:translate = (1, 0, 0)` + +`}` + +`def Mesh` `"tire_2"` `(` + +`prepend references = @tire.obj@){` + +`double3 xformOp:translate = (0, 1, 0)` + +`}` + +`...` + +`}` + + + +Phase 1 w/ References + +[ + +](@todo_image)[ + +](@todo_image) + +[ + +Embed Link Embed Link to Hipchat, Jira, Confluence + +](#) + +1. Update Assimp to support some passthroughs we would need for USD support, specifically a way to override asset resolving. + 1. See USD documentation on Asset Resolution: + 1. [https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution) + 2. [https://graphics.pixar.com/usd/release/api/ar\_page\_front.html](https://graphics.pixar.com/usd/release/api/ar_page_front.html) +2. Update Assimp to generate an Assimp scene by flattening and resolving the O3DE scene + 1. Today a USD loaded via Assimp returns a scene, but if that scene references other USD files, the scene won't contain the geometry of the references. Update Assimp to recursively find references and mark them as a file dependency. + + TinyUSDZ, the 3rd Party USD library used by Assimp, has methods for discovering references geometry. + + [TinyUSDZ tusdcat code example](https://github.com/lighttransport/tinyusdz/blob/83b4ebc90761e60444e55611238fff6cfb6a62c9/examples/tusdcat/main.cc) is able to extract USD references as follows: + + 1. [Load USD as a Layer](https://github.com/lighttransport/tinyusdz/blob/83b4ebc90761e60444e55611238fff6cfb6a62c9/examples/tusdcat/main.cc#L149) + 2. [Do compositions (it also returns a layer)](https://github.com/lighttransport/tinyusdz/blob/83b4ebc90761e60444e55611238fff6cfb6a62c9/examples/tusdcat/main.cc#L212) + 3. Then [construct Stage from Layer](https://github.com/lighttransport/tinyusdz/blob/910c5740008377976a559c4ff1680292475abc85/src/composition.hh#L210)  + + [?](#) + + `bool` `LayerToStage(``const` `Layer &layer, Stage *stage` + + 4. Then call `tinyusdz::tydra::ListPrims ` + + [?](#) + + `std::map meshmap;` + + `tinyusdz::tydra::ListPrims(stage, meshmap);` + + Note: USD can reference other USDs or geometry from other file types (example: obj). As long as the file type is supported by Assimp, we can load this geom and add it to the final flattened Assimp scene. + + + More Information: [Accessing Reference File Geom · Issue #186 · lighttransport/tinyusdz (github.com)](https://github.com/lighttransport/tinyusdz/issues/186) + +3. From here, the current O3DE pipeline would take over, we would consume this flattened scene. + 1. Phase 2: USD w/ References Input/Output + + [ + + ](@todo_image)[ + + ](@todo_image) + + [ + + Embed Link Embed Link to Hipchat, Jira, Confluence + + ](#) + + +### Phase 3 - Prefab output mimicking USD relationship graph + +* Add a feature flag to toggle USD flattening off, so we can deliver the rest of this work iteratively. +* Update Create Jobs in the scene builder to output job dependencies on referenced USD files. + * [See FAQ for why we need both source and job dependencies](#USDintheScenePipeline-Whydoweneedbothsourcedependenciesandjobdependenciesforreferences?). + * Job dependencies will be needed at this point because product assets from one USD file (prefabs, models, etc) will be referenced by other USD files to mimic the USD relationship graph. +* Update the scene builder to generate USD based procedural prefabs. + * We will need to define what these look like. [We may need additional features from procedural prefabs to hit our end goals](https://wiki.agscollab.com/display/lmbr/USD+in+the+Scene+Pipeline#USDintheScenePipeline-WhatdoweneedfromproceduralprefabsforUSDtobefullyfunctional?). +* Decide if we want to either keep the feature flag for USD flattening and turn it off by default, or remove the flag and remove USD flattening support in process job. + +Alternate Solutions +=================== + +Integrate Pixar USD Library Instead of Using Assimp +--------------------------------------------------- + +* Add the Pixar USD library to O3DE as a 3rd party dependency, update Scene Builder to reference this library. + * [See FAQ for why we want to update the scene builder for this support, instead of creating a new builder.](https://wiki.agscollab.com/display/lmbr/USD+in+the+Scene+Pipeline#USDintheScenePipeline-WhybuildUSDsupportintotheSceneBuilder,whynotcreateanewUSDbuilder?) + * We will use the official Pixar library, instead of building our own logic for parsing USD files: [https://github.com/PixarAnimationStudios/USD](https://github.com/PixarAnimationStudios/USD) +* Update the scene builder to load USD files during Create Jobs, so asset references can be examined and resolved. + * Emit references to resolvable USD files as source dependencies. + * Fail or error in the job if the reference can't be resolved. + * [Open question - failure versus error. This is a two way door and can be easily changed at any time.](https://wiki.agscollab.com/display/lmbr/USD+in+the+Scene+Pipeline#USDintheScenePipeline-Shouldwefailajob,oremitanerrorifareferencetoanotherUSDfilecan%27tberesolved?) + * Job dependencies won't yet be needed, because the flattening process will not need to reference product assets from other USD files. +* Update the scene API to reference both Assimp and the USD library. +* Update the scene API to detect USD format files, and branch to specialized USD loading. + * Make sure this allows for both a simple load that just scans for references, independent of the more advanced loading. +* Write a custom asset path resolver, that we pass to the USD library, so it can resolve paths. + * See USD documentation on Asset Resolution: + * [https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution) + * [https://graphics.pixar.com/usd/release/api/ar\_page\_front.html](https://graphics.pixar.com/usd/release/api/ar_page_front.html) +* In process job for a USD file in the Scene Builder, flatten the USD file using the USD library. +* In process job for a USD file in the Scene Builder, convert the flattened USD file to an O3DE scene. + +Open Questions +============== + +Should we move Assimp to a Gem instead of using O3DE 3rdParty system? +--------------------------------------------------------------------- + +Moving forward with USD via Assimp will require regularly pulling new versions of Assimp as new features come online (example: references). Going through the 3rdParty library system is slower as it requires its own approval process via [O3DE's 3rd Party Package repo](https://github.com/o3de/3p-package-source). The downside to moving Assimp into a gem directly, is that we'd reimplement what's already in 3rdParty, for example, [git pulling a particular Assimp commit](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_config.json#L2-L3), [building](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_assimp_windows.cmd), and [packaging](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/install_assimp_windows.json) the library. + +Should we fail a job, or emit an error if a reference to another USD file can't be resolved? +-------------------------------------------------------------------------------------------- + +I'm leaning toward failure. This would be similar to a gap in a nested prefab hierarchy. We could try to preserve what we can from the file, but really we would want to call attention to the file with the broken reference to get it resolved. + +Do we need to emit references to other USD files as job dependencies, or can they just be product dependencies? +--------------------------------------------------------------------------------------------------------------- + +If we're able to fully rely on procedural prefabs to manage the composition of USD file references, and we're able to predict the sub IDs of products from other USD files, then we may not need to emit these references as source dependencies. + +USD has support for [abstract base objects](https://wiki.agscollab.com/display/lmbr/USD+Investigation#USDInvestigation-AbstractObjects), that function sort of as inverted overrides. Our prefab system does not have a similar concept, so we may need to output job or source dependencies to use this information to build a prefab hierarchy. We may find that we instead need to add this sort of functionality to prefabs. + +### Example + +Assume tires.usda has a small\_tire prim and big\_tire prim, which results in the generation of small\_tire.procprefab and big\_tire.procprefab (plus the associated azmodels). Then, assume car.usda has four references to the small\_tire prim. When car.usda is processed, to create car.procprefab, the asset ID of small\_tire.procprefab will have to be resolved + +Why not create a new USD builder upstream of the Scene Builder, and use intermediate assets as a middle step? +------------------------------------------------------------------------------------------------------------- + +Would there be a situation that makes sense where we have USD builder with two collections of outputs? 1) Procedural prefabs to represent the relationship between USD files and 2) Flattened scene data in a format that we can load via Assimp (FBX, Collada, GLTF, etc) output as intermediate assets, to let the scene builder do the rest of the lifting from there? + +Right now I'm leaning toward this being more complicated than it would be worth. It wouldn't save us much time, and the benefits (minimizing the code we change in the scene builder directly) aren't particularly high, especially because we'll still need to make changes to the scene API. + +Can we put most of this logic into a Gem, instead of directly updating the scene API? +------------------------------------------------------------------------------------- + +This would only apply if we decide on not using Assimp's USD implementation. It could establish a pattern for expanding to other, non-Assimp scene formats to load. + +It may also be extra work that makes things more complicated, especially if USD ends up the only reference focused scene format we want to support to this degree, and we're OK with using Assimp or an equivalent for all other file types. + +Do we need to update the scene API to support loading multiple scenes from the same file? +----------------------------------------------------------------------------------------- + +The concept of a stage in a USD file can result in us wanting to generate multiple groupings of product assets, that function as if they came from different source scene files. For example, if we had tires.usda that had one group of prims that generated small\_tire.procprefab and associated other product assets (models, buffers, etc) and another group of prims that generated big\_tire.procprefab, would it be better to handle that separation and splitting in the current scene graph, giving the current scene pipeline one large scene? Or would it make more sense to, at some point in USD processing before we move to the O3DE scene in memory, to generate multiple O3DE scenes and run each of those through the current pipeline? + +Right now I'm leaning toward that second option: We update the scene API, scene builder, etc to support processing multiple O3DE scenes from a single file. + +How configurable can we make the prefab generation? +--------------------------------------------------- + +Right now, based on the relationship of USD files, it seems like we'll need to more heavily lock down the procedural prefabs generated from USD files, and not give as much configuration. + +However, to facilitate technical artists expanding on things, we should still provide some entry points for appending custom components at steps, passing through information like user defined properties to facilitate that. We should also provide an interface to technical artists to generate additional procedural prefabs in response to the creation of our canonical USD generated procedural prefabs. + +What do we need from procedural prefabs for USD to be fully functional? +----------------------------------------------------------------------- + +This needs additional investigation, to see if the current state of procedural prefabs can meet our needs. + +### New Entity Context? + +Will we want to build a new context, and new entity classification for this process, "Scene Entity?" I don't think we will, but I'm putting this here as an open question for additional discussion. + +### Abstract Prefabs + +One feature of USD is abstract prims ([defined as Class in USD terminology](https://graphics.pixar.com/usd/release/glossary.html#class)). + +Right now, procedural prefabs don't support the concept of references to incomplete prefabs. + +Procedural prefabs also do not support the concept of an abstract prefab, that can't itself be instantiated. + +What is the use case here? This allows for building a sort of inverted version of what we accomplish with prefab nesting right now. Instead of having a complex base prefab with simple prefabs that reference it (car.prefab, redcar.prefab), the base can be simple and the references can be complex. + +USD files referencing non-USD file? +----------------------------------- + +A USD file commonly references other file types like obj, fbx, and gltf. + +If we're loading USDs with Assimp, we should be able to support referencing other file types supported by Assimp.   + +Do all USD overrides work with prefabs, or will we have to special case some? +----------------------------------------------------------------------------- + +The concern here is if some USD overrides modify properties that we can't support with prefabs. For example, can a USD override can modify the geometry of a mesh, or the length of an animation? That information is baked into those product assets, so in that case we would need to generate a new azmodel or animation and not be able to use the prefab reference system. + +Would it make sense to do this in phases, flattening in the first part? +----------------------------------------------------------------------- + +Converting USD information to prefabs correctly is going to take a lot of time. Would there be value in us doing a first deliverable where we flatten USD files in process jobs, and functionally treat them as other scene files from there? + +The upside is that, from a content organization standpoint, we would still support the strengths of USD, the references between files. This work wouldn't prevent us from building the robust prefab solution, later. + +When should we do this? +----------------------- + +We may want to wait until more digital content creation tools support USD driven workflows.  + +USD's biggest strength is the cross file referencing, letting content creators work collaboratively in ways they cannot achieve with traditional scene file formats. However, right now, full USD support is largely just Pixar's digital content creation tool and NVidia's _USD Composer_. Max, Maya, and Blender all push content creators to a workflow of treating USD as an export format, and not the primary authoring format. This is a gotcha because the primary authoring format for those files (.blend, .ma, .mb, and .max) do not support USD file references. + +This means that, even if we had full USD support today as proposed in this document (generating prefabs that match the USD relationship graph), it's unlikely that many of O3DE's customers would be able to make use of USD support. + +Out of Scope / Long Term Work +============================= + +Advanced shaders and materials +------------------------------ + +For our initial deliverable, we will generate basic O3DE physically based rendering materials. + +However, the options for shaders and materials from USD files are much bigger than that. As we wrap up our first set of work to support USD, we'll want to start planning what a more fully featured material and shader pipeline from USD files could look like. + +Convert O3DE prefabs to USD files +--------------------------------- + +This initial work would not include the ability to create USD files from O3DE content, like prefabs. A prefab to USD converter could be the first step towards replacing prefabs with USD entirely.  + +This is something that could be looked at long term. + +FAQ +=== + +Why not stop after phase 2: just treat USD like other formats, flatten everything, and then generate product assets? +-------------------------------------------------------------------------------------------------------------------- + +By expanding USD support to create a prefab hierarchy that mimics the USD relationship graph, we will see two main areas of benefit: + +* Performance + * Speed - Asset processing time will improve for USD files, because we won't have to re-flatten the hierarchy for every USD file processed. + * Hard drive space used - This will reduce the duplication of geometry in product assets on disk, because procedural prefabs will allow output to reference already created product assets instead of flatting and re-creating that geometry in multiple product assets. + * Memory usage - Flattening each USD file will result in needing to keep the entire USD file in memory. This will put USD in the same situation as FBX, GLTF, etc, of having an upper limit on what can be processed due to memory available on the local machine. Switching to a prefab focused output mimicking the relationship graph means that in many cases, we will be able to avoid loading dependencies in memory, because we will be able to generate a prefab without loading that other geometry. +* Workflow + * More prefabs output will give content creators in the Editor more options for assembling content in the Editor from scene files. + +Why do we need both source dependencies and job dependencies for references? +---------------------------------------------------------------------------- + +* Source dependencies are necessary because, whenever a referenced USD file changes, the USD file with that reference will need to be re-processed, because those changes to the reference could result in needing to update the product assets for the current file. +* Job dependencies are necessary because some references to other USD files will rely on the product assets of the referenced file. + * Example: If cityblock.usda has an instance of the root prim scene for car.usda placed in it, then cityblock.procprefab will have an instance reference of car.procprefab. To process cityblock.usda and create cityblock.procprefab, the asset ID of car.procprefab will have to be known. + +Why build USD support into the Scene Builder, why not create a new USD builder? +------------------------------------------------------------------------------- + +* We will want a lot of the processing steps in the scene builder to be available for the USD building process: Scene manifests to help control scene settings (although these may be full of unique options for USD files to control composition), Python callbacks at appropriate steps. Product asset generation for USD files will include many of the same output types that other scene files generate: models, materials, buffers, etc. + +Why use an existing library for loading USD files, instead of rolling our own solution? +--------------------------------------------------------------------------------------- + +USD is a complex format, it will take us much longer to write our own logic for loading these files than it would take to use the existing solution. We also will have a lot more bugs to work through, writing our own solution. + +Are there other reasons for customers to want to use USD over our existing, supported scene file formats? +--------------------------------------------------------------------------------------------------------- + +### FBX versus USD + +* FBX is proprietary, so support from non-Autodesk tools is often mixed, including our own. + * FBX being proprietary also makes upgrading to new FBX formats difficult, because there is no spec to build against, just reverse engineering options. +* FBX does not fully support some features we want, like physically based rendering materials. + * Our Maya support is built around a Maya specific extension to FBX. Blender doesn't even attempt to support PBR material properties in FBX files. +* FBX does not support anything similar to the variant management and compositing that USD supports, which is the primary strength of USD. + +### FBX versus GLTF + +* GLTF is built for web and runtime usage. GLTF strips data that isn't necessary to minimize file size, which can reduce its usefulness in a complex pipeline. +* GLTF does have some support for instancing and referencing other content, but it's not as robust as USD's, it does not support variant management to the same degree as USD. + +Would we drop support for other scene file formats with USD support? Why wouldn't we want to only support USD? +-------------------------------------------------------------------------------------------------------------- + +To get the most out of USD would require studios to go all in on it, and to have a need to break up content in that way. + +For many studios, FBX, GLTF, Collada, etc are all good enough. They have existing workflows that work well with these scene file formats, and/or they have libraries of content already built in these formats. + +A large, reference based USD library may require content creators to keep their working assets in USD format, and not really able to use Maya, Max, or Blender native formats. This could be a huge workflow adjustment for some studios that they aren't interested in making. + +USD support would be in addition to other scene file formats, and not a replacement. + +Are there other benefits to supporting USD this way? +---------------------------------------------------- + +Right now we have some big challenges with importing very large scenes. We load the entire scene into memory twice at the same time (Assimp + O3DE). + +Our current pipeline for the most part works best when you give it large, composited scene files. We have no processing tools that can reconstruct a large complex scene from multiple different source scene files. + +USD support would give us that. If a customer had a complex setup that would result in an extremely large single FBX file, they may be able to instead manage that content as multiple, smaller USD files that reference each other to create the same scene. This seems to be a big driving factor in Pixar creating this format to begin with, they were trying to solve these scale problems. \ No newline at end of file From 3c87e3684a45606b4017035d7e83c1eb648db7a9 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Thu, 22 Aug 2024 09:48:39 -0700 Subject: [PATCH 2/8] Code markdown cleanup Signed-off-by: AMZN-Gene --- ...RFC Feature - USD in the Scene Pipeline.md | 215 +++++++----------- 1 file changed, 80 insertions(+), 135 deletions(-) diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md index c7ecb7c..45012d4 100644 --- a/rfcs/RFC Feature - USD in the Scene Pipeline.md +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -79,11 +79,10 @@ Required O3DE Knowledge * Familiarity with O3DE Assimp integration. * [o3de/Code/Tools/SceneAPI/SDKWrapper/AssImpSceneWrapper.cpp at bbc8fa583c34186a9c3ede51b2d6254701c4afca · o3de/o3de (github.com)](https://github.com/o3de/o3de/blob/bbc8fa583c34186a9c3ede51b2d6254701c4afca/Code/Tools/SceneAPI/SDKWrapper/AssImpSceneWrapper.cpp#L91) * Note: You can debug AssetBuilder.exe by enabling --debug and providing USD asset path. (Must have AssetProcessor.exe running) - - [?](#) - - `--project-path=``"D:/prj/o3de/AutomatedTesting"` `--debug` `"D:\prj\o3de\AutomatedTesting\Assets\asset.usd"` `--platform pc --tags dx12 --project-name AutomatedTesting --project-cache-path` `"D:\prj\o3de\AutomatedTesting\Cache"` - + + ``` + --project-path="D:/prj/o3de/AutomatedTesting" --debug "D:\prj\o3de\AutomatedTesting\Assets\asset.usd" --platform pc --tags dx12 --project-name AutomatedTesting --project-cache-path "D:\prj\o3de\AutomatedTesting\Cache" + ``` USD Terminology --------------- @@ -111,22 +110,16 @@ USD Basics ### Basic USD file This usda file represents a stage that defines two prims: A transform named "hello" and a sphere child of that transform named "world". - -[?](#) - -`#usda 1.0` - -`def Xform` `"hello"` - -`{` - -`def Sphere` `"world"` - -`{` - -`}` - -`}` +``` +#usda 1.0 + +def Xform "hello" +{ + def Sphere "world" + { + } +} +``` Visual representation of this file: @@ -138,66 +131,39 @@ This pair of usda files, from [https://graphics.pixar.com/usd/release/tut\_refer _Below: RefExample.usda references HelloWorld.usda and overrides world's color._ -HelloWorld.usda - -[?](#) - -`#usda 1.0` - -`(` - -`defaultPrim =` `"hello"` - -`)` - -`def Xform` `"hello"` - -`{` - -`double3 xformOp:translate = (4, 5, 6)` - -`uniform token[] xformOpOrder = [``"xformOp:translate"``]` - -`def Sphere` `"world"` - -`{` - -`float3[] extent = [(-2, -2, -2), (2, 2, 2)]` - -`color3f[] primvars:displayColor = [(0, 0, 1)]` - -`double` `radius = 2` - -`}` - -`}` - +_HelloWorld.usda_ ``` -RefExample.usda +#usda 1.0 +( + defaultPrim = "hello" +) + +def Xform "hello" +{ + double3 xformOp:translate = (4, 5, 6) + uniform token[] xformOpOrder = ["xformOp:translate"] + + def Sphere "world" + { + float3[] extent = [(-2, -2, -2), (2, 2, 2)] + color3f[] primvars:displayColor = [(0, 0, 1)] + double radius = 2 + } +} ``` -[?](#) - -`#usda 1.0` - -`over` `"refSphere2"` `(` - -`prepend references = @./HelloWorld.usda@` - -`)` - -`{` - -`over` `"world"` - -`{` - -`color3f[] primvars:displayColor = [(1, 0, 0)]` - -`}` - -`}` - +_RefExample.usda_ +``` +#usda 1.0 +over "refSphere2" ( + prepend references = @./HelloWorld.usda@) + { + over "world" + { + color3f[] primvars:displayColor = [(1, 0, 0)] + } + } +``` ### Beyond the Basics @@ -205,7 +171,7 @@ RefExample.usda I recommend reading through the USD tutorials here [https://graphics.pixar.com/usd/release/tut\_usd\_tutorials.html](https://graphics.pixar.com/usd/release/tut_usd_tutorials.html), and experimenting in Blender and Nvidia USD Composer with creating or importing scenes and exporting to usda format files. Note: Blender does not support references: -_    "Like the USD exporter, the importer does not yet handle more advanced USD concepts, such as layers and references." [Universal Scene Description - Blender 4.2 Manual](https://docs.blender.org/manual/en/latest/files/import_export/usd.html)_ +_"Like the USD exporter, the importer does not yet handle more advanced USD concepts, such as layers and references." [Universal Scene Description - Blender 4.2 Manual](https://docs.blender.org/manual/en/latest/files/import_export/usd.html)_ Digital content creation (DCC) tools which takes advantage of USD references are Nvidia Isaac Sim, Nvidia USD Composer, and Pixar tooling.   @@ -271,25 +237,22 @@ For example, the _suzanne.usdc_ geometry file will be processed correctly and ge _usdObj-001.usda which references a separate suzanne.usd geometry file._ -[?](#) - -`def Mesh` `"suzanne"` `(` - -`prepend references = @suzanne.usdc@` - -`)` - -`{}` - +``` +def Mesh "suzanne" ( +prepend references = @suzanne.usdc@ +) +{} +``` [![](@todo_image)suzanne.usdc](@todo_image) [![](@todo_image)usdObj-001.usda](@todo_image) ![](@todo_image) -[?](#) -`Warning | The builder (Scene Builder) has not indicated it handled outputting product dependencies` `for` `file Assets/usdObj-001.usda.  This is a programmer error.` +``` +Warning | The builder (Scene Builder) has not indicated it handled outputting product dependencies for file Assets/usdObj-001.usda.  This is a programmer error. +``` Phase 2 enables referencing... @@ -313,7 +276,6 @@ Current: FBX Scene File to O3DE Entity [ -Embed Link Embed Link to Hipchat, Jira, Confluence ](#) @@ -323,36 +285,24 @@ While ultimately the same O3DE entity hierarchy is produced, notice how unlike b _Car.usd_ -[?](#) - -`def Xform` `"Xform"` - -`{` - -`def Mesh` `"carbody"` `(` - -`prepend references = @carbody.usdc@){}` - -`def Mesh` `"tire_1"` `(` - -`prepend references = @tire.obj@){` - -`double3 xformOp:translate = (1, 0, 0)` - -`}` - -`def Mesh` `"tire_2"` `(` - -`prepend references = @tire.obj@){` - -`double3 xformOp:translate = (0, 1, 0)` - -`}` - -`...` - -`}` - +``` +def Xform "Xform" +{ + def Mesh "carbody" ( + prepend references = @carbody.usdc@){} + + def Mesh "tire_1" ( + prepend references = @tire.obj@){ + double3 xformOp:translate = (1, 0, 0) + } + + def Mesh "tire_2" ( + prepend references = @tire.obj@){ + double3 xformOp:translate = (0, 1, 0) + } + ... +} +``` Phase 1 w/ References @@ -365,7 +315,6 @@ Phase 1 w/ References [ -Embed Link Embed Link to Hipchat, Jira, Confluence ](#) @@ -383,19 +332,16 @@ Embed Link Embed Link to Hipchat, Jira, Confluence 1. [Load USD as a Layer](https://github.com/lighttransport/tinyusdz/blob/83b4ebc90761e60444e55611238fff6cfb6a62c9/examples/tusdcat/main.cc#L149) 2. [Do compositions (it also returns a layer)](https://github.com/lighttransport/tinyusdz/blob/83b4ebc90761e60444e55611238fff6cfb6a62c9/examples/tusdcat/main.cc#L212) 3. Then [construct Stage from Layer](https://github.com/lighttransport/tinyusdz/blob/910c5740008377976a559c4ff1680292475abc85/src/composition.hh#L210)  - - [?](#) - - `bool` `LayerToStage(``const` `Layer &layer, Stage *stage` + + ``` + bool LayerToStage(const Layer &layer, Stage *stage + ``` 4. Then call `tinyusdz::tydra::ListPrims ` - - [?](#) - - `std::map meshmap;` - - `tinyusdz::tydra::ListPrims(stage, meshmap);` - + ``` + std::map meshmap; + tinyusdz::tydra::ListPrims(stage, meshmap); + ``` Note: USD can reference other USDs or geometry from other file types (example: obj). As long as the file type is supported by Assimp, we can load this geom and add it to the final flattened Assimp scene. @@ -412,7 +358,6 @@ Embed Link Embed Link to Hipchat, Jira, Confluence [ - Embed Link Embed Link to Hipchat, Jira, Confluence ](#) From a085fe63bfb23d6a958a68f24558ccae3aee73ff Mon Sep 17 00:00:00 2001 From: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com> Date: Thu, 22 Aug 2024 10:02:47 -0700 Subject: [PATCH 3/8] Adding images Signed-off-by: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com> --- ...RFC Feature - USD in the Scene Pipeline.md | 64 ++++--------------- 1 file changed, 14 insertions(+), 50 deletions(-) diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md index 45012d4..13a9313 100644 --- a/rfcs/RFC Feature - USD in the Scene Pipeline.md +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -66,8 +66,7 @@ Scene files describe 3D content: Geometry, materials, animations. They also incl ### What makes USD interesting, why use USD instead of another format like FBX, GLTF, STL, Collada, STEP, etc? USD is built around a composition feature, the ability to easily reference other files. It's a lot like our prefab system. - -![](@todo_image) +![image](https://github.com/user-attachments/assets/397b92ed-f43c-433e-8cc5-81ada05d33c8) Required O3DE Knowledge ----------------------- @@ -122,8 +121,7 @@ def Xform "hello" ``` Visual representation of this file: - -![](@todo_image) +![image](https://github.com/user-attachments/assets/8a08df9e-213a-4545-a93d-43da8a49b16c) ### Basic USD file references @@ -205,12 +203,10 @@ Do this in three phases: * Workflow - additional procedural prefabs will be available for content creators to use in the Editor. ### Simple Example - -![](@todo_image) +![image](https://github.com/user-attachments/assets/2eaf9c33-0fb0-403a-a954-837695c91c6f) ### Example with References - -![](@todo_image) +![image](https://github.com/user-attachments/assets/ee794185-f07b-4c23-bbb1-99f5c2b94ea7) Solution in Detail ------------------ @@ -225,11 +221,13 @@ Solution in Detail 3. Note: [AssImp latest has compiler error](https://github.com/assimp/assimp/pull/5693) 2. Add USD File Types to Asset Importer Settings 1. Edit _Registry/sceneassetimporter.setreg_ to process _usd, usda, usdz, usdc_ file types. - 2. _![](@todo_image)_ + 2. ![image](https://github.com/user-attachments/assets/fb1a4fea-c13b-4e0d-9044-58374b2b6d4c) + 3. Run Asset Processor 1. Allow asset processor to digest USD files, such as the toy\_biplane.usd file here. 2. [![](@todo_image)toy\_biplane\_idle.usdz](@todo_image) - 3. ![](@todo_image) + 3. ![image](https://github.com/user-attachments/assets/6eeacba2-afe2-4f22-95a2-0dc9b210a917) + At this point, USDs containing just geometry will load, however AssetProcessor will fail to create a product for a simple USD which references that very same geom file. @@ -246,9 +244,7 @@ prepend references = @suzanne.usdc@ [![](@todo_image)suzanne.usdc](@todo_image) [![](@todo_image)usdObj-001.usda](@todo_image) - -![](@todo_image) - +![image](https://github.com/user-attachments/assets/08feb54f-61e1-4faf-ac98-864a0d3f85b0) ``` Warning | The builder (Scene Builder) has not indicated it handled outputting product dependencies for file Assets/usdObj-001.usda.  This is a programmer error. @@ -268,17 +264,9 @@ The Assimp scene builder returns the sub-meshes and O3D breaks them out into sep Current: FBX Scene File to O3DE Entity -[ - -](@todo_image)[ - -](@todo_image) - -[ +![image](https://github.com/user-attachments/assets/7ccf90e4-ec91-42fe-bbef-5f40b3c807c0) -](#) - #### After Phase 2, USD files can reference other USD or OBJ meshes While ultimately the same O3DE entity hierarchy is produced, notice how unlike before, the car asset references external files (_carbody.usd_ and _tire.obj_) allowing multiple content creators to work at the same time. @@ -303,21 +291,8 @@ def Xform "Xform" ... } ``` - - -Phase 1 w/ References - -[ - -](@todo_image)[ - -](@todo_image) - -[ - - -](#) +![image](https://github.com/user-attachments/assets/9c5f4ddd-101b-4b2f-8da8-66daffbd2340) 1. Update Assimp to support some passthroughs we would need for USD support, specifically a way to override asset resolving. 1. See USD documentation on Asset Resolution: 1. [https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution) @@ -348,19 +323,8 @@ Phase 1 w/ References More Information: [Accessing Reference File Geom · Issue #186 · lighttransport/tinyusdz (github.com)](https://github.com/lighttransport/tinyusdz/issues/186) 3. From here, the current O3DE pipeline would take over, we would consume this flattened scene. - 1. Phase 2: USD w/ References Input/Output - - [ - - ](@todo_image)[ - - ](@todo_image) - - [ - - - ](#) - + 1. ![image](https://github.com/user-attachments/assets/e4aac22e-0ec1-463a-96a6-a2b3fbfe7ec7) + ### Phase 3 - Prefab output mimicking USD relationship graph @@ -578,4 +542,4 @@ Right now we have some big challenges with importing very large scenes. We load Our current pipeline for the most part works best when you give it large, composited scene files. We have no processing tools that can reconstruct a large complex scene from multiple different source scene files. -USD support would give us that. If a customer had a complex setup that would result in an extremely large single FBX file, they may be able to instead manage that content as multiple, smaller USD files that reference each other to create the same scene. This seems to be a big driving factor in Pixar creating this format to begin with, they were trying to solve these scale problems. \ No newline at end of file +USD support would give us that. If a customer had a complex setup that would result in an extremely large single FBX file, they may be able to instead manage that content as multiple, smaller USD files that reference each other to create the same scene. This seems to be a big driving factor in Pixar creating this format to begin with, they were trying to solve these scale problems. From 1d6f42a1123149b1013331228083b3ebcd2e4494 Mon Sep 17 00:00:00 2001 From: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com> Date: Fri, 23 Aug 2024 14:43:51 -0700 Subject: [PATCH 4/8] Adding build time stats if we wanted to move assimp out of 3rd party. Signed-off-by: Gene Walters <32776221+AMZN-Gene@users.noreply.github.com> --- rfcs/RFC Feature - USD in the Scene Pipeline.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md index 13a9313..d4daae4 100644 --- a/rfcs/RFC Feature - USD in the Scene Pipeline.md +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -366,7 +366,7 @@ Open Questions Should we move Assimp to a Gem instead of using O3DE 3rdParty system? --------------------------------------------------------------------- -Moving forward with USD via Assimp will require regularly pulling new versions of Assimp as new features come online (example: references). Going through the 3rdParty library system is slower as it requires its own approval process via [O3DE's 3rd Party Package repo](https://github.com/o3de/3p-package-source). The downside to moving Assimp into a gem directly, is that we'd reimplement what's already in 3rdParty, for example, [git pulling a particular Assimp commit](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_config.json#L2-L3), [building](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_assimp_windows.cmd), and [packaging](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/install_assimp_windows.json) the library. +Moving forward with USD via Assimp will require regularly pulling new versions of Assimp as new features come online (example: references). Going through the 3rdParty library system is slower as it requires its own approval process via [O3DE's 3rd Party Package repo](https://github.com/o3de/3p-package-source). The downside to moving Assimp into a gem directly, is that we'd reimplement what's already in 3rdParty, for example, [git pulling a particular Assimp commit](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_config.json#L2-L3), [building](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/build_assimp_windows.cmd), and [packaging](https://github.com/o3de/3p-package-source/blob/main/package-system/assimp/install_assimp_windows.json) the library. This would also increase build time when compiling the editor source; Assimp takes 2 minutes to download and compile on my local 16-Core 3.4GHz PC. Should we fail a job, or emit an error if a reference to another USD file can't be resolved? -------------------------------------------------------------------------------------------- From 4a245958ee0826ff61de2fa8ffc2a3527e2fac54 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Thu, 26 Sep 2024 16:38:11 -0700 Subject: [PATCH 5/8] Don't flatten USDs; merging meshes shouldn't be the responsibility of the importor. Breakout references and variants into separate phases Signed-off-by: AMZN-Gene --- ...RFC Feature - USD in the Scene Pipeline.md | 112 +++++++++++++----- 1 file changed, 83 insertions(+), 29 deletions(-) diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md index d4daae4..cd33018 100644 --- a/rfcs/RFC Feature - USD in the Scene Pipeline.md +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -183,24 +183,34 @@ Build support for USD into the scene pipeline. Do this in three phases: -1. Upgrade O3DE to use Assimp's latest implementation - 1. Input: single USD geometry files that don't use layers or references - 2. Output: azmodel containing the USDs geometry  -2. Contribute back to Assimp and update scene builder to load USD files with references. Stages are flattened by adding in reference geometry. O3DE then processes USD like FBX and other scene file formats. - 1. Input: USD files which contains a hierarchy of references to other geometry inside of other USD files  - 2. Output: Flattened azmodel containing the geometry from the multiple USD files inside one azmodel file -3. Contribute back to Assimp giving API option to flatten stages or keep reference geom separate. Update O3DE to create a procedural prefab that mimics the relationship graph of USD files by matching models and transforms. This could be expanded later to allow O3DE developers to translate USD properties into O3DE component properties as well as add advanced shaders/materials. - 1. Input: USD files which contains a hierarchy of references to other geometry inside of other USD files - 2. Output: Procedural prefabs generated per USD. Root prefab preserves the hierarchy of the original USD. +1. Single USD + 1. Upgrade O3DE to use Assimp's latest implementation + 2. Input: single USD geometry files that don't use layers or references + 3. Output: azmodel containing the USDs geometry  +2. USD with References + 1. Update Assimp scene tree with tags for referenced USD files + 2. Update O3DE scene builder to load USD files and look for reference tags. + 3. Update O3DE to create a procedural prefab that mimics the relationship graph of USD files by matching models and transforms + 4. Input: USD files which contains a hierarchy of references to other geometry inside of other USD files  + 5. Output: Procedural prefabs generated for the root USD as well as referenced USDs. Root prefab preserves the hierarchy of the original USD. +3. USD with Variants + 1. Updated Assimp scene tree with tags for variants within a USD + 2. Update O3DE scene builder to load USD files and look for variants + 3. Input: USD file which contains X variants + 4. Output: + 1. X meshes and X prefabs generated for each variant + 2. Prefabs which references a variant in its hierarchy will nest 1 of the prefab variants ### Why these three phases? * Phase 1 is extremely fast. -* The work for phase 2 is used in phase 3, so there won't be much wasted effort here. * Once phase 2 is completed, there's great benefits to using USD: content creators can start splitting up content on their end. -* Benefits of phase: +* Benefits of phase 2: * Performance - asset processing time, and hard drive space used by product assets: We won't need to re-create product assets, such as models, that are referenced by multiple other USD files. * Workflow - additional procedural prefabs will be available for content creators to use in the Editor. +* Phase 3 and beyond nice-to-have. + * Variants are advanced so makes sense to break them out into a later phase. + * Avoid variant explosion by only selecting 1 of many variants when generating procedural prefabs ### Simple Example ![image](https://github.com/user-attachments/assets/2eaf9c33-0fb0-403a-a954-837695c91c6f) @@ -252,9 +262,9 @@ Warning | The builder (Scene Builder) has not indicated it handled outputting pr Phase 2 enables referencing... -### Phase 2 - Update Assimp to support flattened USD with references +### Phase 2 - Update Assimp to support USD with references -Update Assimp to read USDs, composite all sub-layers and references and flatten all the meshes into one Assimp scene. O3DE will treat the root USD as one big mesh. +Update Assimp to read USDs, composite all sub-layers and tag references in Assimp scene. O3DE will parse the Assimp tree to find references and spin off procedural prefabs. #### The Current Setup @@ -297,8 +307,14 @@ def Xform "Xform" 1. See USD documentation on Asset Resolution: 1. [https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution](https://graphics.pixar.com/usd/release/glossary.html#usdglossary-assetresolution) 2. [https://graphics.pixar.com/usd/release/api/ar\_page\_front.html](https://graphics.pixar.com/usd/release/api/ar_page_front.html) -2. Update Assimp to generate an Assimp scene by flattening and resolving the O3DE scene - 1. Today a USD loaded via Assimp returns a scene, but if that scene references other USD files, the scene won't contain the geometry of the references. Update Assimp to recursively find references and mark them as a file dependency. +2. Update Assimp to generate an Assimp scene containing a way to recognize references + 1. Today a USD loaded via Assimp returns a scene, but if that scene references other USD files, the scene won't contain the geometry of the references. Update Assimp to recursively find references and tag them as a reference (include necessary data such as file path, and reference name) + 1. Assimp outputs the scene graph like it currently does, but when it hits a reference, it annotates that node in the scene graph with a property that describes the reference. + 1. Assimp::Scene root "kitchen" + 1. child 01 + 1. Name: "spoon 01" + 2. Transform: , , + 3. Reference: "c:/project/spoon.usd" TinyUSDZ, the 3rd Party USD library used by Assimp, has methods for discovering references geometry. @@ -317,25 +333,46 @@ def Xform "Xform" std::map meshmap; tinyusdz::tydra::ListPrims(stage, meshmap); ``` - Note: USD can reference other USDs or geometry from other file types (example: obj). As long as the file type is supported by Assimp, we can load this geom and add it to the final flattened Assimp scene. + Note: USD can reference other USDs or geometry from other file types (example: obj). As long as the file type is supported by Assimp, Assimp can tag it, and O3DE can load the geom and add it to the procedural prefab. More Information: [Accessing Reference File Geom · Issue #186 · lighttransport/tinyusdz (github.com)](https://github.com/lighttransport/tinyusdz/issues/186) -3. From here, the current O3DE pipeline would take over, we would consume this flattened scene. +3. From here, the current O3DE pipeline would take over, we would consume this scene. 1. ![image](https://github.com/user-attachments/assets/e4aac22e-0ec1-463a-96a6-a2b3fbfe7ec7) + 2. Update Create Jobs in the scene builder to output job dependencies on referenced USD files. + 3. [See FAQ for why we need both source and job dependencies](#USDintheScenePipeline-Whydoweneedbothsourcedependenciesandjobdependenciesforreferences?). + 4. Job dependencies will be needed at this point because product assets from one USD file (prefabs, models, etc) will be referenced by other USD files to mimic the USD relationship graph. + 5. Update the scene builder to generate USD based procedural prefabs. + 1. We will need to define what these look like. [We may need additional features from procedural prefabs to hit our end goals](https://wiki.agscollab.com/display/lmbr/USD+in+the+Scene+Pipeline#USDintheScenePipeline-WhatdoweneedfromproceduralprefabsforUSDtobefullyfunctional?). + +### Phase 3 - Update Assimp to support USD with Variants + +Variants are switchable references. Ideally, an O3DE component could provide a drop down in editor to quickly select from a list of variants within a variant set, but this phase simply recognizes variants in order to generate a mesh and prefab for each. Any root USD referencing a variant set will make a single variant selection. +1. Update Assimp to generate an Assimp scene containing a way to recognize variant sets, via tagging + 1. Assimp outputs the scene graph like it currently does, but when it hits a variant, it annotates that node in the scene graph with a property. + 1. Assimp::Scene root "kitchen" + 1. child 01 + 1. Name: "spoon 01" + 2. Transform: , , + 3. Reference: "c:/project/spoon.usd" + 4. Variant: "WoodenSpoonA" +2. O3DE pipeline would consume this scene + 1. Update scene builder to look for "variant set" and "variant" tags + 2. Update scene builder to generate a procedural prefab for each variant in a variant set + 3. Update scene builder so root USD prefabs referencing a variant set (referenced inline or another .usd file) will have a variant selected for them in their procedural prefab + 1. In instances where the USD calls out a variant set, but no selection, the 1st variant in the variant set list will be selected + 4. Do not try to combine + 1. A single chair.usd becomes: + 1. chair.procprefab + 2. chairback_01.procprefab -> chairback_01.atommesh + ... + 3. chaircushion_01.procprefab -> chaircushion_01.atommesh + +Note: Variants are not always meshes, they can also just be basic overrides for transforms, material properties, etc. +We should support transforms, but material properties and beyond will wait for future roadmap. -### Phase 3 - Prefab output mimicking USD relationship graph - -* Add a feature flag to toggle USD flattening off, so we can deliver the rest of this work iteratively. -* Update Create Jobs in the scene builder to output job dependencies on referenced USD files. - * [See FAQ for why we need both source and job dependencies](#USDintheScenePipeline-Whydoweneedbothsourcedependenciesandjobdependenciesforreferences?). - * Job dependencies will be needed at this point because product assets from one USD file (prefabs, models, etc) will be referenced by other USD files to mimic the USD relationship graph. -* Update the scene builder to generate USD based procedural prefabs. - * We will need to define what these look like. [We may need additional features from procedural prefabs to hit our end goals](https://wiki.agscollab.com/display/lmbr/USD+in+the+Scene+Pipeline#USDintheScenePipeline-WhatdoweneedfromproceduralprefabsforUSDtobefullyfunctional?). -* Decide if we want to either keep the feature flag for USD flattening and turn it off by default, or remove the flag and remove USD flattening support in process job. - Alternate Solutions =================== @@ -378,7 +415,7 @@ Do we need to emit references to other USD files as job dependencies, or can the If we're able to fully rely on procedural prefabs to manage the composition of USD file references, and we're able to predict the sub IDs of products from other USD files, then we may not need to emit these references as source dependencies. -USD has support for [abstract base objects](https://wiki.agscollab.com/display/lmbr/USD+Investigation#USDInvestigation-AbstractObjects), that function sort of as inverted overrides. Our prefab system does not have a similar concept, so we may need to output job or source dependencies to use this information to build a prefab hierarchy. We may find that we instead need to add this sort of functionality to prefabs. +USD has support for abstract base objects, that function sort of as inverted overrides. Our prefab system does not have a similar concept, so we may need to output job or source dependencies to use this information to build a prefab hierarchy. We may find that we instead need to add this sort of functionality to prefabs. ### Example @@ -464,11 +501,28 @@ Out of Scope / Long Term Work Advanced shaders and materials ------------------------------ - For our initial deliverable, we will generate basic O3DE physically based rendering materials. However, the options for shaders and materials from USD files are much bigger than that. As we wrap up our first set of work to support USD, we'll want to start planning what a more fully featured material and shader pipeline from USD files could look like. +Extra USD Properties and O3DE Components +------------------------------ +Allow developers to read generic scene information, and inject custom components into the procedural prefabs generated by importing USD files. O3DE would ship with built-in components. +1. Update Assimp to output USD property data + 1. Assimp::Scene root "kitchen" + 1. child 01 + 1. Name: "spoon 01" + 2. Transform: , , + 3. Reference: "c:/project/spoon.usd" + 4. Variant: "WoodenSpoonA" + 5. NEW: Other user data/properties/overrides + +1. O3DE scene system already supports storing generic info into each scene node + +Editor Variant Selection Dropdown +------------------------------ +Allow designers to select from a list of available variants from within the editor when a variant set is included in a USD + Convert O3DE prefabs to USD files --------------------------------- From cf0ee2956b219ea11d9ce5bd658411ad8d771838 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Thu, 26 Sep 2024 16:48:08 -0700 Subject: [PATCH 6/8] Minor clean up; moving around information in the 'out-of-scope' section Signed-off-by: AMZN-Gene --- rfcs/RFC Feature - USD in the Scene Pipeline.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md index cd33018..9fe4d8e 100644 --- a/rfcs/RFC Feature - USD in the Scene Pipeline.md +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -505,9 +505,10 @@ For our initial deliverable, we will generate basic O3DE physically based render However, the options for shaders and materials from USD files are much bigger than that. As we wrap up our first set of work to support USD, we'll want to start planning what a more fully featured material and shader pipeline from USD files could look like. -Extra USD Properties and O3DE Components + +USD Property -> O3DE Component Converter ------------------------------ -Allow developers to read generic scene information, and inject custom components into the procedural prefabs generated by importing USD files. O3DE would ship with built-in components. +Allow developers to read generic scene information, and inject custom components into the procedural prefabs generated by importing USD files. 1. Update Assimp to output USD property data 1. Assimp::Scene root "kitchen" 1. child 01 @@ -518,10 +519,8 @@ Allow developers to read generic scene information, and inject custom components 5. NEW: Other user data/properties/overrides 1. O3DE scene system already supports storing generic info into each scene node +1. Provide developers with a way to add to the procedural prefabs recipe, one example we could ship with is an editor component for variant selection dropdown. It allow designers to select from a list of available variants from within the editor when a variant set is included in a USD -Editor Variant Selection Dropdown ------------------------------- -Allow designers to select from a list of available variants from within the editor when a variant set is included in a USD Convert O3DE prefabs to USD files --------------------------------- @@ -533,7 +532,7 @@ This is something that could be looked at long term. FAQ === -Why not stop after phase 2: just treat USD like other formats, flatten everything, and then generate product assets? +Why not just treat USD like other formats, flatten everything, and then generate product assets? -------------------------------------------------------------------------------------------------------------------- By expanding USD support to create a prefab hierarchy that mimics the USD relationship graph, we will see two main areas of benefit: From e74ffe2c17bcc34931fc76b9412aaede1260bab7 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Thu, 26 Sep 2024 16:54:23 -0700 Subject: [PATCH 7/8] Mentioning the ability to export USDs from O3DE based on RFC feedback Signed-off-by: AMZN-Gene --- rfcs/RFC Feature - USD in the Scene Pipeline.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md index 9fe4d8e..be84aa8 100644 --- a/rfcs/RFC Feature - USD in the Scene Pipeline.md +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -525,7 +525,7 @@ Allow developers to read generic scene information, and inject custom components Convert O3DE prefabs to USD files --------------------------------- -This initial work would not include the ability to create USD files from O3DE content, like prefabs. A prefab to USD converter could be the first step towards replacing prefabs with USD entirely.  +This initial work would not include the ability to create USD files from O3DE content, like prefabs. A prefab to USD converter could be the first step towards replacing prefabs with USD entirely. This would give O3DE the ability to modify the imported USD so that the source DCC could receive O3DE specific properties in order to create a back-and-forth workflow. This is something that could be looked at long term. From 811398d0a702daf4c2f69f6c28064f0c47f8cca7 Mon Sep 17 00:00:00 2001 From: AMZN-Gene Date: Thu, 26 Sep 2024 17:28:06 -0700 Subject: [PATCH 8/8] 6. USDs commonly reference files outside of their O3DE project. We should generate AssetProcessor errors to inform users to copy external USD references to their local project folder. Signed-off-by: AMZN-Gene --- rfcs/RFC Feature - USD in the Scene Pipeline.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/rfcs/RFC Feature - USD in the Scene Pipeline.md b/rfcs/RFC Feature - USD in the Scene Pipeline.md index be84aa8..f3aed06 100644 --- a/rfcs/RFC Feature - USD in the Scene Pipeline.md +++ b/rfcs/RFC Feature - USD in the Scene Pipeline.md @@ -345,6 +345,14 @@ def Xform "Xform" 4. Job dependencies will be needed at this point because product assets from one USD file (prefabs, models, etc) will be referenced by other USD files to mimic the USD relationship graph. 5. Update the scene builder to generate USD based procedural prefabs. 1. We will need to define what these look like. [We may need additional features from procedural prefabs to hit our end goals](https://wiki.agscollab.com/display/lmbr/USD+in+the+Scene+Pipeline#USDintheScenePipeline-WhatdoweneedfromproceduralprefabsforUSDtobefullyfunctional?). + 6. USDs commonly reference files outside of their O3DE project. We should generate AssetProcessor errors to inform users to copy external USD references to their local project folder. + 1. Remote reference example from Nvidia's Isaac Sim: + ``` + def Xform "A1" ( + prepend references = @omniverse://localhost/NVIDIA/Assets/Isaac/2023.1.1/Isaac/Robots/Unitree/a1.usd@ + ) + ``` + ### Phase 3 - Update Assimp to support USD with Variants