4D Pipeline Visualizing Reality Blog - trends & breaking news in 3D Visualization, Virtual Reality, Augmented Reality, and eXtended Reality.

Having a way to see apparel and footwear designs on avatars in real-time further empowers designers, brands, and sales teams to create more successful products.

There has been a huge increase in demand to market products online and the need to make commerce more digital. Images and videos are no longer enough. Consumers want to see, to feel, and to interact with the product they are looking to purchase. They want to pick it up and turn it around, they want to zoom in and see the detail. As such, digital sales and marketing are getting richer and more emotional with real-time interactive 3D graphics.

As an example, this could be seeing a pair of shoes in 3D, being able to zoom in and spin them around, customize them, and get a real sense of their style and fit. Or it may mean something even more immersive such as using Augmented Reality to see new furniture in your room at home, or even using Virtual Reality for a deeply immersive experience. In all these examples, the underlining 3D file format is an integral part of the formula for success. It is especially important for those manufactures who have hundreds or even thousands of products that they need to publish and sell.

The need is greater than ever. People spend more time at home but still need to buy things. Customer-at-home shopping means needs have changed so products that were once bought in person now need to be represented well online - whether it be a car, clothing, or furniture. 

With the circumstances we are in, your customers are wanting to interact with your products from home so you need the best visual representation of the item. The good news is the technology is there and has even picked up in the last year and a half.

Now before you decide to push online with interactive 3D models of your products, there are some challenges you should be aware of. You want to make sure the visual output is high quality, your industry is getting behind it, and you have carefully considered which format you want to choose for your 3D files. Essentially, 3D is great only if it can be delivered quickly and at a high enough quality (for the use-case).

Let’s start with breaking down the 3 main file formats that you can choose from for your 3D files: USD, USDz and glTF. Each of these formats is relatively new and is currently receiving a lot of attention.

USD (Universal Screen Description) – interchange format that can be used for file editing in Digital Content Creation Tools such as Maya. For example Apple/Pixar uses this.

USDz – a subset of USD, used by Apple and supported in iOS AR applications (a zip file with USD and image files).

glTF – transmission format for 3D assets that is well suited to the web and mobile devices by removing data that is not important for the efficient display of assets.

Keep in mind, these file types are not in the exact same space. glTF is predominantly web and real-time, whereas USD is a medium meant for bigger, more complex pipelines, involving groups of people that tend to use offline rendering.

Let’s dive deeper into the different file formats:

USD was designed by Pixar Animation Studios (Pixar). Film studios tend to use this format because it allows for non-destructive layering of content. Animators, shot-blocking, set dressing and look dev can all work on the same "file." It's built to let people collaborate. They needed one uniform format that everyone can work on in one place as it was a mess moving from application to application, so this solves a huge problem. All groups on a project, even different departments can work on layers within the same file without tripping over each other.

Pros USD:

1.       Great to use with digital content creation tools like Adobe Dimension/Substance , Maya, Houdini, UE4, Blender

2.       Supports a rich and wide set of features such as 3D models, scene hierarchy, materials, lighting, animations, bones, skinning, and blend shapes.

3.       Supported by other applications to accurately and easily move data from your 3D editor to another tool..

4.       Supported features are very modern.

5.       USD has a notion of variant management, for example to see shoes in different colors or geometric structure.

6.       Great for pipelines between applications.

7.       Massively complex materials can be run offline, not so much run offline. 

8.       When creating a feature film or animations, you can coordinate animators, technical directors, modelers, artists.

Cons USD and USDz:

1.       USDz: If you are publishing an AR experience for Android and iOS, it forces artists to worry about 2 file formats because of iOS’s support for USDz. The Artist has to create 2 file formats to run on Apple (Safari) and Android (Chrome). Web browser AR needs two assets as well.

2.       USD is not suitable for online retail as it is not geared for it.

3.       It is not going to bring you to the web yet.

glTF is great for web-based applications as physically based rendering and computation restrictions are higher on the web for browsers on desktop and mobile. For example, looking at an Ikea couch online, it is simple and easy to render in glTF. There are less computations. The Khronos Group, who defines how materials of your product look online, have upped their game in the last years and now provide sheen properties for fabric, clear coats for a car, etc. Recently, their transmission refraction volume rendering – how light passes through glass, and certain objects like crystal and the absorption of light by the object – are becoming life-like.

Pros glTF:

1.       Great for web based.

2.       Ideal for web, mobile, desktop. Best format the industry has created.

3.       Much simpler material models.

4.       Khronos is driving game engines to conform to a set of Standards.They have been working on asset preparation guidelines to help designers to get a better output. WebGL rendering engines are being held to Khronos Standards. This is an attempt to guarantee a minimum viewing experience for a well-made glTF asset.

5.       Made best for low poly. For example, a car from Honda can display fewer triangles of the car for faster loading.

Cons glTF:

1. Not easy to move data from your 3D editor to another tool. To help with this, some of the DCC packages like Max and Maya have glTF exporter plugins. Blender, in our opinion, has the best one. 

2. Does not have a sophisticated material system. Very simple texture maps and constant values only.

We sat down with our colleague Jason Coleman to get his take on what we have been working on at 4D Pipeline to help our clients use these two formats.

“We worked with Saddington Baynes to build an online car configurator for Honda. This allows Honda customers to design a car online. The original CAD file had 300 million triangles (after tessellation) for the car, far too big for displaying in a WebGL viewer. We processed, optimized, and broke it down to a ~300 thousand triangle glTF file to show up in a web browser. This then allows the customer to change the seating colors, accessories, wheels, trim, change the paint job etc. Making the car to be as realistic as possible and load quickly via web. Ultimately, making the car look great and having the customer engaged as soon as possible. Creating customer engagement, delight and fun will lead to the purchase. Anything that inhibits this process has to be removed. But not at the cost of quality.”

Another example was provided by Jason using the glTF file format for the collaboration between Ikea and Geomagical Labs (both customers of 4D Pipeline).

“Ikea utilizes Geomagical Labs technology to take a photograph of your room. Ideally taking 10 photographs from slightly different points of view. AI will then turn the photo into a 3D scene. Then the customer can take Ikea furniture and drag and drop it into the room where they want it. At 4D Pipeline, in order to accomplish this, we built an optimization system to process all their 3D files into low poly glTF files to come into the system fast and design their rooms. Our system figures out the optimal geometry and material package to yield a good visual experience while decreasing the size of the glTF file. When the customer drops an item in it will react with the scene and lighting. We also made them a glTF viewer and we are training their AI to recognize the data for machine learning.”  

We asked Jason how the interior design industry is adapting to put design technology more into the hands of customers.

“Let’s say you go into a house with an interior designer or to do a renovation, the experience could be virtual to see the vision come to life. The new windows could show with light coming in, you can see your full renovation while making design decisions fully informed.”

Jason also explained how this technology would work.

“There are currently two ways of thinking about capturing 3D – lidar or multi-view camera based. Multi-view camera based is what Geomagical does. I believe the next frontier is making it easier to capture a room by the customer. iPads already have lidar built in. The average person can take a photograph of their room and reduce the friction of needing design software or wondering how it will turn out. With Geomagical, we can now take photographs from different points of view, wait 10 minutes, and start playing with the room! And it works in web browsers and on iOS and Android (through Apps).”

We asked Jason about the future of online retailers and eTailing utilizing 3D.

 “The goal is making objects more realistic and utilizing the best design paradigm – web design needs to focus more on the customer 3D interaction. At 4D, we strive in helping our customers to modernize their pipelines, create better website experiences, making experiences more tangible, bypassing being in the shop, recreate it, showcasing great materials, and great lighting. All for their end customers to have a great interaction with the product in 3D and ultimately buy into it.”

 Jason also shared advice on where a company should go from here.

“Measuring benefits and sales uptick is key in taking a product 3D and online. You want to measure the conversion rate of the purchases your customer’s make. We do this at 4D Pipeline by using analytics to see how long they are looking at the object, Telemetry (measurements various states of the system – dwell time) and metrics. Our goal is to mitigate the risk of getting good-looking assets in front of customers in a timely fashion and enabling them to decide if the product or purchase is good for them. This is an engineering problem that needs artists to help solve it. Everything from lighting, materials, interaction experience all work together to make these beautiful objects come to life on the screen.”

When choosing to bring your product online and in 3D with hopes of captivating your customer’s imagination and facilitating their purchase, it is important to start with the “why” and then choose the right file format with your technology team for your needs. If you want any guidance on which format to choose or how to bring your product into 3D and online, please reach out to us at 4D Pipeline. With our customers including amazing brands like; Adidas, AUDI, Epic Games, Target, IKEA, Adobe, AMD, Intel, Dassault Systèmes, SolidWorks, Saddington Baynes, TurboSquid, we are happy to help you navigate this 3D world.