Build an Immersive Virtual Reality Experience of Amazon Fulfillment Center Tour

Introduction

In the rapidly evolving landscape of warehouse efficiency, inventory planning, and supply chain management, Amazon continues to innovate with its Fulfillment Center (FC) model. Amazon offers free in-person and virtual tours at select locations worldwide, to introduce the people, technology, and processes behind customer order preparation. Amazon Tour allows customers to see different types of robots in action and explain how they make the fulfillment process more efficient! Robots not only benefit the people who work alongside them by reducing the distance they walk, but they also improve the shopping experience for our customers by allowing us to hold more inventory and process orders more quickly. Traditional on-site FC tours, while informative, face limitations in industry personalization, resource and bandwidth constraints, and scalability. To address these challenges, we have deployed an immersive Virtual Reality (VR) tour, using Treedis and Matterport, of Amazon Fulfillment Centers that provides a 24/7, totally immersive experience. The primary goal is to offer customers a realistic and captivating virtual walkthrough of an FC, leveraging the latest VR technology. This technical blog delves into the project’s purpose, solution components, and the art of the possible for AWS customers from diverse industries developing immersive, interactive VR experience for their own operations.

Objectives

The VR FC tour project has a multifaceted objective that aims to demonstrate to AWS customers how Amazon uses physical AI and AWS services to process millions of packages daily. The overarching goals of this project are as follows:

Showcase Technologies and Processes

Explore the innovative technologies and AWS services used in FCs. This virtual tour provides AWS customers with a detailed look at how Amazon integrates machine learning, computer vision, and robotics with AWS services to achieve its operational efficiency.

Scaling the Amazon FC Tour for Every AWS Customer

Develop a fully immersive, interactive, and customizable virtual environment that enables AWS customers to explore the operational dynamics of Amazon FCs. This virtual world will transport users into the heart of FC operations, allowing them to experience the intricate processes and technologies firsthand. By scaling the FC tour experience, AWS customers from diverse industries can gain valuable insights and inspiration for their own operations.

Generating Art of the Possible Ideas

The key elements of the solution, such as enhanced interactive displays, industry-specific overlays, and personalized narration, provide customers with a deeper understanding of operational philosophies, sustainability initiatives, and the overall efficacy of Amazon FCs. Moreover, this immersive experience will foster the generation of new ideas among customers, inspiring them to build predictive maintenance, worker training and safety solutions, and warehouse automation tailored to their unique business needs using AWS services.

By achieving these objectives, the VR FC tour project showcases Amazon’s operational prowess to empower AWS customers to reimagine their own operations. This innovative approach fosters collaboration, knowledge sharing, and the exploration of cutting-edge solutions that leverage the power of AWS and physical AI technologies.

Solution Overview

In building this project, we integrated technologies from AWS, Matterport, and Treedis to offer an end-to-end immersive virtual reality experience of the Amazon FC tour. The following details each architectural component:

Architecture Diagram of Amazon FC tour VR experience

Build Immersive Experiences

To craft an immersive VR FC tour experience, we leveraged the Treedis no code platform, enabling us to create custom-branded digital twins, hybrid experiences, and navigate the virtual environment with ease. Without the need for coding, we harnessed a suite of user-friendly solutions to build our own virtual experience. Treedis’ advanced workflow creator, ‘Flows’, transformed the fulfillment center into a step-by-step onboarding journey, allowing users to effortlessly design guided training flows and explore specific journeys at their discretion – enabling even non-developers to contribute crucial knowledge and expertise to the experience.

The Digital Twin Studio, another no-code solution, empowered us to customize and render a digital twin model of the FC facility with seamless flexibility. The navigation tags feature proved invaluable, enabling automatic user guidance through virtual bubbles that displayed the path to specific locations within the tour. Elevating the immersive experience further, we utilized the 3D Editor to incorporate customer media, such as videos and workflow images, as well as staging 3D assets like packages and robots, demonstrating FC operations. Moreover, we employed the green screen feature to integrate a real person narrating each stage of the FC’s operations, adding an authentic touch to the virtual journey.

Treedis has also built a pipeline that applies all enhancements across both digital and VR devices, ensuring a seamless, immersive experience on any platform, with AR support available for future use. With Treedis’ powerful suite of tools, we created a truly unique and informative VR FC tour experience that blends digital and physical worlds, providing a glimpse into the inner workings of our facilities. To get more details on Treedis services and subscription, please visit AWS marketplace.

3D Modeling and Simulation

To create an immersive and accurate representation of the Amazon FC facility, we leveraged the powerful capabilities of Matterport’s SaaS, 3D Cameras, and professional capture services. These tools enabled us to transform the physical space into a navigable, photorealistic, and dimensionally accurate digital replica. Matterport Capture Services, a fully managed solution, empowered us to capture true-to-life digital twins of the FC facility with ease. We then hosted these digital twins on Matterport’s 3D data platform, utilizing their SaaS solution to ensure seamless access and integration. Leveraging Matterport APIs, we programmatically connected our Treedis systems, granting us direct access to the FC model hosted on the Matterport platform. This integration allowed us to incorporate the highly detailed and accurate digital replica into our VR experience, enhancing the level of immersion and realism saving customer’s time and from walking approximately 1 mile. For those interested in exploring Matterport’s services and subscription options further, we invite you to contact them directly or visit the AWS marketplace, where you can find comprehensive information and pricing details.

Application Hosting and Access

To provide a seamless and accessible experience for users exploring the Amazon FC VR Tour, we leverage AWS Amplify for hosting our React-based website. The application offers interoperability experience across VR and desktop browsers to access the virtual experience. To offer secure access and user management at-scale, we integrated the application with Amazon Cognito, a robust solution for customer identity and access management.

Generative AI Integration

To create an interactive VR experience, we integrated Amazon Bedrock for remote collaboration and knowledge sharing and used Treedis’ additional VR capabilities. A Q&A bot in our virtual environment empowers customers to familiarize themselves with FC processes. The Q&A bot, powered by Amazon Bedrock, was trained on a comprehensive repository of common questions and detailed process information. This training enables users to ask questions using natural language, ensuring an intuitive and conversational experience. Through this approach, we not only provided an immersive virtual tour but also facilitated real-time knowledge sharing and collaboration. Customers could explore the virtual FC environment while simultaneously engaging with the Q&A bot, gaining valuable insights and resolving queries on-the-fly.

End user experience

Whether you’re a new or existing AWS customer, a potential partner, or an aspiring logistics professional, the Amazon FC Tour VR experience promises to be a transformative and unforgettable insight into FC operations and will help you learn the role of AWS services and physical AI to run one of the largest fulfillment centers efficiently. You can use VR devices or your workstation to unveil the intricate processes and technologies powering our facilities.

End user experience of Amazon FC tour VR experience

Conclusion

The Amazon Fulfillment Center Virtual Reality Experience project represents a significant step forward in enabling customers and organizations to leverage AWS and physical AI to drive warehouse efficiency. This groundbreaking experience was launched at the re:Invent 2024 New to AWS Expo booth, with Treedis and Matterport also present for the launch, generating customer excitement and sparking innovative ideas to utilize similar solutions in various domains. From streamlining worker training solutions and seamless site operations to accelerating factory planning processes, predictive maintenance, and property marketing, the potential applications of this technology are vast and far-reaching. The overwhelmingly positive response from attendees underscored the transformative impact this VR experience can have across industries. If you are interested in experiencing this VR tour of an Amazon Fulfillment Center firsthand, we invite you to reach out to us at the Experience Amazon Fulfillment Center (FC) Virtual Reality Experience link. If you would like to develop a customized solution tailored to your specific industry needs, please check out Treedis and Matterport on AWS Marketplace.