I want to see the galaxy of the universe in the room. I want to try on 8 dresses within 1 minute in the morning. I want the "hero" of the game to jump out and have a fight on the desk... Our ideas are really getting more and more.
People's inspiration has inspired many possible creations. Every technological revolution is the evolution of human "cognition". At the Facebook Developer Conference last year, art was driven by AR and SLAM technology to create perfect works of art. Using positioning technology and scene geometric analysis, the Facebook team cleverly projected the works of the famous artist Heather Day in real space. In the middle, with the live bgm melody, beautiful lines and lifelike paintings appeared in front of people.
Heather's artwork is perfectly projected into the real world
Hardware equipment iterations, sensor upgrades, and the development of AR technology have allowed us to break the boundaries of visual space. The challenge of time-space teleportation is no longer an illusion. When it comes to AR, SLAM is a key underlying technology-simultaneous positioning and map construction, combining and integrating real scenes and imaginary spaces, and using the power of computer vision and graphics technology to eliminate the boundary between virtual and real.
AR group photo-dynamic images merged into the real world
AR integrates the information of the real world and the virtual world, and the two kinds of information complement each other, overlap and interact in real time. Positioning virtual materials in three-dimensional space allows different perspectives to see the effect of fusing virtual information with the real world. Therefore, the two key capabilities of spatial 6DoF positioning and map construction must be mentioned:
6DoF positioning-including location information and angle information, based on the mobile phone camera we will see the AR effect, shooting from any angle can see the scene of different perspectives, the field of view seen from different angles can be integrated with the real world. Therefore, it is critical to calculate the current spatial position and angle of the mobile device in real time. This is the 6DoF positioning capability.
Map construction-a full understanding of the three-dimensional space and plane structure of the entire scene, such as where is the floor and where is the wall, quickly and accurately construct the scene online, can support real-time positioning and virtual object implantation, when we recognize these geometric information, we You can see the AR effect combining reality and virtuality such as "put things on the floor and hang murals on the walls".
In addition to the core of simultaneous positioning and map construction, what knowledge points are included in SLAM and 3D vision technology?
Flat/3D object recognition and tracking: Based on industry-leading image recognition and tracking technology, it can quickly identify flat/3D objects in the scene and detect and track obstacles. If a virtual cup is placed in a scene, the algorithm can identify Its location and the three-dimensional structure of the environment, no matter from which angle it is detected and recognized, the objects in the picture will correspond to the real world, which is based on a stable three-dimensional tracking effect.
Motion recovery structure (SFM): For some video images, compared to SLAM for real-time motion tracking, we can also do it offline. The algorithm is similar to SLAM. A good picture or video is processed offline, and the whole scene can be restored with complete 3D points. For some scenes shot in disorder, the three-dimensional structure of the scene can also be restored through SFM.
Real-time depth estimation and 3D reconstruction: If you want the scene structure to be restored online more complete, you must restore a denser 3D point cloud, that is, dense 3D reconstruction. The stereo matching algorithm is used to calculate the spatial 3D distance of each pixel on the screen. The entire scene scanning process integrates the depth information of multiple frames to obtain a complete and accurate 3D model of the scene. This is real-time 3D based on depth fusion Reconstruction algorithm. The algorithm supports real-time depth of field blur, binocular stereoscopic, realistic rendering, shadow synthesis, occlusion and collision of virtual and real objects and other interactive effects on mobile devices, creating a physically real "virtual and real fusion" world.
Dense 3D reconstruction
Lighting estimation: Through computer vision and graphics algorithms, based on the video data of the real scene, the lighting environment of the scene is estimated, including brightness, light source, etc., and the lighting information is superimposed on the virtual model during rendering, so that the rendering effect is more closely related to the real environment. Together.
Scene image + virtual and real combination
Cross-platform lightweight AR/VR engine: Different from the previous SLAM underlying technologies, the cross-platform lightweight AR/VR engine integrates back-end SLAM, 3D reconstruction, and front-end 3D rendering and rendering, and integrates AR/ VR engine. What is finally presented to the user is the fusion effect of the virtual scene drawn in three dimensions and the real world. It has the characteristics of cross-platform, small library size and support for multiple lighting and material models, as well as occlusion of virtual objects, shadow projection, and Avatar expression driving.
SLAM technology in the field of games and robotics fights the battle, we can do more
In order to ensure that the path planning of the machine is more intelligent in some specific environments, in the field of robot application, the existence of SLAM technology also provides technical support for the robot to recognize the surrounding environment, avoid obstacles, plan the path, and construct the scene map in real time.
In the field of autonomous driving, the purpose of SLAM applications is to solve the problems of positioning and 3D reconstruction. In the process of autonomous driving, real-time positioning and path planning capabilities are very important, and information and coordinates must be accurately obtained in real time. Regarding map construction, the SLAM algorithm will restore the high-precision three-dimensional structure of the entire scene through the information collected by the sensors. High-precision maps are also a core part of automatic driving.
SLAM is combined with more and more scenarios, and many users have already perceived a new way to experience AR applications. SLAM and AR technology allow them to experience the subversive interactive experience in video games up close. Originally, our experience was only in a virtual 3D scene. Nowadays, the virtual person in the game can jump out of the mobile phone and fight in the real scene. We can not only take photos with the "hero" in the game, but even with virtual people in the room. Let "Encounter".
OPPO teamed up with Shangtang to create an AR developer platform
King of Glory APP settled
"Hero" jumped out of the mobile phone and competed in the real world
In addition, SLAM's identification and tracking technology is also involved in the field of e-commerce to enhance the shopping experience of our online platform. If we need to buy a refrigerator for our home, we no longer need to worry about trivial issues such as location and size of the refrigerator. Scan the space environment online through the mobile phone camera, and the plane or three-dimensional objects in the real scene will be quickly identified. SLAM The algorithm will cleverly place the "virtual refrigerator" in the most suitable three-dimensional space and place it in a suitable place in the room.
JD APP enters to enhance online platform shopping experience
Recently, OPPO has cooperated with SenseTime to absorb a number of APPs, and jointly promote the popularization of AR applications. The first batch of APPs that are stationed in King Honor and JD.com are to let more consumers feel through SenseTime’s SLAM underlying technology and AR support. A new way to interact with AR applications. Aiming at the characteristics of in-depth customization and optimization of vertical industries, SenseTime will develop corresponding AR enhancement toolkits for mutual entertainment, games, e-commerce, advertising, tourism, education, home improvement and other industries to help APP manufacturers deepen industry AR applications.
In-depth customized cooperation to give users the best experience through APP interaction
The SenseAR underlying engine solution developed by SenseAR provides mobile phone manufacturers with basic AR capabilities. Based on the information and data provided by hardware and mobile phone manufacturers, real-time processing through algorithms, feedback of spatial positioning information and three-dimensional information of spatial scenes, mobile phone manufacturers package the algorithms as a basic capability of mobile phones, and developers based on packaged SDK and mobile platform You can develop more complex logic applications. SenseTime will work with partners to build an AR content creation platform, and provide AR content creation tool chains and AR content materials to APP developers to solve the pain points of lack of AR content and low creation efficiency.
Compared with similar AR platforms, the hardware adaptation and low cost of software development of the SenseAR platform are the core advantages of the platform. Different from ARCore’s high hardware standards, SenseAR supports different hardware configurations and can run stably on sensor devices without a gyroscope. Even popular models with a price of around 1,000 yuan can be well presented on mobile phones. The performance, covering a wide range. And it supports multiple operating systems such as Android, iOS, Windows, Linux, Mac OS, etc. It only needs to develop once to span various operating systems and hardware terminals.
Mbb Tabbing Ribbon Machine,Mbb Round Ribbon Machine,Solar Tinned Copper Ribbon Machine,New Energy Pv Ribbon Machine
Jiangsu Lanhui Intelligent Equipment Technology Co., Ltd , https://www.lanhuisolar.com