Big companies are never in a hurry.
Four years ago, Apple’s ar layout was described in this way.
Now, after continuous recruitment, talent gathering, patent application and technology reserve, it seems that everything is ready.
In January 2021, Bloomberg reported that Apple will launch the long-awaited ar device in 2022, and the information on recently exposed the latest details of an apple MR device (MR, namely hybrid reality, combining the advantages of VR and AR).
The era of AR for apple is coming soon.
Large exposure of MR head display details
According to people familiar with the matter, Apple’s plan is to first launch a VR head display to serve as a “springboard” for future ar devices, according to the company.
It is reported that this VR device will display a full range of 3D digital environment for games, video and social networking. Apple plans to launch the product as early as 2022 to compete with oculus (Facebook), PlayStation VR (Sony) and HTC’s VR head show, according to people familiar with the matter.
It’s Apple’s usual practice to use emerging consumer technologies to build products including music players, smartphones, tablets and smart watches, as pointed out by Bloomberg. This time, however, apple is trying to build a high-end niche product, which is expected to cost much more than its competitors.
Although Apple said it would not comment, this is in line with the product plan previously disclosed by CEO Tim Cook:
There is no denying the potential of VR, but the ultimate goal of the company is ar.
On February 4, 2021, the latest report of the informatio opened the veil of Apple’s first head display (the effect picture is shown below).
I learned that Apple calls it MR head display internally, which can combine VR experience with games and other applications.
In terms of hardware, the device includes dozens of cameras (tracking the movement of hands), lidar sensors (mapping objects in physical space, similar to the AR application on iPad pro and iPhone 12 Pro), and supports VR and MR applications.
In addition, the device is equipped with dual 8K display screen, and its resolution will far exceed any commercial VR head display product on the market.
8K display does not always run at full resolution, it supports foveatedrendering technology based on eye tracking, that is, by tracking the user’s eye movement, rendering the surrounding area at a lower resolution, so as to improve performance.
It is worth mentioning that the chip adopted by the device is expected to be developed by Apple itself, which may be more powerful than the current M1 chip.
According to reports, Apple’s device will try a variety of control methods including manual control and eye control. In addition, there will be a dial design on the side of the device, as well as an accessory similar to a collar.
In addition, the details include:
It is made of net material and light in weight;
The price is high, or about $3000 (which will be much higher than most independent VR head displays, such as oculus Quest 2, which costs $299);
Learn from the design of many apple devices, such as the replaceable watch strap of Apple watch.
In terms of time, JPMorgan securities analysts also revealed that Apple will officially release VR and AR products in the first quarter of 2022, and most of the related parts will start to ship in the fourth quarter of 2021, including semiconductor manufacturer TSMC and lens manufacturer daliguang.
In fact, Apple’s layout on VR and AR has already begun.
The first is talent.
As early as January 2016, apple began to set up a “secret R & D team” with hundreds of people to test VR head display and other technologies.
The team is headed by Mike Rockwell, a former executive vice president of Dolby, who reports to Dan Riccio, who is mainly responsible for the iPhone and iPad hardware engineering team.
Later, internally, apple transferred people from other software and hardware departments to the AR team, including Fletcher Rothkopf, who was responsible for Apple watch design, and Tomlinson Holman, who created the sound standard thx; Outside, from 3D video to wearable hardware, there are big companies joining in, such as codywhite, the former chief engineer of Amazon lumberyard virtual reality platform, Duncan mcroberts, the former director of meta software development, Yury Petrov, the former oculus analyst, and avibar Zeev, who has worked in hololens and Google Earth.
The second is technology.
Since 2013, apple began to acquire a series of related companies, including:
In November 2013, it acquired primesense, an Israeli hardware company, which mainly manufactures Kinect motion capture cameras for Microsoft Xbox.
In December 2014, faceshift, a face recognition company, was acquired. Its technology can capture the user’s face image in real time by using the camera, and even generate a virtual head image.
In April 2015, Linx, an Israeli start-up, was acquired. Linx was founded in 2011. Its technology can increase the depth of field in photos by dividing digital images into three parts.
In May 2015, it acquired German ar company metaio, which mainly develops augmented reality application software for smart phone platform.
In January 2016, it acquired flybymedia, which mainly uses spatial awareness technology to enhance people’s understanding of its surrounding environment.
In August 2018, akonia holography, a Denver start-up company, was acquired. The company’s main business is to build ar equipment lens.
In October 2019, apple acquired a 3D graphics company called ikenema.
It is worth mentioning that in 2016, a patent document named head mounted display accessories for retaining portable electronic device with display was exposed.
The document shows a head display that can display specific images and pair with iPhone. It looks like a pair of ordinary glasses, with built-in headphones, lightning interface connected to iPhone and customized lenses.
At that time, it was still the stage when Facebook, HTC and Sony were divided into three parts, but apple, the sleeping giant, had begun to wake up.
After that, Craig federighi, Apple’s senior vice president of software engineering, said at Apple’s annual Developers Conference WWDC in 2017:
Apple is building the world’s largest ar platform and starting to provide developers with the tools needed to build and enrich virtual content for real scenes such as interactive games, immersive shopping experience and industrial design.
At that time, Apple launched the ar software development kit, which was the first step for apple to enter ar after recruiting and applying for patents.
Since then, Apple’s two important product lines, MAC and iPhone, have started to support VR and AR respectively, and arkit has been updated year by year
In 2018, arkit2 was released, with two major updates: usdz, a new file format cooperated with Pixar, and AR shared by many people;
In 2019, arkit3 will go online, introducing real-time human occlusion, motion capture and tracking, front and rear cameras that can use hardware devices at the same time, facial expression ar experience, etc;
In 2020, arkit3.5 will be launched, and lidar scanner and depth sensing system on iPad Pro will be used to provide a more realistic ar experience.
In October 2019, Mike Rockwell hosted an internal staff meeting.
At the meeting, employees had an experience through Apple’s ar head show: a virtual coffee machine was placed on a real table, and the people behind the coffee machine were blocked.
In addition to showing the effect of AR products, Apple executives also clearly disclosed the roadmap of AR products for the first time: AR head display will be released in 2022, and AR glasses will be released in 2023.
More importantly, Apple executives expressed their vision for the new device at the meeting. They believed that AR might replace iPhone in 10 years, which is consistent with Tim Cook’s idea:
AR is a dream like a smartphone.
Now the details of Apple’s VR and AR products have been exposed, and its layout has become mature. Let’s leave the rest to time.
Editor in charge: Tzh