AVS Forum banner

Apple to Collaborate With Sony for 3D Camera on iPhone XI

2K views 11 replies 3 participants last post by  3DBob 
#1 ·
8:52 am ET January 2, 2019 (Zacks) Print
Reportedly, Apple AAPL is collaborating with Sony Corporation SNE to deploy 3D cameras on iPhone XI. The company plans to leverage Sony’s long-range 3D Camera based on Time of Flight (ToF) technology.

This move is expected to enhance iPhone’s photography, security and gaming features, which will help in luring more customers. Further, it will enable users to precisely map objects in 3D and focus on every part of the picture up to five meters. Moreover, it can also be used to track objects in the dark.

Sony’s 3D camera boasts reliability and consistency. The new camera will be accompanied with a few sensors, which will shrink the notch. Moreover, this collaboration is expected to give Apple a competitive edge and boost user confidence as a 3D map of the users’ face makes hacking extremely difficult.

Notably, the new camera will also be used for Augmented reality (AR) and Virtual Reality (VR) experiences to boost user engagement level. Players can make gestures to manipulate the gaming environment, fight or cast a spell.
 
#2 ·
Well, it could just be for improved facial recognition. FaceID is using a 3D mapped face, using structured light (via an IR projector) to detect depth, similar to how the original Kinect worked. In fact, I believe Apple acquired the Israeli company (PrimeSense?) behind the technology after it was passed up by Microsoft.

This is why it is hard to fake recognition on FaceID compared to most other systems which just rely on a 2d camera.

It would not be surprising if Apple was going with time of flight as an alternative depth mapping camera system to improve FaceID recognition. While it's fairly good now, it can't differentiate between family members very well (kids that look like their parents can fool it, as can twins), so maybe the enhanced depth maps available with time of flight may make it possible to keep family members out.

At least, that's where I see things going at first. Not sure if they will have a time of flight at the back
 
#3 ·
Worf- As per Sony- it's Apple's entry into VR. Sony is the right partnership because of their pioneering consumer stereoscopic cameras in the 21st century. They are working on using the two cameras front and back to create a 360VR 2D and the twin cameras on the back for 3D VR180. This is a new direction that was introduced about 2 years ago that this past year has become very popular. The next step in VR shooting is to combine the capability of these VR cameras with the phone camera. The new super fast processors in the Samsung and Apple X+ series is enabling 4K 60fps VR and AR editing in the phones.

One thing I am seeing also in the VR 3D world is the level of cooperation with VR camera companies to work with the Apple A12 processors by 3rd party companies like Insta and Human Eyes. The apps software seems to be always ahead of the development curve offering new features first on the iphone and a month later on Android. I have both here so I see the race to do VR and AR ahead now with the iphone.

AR is the most developed on Facebook's apps right now but I see Google soon to catch up. Here is a demo of where that technology was last Fall done by a friend, Jim Malcolm.
https://www.facebook.com/xiality/vi...STEwMDAwMTM5OTU2ODIzMjoxNzA0Mjc0MzIyOTYyNTM3/

This can already be accomplished with ease on the new iphone XS.


As a 3D enthusiast, VR and AR is where 3D is heading. No longer will be competing with flat screen TV that used 3D glasses for stereo. 3D is heading into the immersive presentation world where the goal is a dream like reality experience. You feel like you are in there. Apple and Sony's venture is far from just security improvement, it is a goal to have everything on one device. Shoot, edit, publish, and watch it on your iphone. We already can edit, publish, and watch 3D VR on the iphone without goggles, now they want to be able to shoot it as well.
 
  • Like
Reactions: 3DBob
#4 ·
It appears, based on recent news, that Apple is taking a dive because it's running out of tech to implement in it's new yearly phone blowouts. I would say this is part of a scramble to get something to make their next iphones sell...and maybe, just maybe, to help the 3D world...one can hope...:D.
 
#5 ·
Apple is taking a dive with the rest of the market because the market is nose diving due to the Fed's raising interest rates and the President's failure to get a deal on trade with China. These issues have clearly triggered the Apple fall as with all stocks except for few exceptions. When those two issues resolve, you will see the entire market rebound to new highs along with Apple.
 
#6 ·
I agree with all you said, but I was reading that Apple is struggling to find new tech improvements for new releases on a yearly basis that would keep a constant stream of revenue coming in. More and more people are not upgrading their iphones; it appears as the tech is already maxed out. I think they have plateaued like Gopro, unless they can get into augmented reality, etc. with something less than a $1,000, and that would include getting into serious gaming systems, which I don't see them doing.
 
#7 ·
Bob have you been watching? How can you say you don't see Apple dabbling in Gaming and AR and even VR editing is now better on the iOS than on the PC. I don't play Fortnite but my grandsons are addicted to that game. Ever heard of it? The new iOS editor for VR360 and VR180 that allows for keyframing, fast motion and slowmo, image stabilization and color effects, and many others. Users of the Insta360 oneX do most of the VR editing on the iphoneX level processor and then port to Premiere Pro for additional editing, or just upload publish directly from the iphone.

Another thing I did over the holidays was I built a holographic lens and turned my iphone into a holographic projector. These are easy to build to fascinate kids. later I noticed these can be bought to attach to the iphone.


But Apple is also tackling the future with new business models to replace the carrier subsidies. This is in the form of generous cash credits for recycling the older iphone in upgrading and zero interest for 2 year financing with as early as one year upgrades. Apple recognizes the greatest asset is it's growing base and is planning to do keep them with incentives.
 
#8 · (Edited)
Yeah, Don, I know all about Fortnite. My nephew's son plays it all the time. He watches others play on his iPhone at school all the time. He says all of his friends do it--teachers don't seem to care anymore--the world we are living in...:rolleyes: For Apple, I'm mainly thinking about the iphones, although they have a VR/AR project in the works, https://www.macrumors.com/roundup/apple-vr-project/, but I think it's going to be a little too late as Oculist Rift seemed to have that space locked up this past Christmas. On the creation side, like all of us here, the tools might be there, but I don't see the big revenue stream for that side of the business. I think it will mostly be user oriented. They pulled themselves out of the pooper before, so maybe they can do it again through incentives, but the market is really getting saturated. My son got an Oculist Rift for Christmas and loves it. 2018 is the first year in the past 5 years, he hasn't upgraded his iPhone, though. He doesn't see the point.
 
#9 ·
At NAB I had the opportunity to try both the Oculus Rift and the HTC Vive Pro and neither excited me for image quality as it wasn't any better than my HMD here. The one that has those beat has been my Sony Xperia Premium with it's 4K screen. I use it with either the clip on twin lenses which I can carry in my pants pocket for a quick look with Google Cardboard selection and VR Player or YouTube. But in my budget this year is the PiMax 8Kx which has a full immersion 3D 8K screen (4K per Eye) Like the Rift it has to be tethered to a high end graphics card for performance. I think this will be the next level technology I can enjoy over what I have now. It is also supposed to use the Oculus control so it will work with my Adobe Premiere Pro editing.

I also jumped on a new camera today. The Insta360 OneX. Now this is not 3D but it has many features that my new Vuze XR does not have...yet. I also looked into the underwater housing and decided that while I am supposed to get a free one with my XR, the company has been having trouble with the design. Looking closely and the design I can see why. Mechanically it looks like a nightmare for the seal. By contrast the Insta360 OneX housing has a simple standard O ring in an oval grove that I know from my own underwater cameras and lights I built works very well and does not leak. I had my light to 430ft in a trimix dive so I know! While I have no plans to dive anymore, the Vuze XR housing seal looks like a mechanical nightmare compared to the Insta360. Plus, it's on the market now and many divers have vetted it in open water deeps dives. The camera itself uses a better image stabilization than the XR as well. Both are 5.7K but the Insta uses HDR. Too bad it isn't 3D but their 3D 360 camera is way too big and not good for my travels.

Meanwhile the Insta360 One X has a great app for editing on the iphone. They also make a 360 nano that plugs directly into the iphone X. Will these be the GoPro Fusion killer? I wouldn't go that far but Apple is well on the way to entering the VR world as it is the next big thing in consumer shooting. VR and 3D will take off if Apple builds it into their new iphone XI.
 
#11 ·
From what I am seeing VR is taking hold fast and growing among noobies to video. The idea that you can shoot without worry about pointing the camera just right and then ealsily frame the shot on the iphone or android phone, trim the video, add silly effects so easily and intuitive then just publish to Facebook with the push of a button on the screen is very appealing to many. Once that is available all on the iphone from camera to publish, the convenience will sell it to a whole new paradigm.

Something you may not be aware of is that to publish you have several choices-
1. 360VR 2d or 3D
2. 180VR 2D or 3D
3. 1x1 AspectRatio flat 2D
4. 16x9 flat 2D or 3D

#3 is mostly for Facebook now and they may add 3D in the future. The others are hear now depending on the app you use.

Then there is the time lapsed with pan and scan and hyperlapse and slo-mo.

One app I saw even allows for the vlog publish where your scene is 360 degree FOV option and your speaker cameraman image is shot simultaneously as a picture in picture portrait of you narrating.

Some people are doing the editing on an ipad to have a larger screen than the iphone but the ipad doesn't offer any other advantages. The latest iphone XS series is much faster rendering than the ipad however.

We still have a couple of SNAFU's to deal with in VR editing and that is the new cameras are shooting in 120MBS at 5.7K 30fps and the iphones can't display this so to do everything in the iphone you have to shoot at 4K 60fps top quality. To edit 5.7K 30fps, the workflow is the PC with adobe Premiere CC2019. No other platform can do that and the GPU has to be a GTX 1070 or higher.
 
#12 · (Edited)
This supposedly is a render of the new iPhone XI with 3 cameras. I'm assuming the two horizontal cameras would be used for 3D, but they are very close together as we would expect. Some rumors have the 3D as nothing more than 3D face scanning for security--and maybe 3D scanning of objects, usable in some apps to capture 3D objects for gaming, such as your face as an avatar on a game character.

https://www.youtube.com/watch?time_continue=1&v=CassjP_rKpI
 
This is an older thread, you may not receive a response, and could be reviving an old thread. Please consider creating a new thread.
Top